Tail multiple log files.

Satyaveer Arya · 07-02-2013, 04:59 AM

Hi,

I have got a script to tail multiple log files.
Here is the script below:

Code:

$ vi multi-tail.sh
#!/bin/sh

# When this exits, exit all back ground process also.
trap 'kill $(jobs -p)' EXIT

# iterate through the each given file names,
for file in "$@"
do
	# show tails of each in background.
	tail -f $file &
done

# wait .. until CTRL+C
wait

Now, I can run this script like this, mentioned below:

Code:

$ ./multi-tail.sh error_log access_log

As, right now I don't have test environment. So, can anyone please help me in getting the exact meaning of this line in the above mentioned script:

Quote:

trap 'kill $(jobs -p)' EXIT

As, it is using kill command. Will it kill background running processes?

druuna · 07-02-2013, 05:07 AM

Quote:

Originally Posted by Satyaveer Arya

As, it is using kill command. Will it kill background running processes?

Yes, once you exit the script all processes started within the script will also be killed.

You might want to have a look at the bash manual page and search for jobs, the -p option does the following:

Quote:

-p List only the process ID of the job's process group leader.

David the H. · 07-03-2013, 01:44 PM

trap catches process signals like SIGTERM and SIGKILL (produced by things like ctrl+c and closing the terminal), and lets you redefine what happens when they are received. In this case it has only been configured to handle the EXIT pseudo-signal, killing the list of all sub-processes produced by "jobs -p".

BTW, I'm wondering about the output to all these tail commands. As it stands I believe they'll all go to the same terminal mixed together. Is that what you want?

Actually, I think you could drop the loop and just run:

Code:

tail -f "$@"

There will be only one process, so there's no need to background anything, and it also conveniently prints out which file each new line comes from.

Satyaveer Arya · 07-04-2013, 04:59 AM

Hi David,

Quote:

BTW, I'm wondering about the output to all these tail commands. As it stands I believe they'll all go to the same terminal mixed together. Is that what you want?

Yeah, I tested it and it goes on the same terminal, all mixed up. Here is the output mentioned below:

Quote:

[root@localhost ~]# ./logsrun.sh /var/log/boot.log /var/log/audit/audit.log
./logsrun.sh: line 1: $: command not found
type=USER_START msg=audit(1372928401.467:19): user pid=2585 uid=0 auid=0 ses=2 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_open acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=CRED_DISP msg=audit(1372928401.630:20): user pid=2585 uid=0 auid=0 ses=2 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=USER_END msg=audit(1372928401.631:21): user pid=2585 uid=0 auid=0 ses=2 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=USER_ACCT msg=audit(1372928461.652:22): user pid=2590 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:accounting acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=CRED_ACQ msg=audit(1372928461.652:23): user pid=2590 uid=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=LOGIN msg=audit(1372928461.653:24): pid=2590 uid=0 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 old auid=4294967295 new auid=0 old ses=4294967295 new ses=3
type=USER_START msg=audit(1372928461.654:25): user pid=2590 uid=0 auid=0 ses=3 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_open acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=CRED_DISP msg=audit(1372928461.973:26): user pid=2590 uid=0 auid=0 ses=3 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:setcred acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=USER_END msg=audit(1372928461.974:27): user pid=2590 uid=0 auid=0 ses=3 subj=system_u:system_r:crond_t:s0-s0:c0.c1023 msg='op=PAM:session_close acct="root" exe="/usr/sbin/crond" hostname=? addr=? terminal=cron res=success'
type=USER_AUTH msg=audit(1372928838.324:28): user pid=2644 uid=500 auid=500 ses=1 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:unix_chkpwd acct="satyaveer" exe="/sbin/unix_chkpwd" hostname=? addr=? terminal=? res=success'
Starting acpi daemon: [ OK ]
Starting HAL daemon: [ OK ]
Retrigger failed udev events [ OK ]
Enabling Bluetooth devices:
Starting sshd: [ OK ]
Starting postfix: [ OK ]
Starting abrt daemon: [ OK ]
Starting crond: [ OK ]
Starting atd: [ OK ]
Starting rhsmcertd... [ OK ]

And I don't want like this, all mixed up.

And you're suggesting me to put only

Quote:

tail -f"$@"

. Here it is:

Quote:

#$ vi multi-tail.sh
#!/bin/sh

# When this exits, exit all back ground process also.
trap 'kill $(jobs -p)' EXIT

# iterate through the each given file names,
#for file in "$@"
#do
# show tails of each in background.
tail -f $file &
#done

# wait .. until CTRL+C
wait

Satyaveer Arya · 07-04-2013, 11:33 AM

Hi All,

I have to give log files path in the script only. Not while running the script. So, can anyone suggest how and where to add the log files path in the script?

Not during running the script:

Quote:

[root@localhost ~]# ./logsrun.sh /var/log/boot.log /var/log/audit/audit.log

David the H. · 07-05-2013, 12:44 PM

I'm saying you can use a single tail command and remove almost everything else from the script. Process backgrounding and cleanup with trap aren't needed if there's only a single process to manage.

Adding a path prefix is easy, as long as you use bash or another more advanced shell, instead of sh.

Code:

#!/bin/bash

logpath='/var/log/'

tail -f "${@/#/$logpath}"

exit 0

Note that you still have to supply the full sub-path, i.e. "audit/audit.log".

parameter substitution

The only realistic way you could have separate outputs would be to give each tail command its own terminal window. But that would mean having to use a loop again. I think the above is a good compromise, since tail will clearly label which output comes from which log, as I mentioned before.

konsolebox · 07-07-2013, 04:49 AM

Quote:

Originally Posted by Satyaveer Arya

Hi All,

I have to give log files path in the script only. Not while running the script. So, can anyone suggest how and where to add the log files path in the script?

Not during running the script:

Following David the H.'s recommendation of just using the tail command, you could simply just place those paths in an array. if you're using bash you can do:

Code:

#!/bin/bash
LOGFILES=("/var/log/boot.log" "/var/log/audit/audit.log")
tail -f "${LOGFILES[@]}"

Note that path strings need not to be placed within double quotes ("") inside the array declaration if they don't have white space characters but it's better or cleaner that way.

But then again, wouldn't it be just better to place those along with the command instead? And you don't have to depend on any advanced shell for that.

Code:

#!/bin/sh
tail -f "/var/log/boot.log" "/var/log/audit/audit.log"

Also since you don't have to run any other command after that, it would be just better to substitute the calling shell's process with tail's what would become its process:

Code:

#!/bin/sh
exec tail -f "/var/log/boot.log" "/var/log/audit/audit.log"

Satyaveer Arya · 07-13-2013, 02:24 AM

Code:

#!/bin/bash
LOGFILES=("/var/log/boot.log" "/var/log/audit/audit.log")
tail -f "${LOGFILES[@]}"

If I use the above script on linux then both the log files will update periodically.
But if I use it on unix(solaris) then only the first log file specified in script will update periodically.
Why is it so?

Note: Assuming the above mentioned log file in script on unix also.

konsolebox · 07-13-2013, 02:57 AM

Quote:

Originally Posted by Satyaveer Arya

Code:

#!/bin/bash
LOGFILES=("/var/log/boot.log" "/var/log/audit/audit.log")
tail -f "${LOGFILES[@]}"

But if I use it on unix(solaris) then only the first log file specified in script will update periodically.
Why is it so?

First make sure that you're running the script with Bash. If not, use the simpler method which is compatible with all shells based from the original sh.

Code:

#!/bin/sh
exec tail -f "/var/log/boot.log" "/var/log/audit/audit.log"

Second it would depend on the version of tail that you're using. Try running tail manually first then see if it does or doesn't work on two files.

Satyaveer Arya · 07-13-2013, 03:03 AM

I tried the script using bash shell. Which version of tail should work fine for multiple log files on unix?

Satyaveer Arya · 07-13-2013, 03:23 AM

Even I tried to tail two files manually at a time but only first one was showing the output.
Putting in the script doesn't work.

I tried to find the version of tail command using

Code:

tail --version

Even this wasn't showing the version. I'm using Solaris 10 64 bit.

konsolebox · 07-13-2013, 03:25 AM

Quote:

Originally Posted by Satyaveer Arya

I tried the script using bash shell. Which version of tail should work fine for multiple log files on unix?

On newer versions probably, but I'm referring to the one packaged with coreutils (don't know which was for Solaris). And I'm also not sure which version of tail has already worked like the one in your Linux system. You could try building tail from a new version of coreutils but you need to consider the possible impact on the system. Perhaps you could install it separately on a different directory like /usr/local/bin or /opt. You don't necessarily need to install the whole coreutils but you also have to consider the required dependencies, even those which are needed to be updated. I'm sorry if I can't be much of help with this and I also find it sensitive but perhaps starting a new thread that would guide you on installing a new version of tail would be helpful. The ones you would really need to consider are the libraries that you might need to upgrade just to make tail work while not breaking linkage of other binaries that also depend on it.

Satyaveer Arya · 07-13-2013, 06:19 AM

I have to write a script for logs monitoring on production server. But the problem here is we cannot update the package on our own on production server. We're not authorized to do updates on server. So, this is the situation here...

konsolebox · 07-14-2013, 08:07 PM

You would have no choice but consider other solutions like implementing it in other languages like Perl if it's available I guess.