Shell script tail -F

secondhandman · 05-14-2012, 12:13 PM

Hello to the kind people of linuxquestions.org!

I'm trying to create a script that will gather data from a log file that is constantly updating. I only need to include the newest data every time the script runs. This data will need to be emailed to my boss. Right now he receives an email every 5-10 minutes with the data he needs but would prefer to have a single email a day (or a least fewer).

I found this online but don't know what to make of it:

Tail

With no options it shows the last ten lines of a file.

Use tail -n x (where "x" is a number) to display the last x lines.

Try tail -F to use a continually updated version of tail (if the file changes it will be reloaded and displayed), please note that using this option will run tail is a continuous loop so you'll need to use CTRL -C to exit.

For example:
tail -n 20 somelog.txt

The tail -F option seems like it would work, but I'm not sure what to do. Any help would be appropriated.

suicidaleggroll · 05-14-2012, 12:25 PM

One option is to have the script run constantly in an infinite loop with a delay and use "cat file | wc -l" to count the number of lines. Each time it runs, it compares the number of lines in the file to the previous number of lines, then passes the result to tail to only send the updates, IE:

Code:

nlines_old=0
while [[ 1 ]]; do
   nlines_new=$(cat file | wc -l)
   if [[ $nlines_new -gt $nlines_old ]]; then
      nlines=$(expr $nlines_new - $nlines_old)
      updates=$(tail -n $nlines)
   
      # Mail $updates
   
      nlines_old=$nlines_new
   fi
   sleep 10
done

It's not perfect though because any delay between the cat | wc -l and the tail will cause it to skip lines. I guess it just depends on how quickly this file is being updated.

secondhandman · 05-14-2012, 12:32 PM

Thanks for the help!

I'm very new to scripting and am not sure what the commands in the example you gave are doing. Some solid advice though, I'll start looking into that now!

Any links to websites that can explain it to me like I'm an idiot would appreciated

suicidaleggroll · 05-14-2012, 03:03 PM

It's a very simple script, there are probably better ways to do it

Essentially it runs forever (hence the "while [[ 1 ]]"), with a delay between loops of 10 seconds (the "sleep 10"). Each time it runs through the loop, it checks the number of lines in the file ("cat file | wc -l"). If there are more lines now than the last time the loop ran 10 seconds ago, it enters the if statement. At that point is subtracts the previous number of lines from the current number of lines and stores the result (the number of lines that have been added) in nlines. Then it does a "tail -n $nlines" to grab those new lines and stores them in the variable "updates". It's then up to you to do whatever it is you want to do with that data.

ratotopi · 05-14-2012, 05:22 PM

why not to mail the required log file through cron job once a day to your boss ?

secondhandman · 05-15-2012, 08:50 AM

Still trying to get my head around this. Will update when I do!

suicidaleggroll · 05-15-2012, 09:09 AM

Another option is to copy the log file to a backup at the end of the script, and email the results of a diff at the beginning of the script. IE:

Code:

cp file tempfile
updates=$(diff tempfile difffile)

# Mail $updates

mv tempfile difffile

catkin · 05-15-2012, 09:28 AM

logtail "reads a specified file (usually a log file) and writes to the standard output that part of it which has not been read by previous runs of logtail".

secondhandman · 05-15-2012, 12:26 PM

Thanks for all the reply's. While looking into logtail, I came across a program called logwatch that seems to allow the data to be emailed. It seems like a simpler setup than having to write the script myself, but I'll figure it out! I'll let you know how it goes!

Total noob here by the way