-   Linux - Newbie (
-   -   Shell script to Continuously scan a log file for success (

novice82 10-08-2009 09:06 PM

Shell script to Continuously scan a log file for success
Hello members,

I have some doubts on how to write a script that can reports success / failure of a batch job ?

1. Run a batch job:

2. Wait and search for a particular string in the Log file:

tail -f log01*.txt | egrep -v "^SUCCESSFUL"
  echo "continue with the other tasks"
    tail -f log01*.txt | egrep -v "^FAILURE"
  echo "Process failed!!"

My questions are :
1. My batch job is likely to create a couple of logs with of type log01*.txt. How do I ensure the script scans through each of those logs to find either of the strings? I'm assuming the '*' can help me with this.

2. Do I continuously scan the log file ( that might still be in the "generation" state ) or wait until the logfile is created completely and then scan for a string ? In this case, how does the script detect that a particular log file has generated successfully, so that it can search for a string ?



GrapefruiTgirl 10-08-2009 10:34 PM

Some tips..
Hi Kris, welcome to LQ!

I'm not 100% sure I understand exactly what you're trying to do, but I do understand that you want to scan the log files for the SUCCESSFUL message.

First, a few tips to get you started:

1) in bash, use ELIF or ELSE, not ELSIF, and either way, these need to be accompanied by an opening IF and a closing FI.


some code here...
if [ something A ]; then something B..
elif [ something else ]; then something C..

2) the tail command (or any command for that matter) will keep control of execution until it is killed, unless you put an ampersand after a command. If you don't use an ampersand, the tail command will start, but the rest of your script will not execute. Example:

shell$ tail -f somefile.txt #this will run forever until killed
shell$ tail -f somefile.txt & # the & will make the command 'fork' into the background, so execution will continue on..

3) tail will not work if the file does not exist yet, so you have to make a choice of some sort: either create the logfiles initially, using the "touch" command (assuming you know what they will be called) or wait until you're sure the logfiles exist, and THEN do a little search to locate all the logfiles and either do a tail on them, or just grep them for the success/failure messages.

4) Unless you need to know immediately whether there's a success/failure message, you don't need to use tail; if you don't need to know the instant a message occurs, just grep the logfiles after the job is done.

5) If you can wait till the process is done, to check the logs, you could do something like:


for filename in /path/to/log01*.txt; do
 if [ "$(grep "SUCCESSFUL" $filename)" ]; then
  echo "Woohoo! Success!"
  echo "Boo!! Failure.."

I hope this helps you get going but if you need more help, just ask away!


novice82 10-08-2009 11:17 PM

Hello Sasha,

Thanks a ton for the detailed reply. I've just sample run the code on existing logfiles and it works fine. I want to scan the logs for Success / Failure, only once the job is finished its processing. If that be the case, can you show me how to "wait until the job has finished processing, before i grep in the log files " ? I'm assuming, I could find if the process has finished or not by waiting for some exit status ? how do I go about reading the exit status of a submitted job ?

A small change in your code above ; It seems to be missing the "fi" to close the "if" statement.



chrism01 10-09-2009 01:50 AM

A simple soln is to write a wrapper script thus


# run your main app prog here
/somedir/prog > /somedir/prog.log 2>&1

# now run the grep; I always arrange my app progs,
# so that 'Success' or 'Failure' appears in the last line of the logfile...

result=$(grep Success /somedir/prog.log|tail -1)
if [[ $? -eq 0 ]]
    echo OK
    echo Ooops

This is just conceptual code, amend to suit.

Alternately, in app progs I've written myself, so I have the src, I actually do the logging and email as part of the program's controlled cleanup/shutdown fn.
The above external check is only reqd if the prog may crash uncontrollably.

novice82 10-09-2009 02:20 AM


Originally Posted by chrism01 (Post 3713134)
A simple soln is to write a wrapper script thus

Thanks Chris. Just what the doctor ordered :)

All times are GMT -5. The time now is 03:26 AM.