Latest LQ Deal: Latest LQ Deals
Go Back > Forums > Linux Forums > Linux - Newbie
User Name
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!


  Search this Thread
Old 10-08-2009, 09:06 PM   #1
LQ Newbie
Registered: Oct 2009
Posts: 13

Rep: Reputation: 0
Shell script to Continuously scan a log file for success

Hello members,

I have some doubts on how to write a script that can reports success / failure of a batch job ?

1. Run a batch job:

2. Wait and search for a particular string in the Log file:
tail -f log01*.txt | egrep -v "^SUCCESSFUL" 
   echo "continue with the other tasks"
     tail -f log01*.txt | egrep -v "^FAILURE"
   echo "Process failed!!"
My questions are :
1. My batch job is likely to create a couple of logs with of type log01*.txt. How do I ensure the script scans through each of those logs to find either of the strings? I'm assuming the '*' can help me with this.

2. Do I continuously scan the log file ( that might still be in the "generation" state ) or wait until the logfile is created completely and then scan for a string ? In this case, how does the script detect that a particular log file has generated successfully, so that it can search for a string ?


Old 10-08-2009, 10:34 PM   #2
LQ Guru
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 553Reputation: 553Reputation: 553Reputation: 553Reputation: 553Reputation: 553
Some tips..

Hi Kris, welcome to LQ!

I'm not 100% sure I understand exactly what you're trying to do, but I do understand that you want to scan the log files for the SUCCESSFUL message.

First, a few tips to get you started:

1) in bash, use ELIF or ELSE, not ELSIF, and either way, these need to be accompanied by an opening IF and a closing FI.

some code here...
if [ something A ]; then something B..
elif [ something else ]; then something C..
2) the tail command (or any command for that matter) will keep control of execution until it is killed, unless you put an ampersand after a command. If you don't use an ampersand, the tail command will start, but the rest of your script will not execute. Example:
shell$ tail -f somefile.txt #this will run forever until killed
shell$ tail -f somefile.txt & # the & will make the command 'fork' into the background, so execution will continue on..
3) tail will not work if the file does not exist yet, so you have to make a choice of some sort: either create the logfiles initially, using the "touch" command (assuming you know what they will be called) or wait until you're sure the logfiles exist, and THEN do a little search to locate all the logfiles and either do a tail on them, or just grep them for the success/failure messages.

4) Unless you need to know immediately whether there's a success/failure message, you don't need to use tail; if you don't need to know the instant a message occurs, just grep the logfiles after the job is done.

5) If you can wait till the process is done, to check the logs, you could do something like:

for filename in /path/to/log01*.txt; do
 if [ "$(grep "SUCCESSFUL" $filename)" ]; then
   echo "Woohoo! Success!"
   echo "Boo!! Failure.."
I hope this helps you get going but if you need more help, just ask away!


Last edited by GrapefruiTgirl; 10-12-2009 at 12:35 PM. Reason: formatting + added the "fi" I forget!! Sorry.
Old 10-08-2009, 11:17 PM   #3
LQ Newbie
Registered: Oct 2009
Posts: 13

Original Poster
Rep: Reputation: 0
Hello Sasha,

Thanks a ton for the detailed reply. I've just sample run the code on existing logfiles and it works fine. I want to scan the logs for Success / Failure, only once the job is finished its processing. If that be the case, can you show me how to "wait until the job has finished processing, before i grep in the log files " ? I'm assuming, I could find if the process has finished or not by waiting for some exit status ? how do I go about reading the exit status of a submitted job ?

A small change in your code above ; It seems to be missing the "fi" to close the "if" statement.


Old 10-09-2009, 01:50 AM   #4
LQ Guru
Registered: Aug 2004
Location: Sydney
Distribution: Centos 7.7 (?), Centos 8.1
Posts: 17,843

Rep: Reputation: 2572Reputation: 2572Reputation: 2572Reputation: 2572Reputation: 2572Reputation: 2572Reputation: 2572Reputation: 2572Reputation: 2572Reputation: 2572Reputation: 2572
A simple soln is to write a wrapper script thus

# run your main app prog here
/somedir/prog > /somedir/prog.log 2>&1

# now run the grep; I always arrange my app progs, 
# so that 'Success' or 'Failure' appears in the last line of the logfile...

result=$(grep Success /somedir/prog.log|tail -1)
if [[ $? -eq 0 ]]
    echo OK
    echo Ooops
This is just conceptual code, amend to suit.

Alternately, in app progs I've written myself, so I have the src, I actually do the logging and email as part of the program's controlled cleanup/shutdown fn.
The above external check is only reqd if the prog may crash uncontrollably.

Last edited by chrism01; 10-09-2009 at 01:52 AM.
Old 10-09-2009, 02:20 AM   #5
LQ Newbie
Registered: Oct 2009
Posts: 13

Original Poster
Rep: Reputation: 0
Originally Posted by chrism01 View Post
A simple soln is to write a wrapper script thus
Thanks Chris. Just what the doctor ordered


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Shell Script or perl help. to write sections of a log to a tmp file for mailing pobman Programming 2 02-02-2009 03:30 PM
shell script to monitor log file calipryss Linux - Newbie 14 08-05-2008 10:46 PM
Filtering a CSV file from web log with shell script? Micro420 Programming 8 08-22-2007 03:13 AM
Scan log file for errors huskerharry Linux - Newbie 7 10-20-2006 01:43 AM
F-prot Anti-virus scan log, suspicous file question. webwolf70 Linux - Security 5 11-16-2004 09:15 AM > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 11:35 AM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration