script to pull string from log file and email it only once

pnorth72 · 09-11-2017, 09:56 AM

Hello All,

this is my first post here and I am new to scripting so please forgive the ignorance.

So i'm trying to write a simple script to send me an error message from a log file.

I have searched on this site and did find something that may work but since i'm so new to scripting I don't quite understand exactly what I need to do.

This is the script that someone posted here before

#!/bin/bash

string=fail

tail -n0 -F /pat/to/file | \
while read LINE
do
if echo "$LINE" | grep "$string" 1>/dev/null 2>&1
then
echo "String found on $HOSTNAME" | mail -s "Subject" dummy@iam.com
fi
done

my problem is I don't understand some of the info/arguments here.

I just need a simple script that is looking at a log file when it is kicked off.
once the line comes up that I am looking for it will send that line in an email to me.

I also don't want to get the same info over and over again, so once the line is emailed to me, I don't want it to send me the same error again, only new errors.

Here is the actual paths file and string i'm looking for
path=/opt/IBM/WebSphere/AppServer/profiles/Custom01/logs/WCProd*/
LogFile= SystemOut.log
string= "authorization is declined by the back-end system, the response reason code is {0}"

if anyone can help me it would be huge! I really do appreciate it.

thanks
Paul.

Turbocapitalist · 09-11-2017, 10:25 AM

It helps if you post in [code] [/code] tags so that the indentations are kept:

Code:

#!/bin/bash

string="fail";

tail -n0 -F /path/to/file | \
while read LINE
do
    if echo "$LINE" | grep -q "$string" 2>/dev/null
    then
        echo "String found on $HOSTNAME" | mail -s "Subject" dummy@iam.com
    fi
done

Well you can look at the manual pages for tail, grep, and mail to see an explanation of the specific options.

The -n 0 for tail means that only the lines added to the file after tail is up and running will be passed on to read via the pipe.

Then also look at the manual page for bash to see about "while" and "read". However, that manual is quite daunting and you'll have to remember that it is a reference document to be navigated and not a tutorial to be walked through one line at a time.

pnorth72 · 09-13-2017, 07:32 AM

Excellent

thanks for the reply, I will give that a try.

pan64 · 09-13-2017, 08:11 AM

Quote:

Originally Posted by pnorth72

I also don't want to get the same info over and over again, so once the line is emailed to me, I don't want it to send me the same error again, only new errors.

Using while to read a file and grep line by line is not really suggested/recommanded. I would say this is a common error, grep can itself scan the whole file, there is no reason to grep each line one by one.
Actually

Code:

if tail -n0 -F /path/to/file | grep -q "$string" 2>/dev/null; then
then
    echo "String found on $HOSTNAME" | mail -s "Subject" dummy@iam.com
fi

may work and will find $string only once.

pnorth72 · 09-13-2017, 02:51 PM

Quote:

Originally Posted by pan64

Using while to read a file and grep line by line is not really suggested/recommanded. I would say this is a common error, grep can itself scan the whole file, there is no reason to grep each line one by one.
Actually

Code:

if tail -n0 -F /path/to/file | grep -q "$string" 2>/dev/null; then
then
    echo "String found on $HOSTNAME" | mail -s "Subject" dummy@iam.com
fi

may work and will find $string only once.

So this works once, but what if I needed it to run 24x7 to catch any new errors?

ideally it would be great to constantly read the last line of the log file looking for the error, once it finds it send an email out, but the script should still continue to read the log file looking for the error in case it occurs again.

thanks so much for the help

lsalab · 09-13-2017, 04:56 PM

Why use a custom script at all?

Have you tried using rsyslog directly, it actually reads every log in your OS as it's generated, and with the ommail module you can send an email whenever a certain condition is met. Take a look:

http://www.rsyslog.com/doc/v8-stable...es/ommail.html

chrism01 · 09-15-2017, 01:05 AM

To check 24/x7 & not repeat error msg/mail:

Try it in Perl using this module http://search.cpan.org/~mgrabnar/File-Tail-1.3/Tail.pm.

In the loop, check if error exists in db (or similar eg hash), if it exists its already been emailed, do next iteration.
If not, email error, add to db, then next iteration.

syg00 · 09-15-2017, 01:22 AM

Quote:

Originally Posted by pnorth72

this is my first post here and I am new to scripting

I'm thinking adding perl to the mix is asking a bit much of the OP.
Small steps, one at a time.

TenTenths · 09-15-2017, 06:22 AM

I've had to do this before and these are the high level steps I took.

Create the script and touch /var/log/raisedalerts

The script should:

Create a temporary file in /tmp
grep the whole log file for the error conditions and store them in the temporary file.
diff the temporary file against /var/log/raisedalerts
If there is a difference then it means that either there's a new alert or the log file has rotated, send the mail with all the lines in the diff that begin with + as these are NEW
Copy your temp file to /var/log/raisedalerts
Tidy up any temporary files.

joe_2000 · 09-15-2017, 05:15 PM

There is also logwatch which does this in a very handy way.

EDIT: Well, it doesn't exactly do what you asked for, but essentially it helps you to monitor logs and get email summaries.

If you want to be notified immediately upon occurrence of a certain message, it's not the right solution.

chrism01 · 09-18-2017, 01:00 AM

@syg00 : I know what you mean, but doing this properly is non-trivial and that perl module does all the tricky bit for you.

It's up to the OP: writing your own is certainly educational