LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-11-2017, 09:56 AM   #1
pnorth72
LQ Newbie
 
Registered: Dec 2012
Posts: 3

Rep: Reputation: Disabled
script to pull string from log file and email it only once


Hello All,

this is my first post here and I am new to scripting so please forgive the ignorance.

So i'm trying to write a simple script to send me an error message from a log file.

I have searched on this site and did find something that may work but since i'm so new to scripting I don't quite understand exactly what I need to do.

This is the script that someone posted here before

#!/bin/bash

string=fail

tail -n0 -F /pat/to/file | \
while read LINE
do
if echo "$LINE" | grep "$string" 1>/dev/null 2>&1
then
echo "String found on $HOSTNAME" | mail -s "Subject" dummy@iam.com
fi
done


my problem is I don't understand some of the info/arguments here.

I just need a simple script that is looking at a log file when it is kicked off.
once the line comes up that I am looking for it will send that line in an email to me.

I also don't want to get the same info over and over again, so once the line is emailed to me, I don't want it to send me the same error again, only new errors.

Here is the actual paths file and string i'm looking for
path=/opt/IBM/WebSphere/AppServer/profiles/Custom01/logs/WCProd*/
LogFile= SystemOut.log
string= "authorization is declined by the back-end system, the response reason code is {0}"

if anyone can help me it would be huge! I really do appreciate it.

thanks
Paul.
 
Old 09-11-2017, 10:25 AM   #2
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 5,614
Blog Entries: 3

Rep: Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859Reputation: 2859
It helps if you post in [code] [/code] tags so that the indentations are kept:

Code:
#!/bin/bash

string="fail";

tail -n0 -F /path/to/file | \
while read LINE
do
    if echo "$LINE" | grep -q "$string" 2>/dev/null
    then
        echo "String found on $HOSTNAME" | mail -s "Subject" dummy@iam.com
    fi
done
Well you can look at the manual pages for tail, grep, and mail to see an explanation of the specific options.

The -n 0 for tail means that only the lines added to the file after tail is up and running will be passed on to read via the pipe.

Then also look at the manual page for bash to see about "while" and "read". However, that manual is quite daunting and you'll have to remember that it is a reference document to be navigated and not a tutorial to be walked through one line at a time.

Last edited by Turbocapitalist; 09-11-2017 at 10:27 AM.
 
1 members found this post helpful.
Old 09-13-2017, 07:32 AM   #3
pnorth72
LQ Newbie
 
Registered: Dec 2012
Posts: 3

Original Poster
Rep: Reputation: Disabled
Excellent

thanks for the reply, I will give that a try.
 
Old 09-13-2017, 08:11 AM   #4
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 16,493

Rep: Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532Reputation: 5532
Quote:
Originally Posted by pnorth72 View Post
I also don't want to get the same info over and over again, so once the line is emailed to me, I don't want it to send me the same error again, only new errors.
Using while to read a file and grep line by line is not really suggested/recommanded. I would say this is a common error, grep can itself scan the whole file, there is no reason to grep each line one by one.
Actually
Code:
if tail -n0 -F /path/to/file | grep -q "$string" 2>/dev/null; then
then
    echo "String found on $HOSTNAME" | mail -s "Subject" dummy@iam.com
fi
may work and will find $string only once.
 
Old 09-13-2017, 02:51 PM   #5
pnorth72
LQ Newbie
 
Registered: Dec 2012
Posts: 3

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
Using while to read a file and grep line by line is not really suggested/recommanded. I would say this is a common error, grep can itself scan the whole file, there is no reason to grep each line one by one.
Actually
Code:
if tail -n0 -F /path/to/file | grep -q "$string" 2>/dev/null; then
then
    echo "String found on $HOSTNAME" | mail -s "Subject" dummy@iam.com
fi
may work and will find $string only once.
So this works once, but what if I needed it to run 24x7 to catch any new errors?

ideally it would be great to constantly read the last line of the log file looking for the error, once it finds it send an email out, but the script should still continue to read the log file looking for the error in case it occurs again.

thanks so much for the help
 
Old 09-13-2017, 04:56 PM   #6
lsalab
LQ Newbie
 
Registered: Jan 2009
Posts: 24

Rep: Reputation: 3
Why use a custom script at all?

Have you tried using rsyslog directly, it actually reads every log in your OS as it's generated, and with the ommail module you can send an email whenever a certain condition is met. Take a look:

http://www.rsyslog.com/doc/v8-stable...es/ommail.html
 
Old 09-15-2017, 01:05 AM   #7
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 7.7 (?), Centos 8.1
Posts: 17,862

Rep: Reputation: 2598Reputation: 2598Reputation: 2598Reputation: 2598Reputation: 2598Reputation: 2598Reputation: 2598Reputation: 2598Reputation: 2598Reputation: 2598Reputation: 2598
To check 24/x7 & not repeat error msg/mail:

Try it in Perl using this module http://search.cpan.org/~mgrabnar/File-Tail-1.3/Tail.pm.

In the loop, check if error exists in db (or similar eg hash), if it exists its already been emailed, do next iteration.
If not, email error, add to db, then next iteration.

Last edited by chrism01; 09-18-2017 at 12:58 AM. Reason: typo
 
Old 09-15-2017, 01:22 AM   #8
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 19,706

Rep: Reputation: 3548Reputation: 3548Reputation: 3548Reputation: 3548Reputation: 3548Reputation: 3548Reputation: 3548Reputation: 3548Reputation: 3548Reputation: 3548Reputation: 3548
Quote:
Originally Posted by pnorth72 View Post
this is my first post here and I am new to scripting
I'm thinking adding perl to the mix is asking a bit much of the OP.
Small steps, one at a time.
 
Old 09-15-2017, 06:22 AM   #9
TenTenths
Senior Member
 
Registered: Aug 2011
Location: Dublin
Distribution: Centos 5 / 6 / 7
Posts: 3,185

Rep: Reputation: 1377Reputation: 1377Reputation: 1377Reputation: 1377Reputation: 1377Reputation: 1377Reputation: 1377Reputation: 1377Reputation: 1377Reputation: 1377
I've had to do this before and these are the high level steps I took.
  • Create the script and touch /var/log/raisedalerts

The script should:
  • Create a temporary file in /tmp
  • grep the whole log file for the error conditions and store them in the temporary file.
  • diff the temporary file against /var/log/raisedalerts
  • If there is a difference then it means that either there's a new alert or the log file has rotated, send the mail with all the lines in the diff that begin with + as these are NEW
  • Copy your temp file to /var/log/raisedalerts
  • Tidy up any temporary files.
 
Old 09-15-2017, 05:15 PM   #10
joe_2000
Senior Member
 
Registered: Jul 2012
Location: Aachen, Germany
Distribution: Void, Debian
Posts: 1,016

Rep: Reputation: 308Reputation: 308Reputation: 308Reputation: 308
There is also logwatch which does this in a very handy way.

EDIT: Well, it doesn't exactly do what you asked for, but essentially it helps you to monitor logs and get email summaries.

If you want to be notified immediately upon occurrence of a certain message, it's not the right solution.

Last edited by joe_2000; 09-15-2017 at 05:25 PM.
 
Old 09-18-2017, 01:00 AM   #11
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 7.7 (?), Centos 8.1
Posts: 17,862

Rep: Reputation: 2598Reputation: 2598Reputation: 2598Reputation: 2598Reputation: 2598Reputation: 2598Reputation: 2598Reputation: 2598Reputation: 2598Reputation: 2598Reputation: 2598
@syg00 : I know what you mean, but doing this properly is non-trivial and that perl module does all the tricky bit for you.

It's up to the OP: writing your own is certainly educational
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Sed/awk/cut to pull a repeating string out of a longer string StupidNewbie Programming 4 09-13-2018 03:41 AM
[SOLVED] Pull string from a file into a variable using bash DuskFall Linux - Newbie 13 08-06-2011 03:06 PM
creating a script to email daily log file sittnduck Linux - Server 4 07-15-2008 08:59 PM
Need assistance with shell script to pull MP3 file Cpare Linux - Newbie 5 06-28-2008 08:53 PM
Bash script to put log files into single file and email DragonM15 Programming 13 11-08-2007 03:27 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 04:44 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration