LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-18-2013, 04:04 AM   #1
Sony123
LQ Newbie
 
Registered: Jul 2013
Posts: 4

Rep: Reputation: Disabled
Avoid duplicate pids while grepping for a pattern in a shell script


Hi,

I have written a Linux shell script that reads the updating logs and sends an email notification when there are OOM using do while loop.

I am able to get the email notification for OOM errors but every time a duplicate PID is getting generated for the email sent which is creating a huge no. of duplicate pids while I am grepping for that particular process/ Admin server (weblogic). PFB my script, is there any way I can avoid the duplicate PIDs and can get only parent PID while doing a grep.

#!/bin/sh
# Script to read the updating log file and send mail for any errors : LogCheck.sh
# SET VARIABLES
logfile=BEA_HOME/SERVERS/Admin.log
pattern="java.lang.OutOfMemoryError: Java heap space"

#Read each line as it gets updating to the log file
tail -fn0 $logfile | while read line ; do

#check each line for our pattern
echo "$line" | grep -i "$pattern"

#Perform the below action if a line matches with our pattern
if [ $? = 0 ];

then
#Send an email
echo "Found an error: $line" | mailx -s "please check the error" emailID
fi

done
 
Old 07-18-2013, 04:12 AM   #2
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,471

Rep: Reputation: 444Reputation: 444Reputation: 444Reputation: 444Reputation: 444
do you have sample log?

my first guess would be to exclude your emails

Code:
echo "$line" | grep -v "Found an error:" | grep -i "$pattern"
Oh, wrap in [ code ] tags http://www.linuxquestions.org/questi....php?do=bbcode
 
Old 07-18-2013, 04:27 AM   #3
Sony123
LQ Newbie
 
Registered: Jul 2013
Posts: 4

Original Poster
Rep: Reputation: Disabled
Hi,

Thank you for the reply...

I dont have the sample log currently..Actually I need the email notification sent to my ID in order to either trouble shoot/bounce my server for OOM(in this case its weblogic server)..Whenever I do a grep of that particular server like Adminserver, along with the parent PID with memory arguments I also get to see a huge no. of duplicate PIDs like below which is creating a problem as I beleive there will be limit for the no. of PIDs generated in Linux..
Below is the output for grep..

ps -ef | grep AdminSerever
20375 21013 10 17Jun ? 02:14:23 tail -fn0 BEA_HOME/SERVERS/Admin.log
Everytime I am getting this 'tail -fn0' PID too while I am doing a grep for AdminServer along with the parent PID..Is there any way I can avoid these duplicate PIDs to be while doing a grep..

Thanks
 
Old 07-18-2013, 04:46 AM   #4
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,471

Rep: Reputation: 444Reputation: 444Reputation: 444Reputation: 444Reputation: 444
make a grep exclude it

Code:
ps -ef | grep -v "tail -fn0" |grep AdminSerever
but typo's aside, I don't understand why you get that line from your grep

and yes, a limit to PIDs *BUT* they are recycled , so unless nothing ever exits you have infinite.
 
Old 07-18-2013, 05:26 AM   #5
Sony123
LQ Newbie
 
Registered: Jul 2013
Posts: 4

Original Poster
Rep: Reputation: Disabled
Hi,

Not sure why I am getting the duplicate pids too..whenever i receive a mail for out of memory and if i dont restart the weblogic server, these duplicate pids go on increasing..i tried the below code too but still i am seeing the duplicate pids:

code: I used uniq and sort to grep for only parent pid lines but still i could see duplicate pids..
echo "$line" | grep -i "$pattern" | uniq

I get a duplicate pids like below everytime i grep for adminserver:

29872 1 0 07:35 ? 00:00:00 tail -fn0 BEA_HOME/SERVERS/Admin.log

Thanks - Please excuse for any typo's..
 
Old 07-18-2013, 06:17 AM   #6
Firerat
Senior Member
 
Registered: Oct 2008
Distribution: Debian Jessie / sid
Posts: 1,471

Rep: Reputation: 444Reputation: 444Reputation: 444Reputation: 444Reputation: 444
you are looking for $pattern in log
then sending that line via email
if that email gets logged, your pattern will match again, resulting in email
if that email gets logged, your pattern will match again, resulting in email
if that email gets logged, your pattern will match again, resulting in email
if that email gets logged, your pattern will match again, resulting in email



assuming the email is making it into the log, exclude those lines
Code:
echo "$line" | grep -v "Found an error:" | grep -i "$pattern"
This is just a guess,
I don't know if the email is logged, which is why I asked for sample log.

as for
>> I get a duplicate pids like below everytime i grep for adminserver:

Well, if you grep for BEA_HOME/SERVERS/Admin.log in ps -ef, yes you will get

Code:
29872 1 0 07:35 ? 00:00:00 tail -fn0 BEA_HOME/SERVERS/Admin.log
because, it matches the pattern
if you first remove anything with tail -fn0, you won't get it

Code:
ps -ef | grep -v "tail -fn0" |grep "BEA_HOME/SERVERS/Admin.log"
Code:
grep --help
<snip>
Miscellaneous:
  -s, --no-messages         suppress error messages
  -v, --invert-match        select non-matching lines
  -V, --version             print version information and exit
      --help                display this help and exit
      --mmap                deprecated no-op; evokes a warning
<snip>
 
Old 07-18-2013, 07:47 PM   #7
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,245

Rep: Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684Reputation: 2684
Maybe this could all be solved by using pgrep??
 
Old 07-19-2013, 06:22 PM   #8
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,602

Rep: Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241Reputation: 1241
If you are getting lots of OOM problems, perhaps you should add memory to the system. Either addition memory chips, or at least a bigger swap file (or an additional one).

You can also look at the kernel control parameter. If you are having difficulty identifying WHAT is causing the OOM (and believe me, the OOM messages don't necessarily refer to the cause), you need to turn off oversubscription. With oversubscription disabled, the system will abort the process causing the OOM when it is recognized - and that will tell you what is forcing the system OOM.

reference: http://www.codeproject.com/Articles/...agement-Mechan
 
Old 07-26-2013, 07:56 AM   #9
Sony123
LQ Newbie
 
Registered: Jul 2013
Posts: 4

Original Poster
Rep: Reputation: Disabled
Hi All,

Thank you for the suggestions. Marking this as solved as we added more jvm heap to the server by which the oom errors stopped.

-Please excuse for typos..
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Need a shell script help - grepping list of entries from another file? Jykke Linux - General 3 07-12-2011 08:40 AM
[SOLVED] How to replace newline pattern in file by other newline pattern in a shell script XXLRay Linux - Software 9 11-29-2010 08:57 AM
[SOLVED] a short shell script related to PIDs gujedan Programming 3 11-22-2010 05:54 AM
[SOLVED] Need a shell script "grepping" IP adress, and act according to this IP. Linux.tar.gz Programming 15 11-06-2009 11:27 AM
how to find a pattern using shell script sharad Linux - General 1 05-23-2006 04:50 AM


All times are GMT -5. The time now is 10:37 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration