LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 07-23-2013, 12:51 PM   #1
NetEng1
Member
 
Registered: Aug 2011
Distribution: ubuntu
Posts: 40

Rep: Reputation: Disabled
Bash script to scrape logs


I have been tasked with scraping postgres logs and looking for errors of a specific type and returning information that nagios can alarm on. I started using grep in a bash script and was able to get it to list the instances of "ERROR:", but to find out what function caused the error, the line above it would also need to be grep'd for a string. For example, below is a few lines from the postgres log:


2013-07-22 12:55:07 EDT STATEMENT: select parser."ProcessMail"()
2013-07-22 12:58:13 EDT ERROR: canceling statement due to user request
2013-07-22 12:58:13 EDT CONTEXT: PL/pgSQL function "getsamples" line 30 at RETURN QUERY

Grep catches the second line, but we only want to error if the line above it has "ProcessMail". sometimes there is an error that we do not want to alarm on and the line above it may contain something else.

I tried the -B switch, but that only prints/echos the line and not searches it for the interested string "ProcessMail".

What is the best way of going about this task?

Thank you in advanced.
 
Old 07-23-2013, 01:17 PM   #2
Kustom42
Senior Member
 
Registered: Mar 2012
Distribution: Red Hat
Posts: 1,604

Rep: Reputation: 415Reputation: 415Reputation: 415Reputation: 415Reputation: 415
You could look at some pre-written nagios checks for pgsql.

http://exchange.nagios.org/directory...stgres/details

That plugin is packed with features and should work for you.


Besides that you can either try to put together a very simple bash script, you could go as simple as to write your grep output to a tmp file and then regex it for any error patterns you want to see but the best option is to write something a little more robust in something like perl or python.

You can even use exit codes in your grep's, its not pretty and by far is not the best way to do it but its quick and simple. Put all of your error patterns you want to search for in a text file, one patter per line.

Code:
#!/bin/bash
ERRORPATTERNS="/path/to/error/patterns"
TMPFILE="/tmp/pgsql_error_check_$(date +%m_%d_%y)"
LOGFILE="/var/log/pgsql/error_check.log"
grep -i -A2 -B2 error /var/log/pgsql/* > $TMPFILE
egrep -i -q -f $ERRORPATTERNS $TMPFILE >> $LOGFILE
    if [ $? -eq 0 ]
    then
    rm -f $TMPFILE
    exit 2
    elif [ $? -ne 0 ]
    then
    rm -f $TMPFILE
    exit 0
    fi
Then you can just have nagios just run that bash script. Nagios recoginizes exit code 2 as critical, 1 as a warning, and 0 as OK. So basically if your egrep does not match anything that process will exit with status code 1, that means you have no errors that you care about so we exit the bash script with exit code 0 telling nagios everything is ok. On the other side, if egrep matches anything it will exit with status code 0 and then we can have your script exit with status code 2 which tells nagios there is a problem.



Keep in mind, this above script is extremely dirty and I just threw it up there super fast to give you some context. There is no sanitization of the input nor any sort of confirmation besides exit codes which are not the most reliable way to test for things although they do have their use.
 
1 members found this post helpful.
Old 07-23-2013, 01:20 PM   #3
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 26,604

Rep: Reputation: 7960Reputation: 7960Reputation: 7960Reputation: 7960Reputation: 7960Reputation: 7960Reputation: 7960Reputation: 7960Reputation: 7960Reputation: 7960Reputation: 7960
Quote:
Originally Posted by NetEng1 View Post
I have been tasked with scraping postgres logs and looking for errors of a specific type and returning information that nagios can alarm on. I started using grep in a bash script and was able to get it to list the instances of "ERROR:", but to find out what function caused the error, the line above it would also need to be grep'd for a string. For example, below is a few lines from the postgres log:
Code:
2013-07-22 12:55:07 EDT STATEMENT:  select parser."ProcessMail"()
2013-07-22 12:58:13 EDT ERROR:  canceling statement due to user request
2013-07-22 12:58:13 EDT CONTEXT:  PL/pgSQL function "getsamples" line 30 at RETURN QUERY
Grep catches the second line, but we only want to error if the line above it has "ProcessMail". sometimes there is an error that we do not want to alarm on and the line above it may contain something else.

I tried the -B switch, but that only prints/echos the line and not searches it for the interested string "ProcessMail".
Why don't you try the "-C 1" flag, and look for ProcessMail instead? That will get that line, plus the line below it. You could process it further from there.
 
Old 07-23-2013, 04:19 PM   #4
NetEng1
Member
 
Registered: Aug 2011
Distribution: ubuntu
Posts: 40

Original Poster
Rep: Reputation: Disabled
Thanks. -C 1 gets the line before and after. We are just interested in certain functions that caused the error and that comes before so B 1 seems to work. I am currently looking at using the following egrep statement and maybe some of the concepts in the code Kustom42 provided. Although I am being told that sometimes there is a blank line above the ERROR line, so I am unsure of how to handle that.

I guess I could go back to reading each line and "remembering" the previous two as line variables, but I am not sure how to "grep" within a memory variable for a string.

Thanks
 
Old 07-23-2013, 04:46 PM   #5
Kustom42
Senior Member
 
Registered: Mar 2012
Distribution: Red Hat
Posts: 1,604

Rep: Reputation: 415Reputation: 415Reputation: 415Reputation: 415Reputation: 415
You can use

Code:
sed '/^$/d'
To sanitize the input and remove any blank lines, such as a blank line above the error statement you are concerned with.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
pls help in my bash script to transfer logs to aws s3,kindly suggest changes... sudi267 Linux - Newbie 1 03-05-2013 08:25 AM
BASH script won't die when user logs out. arizonagroovejet Linux - General 14 11-06-2009 02:37 AM
bash script to use sed for filter mutiples patterns from apache access logs matyu Programming 5 02-06-2008 10:28 PM
Iptables (with masq) troubleshooting, very simple script attached script and logs. xinu Linux - Networking 13 11-01-2007 04:19 AM
Scrape Website for TV listings drspangle Linux - Software 4 07-13-2004 04:20 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 03:01 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration