LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Closed Thread
  Search this Thread
Old 05-16-2018, 04:39 AM   #1
ytd
Member
 
Registered: Jan 2009
Posts: 200

Rep: Reputation: 31
Script that e-mails errors only in the last 500 lines


Hello everyone,

I'm trying to make a script that runs every 5 minute and I want to be e-mail-ed only if it finds an error in the last 500 lines of a log.

Down below is the start of the script. Please help me complete it.
I know that if I want to display only the last 500 lines in catalina.out then I need to:
tail -f 500 catalina.out

but I only want to receive the e-mail if the error repeats itself in the last 500 lines.
because if I solve the problem, I will still receive the message over and over again because this script searches again the whole log from 5 to 5 minutes.

I mean I could delete the log file but that's not the ideea.



#!/bin/bash
# Mail out Tomcat errors that are in catalina.out

errorLog=/home/apache-tomcat-8.5.8/logs/catalina.out # Error log location

email=myemailaddress@yahoo.com # Send report here

# Pull out the lines that mention PHP, and use AWK to get the column we're interested in
errors=$(cat catalina.out | grep Memory$allocation$failed$during$query$processing)
# Remove referer information, sort, and remove duplicate entries
errors=$(echo "$the_error" | awk -F', referer' '{print $1}' | sort | uniq)
# Check that we actually have some errors



if [ -n "$the_error" ]
then
echo "$the_error" | mail "$email" -s "PHP Errors"
fi

Last edited by ytd; 05-16-2018 at 05:38 AM.
 
Old 05-16-2018, 07:11 AM   #2
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 20,015

Rep: Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766
Quote:
Originally Posted by ytd View Post
Hello everyone,
I'm trying to make a script that runs every 5 minute and I want to be e-mail-ed only if it finds an error in the last 500 lines of a log.

Down below is the start of the script. Please help me complete it. I know that if I want to display only the last 500 lines in catalina.out then I need to:
tail -f 500 catalina.out

but I only want to receive the e-mail if the error repeats itself in the last 500 lines. because if I solve the problem, I will still receive the message over and over again because this script searches again the whole log from 5 to 5 minutes. I mean I could delete the log file but that's not the ideea.
Code:
#!/bin/bash
# Mail out Tomcat errors that are in catalina.out

errorLog=/home/apache-tomcat-8.5.8/logs/catalina.out # Error log location

email=myemailaddress@yahoo.com # Send report here

# Pull out the lines that mention PHP, and use AWK to get the column we're interested in
errors=$(cat catalina.out | grep Memory$allocation$failed$during$query$processing)
# Remove referer information, sort, and remove duplicate entries
errors=$(echo "$the_error" | awk -F', referer' '{print $1}' | sort | uniq)
# Check that we actually have some errors
if [ -n "$the_error" ]
then
    echo "$the_error" | mail "$email" -s "PHP Errors"
fi
You've been a member here for nine years, so use CODE tags when posting scripts, please. And if you want to find out how many occurrences of a word are in a particular variable, you can pipe it through "wc -l", which will give you a number. Perform a test on that number to act accordingly.
 
1 members found this post helpful.
Old 05-16-2018, 07:58 AM   #3
ytd
Member
 
Registered: Jan 2009
Posts: 200

Original Poster
Rep: Reputation: 31
Hi,

Apart from beings member for 9 years, the rest I didn't understand anything.

Here's where I am now:



And the crontab (for suse) is:

-*/5 7-18 * * * root /home/test/tomcat >/dev/null 2>&1

it runs a cron job every 5 minutes between 7:00 and 17:00 (line added in /etc/crontab)

Script:


errorLog=/home/test/apache-tomcat-8.5.8/logs/catalina.out # Error log location

email=dmyemail@yahoo.com # Send report here

# Pull out the lines that mention PHP, and use AWK to get the column we're interested in



errors=$(cat /home/test/apache-tomcat-8.5.8/logs/catalina.out | grep Memory$allocation$failed$during$query$processing)

# Remove referer information, sort, and remove duplicate entries
errors=$(echo "$errors" | awk -F', referer' '{print $1}' | sort | uniq)
# Check that we actually have some errors



if [ -n "$errors" ]
then
echo "$errors" | mail "$email" -s "e-mail subject - something is not working"
fi



It would be interresting if I could find a method for not spaming myself with one e-mail every 5 min if I am AFK for 2 hours.
And I do need to check every 5 min if an error occurs in tomcat.

Any advice?

Last edited by ytd; 05-16-2018 at 08:01 AM.
 
Old 05-16-2018, 07:41 PM   #4
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 20,015

Rep: Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766
Quote:
Originally Posted by ytd View Post
Hi,
Apart from beings member for 9 years, the rest I didn't understand anything.
Then you really should read it again until you do. After nine years, these are things you should know. Putting code in CODE tags makes it easier to read; saying you don't understand that after someone tells you makes zero sense. Mousing over the # sign in the posting window even tells you what it does.
Quote:
Here's where I am now:
And the crontab (for suse) is:
Code:
-*/5 7-18 * * * root /home/test/tomcat >/dev/null 2>&1
it runs a cron job every 5 minutes between 7:00 and 17:00 (line added in /etc/crontab)
Ok..running it doesn't appear to be a problem
Quote:
Script:
Code:
errorLog=/home/test/apache-tomcat-8.5.8/logs/catalina.out # Error log location

email=dmyemail@yahoo.com # Send report here

# Pull out the lines that mention PHP, and use AWK to get the column we're interested in
errors=$(cat /home/test/apache-tomcat-8.5.8/logs/catalina.out | grep Memory$allocation$failed$during$query$processing)

# Remove referer information, sort, and remove duplicate entries
errors=$(echo "$errors" | awk -F', referer' '{print $1}' | sort | uniq)
# Check that we actually have some errors
if [ -n "$errors" ]
then
    echo "$errors" | mail "$email" -s "e-mail subject - something is not working"
fi
It would be interresting if I could find a method for not spaming myself with one e-mail every 5 min if I am AFK for 2 hours. And I do need to check every 5 min if an error occurs in tomcat. Any advice?
Yes; again, put your code in CODE tags. The "wc -l" I mentioned earlier will tell you how many matching lines you have...so if you want more than 1, write your script to look at that variable to see if it's greater than/less than, and act accordingly, which is exactly what addresses your original question. Past that, there are MANY ways to proceed, from writing the time of the last email to a temp file and reading it on each loop, putting a sleep statement in your script to have it pause every five minutes and checking a variable, among others.

I'd go with writing a file in /tmp, and leaving it there. If it exists, don't send another email...when you want the script to resume, delete the file from /tmp.
 
1 members found this post helpful.
Old 05-16-2018, 09:16 PM   #5
scasey
Member
 
Registered: Feb 2013
Location: Tucson, AZ, USA
Distribution: CentOS 7.4
Posts: 846

Rep: Reputation: 257Reputation: 257Reputation: 257
Everything TB0ne has said is relevant. Please use tags when you post code.

If you tail -50 or otherwise review the last 50 lines in a log, and the log hasn't changed, then, yes, you will spam yourself with the same message every 5 minutes.
Consider a couple of ideas
...increase the interval: if 5 minutes gives you the same report with no changes every time, then 5 minutes is too short.
...put the output of the scan into a text file, compare the text file to the scan next time it runs, do nothing if there's no change.
...and, ultimately, why are you checking for errors in tomcat at all...that is, what problem are you trying to solve.
 
1 members found this post helpful.
Old 05-16-2018, 09:40 PM   #6
Turbocapitalist
Senior Member
 
Registered: Apr 2005
Distribution: Ubuntu, Devuan, OpenBSD
Posts: 3,070
Blog Entries: 3

Rep: Reputation: 1327Reputation: 1327Reputation: 1327Reputation: 1327Reputation: 1327Reputation: 1327Reputation: 1327Reputation: 1327Reputation: 1327Reputation: 1327
In addition to that, simplify your error collection by removing cat and grep.

Code:
awk -F', referer' '/Memory allocation failed during query processing/ {print $1}' $errorLog | sort | uniq)
 
1 members found this post helpful.
Old 05-17-2018, 12:14 AM   #7
ytd
Member
 
Registered: Jan 2009
Posts: 200

Original Poster
Rep: Reputation: 31
TB0ne, I won't read what you said, it's too much to read.

I think I will consider this closed, I am noob and don't understand what to write in script. I wanned to be helped (you guys continue to change the script from top to bottom).

5 min is big for me, it matters alot if the application is not working for more than 5 min so I need it to be 5 min (or even less).

Since the script only runs between 7 and 17 I am at work and I should be able to instant check my mail.
When I am in holidays, I have created a separated folder in my e-mail and the allerts are going there not in inbox so it should be good.

I can also create a group e-mail and when I am not at my desk someone else can fix the problem.

Uhm... yes, this is indeed a good strategy for me and it works well.

Thread closed.
 
Old 05-17-2018, 06:52 AM   #8
TB0ne
LQ Guru
 
Registered: Jul 2003
Location: Birmingham, Alabama
Distribution: SuSE, RedHat, Slack,CentOS
Posts: 20,015

Rep: Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766Reputation: 4766
Quote:
Originally Posted by ytd View Post
TB0ne, I won't read what you said, it's too much to read.
Then don't bother posting to a forum asking for help, if you don't want to read it.
Quote:
I think I will consider this closed, I am noob and don't understand what to write in script. I wanned to be helped (you guys continue to change the script from top to bottom).
No, we WILL NOT write your scripts for you; this is, again, spelled out in the "Question Guidelines". Read those, unless you consider the forum rules 'too much to read' also.
Quote:
5 min is big for me, it matters alot if the application is not working for more than 5 min so I need it to be 5 min (or even less).
Then have it run whenever you want to...write YOUR SCRIPT YOURSELF, and make it do what you want.
Quote:
Since the script only runs between 7 and 17 I am at work and I should be able to instant check my mail. When I am in holidays, I have created a separated folder in my e-mail and the allerts are going there not in inbox so it should be good.I can also create a group e-mail and when I am not at my desk someone else can fix the problem.
So someone else will have to write the script; apparently, handing you commands that will work, answering your questions, and giving you advice on how to accomplish your goals isn't enough for you, so you complain about it being 'too much to read'??

This isn't the place to come for handouts; if you want your scripts written for you, hire someone. Otherwise, show your own effort.
 
2 members found this post helpful.
Old 05-17-2018, 11:27 AM   #9
rtmistler
Moderator
 
Registered: Mar 2011
Location: Sutton, MA. USA
Distribution: MINT Debian, Angstrom, SUSE, Ubuntu
Posts: 6,620
Blog Entries: 12

Rep: Reputation: 2298Reputation: 2298Reputation: 2298Reputation: 2298Reputation: 2298Reputation: 2298Reputation: 2298Reputation: 2298Reputation: 2298Reputation: 2298Reputation: 2298
Thread has been closed because this discussion should not continue.

@ytd,

It is inappropriate for you to ask a detailed question and then complain that you cannot be bothered to read details from other members.

Please do not repeat this pattern of behavior.
 
1 members found this post helpful.
  


Closed Thread


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Which lines are errors nix84 Linux - Networking 3 05-25-2014 09:45 AM
CGI scripts - Persistent Server 500 errors DLHOMME Linux - Networking 9 05-22-2006 02:56 PM
CGI scripts - Persistent Server 500 errors DLHOMME Linux - Networking 1 05-19-2006 07:56 AM
IBM Contributes 500,000 Lines Of Code To Apache jamaso General 2 08-04-2004 11:58 AM
Apache 1.3.20 close connection on HTTP 1.1 400 500 errors dafri Linux - Software 4 01-14-2003 08:50 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 05:48 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration