Script that e-mails errors only in the last 500 lines
 

Hello everyone,

I'm trying to make a script that runs every 5 minute and I want to be e-mail-ed only if it finds an error in the last 500 lines of a log.

Down below is the start of the script. Please help me complete it.
I know that if I want to display only the last 500 lines in catalina.out then I need to:
tail -f 500 catalina.out

but I only want to receive the e-mail if the error repeats itself in the last 500 lines.
because if I solve the problem, I will still receive the message over and over again because this script searches again the whole log from 5 to 5 minutes.

I mean I could delete the log file but that's not the ideea.



#!/bin/bash
# Mail out Tomcat errors that are in catalina.out

errorLog=/home/apache-tomcat-8.5.8/logs/catalina.out # Error log location

email=myemailaddress@yahoo.com # Send report here

# Pull out the lines that mention PHP, and use AWK to get the column we're interested in
errors=$(cat catalina.out | grep Memory$allocation$failed$during$query$processing)
# Remove referer information, sort, and remove duplicate entries
errors=$(echo "$the_error" | awk -F', referer' '{print $1}' | sort | uniq)
# Check that we actually have some errors



if [ -n "$the_error" ]
then
echo "$the_error" | mail "$email" -s "PHP Errors"
fi

You've been a member here for nine years, so use CODE tags when posting scripts, please. And if you want to find out how many occurrences of a word are in a particular variable, you can pipe it through "wc -l", which will give you a number. Perform a test on that number to act accordingly.

Hi,

Apart from beings member for 9 years, the rest I didn't understand anything.

Here's where I am now:



And the crontab (for suse) is:

-*/5 7-18 * * * root /home/test/tomcat >/dev/null 2>&1

it runs a cron job every 5 minutes between 7:00 and 17:00 (line added in /etc/crontab)

Script:


errorLog=/home/test/apache-tomcat-8.5.8/logs/catalina.out # Error log location

email=dmyemail@yahoo.com # Send report here

# Pull out the lines that mention PHP, and use AWK to get the column we're interested in



errors=$(cat /home/test/apache-tomcat-8.5.8/logs/catalina.out | grep Memory$allocation$failed$during$query$processing)

# Remove referer information, sort, and remove duplicate entries
errors=$(echo "$errors" | awk -F', referer' '{print $1}' | sort | uniq)
# Check that we actually have some errors



if [ -n "$errors" ]
then
echo "$errors" | mail "$email" -s "e-mail subject - something is not working"
fi



It would be interresting if I could find a method for not spaming myself with one e-mail every 5 min if I am AFK for 2 hours.
And I do need to check every 5 min if an error occurs in tomcat.

Any advice?

Then you really should read it again until you do. After nine years, these are things you should know. Putting code in CODE tags makes it easier to read; saying you don't understand that after someone tells you makes zero sense. Mousing over the # sign in the posting window even tells you what it does.
Ok..running it doesn't appear to be a problem
Yes; again, put your code in CODE tags. The "wc -l" I mentioned earlier will tell you how many matching lines you have...so if you want more than 1, write your script to look at that variable to see if it's greater than/less than, and act accordingly, which is exactly what addresses your original question. Past that, there are MANY ways to proceed, from writing the time of the last email to a temp file and reading it on each loop, putting a sleep statement in your script to have it pause every five minutes and checking a variable, among others.

I'd go with writing a file in /tmp, and leaving it there. If it exists, don't send another email...when you want the script to resume, delete the file from /tmp.

Everything TB0ne has said is relevant. Please use tags when you post code.

If you tail -50 or otherwise review the last 50 lines in a log, and the log hasn't changed, then, yes, you will spam yourself with the same message every 5 minutes.
Consider a couple of ideas
...increase the interval: if 5 minutes gives you the same report with no changes every time, then 5 minutes is too short.
...put the output of the scan into a text file, compare the text file to the scan next time it runs, do nothing if there's no change.
...and, ultimately, why are you checking for errors in tomcat at all...that is, what problem are you trying to solve.

In addition to that, simplify your error collection by removing cat and grep.

TB0ne, I won't read what you said, it's too much to read.

I think I will consider this closed, I am noob and don't understand what to write in script. I wanned to be helped (you guys continue to change the script from top to bottom).

5 min is big for me, it matters alot if the application is not working for more than 5 min so I need it to be 5 min (or even less).

Since the script only runs between 7 and 17 I am at work and I should be able to instant check my mail.
When I am in holidays, I have created a separated folder in my e-mail and the allerts are going there not in inbox so it should be good.

I can also create a group e-mail and when I am not at my desk someone else can fix the problem.

Uhm... yes, this is indeed a good strategy for me and it works well.

Thread closed.

Then don't bother posting to a forum asking for help, if you don't want to read it.
No, we WILL NOT write your scripts for you; this is, again, spelled out in the "Question Guidelines". Read those, unless you consider the forum rules 'too much to read' also.
Then have it run whenever you want to...write YOUR SCRIPT YOURSELF, and make it do what you want.
So someone else will have to write the script; apparently, handing you commands that will work, answering your questions, and giving you advice on how to accomplish your goals isn't enough for you, so you complain about it being 'too much to read'??

This isn't the place to come for handouts; if you want your scripts written for you, hire someone. Otherwise, show your own effort.

Thread has been closed because this discussion should not continue.

@ytd,

It is inappropriate for you to ask a detailed question and then complain that you cannot be bothered to read details from other members.

Please do not repeat this pattern of behavior.