Script that e-mails errors only in the last 500 lines
Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Script that e-mails errors only in the last 500 lines
Hello everyone,
I'm trying to make a script that runs every 5 minute and I want to be e-mail-ed only if it finds an error in the last 500 lines of a log.
Down below is the start of the script. Please help me complete it.
I know that if I want to display only the last 500 lines in catalina.out then I need to:
tail -f 500 catalina.out
but I only want to receive the e-mail if the error repeats itself in the last 500 lines.
because if I solve the problem, I will still receive the message over and over again because this script searches again the whole log from 5 to 5 minutes.
I mean I could delete the log file but that's not the ideea.
#!/bin/bash
# Mail out Tomcat errors that are in catalina.out
# Pull out the lines that mention PHP, and use AWK to get the column we're interested in
errors=$(cat catalina.out | grep Memory$allocation$failed$during$query$processing)
# Remove referer information, sort, and remove duplicate entries
errors=$(echo "$the_error" | awk -F', referer' '{print $1}' | sort | uniq)
# Check that we actually have some errors
if [ -n "$the_error" ]
then
echo "$the_error" | mail "$email" -s "PHP Errors"
fi
Hello everyone,
I'm trying to make a script that runs every 5 minute and I want to be e-mail-ed only if it finds an error in the last 500 lines of a log.
Down below is the start of the script. Please help me complete it. I know that if I want to display only the last 500 lines in catalina.out then I need to:
tail -f 500 catalina.out
but I only want to receive the e-mail if the error repeats itself in the last 500 lines. because if I solve the problem, I will still receive the message over and over again because this script searches again the whole log from 5 to 5 minutes. I mean I could delete the log file but that's not the ideea.
Code:
#!/bin/bash
# Mail out Tomcat errors that are in catalina.out
errorLog=/home/apache-tomcat-8.5.8/logs/catalina.out # Error log location
email=myemailaddress@yahoo.com # Send report here
# Pull out the lines that mention PHP, and use AWK to get the column we're interested in
errors=$(cat catalina.out | grep Memory$allocation$failed$during$query$processing)
# Remove referer information, sort, and remove duplicate entries
errors=$(echo "$the_error" | awk -F', referer' '{print $1}' | sort | uniq)
# Check that we actually have some errors
if [ -n "$the_error" ]
then
echo "$the_error" | mail "$email" -s "PHP Errors"
fi
You've been a member here for nine years, so use CODE tags when posting scripts, please. And if you want to find out how many occurrences of a word are in a particular variable, you can pipe it through "wc -l", which will give you a number. Perform a test on that number to act accordingly.
# Remove referer information, sort, and remove duplicate entries
errors=$(echo "$errors" | awk -F', referer' '{print $1}' | sort | uniq)
# Check that we actually have some errors
if [ -n "$errors" ]
then
echo "$errors" | mail "$email" -s "e-mail subject - something is not working"
fi
It would be interresting if I could find a method for not spaming myself with one e-mail every 5 min if I am AFK for 2 hours.
And I do need to check every 5 min if an error occurs in tomcat.
Hi,
Apart from beings member for 9 years, the rest I didn't understand anything.
Then you really should read it again until you do. After nine years, these are things you should know. Putting code in CODE tags makes it easier to read; saying you don't understand that after someone tells you makes zero sense. Mousing over the # sign in the posting window even tells you what it does.
Quote:
Here's where I am now:
And the crontab (for suse) is:
it runs a cron job every 5 minutes between 7:00 and 17:00 (line added in /etc/crontab)
Ok..running it doesn't appear to be a problem
Quote:
Script:
Code:
errorLog=/home/test/apache-tomcat-8.5.8/logs/catalina.out # Error log location
email=dmyemail@yahoo.com # Send report here
# Pull out the lines that mention PHP, and use AWK to get the column we're interested in
errors=$(cat /home/test/apache-tomcat-8.5.8/logs/catalina.out | grep Memory$allocation$failed$during$query$processing)
# Remove referer information, sort, and remove duplicate entries
errors=$(echo "$errors" | awk -F', referer' '{print $1}' | sort | uniq)
# Check that we actually have some errors
if [ -n "$errors" ]
then
echo "$errors" | mail "$email" -s "e-mail subject - something is not working"
fi
It would be interresting if I could find a method for not spaming myself with one e-mail every 5 min if I am AFK for 2 hours. And I do need to check every 5 min if an error occurs in tomcat. Any advice?
Yes; again, put your code in CODE tags. The "wc -l" I mentioned earlier will tell you how many matching lines you have...so if you want more than 1, write your script to look at that variable to see if it's greater than/less than, and act accordingly, which is exactly what addresses your original question. Past that, there are MANY ways to proceed, from writing the time of the last email to a temp file and reading it on each loop, putting a sleep statement in your script to have it pause every five minutes and checking a variable, among others.
I'd go with writing a file in /tmp, and leaving it there. If it exists, don't send another email...when you want the script to resume, delete the file from /tmp.
Everything TB0ne has said is relevant. Please use tags when you post code.
If you tail -50 or otherwise review the last 50 lines in a log, and the log hasn't changed, then, yes, you will spam yourself with the same message every 5 minutes.
Consider a couple of ideas
...increase the interval: if 5 minutes gives you the same report with no changes every time, then 5 minutes is too short.
...put the output of the scan into a text file, compare the text file to the scan next time it runs, do nothing if there's no change.
...and, ultimately, why are you checking for errors in tomcat at all...that is, what problem are you trying to solve.
TB0ne, I won't read what you said, it's too much to read.
I think I will consider this closed, I am noob and don't understand what to write in script. I wanned to be helped (you guys continue to change the script from top to bottom).
5 min is big for me, it matters alot if the application is not working for more than 5 min so I need it to be 5 min (or even less).
Since the script only runs between 7 and 17 I am at work and I should be able to instant check my mail.
When I am in holidays, I have created a separated folder in my e-mail and the allerts are going there not in inbox so it should be good.
I can also create a group e-mail and when I am not at my desk someone else can fix the problem.
Uhm... yes, this is indeed a good strategy for me and it works well.
TB0ne, I won't read what you said, it's too much to read.
Then don't bother posting to a forum asking for help, if you don't want to read it.
Quote:
I think I will consider this closed, I am noob and don't understand what to write in script. I wanned to be helped (you guys continue to change the script from top to bottom).
No, we WILL NOT write your scripts for you; this is, again, spelled out in the "Question Guidelines". Read those, unless you consider the forum rules 'too much to read' also.
Quote:
5 min is big for me, it matters alot if the application is not working for more than 5 min so I need it to be 5 min (or even less).
Then have it run whenever you want to...write YOUR SCRIPT YOURSELF, and make it do what you want.
Quote:
Since the script only runs between 7 and 17 I am at work and I should be able to instant check my mail. When I am in holidays, I have created a separated folder in my e-mail and the allerts are going there not in inbox so it should be good.I can also create a group e-mail and when I am not at my desk someone else can fix the problem.
So someone else will have to write the script; apparently, handing you commands that will work, answering your questions, and giving you advice on how to accomplish your goals isn't enough for you, so you complain about it being 'too much to read'??
This isn't the place to come for handouts; if you want your scripts written for you, hire someone. Otherwise, show your own effort.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.