log monitor shell script

arunganga · 03-31-2015, 03:03 PM

Hi Guys,

i have requirement for log monitor shell script as per below
a)grep WARNING or EXCEPTION or SEVERE from log file
b) count the grep results
c)send an email notification for grep results

please help me .

thanks,
Arun

T3RM1NVT0R · 03-31-2015, 03:12 PM

Welcome to LQ!!!

As you are new to LQ go through LQ rules here

We will not do your homework or the job / work which has been assigned to you. We as a community are here to help / assist you but you have to let us know what you have tried so far and where you are stuck.

arunganga · 03-31-2015, 03:22 PM

failed to open

T3RM1NVT0R · 03-31-2015, 03:28 PM

You have pasted the script but you did not mention which section of the script you are facing issue with? Whenever you are pasting a script it is good idea to use code tags as I have used below to keep indentation:

Code:

#!/bin/bash
EMAIL_SUBJECT="Found several log file messages matching Exception"
EMAIL_FROM_ADDRESS=abc.com
EMAIL_TO_ADDRESS=test.com
COMMAND_DIR=log location
StatusFile_DIR=/home/user001/

for ((i = 0 ; i < 10000 ; i++ ));
    do
    statusvar=`tail -f log location` >> /home/user001/Error_Temp_details.txt
    echo $i;
    mv /home/user001/Error_Temp_details.txt /home/user001/Error_details.txt
    done ;

E_count = grep -c "ERROR" /home/user001/Error_details.txt
W_count = grep -c "WARNING" /home/user001/Error_details.txt

if ($E_count>=0)
   then
   {
   echo -e "Dear Team,"
   echo -e "\n Found" $E_count "\n Found several log file messages matching ERROR"
   mailx -s "$EMAIL_SUBJECT" -r $EMAIL_FROM_ADDRESS $EMAIL_TO_ADDRESS
   }
if ($W_count>=0)
   then
   {
   echo -e "Dear Team,"
   echo -e "\n Found" $W_count "\n Found several log file messages matching WARNING"
   mailx -s "$EMAIL_SUBJECT" -r $EMAIL_FROM_ADDRESS $EMAIL_TO_ADDRESS
   }

arunganga · 03-31-2015, 03:40 PM

failed to open

T3RM1NVT0R · 03-31-2015, 03:59 PM

I have put comment in your script:

Code:

#!/bin/bash
EMAIL_SUBJECT="Found several log file messages matching Exception"
EMAIL_FROM_ADDRESS=abc.com
EMAIL_TO_ADDRESS=test.com
COMMAND_DIR=log location
StatusFile_DIR=/home/user001/

for ((i = 0 ; i < 10000 ; i++ ));
    do
    statusvar=`tail -f log location` >> /home/user001/Error_Temp_details.txt
    echo $i;
    mv /home/user001/Error_Temp_details.txt /home/user001/Error_details.txt
    done ;

E_count = grep -c "ERROR" /home/user001/Error_details.txt #Doing grep this way will return non-numerical value. You should do as:
#E_count = grep -c "ERROR" /home/user001/Error_details.txt | wc -l # Like this
W_count = grep -c "WARNING" /home/user001/Error_details.txt #Same as above
#W_count = grep -c "WARNING" /home/user001/Error_details.txt | wc -l # This way it will return the number of ERROR it found.

if ($E_count>=0)
   then
   {
   echo -e "Dear Team,"
   echo -e "\n Found" $E_count "\n Found several log file messages matching ERROR"
   mailx -s "$EMAIL_SUBJECT" -r $EMAIL_FROM_ADDRESS $EMAIL_TO_ADDRESS
   }
if ($W_count>=0)
   then
   {
   echo -e "Dear Team,"
   echo -e "\n Found" $W_count "\n Found several log file messages matching WARNING"
   mailx -s "$EMAIL_SUBJECT" -r $EMAIL_FROM_ADDRESS $EMAIL_TO_ADDRESS
   }

Give it a try and let us know how far it goes. Also run the script with -x to get the detailed output on what script is doing. Like sh -x <script.sh>

arunganga · 03-31-2015, 04:50 PM

checking

joe_2000 · 03-31-2015, 04:51 PM

This here won't work:

Code:

E_count = grep -c "ERROR" /home/user001/Error_details.txt

First of all: for variable assignments in bash you cannot have the = sign enclosed with white spaces. It should always be

Code:

var=value

not

Code:

var = value

Secondly, you cannot assign the output of a command to a variable like that. It should be something like

Code:

var=$(grep whatever)

Next, this if condition looks wrong to me:

Code:

if($E_count>=0)

I would do something like

Code:

if [ $E_count -ge 0 ];then

Same goes for the second if condition below.
The initial loop looks very strange to me, too, I don't understand what exactly you are trying to accomplish there, but whatever it is I am sure it is very inefficiently coded...

joe_2000 · 03-31-2015, 04:56 PM

Maybe one additional consideration. Put together a grep command that only gives you output if something is wrong.
Put that into a cronjob and set up cron such that it can email you.
Whenever your grep produces output it will be emailed to you automatically.
That should get you going and is probably an order of magnitude simpler and cleaner than what you are looking at right now...

T3RM1NVT0R · 03-31-2015, 05:14 PM

@joe_2000: Good catch.

Yes it should be without spaces. Infact you could put it like this and it should work:

Code:

E_count=`grep "ERROR" /home/user001/Error_details.txt | wc -l`

Using back ticks.

T3RM1NVT0R · 03-31-2015, 05:29 PM

Code:

for ((i = 0 ; i < 10000 ; i++ ));
do
statusvar=`tail -f /DBA/capsqa2/JavaCAPS62/appserver/domains/domain1/logs/server.log` >> /VZ/EAIworkspace/narender/Error_Temp_details.txt
echo $i;
mv /VZ/EAIworkspace/narender/Error_Temp_details.txt /VZ/EAIworkspace/narender/Error_details.txt
done ;

This is totally wrong. You are using for to run a loop for 10000 time but it will get stuck with the first run, the reason being you are using tail -f which will continuously keep on updating /VZ/EAIworkspace/narender/Error_Temp_details.txt. Basically you are getting into infinite condition. Even where there is no update on /DBA/capsqa2/JavaCAPS62/appserver/domains/domain1/logs/server.log it will just sit there. You will never reach 10000 as i=0 will run indefinitely. Fix that first.

Pointer: If you want to record 10000 lines anyways you can go with tail -n 1 instead. Example:

Code:

statusvar=`tail -n 1 /DBA/capsqa2/JavaCAPS62/appserver/domains/domain1/logs/server.log` >> /VZ/EAIworkspace/narender/Error_Temp_details.txt

I am not reading the whole code obviously this is something you have to debug. We guys can only give you pointers.

arunganga · 03-31-2015, 11:54 PM

will check and provide the details

joe_2000 · 04-01-2015, 10:32 AM

Quote:

Originally Posted by T3RM1NVT0R

Using back ticks.

Backticks work, too. I personally prefer the

Code:

var=$(grep whatever)

I already posted because I find it more readable. An additional bonus is that it can be nested