analyse a shell script

consty · 02-23-2007, 07:01 AM

Hi all,
I have a file with patterns like the one hereafter, patterns begin and end with ======, there are many lines inside a pattern. It's a log and what I want to do is to check every pattern to look if the string ERROR exists and if so print that string and the date, hour and Reference contained in the pattern.
I need help to write the script shell.
Thanks in advance
Consty

======
05/02/2007 04:51:02 [8714]: abdndala.c01.07: SAMS can not be sent, TYPE=DOUBE, STATE: 342184546 (slipt 6,Entity:ANY).
ERROR: 4685 Reference :=172910706273
absolute MM :=)(A1225:clAd=0000)(29746:messi :=624020100762629)(2747:hel_lang=3)(K2749:
sams_id=398)
======

waelaltaqi · 02-23-2007, 07:41 AM

try to read some information about sed and grep commands. they are great text filtering and searching tools. google and sed and grep and you'll get the results you need. post the log file if you can.
the best i can think off for now if trying the following:

Code:

cat error file | grep ERROR*

taylor_venable · 02-23-2007, 08:02 AM

The problem with grep is that it's exclusively line-based. He needs to know the date and time, but these are contained on a line that comes before the one that tells him if he needs to keep that information at all. This is a situation where Perl comes in really handy. Not knowing the fully syntax of the log file, here's a simple script I came up with:

Code:

#!/usr/bin/env perl

use warnings;
use strict;

my $line = readline STDIN;

while (defined $line) {
    if ($line =~ m!======!) {
        my $date = '';
        my $time = '';
        my $error = 0;
        my $reference = '';
        $line = readline STDIN;
        while (defined $line && $line !~ m!======!) {
            if ($line =~ m!(\d\d/\d\d/\d\d\d\d) (\d\d:\d\d:\d\d)!) {
                $date = $1; $time = $2;
            }
            elsif ($line =~ m!^ERROR!) {
                $error = 1;
                $line =~ m!Reference :=(\d+)!;
                $reference = $1;
            }
            $line = readline STDIN;
        }
        if ($error) {
            print "ERROR: date = $date | time = $time | reference = $reference\n";
        }
    }
    else {
        $line = readline STDIN;
    }
}

This is the log file I used for testing:

Code:

======
05/02/2007 04:51:02 [8714]: abdndala.c01.07: SAMS can not be sent, TYPE=DOUBE, STATE: 342184546 (slipt 6,Entity:ANY).
ERROR: 4685 Reference :=172910706273
absolute MM :=)(A1225:clAd=0000)(29746:messi :=624020100762629)(2747:hel_lang=3)(K2749:
sams_id=398)
======
05/03/2007 04:51:02 [8714]: blahblah.c01.07: SAMS can not be sent, TYPE=DOUBE, STATE: 342184546 (slipt 6,Entity:ANY).
ERROR: 4685 Reference :=12345678
absolute MM :=)(A1225:clAd=0000)(29746:messi :=624020100762629)(2747:hel_lang=3)(K2749:
sams_id=398)
======
05/03/2007 04:51:02 [8714]: foobar.c01.07: SAMS sent successfully, TYPE=DOUBE, STATE: 342184546 (slipt 6,Entity:ANY).
absolute MM :=)(A1225:clAd=0000)(29746:messi :=624020100762629)(2747:hel_lang=3)(K2749:
sams_id=398)
======

Where the first two contain errors and the last one doesn't. Even if that's not quite correct, the script may still work right.

muha · 02-23-2007, 08:21 AM

This might get you started.
Print paragraphs that contain ERROR:

Code:

sed -e '/./{H;$!d;}' -e 'x;/ERROR/!d;' my_file

Then drill down, maybe using more sed commands to get the data and blah.
Some people might tell you to use perl.

If you need more info for sed: post more input and intended output.
Good link for sed:
http://www.linuxquestions.org/bookmarks/tags/sed

Hobbletoe · 02-23-2007, 11:26 AM

Quote:

Originally Posted by taylor_venable

The problem with grep is that it's exclusively line-based.

Actually, grep can be used in this case as well using the -B option to print out lines that occur before the match occurs (-A if you want lines after you match). Ex:

Code:

#!/bin/bash
# Create the error listing.  By default, grep 
#  will separate the lines with a '--' string
error_string=$(grep -B 1 "ERROR" log.file)

# Loop through the error_string variable until
#  there is no more information in it.
while [ ${#error_string} -gt 0 ]
do
 # Pull out the first error string
 error_string2=${error_string%%--*}

 # Remove the first error string, enclose in double
 #  quotes to handle slashes
 error_string=${error_string//"${error_string2}"/}

 # Remove the -- from the string.  Can not be done
 #  on the previous line as there is no -- at the 
 #  end of the string, and you end up in an 
 #  infinite loop
 error_string=${error_string/--/}

 # echo out the date, time, error, and reference
 echo $(echo ${error_string2} | cut -d' ' -f1,2) ERROR$(echo ${error_string2##*ERROR})

done

This is obviously not the best way to parse the error log (a big log would totally crush this code to the point of it being unusable, trust me), and would suggest using one pearl or sed as mentioned, but wanted to point out that grep can be used for more than displaying just the line that matches your text.