I need to pick out the first, last and how many from a file...?!!?

vous · 03-22-2005, 08:46 AM

Hello All,

I am having a h3ll of a time redoing my log analysis scripts in bash....please help me :-(....

OK this is what I need to do...

I get a file copied to a directory every day with this content:

<<<<<<<<<<<<START of FILE>>>>>>>>>>>>>>>>>

20050312091225,JOE,0015557
20050312091225,JOE,0015557
20050312091226,JOE,0015557
20050314183506,JOE,0015633
20050309103330,BOB,0016744
20050310112812,BOB,0016744
20050311121502,BOB,0016744
20050312140003,BOB,0016744
20050313114504,BOB,0016744
20050314110005,BOB,0016744
20050315160004,BOB,0016744
20050100034434,BOB,0016744

<<<<<<<<<<<<END of FILE>>>>>>>>>>>>>>>>>>>

From this file I have to write a script that does the following things AND they have to be done from a script written for the bash shell:

1) Based on the last 2 arguments "JOE,0015557" figure out how many times it comes up in the file.

2) From the ones that occur more than once, when was the first and when was the last time it occured. This you figure out based on the first field which is a timestamp.

So the file that has to come as output from the bash scipt will be something like:

JOE,0015557------3 times-----First Incidence:20050312091225-----Lastincidence:20050312091226
BOB,0016744------8 times-----First Incidence:20050100034434-----Lastincidence:20050315160004

Any thoughts on how to do this in bash shell?

innuendo_98 · 03-22-2005, 08:53 AM

try using cut and use as your delimiter the ',' asign fields 2 and 3 to a variable and loop throgh the file, if you have a counter variable whenever you find a match you increase the counter

cut --delimiter='/' --output-delimiter='/' --fields=2,3

enemorales · 03-22-2005, 11:34 AM

Perl is your man.

Code:

#!/usr/bin/perl

while(<STDIN>) {
  chomp;
  ($date,$user,$number) = split /,/;
  $key = "$user,$number";
  $frec{$key}++;
  if ($first{$key}) {
    $first{$key} = $date if ($first{$key} ge $date);
  } else {
    $first{$key} = $date;
  }
  if ($last{$key}) {
    $last{$key} = $date if ($last{$key} le $date);
  } else {
    $last{$key} = $date;
  }
}

for $i (keys %frec) {
  if ($frec{$i}>1) {
    print "$i---$frec{$i}---First incidende:$first{$i}----Last incidende:$last{$i}\n";
  }
}

If you save a file named "yourscript" with this and give it excecution permissions, you can run

Code:

cat yourfile.txt | yourscript

PS: I'm very new to perl, but from what I now of its, there is plenty of other ways to write this problem, many of them better than mine. Anyway, at least I can explain you how this solution works, so dont hesitate to ask...

TheLinuxDuck · 03-22-2005, 01:44 PM

Isn't this the exact same problem you asked in this thread, minus slight data modification?