generate script perl mail log not work

puka · 05-30-2011, 12:40 PM

Dear unix forum members,

I'm working on a script that will parse a mail machine's logs and print a list of email addresses in this format:

authen@domain.com | sender@domain | recipient@domain
exam
account1@domain1.com | sender2@domain2.com | brian5841@yahoo.com,brooke_leee@yaho...0809@yahoo.com
or
account1@domain1.com | sender2@domain2.com | brian5841@yahoo.com
account1@domain1.com | sender2@domain2.com | brooke_leee@yahoo.com
account1@domain1.com | sender2@domain2.com | c0809@yahoo.com

The logs look something like this:

Feb 18 20:00:11 mail postfix/smtpd[27053]: F33E01715C9: client=unknown[22.22.22.22], sasl_method=LOGIN, sasl_username=account1@domain1.com
Feb 18 20:00:14 mail postfix/cleanup[27072]: F33E01715C9: message-id=<JGWU@domain2.com>
Feb 18 20:00:14 mail postfix/qmgr[27047]: F33E01715C9: from=<sender2@domain2.com>, size=5578, nrcpt=30 (queue active)
Feb 18 20:00:20 mail postfix/smtp[27117]: F33E01715C9: to=<brian5841@yahoo.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=8.5, delays=2.6/0.01/0.01/5.9, dsn=2.0.0, status=sent (250 2.0.0 Ok, id=13740-01, from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 5AEE71715D0)
Feb 18 20:00:20 mail postfix/smtp[27117]: F33E01715C9: to=<brooke_leee@yahoo.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=8.5, delays=2.6/0.01/0.01/5.9, dsn=2.0.0, status=sent (250 2.0.0 Ok, id=13740-01, from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 5AEE71715D0)
Feb 18 20:00:20 mail postfix/smtp[27117]: F33E01715C9: to=<c0809@yahoo.com>, relay=127.0.0.1[127.0.0.1]:10024, delay=8.5, delays=2.6/0.01/0.01/5.9, dsn=2.0.0, status=sent (250 2.0.0 Ok, id=13740-01, from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 5AEE71715D0)
Feb 18 20:00:21 mail postfix/qmgr[27047]: F33E01715C9: removed

The "from" and "to" are on different lines and there is another challenge which is that the results should be limited to messages who have 3 or fewer recipients.

I thought it would be easy enough, and I wrote a script that first gets a list of the tag numbers ( F33E01715C9 which belong to messages with 3 or fewer recipients

Very crude and spaghetti like...and even worse is the FOR loop that follows, which involves grepping through the entire 6000mb maillog file 73,000 times in order to print the sender and recipient addresses.

Needless to say, its not an efficient script, there must be a better way. Please help!! Any responses are appreciated, maybe someone can just point me in the right direction?
Thanks,
Puka

Tinkster · 05-30-2011, 07:27 PM

Hi, welcome to LQ!

Here's an awk snippet that does part of what you're asking, it works
w/o loops and iterations over the same file:

Code:

$7~/client=unknown/{
  id[$6]=0
}
$7~/from=/{
  mail[$6]=gensub( /from=(.*)/, "\\1",1,$7)
}
$7~/to=/{
  mail[$6]=mail[$6]"|"gensub( /to=(.*)/, "\\1",1,$7)
  id[$6]++
}
$7~/removed/{
  if(id[$6] >2){
    print mail[$6]
  }else{
    delete id[$6]
    delete mail[$6]
  }
}

Should get you on the way, anyway ... convert to perl if you wish ;}

I took your data, replicated the lot, removed a line and changed
the ID from F33E01715C9 to F34E01715C9.

The line w/ three recipients got printed like so:

Code:

<sender2@domain2.com>,|<brian5841@yahoo.com>,|<brooke_leee@yahoo.com>,|<c0809@yahoo.com>,

Cheers,
Tink

puka · 05-30-2011, 09:47 PM

Tinkster · 05-30-2011, 11:36 PM

Right ... assuming the records are always in the same form & order:

Code:

#!/usr/bin/perl
while (<>) {
  ($Fld1,$Fld2,$Fld3,$Fld4,$Fld5,$Fld6,$Fld7,$Fld8,$Fld9) = split('[[:space:]]+', $_, -1);
  if ($Fld7 =~ /client=unknown/) {
    $id{$Fld6} = 0;
    $Fld9 =~ m/sasl_username=(.*)/;
    $mail{$Fld6} = $1."|";
  }
  if ($Fld7 =~ /from=/) {
    $Fld7 =~ m/from=<([^>]+)>,/;
    $mail{$Fld6} .= $1."|";
  }
  if ($Fld7 =~ /to=/) {
    $Fld7 =~ m/to=<([^>]+)>,/;
    $mail{$Fld6} .= $1.",";
    $id{$Fld6}++;
  }
  if ($Fld7 =~ /removed/) {
    if ($id{$Fld6} > 2) {
      print $mail{$Fld6}."\n";
    }
    else {
      delete $id{$Fld6};
      delete $mail{$Fld6};
    }
  }
}

Here's the output:

Code:

account1@domain1.com|sender2@domain2.com|brian5841@yahoo.com,brooke_leee@yahoo.com,c0809@yahoo.com,

I'm certain there's prettier or more efficient ways to do
this in perl; I just hacked what awk does up in perl like
lingo ... enjoy.

Cheers,
Tink

Tinkster · 05-31-2011, 11:54 AM

So .. did that help?