LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 03-17-2003, 02:27 AM   #1
markus1982
Senior Member
 
Registered: Aug 2002
Location: Stuttgart (Germany)
Distribution: Debian/GNU Linux
Posts: 1,467

Rep: Reputation: 46
parsing mail.log with perl and calculate mail traffic on domain base


I'm trying to set up a perl program to be able to calculate the mail traffic per domain. I'd like to seperate mail sent/received traffic. It should also support multiple domains (stored in an SQL table).

I thought about the message id as base for everything since it's unique, right?

Here's a example program I've come up so far with:
Quote:
#!/usr/bin/perl -w
use strict;

my $client_ip;
my $mail_from;
my $mail_to;
my $message_id;
my $message_size;
my $message_time;
my $nr_recipient;


open(FILE, "mail.log") or die "Couldn't open mail.log: $!; aborting";
while (<FILE>) {
if ($_ =~ /(\S+ .* \d+:\d+:\d+) .* postfix\/smtpd\[.*\]: (\S+): client=.*\[(\d+\.\d+\.\d+\.\d+)\]$/) {
$message_time = $1;
$message_id = $2;
$client_ip = $3;

print "Message-ID: \t", $message_id,"\n";
print "Message-Time: \t", $message_time,"\n";
print "Client-IP: \t", $client_ip,"\n\n";
} elsif ($_ =~ /.* postfix\/qmgr\[.*\]: (\S+): from=<(\S+)>, size=(\d+), nrcpt=(\d+)/) {
$message_id = $1;
$mail_from = $2;
$message_size = $3;
$nr_recipient = $4;

print "Message-ID: \t", $message_id, "\n";
print "Mail from: \t", $mail_from, "\n";
print "Messagesize: \t", $message_size, "\n";
print "No recipients:\t", $nr_recipient, "\n\n";
} elsif ($_ =~ /.* postfix\/.*: (\S+): to=<(\S+)>/) {
$message_id = $1;
$mail_to = $2;

print "Message-ID: \t", $message_id, "\n";
print "Mail to: \t", $mail_to, "\n\n";
}
}
close(FILE);
And an example log file:
Quote:
Mar 17 07:11:43 box postfix/pickup[489]: 2C2C7C0A8: uid=1000 from=<markus>
Mar 17 07:11:43 box postfix/cleanup[493]: 2C2C7C0A8: message-id=<20030317061143.GA481@box.example.de>
Mar 17 07:11:43 box postfix/qmgr[451]: 2C2C7C0A8: from=<markus@example.com>, size=435, nrcpt=7 (queue active)
Mar 17 07:11:44 box postfix/smtp[495]: 2C2C7C0A8: to=<recipient1@example.net>, relay=XXX.XXX.net[XXX.XXX.XXX.XXX], delay=1, status=sent (250 2.0.0 h2H72LG00567 Message accepted for delivery)
Mar 17 07:11:44 box postfix/smtp[495]: 2C2C7C0A8: to=<recipient2@example.net>, relay=XXX.XXX.net[XXX.XXX.XXX.XXX], delay=1, status=sent (250 2.0.0 h2H72LG00567 Message accepted for delivery)
Mar 17 07:11:44 box postfix/smtp[494]: 2C2C7C0A8: to=<recipient3@example.net>, relay=XXX.XXX.net[XXX.XXX.XXX.XXX], delay=1, status=sent (250 2.0.0 h2H72LG00566 Message accepted for delivery)
Mar 17 07:11:44 box postfix/smtp[494]: 2C2C7C0A8: to=<recipient4@example.net>, relay=XXX.XXX.net[XXX.XXX.XXX.XXX], delay=1, status=sent (250 2.0.0 h2H72LG00566 Message accepted for delivery)
Mar 17 07:11:44 box postfix/smtp[494]: 2C2C7C0A8: to=<recipient5@example.de>, relay=XXX.XXX.net[XXX.XXX.XXX.XXX], delay=1, status=sent (250 2.0.0 h2H72LG00566 Message accepted for delivery)
Mar 17 07:11:44 box postfix/smtp[494]: 2C2C7C0A8: to=<recipient6@example.de>, relay=XXX.XXX.net[XXX.XXX.XXX.XXX], delay=1, status=sent (250 2.0.0 h2H72LG00566 Message accepted for delivery)
Mar 17 07:11:44 box postfix/smtp[494]: 2C2C7C0A8: to=<recipient7@example.de>, relay=XXX.XXX.net[XXX.XXX.XXX.XXX], delay=1, status=sent (250 2.0.0 h2H72LG00566 Message accepted for delivery)
Example output of the current perl program:
Quote:
Message-ID: 2C2C7C0A8
Mail from: markus@example.com
Messagesize: 435
No recipients: 7

Message-ID: 2C2C7C0A8
Mail to: recipient1@example.net

Message-ID: 2C2C7C0A8
Mail to: recipient2@example.net

Message-ID: 2C2C7C0A8
Mail to: recipient3@example.net

Message-ID: 2C2C7C0A8
Mail to: recipient4@example.net

Message-ID: 2C2C7C0A8
Mail to: recipient5@example.de

Message-ID: 2C2C7C0A8
Mail to: recipient6@example.de

Message-ID: 2C2C7C0A8
Mail to: recipient7@example.de
So the regular expressions work fine ... now what needs to be done is the following:

- check if example.com, example.de, example.net is in file accounting_domains.txt
- if it is in there calculate traffic from / to the domains


I'm no perl expert so I need advice here ... also with the whole concept in general. Suggestions ?
 
Old 03-18-2003, 06:22 AM   #2
j-ray
Senior Member
 
Registered: Jan 2002
Location: germany
Distribution: ubuntu
Posts: 1,434

Rep: Reputation: 100Reputation: 100
i couldnt test it but i guess sth like that should work:
(it doesnt differ between mail:to and :from but u will know how to use regexp yourself)

open a filehandle on the log file as before
write the mailaddress into $mailaddress

my ($mailaddress, $read,$num, $sum);

while(<INFILE>) {
if (/$mailadress/) {
$read = 1;}
else {read=0;}

if ((/Messagesize: /) && $read == 1) {
$num = substr($_,13);

$num =~tr/[1-9]//cd;
$sum += $num;
}
}
close (INFILE);
print "traffic of user $mailaddress was $sum Bytes\n";

Last edited by j-ray; 03-18-2003 at 06:26 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
I can only get mail for myhost.domain.com. I cannot get my mail for domain.com. adamrau Linux - Software 1 04-22-2007 03:07 PM
Mail file parsing paddyjoy Programming 4 08-07-2005 10:52 PM
how to find http traffic and mail traffic alone? basbosco Linux - General 1 06-07-2005 10:29 PM
qmail log says mail delivered, mail nowhere to be found svartrev Linux - Software 0 03-24-2004 05:12 PM
ISP mail aliases + Domain mail how? HoundDog Linux - Networking 1 08-09-2003 07:17 AM


All times are GMT -5. The time now is 11:52 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration