Logwatch and Postfix

fukawi2 · 03-20-2009, 01:04 AM

I have Logwatch installed from my distro repos, but it refuses to display any Postfix output:

Code:

root@dingo ! # logwatch --print

 ################### Logwatch 7.3.6 (05/19/07) #################### 
        Processing Initiated: Fri Mar 20 16:47:17 2009
        Date Range Processed: yesterday
                              ( 2009-Mar-19 )
                              Period is day.
      Detail Level of Output: 5
              Type of Output: unformatted
           Logfiles for Host: dingo.DOMAIN.net
  ################################################################## 
 
 --------------------- httpd Begin ------------------------ 
<SNIP> 
 ---------------------- httpd End ------------------------- 

 
 --------------------- Disk Space Begin ------------------------ 
<SNIP>
 ---------------------- Disk Space End ------------------------- 
 ###################### Logwatch End #########################

My 'service' line is set to 'all' as per the default:

Code:

root@dingo ~ # grep '^Service' /usr/share/logwatch/default.conf/logwatch.conf 
Service = All
Service = "-zz-network"     # Prevents execution of zz-network service, which
Service = "-zz-sys"         # Prevents execution of zz-sys service, which
Service = "-eximstats"      # Prevents execution of eximstats service, which

My postfix definately has data:

Code:

root@dingo ~ # tail /var/log/mail.log
2009-03-20T16:35:46+11:00 dingo postfix/smtpd[5090]: connect from wf-out-1314.google.com[209.85.200.168]
2009-03-20T16:35:46+11:00 dingo postfix/smtpd[5090]: 893C31C88: client=wf-out-1314.google.com[209.85.200.168]
2009-03-20T16:35:46+11:00 dingo postfix/cleanup[5109]: 893C31C88: message-id=<f9f9bf9a0903192235y7ab7e22cg2c7e6a66cff93bff@mail.gmail.com>
2009-03-20T16:35:46+11:00 dingo postfix/qmgr[10274]: 893C31C88: from=<CENSORED-EMAIL-ADDRESS>, size=8523, nrcpt=1 (queue active)
2009-03-20T16:35:48+11:00 dingo postfix/qmgr[10274]: 893C31C88: removed
2009-03-20T16:35:48+11:00 dingo postfix/smtp[5111]: 893C31C88: to=<CENSORED-EMAIL-ADDRESS>, orig_to=<CENSORED-EMAIL-ADDRESS>, relay=gmail-smtp-in.l.google.com[74.125.45.114]:25, delay=2.4, delays=0.38/0.01/0.33/1.7, dsn=2.0.0, status=sent (250 2.0.0 OK 1237527348 9si4841371yxs.36)
2009-03-20T16:36:16+11:00 dingo postfix/smtpd[5090]: disconnect from wf-out-1314.google.com[209.85.200.168]
2009-03-20T16:38:33+11:00 dingo pop3d: Connection, ip=[::ffff:118.208.137.186]
2009-03-20T16:38:33+11:00 dingo pop3d: LOGOUT, ip=[::ffff:118.208.137.186]
2009-03-20T16:38:33+11:00 dingo pop3d: Disconnected, ip=[::ffff:118.208.137.186]
2009-03-20T16:39:36+11:00 dingo postfix/anvil[5094]: statistics: max connection rate 1/60s for (smtp:202.14.166.251) at Mar 20 16:35:05
2009-03-20T16:39:36+11:00 dingo postfix/anvil[5094]: statistics: max connection count 1 for (smtp:202.14.166.251) at Mar 20 16:35:05
2009-03-20T16:39:36+11:00 dingo postfix/anvil[5094]: statistics: max cache size 2 at Mar 20 16:35:46
2009-03-20T16:42:39+11:00 dingo pop3d: LOGIN, user=CENSORED-EMAIL-ADDRESS, ip=[::ffff:118.208.137.186], port=[41773]
2009-03-20T16:42:39+11:00 dingo pop3d: LOGIN, user=CENSORED-EMAIL-ADDRESS, ip=[::ffff:118.208.137.186], port=[41774]
2009-03-20T16:42:40+11:00 dingo pop3d: LOGOUT, user=CENSORED-EMAIL-ADDRESS, ip=[::ffff:118.208.137.186], port=[41773], top=0, retr=0, rcvd=12, sent=39, time=1
2009-03-20T16:42:40+11:00 dingo pop3d: LOGOUT, user=CENSORED-EMAIL-ADDRESS, ip=[::ffff:118.208.137.186], port=[41774], top=0, retr=0, rcvd=12, sent=39, time=1

unSpawn · 03-22-2009, 06:06 AM

What shows if you would 'logwatch --service postfix --print --Detail <level> --debug <level> --range All --numeric'? (Play with "level" being Med and High)

fukawi2 · 03-22-2009, 04:56 PM

Code:

root@dingo ~ # logwatch --service postfix --print --Detail Med --debug Med --range All --numeric
export LOGWATCH_DATE_RANGE='all'
export LOGWATCH_GLOBAL_DETAIL='5'
export LOGWATCH_OUTPUT_TYPE='unformatted'
export LOGWATCH_TEMP_DIR='/var/cache/logwatch/logwatch.nuztvtcI/'
export LOGWATCH_DEBUG='5'

Preprocessing LogFile: maillog
/var/cache/logwatch/logwatch.nuztvtcI/maillog-archive /var/log/mail.log  | /usr/bin/perl /usr/share/logwatch/scripts/shared/expandrepeats ''| /usr/bin/perl /usr/share/logwatch/scripts/shared/onlyhost ''| /usr/bin/perl /usr/share/logwatch/scripts/shared/applystddate ''>/var/cache/logwatch/logwatch.nuztvtcI/maillog
export postfix_syslog_name='postfix'

High output is here:
http://www.pastebin.ca/1368452

unSpawn · 03-22-2009, 07:15 PM

Thanks for the detailed logs. Looking a bit further I found my Logwatch is rather stale (7.3.6) for testing compared to the standalone parser at http://www.mikecappella.com/logwatch/. See if running that one works "better": unpack, install (or not, move to any tempdir for testing), change location of ^my.$config_file in postfix-logwatch, then run as 'perl -T ./postfix-logwatch /var/log/maillog'. If that produces any or more output (no --debug) you know the cause. If not then maybe post a piece of (anonymized) log for testing?

fukawi2 · 03-22-2009, 07:42 PM

Thanks for your interest and help so far unSpawn

No luck with that either though

Code:

root@dingo /tmp/postfix-logwatch-1.37.08 # grep -E '^my \$config_file' postfix-logwatch
my $config_file = "/usr/share/logwatch/default.conf/logwatch.conf";
root@dingo /tmp/postfix-logwatch-1.37.08 # perl -T ./postfix-logwatch /var/log/mail.log
root@dingo /tmp/postfix-logwatch-1.37.08 #

How many lines of logging would be good for testing?

unSpawn · 03-22-2009, 07:56 PM

Aw, say onehundred. If you don't want to dump 'em here or on pastebin you're invited to send me an email. BTW, what Logwatch version do you run?

fukawi2 · 03-22-2009, 08:04 PM

Code:

Name           : logwatch
Version        : 7.3.6-1

Email addresses have been changed to protect the innocent

http://www.pastebin.ca/1368626

As I was putting that up to pastebin, I had the thought that perhaps Logwatch can't handle the ISO timestamps? That would be annoying, but I think I can work around it in syslog-ng by logging twice to 2 different files, an ISO version and a 'normal' version

unSpawn · 03-22-2009, 08:23 PM

Quote:

Originally Posted by fukawi2

As I was putting that up to pastebin, I had the thought that perhaps Logwatch can't handle the ISO timestamps? That would be annoying, but I think I can work around it in syslog-ng by logging twice to 2 different files, an ISO version and a 'normal' version

Good call! It is the ISO date. No need to log twice. You only need a wee script to massage the date into a format Logwatch is willing to read. In the long run it should be patched to accept ISO dates ofcourse. Maybe something for their bug tracker or mailing list?

fukawi2 · 03-22-2009, 08:26 PM

Hmm, buggah!

Thinking back, I actually only changed the date format for an exercise I was doing with logging to SQL. I'm over that 'project' so I can just change it back...

Thanks again for your help, I'll mark this one as SOLVED

unSpawn · 03-22-2009, 08:41 PM

Cool. Glad you've solved that. For anyone looking, in shell the conversion from ISO.* to ancient syslog-style date should go something like 'ISO2syslog() { DATE="$1"; DATE=${DATE/+*/}; date --date=${DATE} "+%b %e %H:%M:%S"; }'.

fukawi2 · 03-22-2009, 08:56 PM

Close... I'm not sure what's borked though...

Code:

root@dingo /var/log # for LNE in `cat mail.log` ; do DATE="$1"; DATE=${DATE/+*/}; date --date=${DATE} "+%b %e %H:%M:%S"; done | uniq 
Mar 23 00:00:00
root@dingo /var/log #

unSpawn · 03-23-2009, 04:01 AM

Try

Code:

cat mail.log|while read LINE; do LINE=(${LINE})
 DATE="${LINE[0]}"; DATE=${DATE/+*/}
 LINE[0]=$(date --date=${DATE} "+%b %e %H:%M:%S")
 echo "${LINE[*]}"
done

fukawi2 · 03-23-2009, 06:20 PM

Mr. C. · 03-29-2009, 01:47 PM

FYI: postfix-logwatch runs both standalone, and replaces the postfix filter shipped with logwatch. Due to licensing differences, I am no longer updating the logwatch source with the postfix filter, so the most up to date will be available at the link above.