LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 11-30-2009, 02:41 PM   #1
grep65535
LQ Newbie
 
Registered: Jul 2006
Location: Seattle, WA
Distribution: Fedora, RHEL, CentOS
Posts: 29

Rep: Reputation: 18
awstats not reading maillog properly?


RHEL 5.4
postfix-2.3.3-2.1.el5_2
awstats-6.9-1.el5

I've setup awstats on the same server over and over now with both the tar & rpm packages. At the moment I'm back to using the RHEL5 repo version. I've come to the conclusion that my awstats setup may be correct, but maybe my log isn't being parsed correctly.

This server was running RHEL3 earlier this month and I setup awstats for tracking the mail server's stats. It worked perfectly showing very accurate (exactly what we expected) numbers in the thousands every day. Then I upgraded to RHEL 5.4 and as far as I know set everything back to what it was. But now awstats only shows like 10-50 emails sent each day from the server, even though it's actually 1600~3000. I even used the pflogsumm package to verify that I wasn't just crazy, and it reported exactly what we expected.

Postfix settings in the main.cf are exactly as they were before.
The awstats conf file is exactly as it was before. I did a direct copy initially of the /var/www/cgi-bin/awstats stuff so that started out exactly the same with permissions and everything.

We're using mailman-2.1.9-4.el5 to send out 99.99% of emails on this server and those settings are all equally transferred right over.
I've tested mailman out using a test list...I sent about 3000 emails to the test list which had 2 recipients configured on it, and it reported about 3000 emails in awstats. I had thought that it worked at the time but now I'm realizing that since I sent it 3000 times, the server received 3000 emails, and sent 3000 more, so should have recorded over 6000. Mailman uses the aliases file for sending to mailing lists. For ex:

mailman: "|/usr/lib/mailman/mail/mailman post mailman"
mailman-admin: "|/usr/lib/mailman/mail/mailman admin mailman"
mailman-bounces: "|/usr/lib/mailman/mail/mailman bounces mailman"

Maybe one of the perlscripts parses these sort of mails differently?


I've been going over and over the logs and it seems to only be recording (or marking as valid) the mail that the server receives back...bounces, automated responses, etc. whereas all email going out (according to the log) is just discarded. I'm assuming this is something to do with the log file format...but then I'm not sure what I can do about it since I can't seem to find any other instance of this on google.


in my awstats.postfix.conf file I have:


LogFile="perl /var/www/cgi-bin/awstats/maillogconvert.pl standard < /var/log/maillog |"
LogType=M
LogFormat="%time2 %email %email_r %host %host_r %method %url %code %bytesd"
LogSeparator=" "
SiteDomain="interestingdomain.com"
HostAliases="interestingdomain.com"
LogType=M
LevelForBrowsersDetection=0
LevelForOSDetection=0
LevelForRefererAnalyze=0
LevelForRobotsDetection=0
LevelForWormsDetection=0
LevelForSearchEnginesDetection=0
LevelForFileTypesDetection=0
ShowMenu=1
ShowSummary=HB
ShowMonthStats=HB
ShowDaysOfMonthStats=HB
ShowDaysOfWeekStats=HB
ShowHoursStats=HB
ShowDomainsStats=0
ShowHostsStats=HBL
ShowAuthenticatedUsers=0
ShowRobotsStats=0
ShowEMailSenders=HBML
ShowEMailReceivers=HBML
ShowSessionsStats=0
ShowPagesStats=0
ShowFileTypesStats=0
ShowFileSizesStats=0
ShowBrowsersStats=0
ShowOSStats=0
ShowOriginStats=0
ShowKeyphrasesStats=0
ShowKeywordsStats=0
ShowMiscStats=0
ShowHTTPErrorsStats=0
ShowSMTPErrorsStats=1


I've also tried for the logfile:
LogFile="cat /var/log/maillog | /etc/awstats/prepflog.pl | perl /etc/awstats/maillogconvert.pl standard |"

I don't know much perl, but I did an sdiff on the maillogconvert.pl and the awstats.pl between the RHEL version and the tarball from the web site, and there are a few differences.
I've just tried using both versions of these files to update stats (deleting the awstats db files in between) to no avail.

Anyone have any idea what I may be missing?
 
Old 12-01-2009, 05:44 PM   #2
grep65535
LQ Newbie
 
Registered: Jul 2006
Location: Seattle, WA
Distribution: Fedora, RHEL, CentOS
Posts: 29

Original Poster
Rep: Reputation: 18
Maybe it has something to do with mailman "personalization" and how those emails are differentiated (if they even are) in /var/log/maillog . I've been over the logs many times and I can't find any obvious differences in the formatting of before when it worked vs now when it's not working.

Makes me point more to the scripts, but then wouldn't this be a wider issue with more attention on the web that what I've found...which is nothing similar to this issue I'm having.
 
Old 12-04-2009, 10:45 PM   #3
grep65535
LQ Newbie
 
Registered: Jul 2006
Location: Seattle, WA
Distribution: Fedora, RHEL, CentOS
Posts: 29

Original Poster
Rep: Reputation: 18
It ends up that apparently when postfix relays email off to an exchange server, it switches each log entry's DSN from 2.0.0 to 2.6.0 for some legitimate but unclear reason. The script maillogconvert.pl discards all log entries marked with a DSN of 2.6.0 because it's considered to be a "possible duplicate message". But in this case they are not duplicate messages, they're ALL outgoing messages from the server.

So what we did was just change the if/ifelse statement in the perlscript to check for "2._6.0" so it just fails. This works in our case because it's purely a listserv that sends out email notifications, not a server that user actively interact with like a regular email server. I still don't know why it's different than before...because it worked before. This was an upgrade from RHEL3 to RHEL5.
 
  


Reply

Tags
awstats, mailman, postfix, rhel


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
maillog query linuxjamil Linux - Server 1 05-28-2009 02:51 AM
help for maillog ayush1440 Linux - Server 0 05-22-2008 11:29 AM
why is maillog on my server getting created in /var/log/maillog.3 ? weblink_dipti Linux - Software 2 06-16-2007 05:47 AM
System won't boot properly knoppix doesn't initialize properly rodewan Linux - Software 0 01-23-2005 04:35 PM
CD Rom drive not reading properly snecklifter Linux - Newbie 0 05-03-2004 06:50 AM


All times are GMT -5. The time now is 11:00 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration