Rotating Apache logs and Webalizer

xMeta4x · 02-21-2008, 02:48 AM

*edit* - Sorry, could a mod please move to the server forum?

Hi All,

I have a site whose "access_log" file is growing rapidly. I'd like to rotate the logs, but I'm not sure how to do so and still be able to run webalizer. I've googled but didn't find any clear instructions.

I looked at rotate logs (piped from apache), but rather then dumping the old log off as a renamed file and continuing with "access_log", it seems to start logging to the alternatively named file which stops webalizer from working.

I run webalizer once a night as a cron job. Could I simply enter a subsequent command to rename "access_log" and then let apache re-create it?

I'd like to run it past you guys first before I start messing about on a live server!

Cheers,

Andy

(For information, the server is CentOS 5 with Apache 2)

theNbomr · 02-21-2008, 02:25 PM

This is my logrotate script for an Apache web server. It runs at midnight, and I set up a cron job to run at 2 minutes after midnight to run webalizer. I just ignore any missed hits during that 2 minute interval. The logrotate script creates a datestamp filename, for which I have other parsers to further analyze the logs.

/etc/logrotate.d/httpd

Code:

/var/log/httpd/*_log {
    daily
    rotate 368
    missingok
    notifempty
    sharedscripts
    postrotate
        /bin/kill -HUP `cat /var/run/httpd.pid 2>/dev/null` 2> /dev/null || true
    endscript
    lastaction
        yesterday=ac$(date -d yesterday +%y%m%d).log; logdir=/var/log/httpd; mv ${logdir}/access_log.1 ${logdir}/${yesterday}
    endscript
}

This runs on a FC5 based host.
--- rod.

griffey · 02-21-2008, 03:39 PM

Hi Andy.

I can't for sure answer the second part of your question, but I'll see if I can help with the first.

In my virtual host section for each of my hosts, I have this for the logging:

Code:

CustomLog "| /usr/sbin/rotatelogs /var/log/httpd/website-access_log.%B 2629800" combined

The %B appends a month name, and the 2629800 is around the normal number of seconds for a month.

This gets me:
website-access_log.January
website-access_log.February
website-access_log.March

Etc., etc.

I use Analog to do stats and in the configuration file when I tell it what log to look at I can use a wildcard and tell it to look in "path/to/logs/website-access_log.*" and it will analyze all of the logs that match as if they were one big file. Again, not sure about webalizer or how it is configured.

Hope that helps (partially, maybe, at least).

Good luck.

G.--

xMeta4x · 02-22-2008, 01:38 AM

Thanks for the replies!

I've gotten a little further by using logrotate (not rotatelogs). I run this cron job:

Code:

01 00 * * * root /usr/sbin/logrotate /etc/logrotate.d/httpd

The contents of httpd referenced above is:

Code:

/var/log/httpd/*log {
    missingok
    notifempty
    sharedscripts
    postrotate
        /sbin/service httpd reload > /dev/null 2>/dev/null || true
    endscript
}

This seems to have deleted a file called access_log.1 (which suggests that logrotate has run before) and created one called access_log.2 and emptied access_log.

Whilst this is good, I wouldn't mind saving the old logs! Could I prevent the historic logs from being deleted?

Cheers,

Andy

theNbomr · 02-22-2008, 01:51 PM

Logrotate will do a rolling rotation, moving access_log to access_log.1, access_log.1 to access_log.2, and so on. The good news is that you haven't lost any log files, yet. As soon as logrotate reaches its default depth of file retention, it will drop the oldest log files. The 'lastaction' clause that I put into my logrotate script for httpd renames the log files to a date stamped version outside the domain of logrotate, and are thus retained perpetually. I compress a years worth of log files into one zip file and save them.
--- rod.

tylerhoadley · 03-20-2009, 12:42 AM

add prerotate to the logrotate file

/var/log/httpd/*log {
missingok
rotate 12
compress
delaycompress
notifempty
create 640 root adm
sharedscripts
postrotate
/sbin/service httpd reload > /dev/null 2>/dev/null || true
endscript
monthly
mail *********@*****.***
prerotate
webalizer -c /etc/webalizer/webalizer.conf
endscript

}

prerotate will run webalizer then rotate collecting all the data. verify where your webalizer.conf file is... could be /etc/webalizer.conf . This snippet will also rotate them once a month and keep them up for to one year/12 months. it will then compress after the second log has been rotated. the 13th log is also emailed to you before deletion, just add your email address. compression can be important depending on how much traffic and errors. use what you can.

Cheers