rsyslog writes to disk & log rotation

hattori.hanzo · 05-08-2013, 09:20 PM

Hello,

I am using rsyslog to received syslog messages from my firewalls. I have setup logrotation to happen as per rsyslog's example:

Code:

/etc/rsyslog.conf:

#####################################################
# Log everything to a per host daily logfile        #
#####################################################
$template DailyPerHostLogs,"/var/log/rsyslog/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%.log"
*.* -?DailyPerHostLogs

/etc/cron.hourly/syslog-bzip2:

# Compress *.log-files not changed in more than 24 hours:
find /var/log/syslog/2008 -type f -mtime +1  -name "*.log" -exec gzip '{}' \;

The cron job checks if the log file has not changed in more than 24 hours to them gzip it. Since the firewalls are very active and always logging packets this condition will never be true.

How can I do log rotation but not drop any syslog events? I did a test by removing -mtime +1 and it dropped about 7 seconds of logs.

I looked at rsyslog's disk queuing but that only applies if the destination is unavaliable but that will never be when it writes to disk.

Thanks.

chrism01 · 05-08-2013, 11:17 PM

Try -mmin http://linux.die.net/man/1/find

hattori.hanzo · 05-08-2013, 11:38 PM

Thanks Chris. Will try that.

unSpawn · 05-09-2013, 12:16 AM

Quote:

Originally Posted by hattori.hanzo

How can I do log rotation but not drop any syslog events?

You already let Rsyslogd handle log rotation by using a per-day, per-host log file name template.
Ergo Rsyslogd will close the old file and open a new one on day start.
So the Rsyslogd part is not what you're having problems with.

Quote:

Originally Posted by hattori.hanzo

I looked at rsyslog's disk queuing but that only applies if the destination is unavaliable but that will never be when it writes to disk.

About $MainMsgQueue.*: "In this mode, receiver and output modules are de-coupled via an in-memory queue. This queue buffers messages when the output modules are not capable to process them as fast as they are received". So it's not availability of the output module but speed that dictates it.

Depending on how many hosts you need to handle, how long log files need to be stored and where they should be stored after processing I would rewrite the template as it IMHO makes no sense duplicating %HOSTNAME and having that much tree depth. With your current directory structure I would rewrite the cronjob this way:

Code:

#!/bin/sh --
# /var/log/rsyslog/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/%HOSTNAME%.log"
DATE_STRING=$(/bin/date +"%Y/%m/%d" --date="yesterday")

find /var/log/syslog/ -type d -iname \*${DATE_STRING}\* | while read LOG_LOG_DIRNAME; do
 # Loop over yesterdays logs:
 find "${LOG_LOG_DIRNAME}" -type f | while read OLD_LOG_FILENAME; do
  # Superfluous now but loop only over closed files anyway:
  /sbin/fuser "${OLD_LOG_FILENAME}" >/dev/null 2>&1
  # Check if you want reporting else uncomment and add " || bzip2 "${OLD_LOG_FILENAME}"" to the line above
  case $? in
   0) /usr/bin/logger -t logrotate.custom "In use: "${OLD_LOG_FILENAME}".";;
   1) bzip2 "${OLD_LOG_FILENAME}";;
   *) /usr/bin/logger -t logrotate.custom "Unknown problem: "${OLD_LOG_FILENAME}". Investigate.";;
  esac
 done
done
exit 0

Yes it's convoluted but looping only over yesterdays logs and checking if they're not in use IMHO is the safest way to rotate these logs.

hattori.hanzo · 05-17-2013, 01:33 AM

Thanks alot for the input. I will look into this further with some more testing.

Cheers.