Problem with the rotation of logs

yzT! · 03-31-2014, 01:27 AM

I'm facing a problem with one of my servers related to the rotation of logs. When it reaches the log.2, it flushes all logs and stops logging unless I restart syslog.

For instance:

Mar 8 - secure
Mar 15 - secure.1
Mar 22 - secure.2
Mar 29 - previous logs lost and not logging anymore.

I manage like 50 servers and only this one is acting like this. What could be happening?

TenTenths · 03-31-2014, 02:10 AM

Check your logrotate settings are the same on this server.
Check your logs to see if syslog is failing at any point.
When you have to (re)start syslog is it actually running at the time.

As you haven't posted any of your logrotate settings relevant to that log that's about the only suggestions I can make.

yzT! · 03-31-2014, 02:23 AM

Just noticed that this server had "rotate 2" while others had "rotate 4". That explains why the latest file with logs is the .2, however, that's not a reason to stop logging at all, it should start logging again in the first file.

Code:

# see "man logrotate" for details
# rotate log files weekly
weekly

# keep 4 weeks worth of backlogs
rotate 4

# create new (empty) log files after rotating old ones
create

# uncomment this if you want your log files compressed
#compress

# RPM packages drop log rotation information into this directory
include /etc/logrotate.d

# no packages own wtmp -- we'll rotate them here
/var/log/wtmp {
    monthly
    create 0664 root utmp
    rotate 1
}

# system-specific logs may be also be configured here.

TenTenths · 03-31-2014, 02:32 AM

Ok, that's your logrotate config file, as you can see it includes files in /etc/logrotate.d you'll probably find a "syslog" file in there.

For reference on my CentOS 6.5 box this is:

Code:

/var/log/cron
/var/log/maillog
/var/log/messages
/var/log/secure
/var/log/spooler
{
    sharedscripts
    postrotate
        /bin/kill -HUP `cat /var/run/syslogd.pid 2> /dev/null` 2> /dev/null || true
    endscript
}

yzT! · 03-31-2014, 03:13 AM

In the server where this is happening, I got no syslog file. Instead I have syslog-ng. Content is the same as the others, just changing syslog for syslog-ng.

Code:

/var/log/messages /var/log/secure /var/log/maillog /var/log/spooler /var/log/boot.log /var/log/cron {
    sharedscripts
    postrotate
        /bin/kill -HUP `cat /var/run/syslog-ng.pid 2> /dev/null` 2> /dev/null || true
    endscript
}

TenTenths · 03-31-2014, 03:16 AM

Ok, that's fine and expected if you're using syslog-ng.

At least we've your 2 / 4 sorted, I'd still check your logs to see if there's any errors related to syslog(-ng) and also my comments about checking to see if syslog is actually stopped or "hung" next time it happens.

yzT! · 03-31-2014, 04:07 AM

I cannot check the logs because they are empty after this happens.

I'm adding a sticky note to remember to check back in four weeks whether it's stopped or not after it reaches the final log.