Bizarre syslog problem

forrie · 01-30-2012, 01:55 PM

The package sysklogd has not changed - but recently, without changes to the /etc/syslog.conf file, my data for local6.* and local4.* are showing up in /var/log/messages. They weren't before, and I can't seem to figure out why. Version info:

Name : sysklogd
Arch : x86_64
Version : 1.4.1
Release : 46.el5
Size : 125 k
Repo : installed

This is on RHEL 5.7. The syslog.conf file (with whitespace removed):

Quote:

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none /var/log/messages
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* -/var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg *
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log
local4.* /var/log/puppet/puppet.log
local6.* /local/log/firewall/firewall.log

I rebooted the system just now and the phenomenon is still happening.

It first showed up when I was reviewing our firewall logs -- the Cisco ASA is configured to send to "facility 22" (local6) -- that had not changed, either... I had noticed really huge /var/log/messages files and the configured location was empty.

Have I hit a bug?

Yes, I realize rsyslog and syslog-ng may be superior, but I want to fix this bug before introducing more variables :-)

Thanks.

acid_kewpie · 01-30-2012, 02:25 PM

"bizarre problem" tends to mean "thing I don't understand" ;-)

your config matches exactly with what you say is happening. All info level messages are clearly configured to be sent to /var/log/messages. I see nothing "phenomenonal" in any way at all.

forrie · 01-30-2012, 02:27 PM

I don't understand how this could be working for months, and then suddenly change. So, I can only assume there was/is a bug somewhere.

But shouldn't this also log *.info and local6.* to the defined locations or is this a first-match scenario?

I looked at the syslog.conf manpage, and I don't see an easy way to negate *.info from the LocalX facilities I configured. More impetus for me to consider a different syslog engine for that type of granularity :-)

acid_kewpie · 01-30-2012, 02:31 PM

it should log it to all appropriate destinations yes, but then you didn't mention that that wasn't happening originally. Do you know that the facilities are actually what you want them to be?

forrie · 01-30-2012, 02:33 PM

I posted the config. What originally alerted me to this were gazillions of Cisco firewall logs going to /var/log/messages and not to the location I defined /local/firewall/log/firewall.log where it has been going all along. In this scenario, the data are going to /var/log/messages and not to their named destinations otherwise....

I made sure the whitespace between the LHS and RHS are tabs, too. Restarted syslogd, no luck.

acid_kewpie · 01-30-2012, 02:36 PM

OK, well if they are coming in from IOS boxes, my angle to investigate would be a tcpdump on port 514 and see what the facility of the UDP traffic is listed as. I think a -v on tcpdump will show this, if not tshark / wireshark will.

forrie · 01-30-2012, 02:37 PM

I verified the config and it hasn't been changed, it's still "facility 22" which in IOS is LOCAL6. Also, the puppet logs, which were working fine, are also being shipped to /var/log/messages.

acid_kewpie · 01-30-2012, 02:42 PM

right, but I didn't mention the config. Is the traffic clearly on the right facility as the UDP packets hit you?

(btw, just install syslog-ng)