log files are empty -- messages,syslog,mail.err
 

Hi,

I have noticed that for couple of days/more than week I have empty log files:
messages,syslog,mail.err.

Could any body please help me to restore the logging ?


Thank you,
Kind Regards,
Martin

--- rsyslogd daemon runs.

---------------------------------------------/etc/syslog.conf:
# /etc/syslog.conf Configuration file for inetutils-syslogd.
#
# For more information see syslog.conf(5) manpage.

#
# First some standard logfiles. Log by facility.
#

auth,authpriv.* /var/log/auth.log
*.*;auth,authpriv.none /var/log/syslog
#cron.* /var/log/cron.log
daemon.* /var/log/daemon.log
kern.* /var/log/kern.log
lpr.* /var/log/lpr.log
mail.* /var/log/mail.log
user.* /var/log/user.log
uucp.* /var/log/uucp.log

#
# Logging for the mail system. Split it up so that
# it is easy to write scripts to parse these files.
#
mail.info /var/log/mail.info
mail.warn /var/log/mail.warn
mail.err /var/log/mail.err

# Logging for INN news system
#
news.crit /var/log/news/news.crit
news.err /var/log/news/news.err
news.notice -/var/log/news/news.notice

#
# Some `catch-all' logfiles.
#
*.=debug;\
auth,authpriv.none;\
news.none;mail.none /var/log/debug
*.=info;*.=notice;*.=warn;\
auth,authpriv.none;\
cron,daemon.none;\
mail,news.none /var/log/messages

#
# Emergencies are sent to everybody logged in.
#
*.emerg *

#
# I like to have messages displayed on the console, but only on a virtual
# console I usually leave idle.
#
daemon,mail.*;\
news.=crit;news.=err;news.=notice;\
*.=debug;*.=info;\
*.=notice;*.=warn /dev/tty5

# The named pipe /dev/xconsole is for the `xconsole' utility. To use it,
# you must invoke `xconsole' with the `-file' option:
#
# $ xconsole -file /dev/xconsole [...]
#
# NOTE: adjust the list below, or you'll go crazy if you have a reasonably
# busy site..
#
daemon.*;mail.*;\
news.crit;news.err;news.notice;\
*.=debug;*.=info;\
*.=notice;*.=warn |/dev/xconsole

# --- my additions:
#local3.* /var/log/smartd.log




-------------------------------------/etc/rsyslog.d/50-default.conf:
# Default rules for rsyslog.
#
# For more information see rsyslog.conf(5) and /etc/rsyslog.conf

#
# First some standard log files. Log by facility.
#
auth,authpriv.* /var/log/auth.log
*.*;auth,authpriv.none /var/log/syslog
cron.* /var/log/cron.log
daemon.* /var/log/daemon.log
kern.* /var/log/kern.log
lpr.* /var/log/lpr.log
mail.* /var/log/mail.log
user.* /var/log/user.log

#
# Logging for the mail system. Split it up so that
# it is easy to write scripts to parse these files.
#
mail.info /var/log/mail.info
mail.warn /var/log/mail.warn
mail.err /var/log/mail.err

#
# Logging for INN news system.
#
news.crit /var/log/news/news.crit
news.err /var/log/news/news.err
news.notice /var/log/news/news.notice

#
# Some "catch-all" log files.
#
*.=debug;\
auth,authpriv.none;\
news.none;mail.none /var/log/debug
*.=info;*.=notice;*.=warn;\
auth,authpriv.none;\
cron,daemon.none;\
mail,news.none /var/log/messages

#
# Emergencies are sent to everybody logged in.
#
*.emerg *

#
# I like to have messages displayed on the console, but only on a virtual
# console I usually leave idle.
#
daemon,mail.*;\
news.=crit;news.=err;news.=notice;\
*.=debug;*.=info;\
*.=notice;*.=warn /dev/tty5

# The named pipe /dev/xconsole is for the `xconsole' utility. To use it,
# you must invoke `xconsole' with the `-file' option:
#
# $ xconsole -file /dev/xconsole [...]
#
# NOTE: adjust the list below, or you'll go crazy if you have a reasonably
# busy site..
#
daemon.*;mail.*;\
news.err;\
*.=debug;*.=info;\
*.=notice;*.=warn |/dev/xconsole

well is rsyslog even running? Are the file handles open for writing by the rsyslog process? Can you use the logger tool to get messages into them? are there any log errors when rsyslog is restarted? can you run it in foreground debug mode?

thank you for helping me:

-- well is rsyslog even running?
YES
sudo ps aux |grep rsyslog
syslog 1981 0.0 0.0 126668 2376 ? Sl 00:08 0:01 rsyslogd -c5

-- Are the file handles open for writing by the rsyslog process?
YES
ls -l mail.err syslog messages
-rw-r----- 1 syslog adm 0 Feb 19 07:39 mail.err
-rw-rw-rw- 1 u1 u1 0 Oct 16 09:39 messages
-rw-r----- 1 syslog adm 0 Feb 24 00:14 syslog

-- Can you use the logger tool to get messages into them?
NO
gnome-system-log
(gnome-system-log:14070): GLib-GObject-WARNING **: /build/buildd/glib2.0-2.30.0/./gobject/gsignal.c:2412: instance `0x7f41fc0148c0' has no handler with id `1709

-- are there any log errors when rsyslog is restarted?
NO
sudo service rsyslog restart
rsyslog start/running, process 15313

-- foregroung debug mode:
/usr/sbin/rsyslogd -c5 -dn >~/rsyslog-1.log
rsyslogd: Could no open output pipe '/dev/xconsole' [try http://www.rsyslog.com/e/2039 ]



--------------------------------------
just part of logging -- looked for error , fail key words:

8411.321228551:7fd36961a720: Error opening log pipe: /dev/xconsole
8411.321238286:7fd36961a720: Called LogError, msg: Could no open output pipe '/dev/xconsole'

Actions:
8411.324473300:7fd36961a720: builtin-file: /var/log/syslog
8411.324483498:7fd36961a720: template='/var/log/syslog'
8411.324490721:7fd36961a720: use async writer=0
8411.324497898:7fd36961a720: flush on TX end=1
8411.324505026:7fd36961a720: flush interval=1
8411.324512328:7fd36961a720: file cache size=10
8411.324519464:7fd36961a720: create directories: yes
8411.324526749:7fd36961a720: file owner 101, group 4
8411.324534186:7fd36961a720: force chown() for all files: no
8411.324541394:7fd36961a720: directory owner 0, group 0
8411.324548681:7fd36961a720: dir create mode 0755, file create mode 0640
8411.324555747:7fd36961a720: fail if owner/group can not be set: no
8411.324562837:7fd36961a720:
Instance data: 0x1426fe0
8411.324570039:7fd36961a720: RepeatedMsgReduction: 1
8411.324577067:7fd36961a720: Resume Interval: 30
8411.324584189:7fd36961a720: State: rdy
8411.324591387:7fd36961a720: Exec only when previous is suspended: 0
8411.324603197:7fd36961a720: submission mode: slow, but feature-rich
8411.324610750:7fd36961a720:
8411.324618087:7fd36961a720:
8411.324625192:7fd36961a720:
8411.324632818:7fd36961a720: rule 0x1427be0: rsyslog rule:
8411.324642370:7fd36961a720: X X X X X X X X X FF X X X X X X X X X X X X X X X

Actions:
8411.326555645:7fd36961a720: builtin-file: /var/log/mail.err
8411.326565828:7fd36961a720: template='/var/log/mail.err'
8411.326573128:7fd36961a720: use async writer=0
8411.326580413:7fd36961a720: flush on TX end=1
8411.326587688:7fd36961a720: flush interval=1
8411.326594891:7fd36961a720: file cache size=10
8411.326602143:7fd36961a720: create directories: yes
8411.326609531:7fd36961a720: file owner 101, group 4
8411.326616629:7fd36961a720: force chown() for all files: no
8411.326623889:7fd36961a720: directory owner 0, group 0
8411.326631321:7fd36961a720: dir create mode 0755, file create mode 0640
8411.326638546:7fd36961a720: fail if owner/group can not be set: no
8411.326645639:7fd36961a720:
Instance data: 0x142e100
8411.326652996:7fd36961a720: RepeatedMsgReduction: 1
8411.326660486:7fd36961a720: Resume Interval: 30
8411.326667729:7fd36961a720: State: rdy
8411.326675054:7fd36961a720: Exec only when previous is suspended: 0
8411.326682390:7fd36961a720: submission mode: slow, but feature-rich
8411.326689790:7fd36961a720:
8411.326696944:7fd36961a720:
8411.326703917:7fd36961a720:
8411.326711745:7fd36961a720: rule 0x142ed00: rsyslog rule:
8411.326721435:7fd36961a720: X X X X X X X 7 X X X X X X X X X X X X X X X X X



8411.333874064:7fd36961a720: Command 'failonchownfailure':
8411.333881339:7fd36961a720: type : 4
8411.333888582:7fd36961a720: pData: 0x65c90c
8411.333895779:7fd36961a720: Hdlr : 0x0
8411.333903054:7fd36961a720: Owner: 0x410e10


8411.317893748:7fd36961a720: action 1 queue[DA]: error -2040 reading .qi file - can not read persisted info (if any)

8555.843018812:7fd36961a720: error 14 unlinking '(null)' - ignored: Bad address


-----------------------and now after CTRL-C finally ... no data to flush ... :
8555.842866458:7fd36961a720: strm 0x1420fd0: file -1(rsyslog) closing
8555.842899523:7fd36961a720: strm 0x1420fd0: file -1(rsyslog) flush, buflen 0 (no need to flush)
8555.842931438:7fd36961a720: strm 0x14221f0: file -1(rsyslog) closing
8555.842964014:7fd36961a720: strm 0x1423410: file -1(rsyslog) closing
8555.843018812:7fd36961a720: error 14 unlinking '(null)' - ignored: Bad address
8555.843053278:7fd36961a720: action 1 queue: queue (type 1) will lose 0 messages, destroying...
8555.843099648:7fd36961a720: strm 0x14258b0: file -1(ufw.log) closing
8555.843132631:7fd36961a720: strm 0x14258b0: file -1(ufw.log) flush, buflen 0 (no need to flush)
8555.843167957:7fd36961a720: strm 0x14265c0: file -1(auth.log) closing
8555.843199374:7fd36961a720: strm 0x14265c0: file -1(auth.log) flush, buflen 0 (no need to flush)
8555.843234414:7fd36961a720: strm 0x1427160: file -1(syslog) closing
8555.843266205:7fd36961a720: strm 0x1427160: file -1(syslog) flush, buflen 0 (no need to flush)
8555.843299933:7fd36961a720: strm 0x1427df0: file -1(cron.log) closing
8555.843332171:7fd36961a720: strm 0x1427df0: file -1(cron.log) flush, buflen 0 (no need to flush)
8555.843366571:7fd36961a720: strm 0x1428a80: file -1(daemon.log) closing
8555.843397406:7fd36961a720: strm 0x1428a80: file -1(daemon.log) flush, buflen 0 (no need to flush)
8555.843432727:7fd36961a720: strm 0x1429710: file -1(kern.log) closing
8555.843473037:7fd36961a720: strm 0x1429710: file -1(kern.log) flush, buflen 0 (no need to flush)
8555.843505328:7fd36961a720: strm 0x142a3a0: file -1(lpr.log) closing
8555.843535311:7fd36961a720: strm 0x142a3a0: file -1(lpr.log) flush, buflen 0 (no need to flush)
8555.843571901:7fd36961a720: strm 0x142b030: file -1(mail.log) closing
8555.843602432:7fd36961a720: strm 0x142b030: file -1(mail.log) flush, buflen 0 (no need to flush)
8555.843637432:7fd36961a720: strm 0x142bcc0: file -1(user.log) closing
8555.843668405:7fd36961a720: strm 0x142bcc0: file -1(user.log) flush, buflen 0 (no need to flush)
8555.843700503:7fd36961a720: strm 0x142c990: file -1(mail.info) closing
8555.843729466:7fd36961a720: strm 0x142c990: file -1(mail.info) flush, buflen 0 (no need to flush)
8555.843763778:7fd36961a720: strm 0x142d5f0: file -1(mail.warn) closing
8555.843793921:7fd36961a720: strm 0x142d5f0: file -1(mail.warn) flush, buflen 0 (no need to flush)
8555.843827632:7fd36961a720: strm 0x142e280: file -1(mail.err) closing
8555.843856200:7fd36961a720: strm 0x142e280: file -1(mail.err) flush, buflen 0 (no need to flush)
8555.843889560:7fd36961a720: strm 0x142ef10: file -1(news.crit) closing
8555.843919646:7fd36961a720: strm 0x142ef10: file -1(news.crit) flush, buflen 0 (no need to flush)
8555.843965833:7fd36961a720: strm 0x142fba0: file -1(news.err) closing
8555.843997749:7fd36961a720: strm 0x142fba0: file -1(news.err) flush, buflen 0 (no need to flush)
8555.844031752:7fd36961a720: strm 0x1430830: file -1(news.notice) closing
8555.844064762:7fd36961a720: strm 0x1430830: file -1(news.notice) flush, buflen 0 (no need to flush)
8555.844100168:7fd36961a720: strm 0x14314c0: file -1(debug) closing
8555.844132043:7fd36961a720: strm 0x14314c0: file -1(debug) flush, buflen 0 (no need to flush)
8555.844165971:7fd36961a720: strm 0x1432150: file -1(messages) closing
8555.844197004:7fd36961a720: strm 0x1432150: file -1(messages) flush, buflen 0 (no need to flush)
8555.844232366:7fd36961a720: strm 0x1433460: file -1(tty5) closing
8555.844263337:7fd36961a720: strm 0x1433460: file -1(tty5) flush, buflen 0 (no need to flush)
8555.844307693:7fd36961a720: all primary multi-thread sources have been terminated - now doing aux cleanup...
8555.844399996:7fd36961a720: file syslogd.c released module 'lmnet', reference count now 2
8555.844431016:7fd36961a720: destructing parser 'rsyslog.rfc5424'
8555.844455277:7fd36961a720: destructing parser 'rsyslog.rfc3164'
8555.844484875:7fd36961a720: file conf.c released module 'lmnet', reference count now 1
8555.844537978:7fd36961a720: rsyslog runtime de-initialized, current users 0
8555.844564678:7fd36961a720: module lmnet NOT unloaded because it still has a refcount of 1
8555.844585704:7fd36961a720: Unloading module builtin-file
8555.844610489:7fd36961a720: module lmnet NOT unloaded because it still has a refcount of 1
8555.844632846:7fd36961a720: Unloading module builtin-pipe
8555.844658264:7fd36961a720: module lmnet NOT unloaded because it still has a refcount of 1
8555.844680629:7fd36961a720: Unloading module builtin-fwd
8555.844707062:7fd36961a720: file omfwd.c released module 'lmnet', reference count now 0
8555.844729570:7fd36961a720: module 'lmnet' has zero reference count, unloading...
8555.844751716:7fd36961a720: Unloading module lmnet
8555.844815656:7fd36961a720: Unloading module builtin-shell
8555.844843769:7fd36961a720: Unloading module builtin-discard
8555.844865407:7fd36961a720: Unloading module builtin-usrmsg
8555.844887277:7fd36961a720: Unloading module builtin-pmrfc5424
8555.844910222:7fd36961a720: Unloading module builtin-pmrfc3164
8555.844933900:7fd36961a720: Unloading module builtin-smfile
8555.844958598:7fd36961a720: Unloading module builtin-smtradfile
8555.844981131:7fd36961a720: Unloading module builtin-smfwd
8555.845004031:7fd36961a720: Unloading module builtin-smtradfwd
8555.845028406:7fd36961a720: Unloading module imuxsock
8555.845090707:7fd36961a720: Unloading module imklog
8555.845143373:7fd36961a720: Clean shutdown completed, bye




-------------------
Could you please help where should I look and what - more deeply ?
Is /dev/xconsole problem ? I do not know if it corresponding but xconsole is located in /usr/bin/xconsole. Not even soft link did not help.
If it helps I can give log file - I do not mind if it contains sensitive data. Should I run debug mode longer ?

Thank you for any clue.
Kind Regards,
Martin

I would certainly look at commenting out the xconsole line and restarting. /dev/xconsole is read by /usr/bin/xconsole i believe, so they are not the same thing, so not interchangeable at all. It doesn't look relevant but are there and qi spool files in (i think) /var/spool/rsyslog ?

-- I just realized that I saw this message: "file '/dev/xconsole' did not exist" for a long time before the logging stopped working( at the begining of some log file - but I do not remmember which one it was).

-- I did not succedded to commeneted out xconsole because I did not find it in /etc/init.d/rsyslog script.
(Previously I wrongly looked into old version - syslog - file /etc/init.d/inetutils-syslogd)


when I read man mkfifo - there is some info about SELinux. Because I installed it couple of days ago. Maybe it could have influence on it - some permissions ?

.qi spool files - I did not find any:
sudo locate -b '\.qi'
empty

you just install selinux?? If this happened at the same time, I would put a hell of a lot of money on that being the reason! turn it to permissive and try again.

I am sorry getenforce gave me it is disabled. So It is not enabled yet.

Hi,

I would like to thank you for helping me to solve it. I solved it by removing mysql.conf file from directory /etc/rsyslog.d - this file is wrongly configured so I have to figure out how to properly set it up but temporarily without him it works.

Regards,
M.