[SOLVED] Discard message on Rsyslog

trungmv · 05-04-2012, 04:40 AM

Hi all,

I searching the help from many forums and authors long time but discard message unwanted yet.
Now i am very stuck and confused.
This is my rsyslog.conf:

$ModLoad impstats.so
$PStatsInterval 300
syslog.info /var/log/rsyslog-stats

#--------------------------------------------------This line is comment
$ModLoad immark.so # provides --MARK-- message capability
$ModLoad imuxsock.so # provides support for local system logging (via logger command)
$ModLoad imklog.so # provides kernel logging support (previously done by rklogd)

#--------------------------------------------------This line is comment
$ModLoad imudp.so # provides UDP syslog reception
$UDPServerAddress * # all local interfaces
$UDPServerRun 514 # start UDP server (log server receiver)

#--------------------------------------------------This line is comment
$ModLoad imtcp.so # provides TCP syslog reception and GSS-API (if compiled)
$InputTCPServerRun 514 # start TCP server (log server receiver)

#--------------------------------------------------This line is comment
$ModLoad imrelp.so # RELP input
$InputRELPServerRun 20514 # start RELP Protocol

#--------------------------------------------------This line is comment
$ModLoad imfile.so # Text file input
$InputFileName /var/log/i-am-a-text-file.log
$InputFileTag my-text-file:
$InputFileStateFile stat-file1
$InputFileSeverity error
$InputFileFacility local7
$InputFilePollInterval 10 # check for new lines every 10 seconds
$InputRunFileMonitor

#--------------------------------------------------This line is comment
$ModLoad ommysql.so # Log to MySQL

#--------------------------------------------------This line is comment
$ModLoad omrelp.so # Send to another host via RELP

# Globals -----------------------------------------This line is comment
$umask 0000
$DirCreateMode 0640
$FileCreateMode 0640
$RepeatedMsgReduction on

$WorkDirectory /var/log/rsyslog # default location for work (spool) files
$ActionQueueType LinkedList # use asynchronous processing
$ActionQueueFileName queue # set file name, also enables disk mode
$ActionResumeRetryCount -1 # infinite retries on insert failure
$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down
$MainMsgQueueMaxFileSize 100M
$ActionQueueMaxFileSize 5M

# A template that resambles traditional syslogd file output:
$template TraditionalFormat,"%timegenerated% %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n"

# a template useful for debugging format issues
$template DEBUG,"Debug line with all properties:\nFROMHOST: '%FROMHOST%', HOSTNAME: '%HOSTNAME%', PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%', APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%',\nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%',\nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\nrawmsg: '%rawmsg%'\n\n"

# A template that resembles RFC 3164 on-the-wire format:
# (yes, there is NO space betwen syslogtag and msg! that's important!)
$template RFC3164fmt,"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag%%msg%"

# a template resembling traditional wallmessage format:
$template wallmsg,"\r\n\7Message from syslogd@%HOSTNAME% at %timegenerated% ...\r\n %syslogtag%%msg%\n\r"

$template WinSyslogFmt,"%HOSTNAME%,%timegenerated:1:10:date-rfc3339%,%timegenerated:12:19:date-rfc3339%,%timegenerated:1:10:date-rfc3339%,%timegenerated:12:19:date-rfc3339%,%syslogfacility%,%syslogpriority%,%syslogtag%%msg%\n"
# A template used for database writing (notice it *is* an actual
# sql-statement):

$template dbFormat,"insert into SystemEvents (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag, EventLogType, EventID) values ('%msg%', %syslogfacility%, '%HOSTNAME%',%syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",sql

$template FileFormat,"%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"

$template ForwardFormat,"<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"

####Discard Message
:msg, contains, "861: NT AUTHORITY\SYSTEM:" ~
:msg, !contains, "861: NT AUTHORITY\SYSTEM:" ~
:msg, startswith, "861: NT AUTHORITY\SYSTEM:" ~

# Store all log files in MySQL DB :
*.* mmysql:127.0.0.1,Syslog,rsyslog,mypassword
#--------------------------------------------------This line is comment

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.* /dev/console;TraditionalFileFormat

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none /var/log/messages;TraditionalFormat

# The authpriv file has restricted access.
authpriv.* /var/log/secure

# Log all the mail messages in one place.
mail.* -/var/log/maillog

# Log cron stuff
cron.* /var/log/cron

# Everybody gets emergency messages
*.emerg *

# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler

# Save boot messages also to boot.log
local7.* /var/log/boot.log

#--------------------------------------------------This line is comment
$IncludeConfig /etc/rsyslog.d/*.conf

I already added more line at below ####Discard Message but did not successful.
Anyone using rsyslog can help me to this issue?
Any help is appreciated.

Best regards,

kbp · 05-04-2012, 09:46 PM

It looks like rsyslog can't handle '\', this regex will probably do what you want though:

Code:

:msg, regex, ".*861.*NT.*AUTHORITY.*SYSTEM.*"   ~

trungmv · 05-07-2012, 01:47 AM

Hi kbp,

Thank you feedback.
Seems your help is useless in this case. I still did not successful.

Code:

####Discard Message
#:msg, contains, "861: NT AUTHORITY\SYSTEM:" ~
#:msg, regex, "861" ~

:msg, regex, ".*861.*NT.*AUTHORITY.*SYSTEM.*"   ~

#*.* /var/log/rsyslog_debug.log;RSYSLOG_DebugFormat
# Store all log files in MySQL DB  :

*.*        :ommysql:127.0.0.1,Syslog,rsyslog,mypasswordhere

And after each time restart rsyslog, i saw:
http://nl6.upanh.com/b4.s26.d2/bd13d...36.rsyslog.jpg
Do you have any other assistance?
Any help is appreciate,

Best regards,

kbp · 05-07-2012, 02:53 AM

Try moving the line right to the top of rsyslog.conf, it worked on my system running 5.8.7

trungmv · 05-07-2012, 03:43 AM

Yeah, excellence!
I can do it
Thank you very much kbp
P/S: As you known my gathering data will be stored in Message folder and Mysql.
So, data will grow bigger and occupy much of the hard drive capacity.
Do you have any idea how to decrease this data on each day/week/month?
Ho do i do?

Best regards,

kbp · 05-07-2012, 08:04 AM

Database archiving I would guess but I'm not a dba ... maybe someone else could provide archiving/truncation methods.

trungmv · 05-07-2012, 08:57 AM

So, Could you tell me how to configure rsyslog can send email alert or notification in past 24 hours or something like that?

kbp · 05-07-2012, 06:06 PM

That sounds more like logwatch ..

trungmv · 05-07-2012, 10:09 PM

I never used logwatch
So,Could you tell me how to integrate logwatch and rsyslog to send email if have any alert?

kbp · 05-08-2012, 05:01 AM

What exactly are you trying to achieve? Logwatch will send you an email every day with info regarding the systems health and services from the previous day, are you expecting instant alerts based on specific log messages?

trungmv · 05-08-2012, 05:29 AM

My means is rsyslog will gathering data from other clients.
I expected instant alerts based on specific log messages from rsyslog system.
Such as Severity is warning, error .... it can send email notification or at least send email report in past 24h(based on ryslog)