Share your knowledge at the LQ Wiki.
Go Back > Forums > Linux Forums > Linux - Newbie
User Name
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!


  Search this Thread
Old 01-21-2009, 08:12 AM   #1
Registered: Aug 2006
Location: England Somewhere
Distribution: Mandriva, PCLinuxOS, Karoshi, Suse, Redhat, Ubuntu
Posts: 518

Rep: Reputation: 39
Question Syslog-ng, Central log server, trying to split logs out using filters

Hi All,
I've been working on this for a short while, but still not fully been able to get round the problem i'm having. Below is snipit of syslog-ng.conf file, i'm sure it contains the parts relevant to this question.

#@SuSEconfig@ This is a template file used by SuSEconfig
#@SuSEconfig@ to generate the final syslog-ng.conf.
#@SuSEconfig@ SuSEconfig adds additional log sockets from
#@SuSEconfig@ /etc/sysconfig/syslog to the source bellow.
# File format description can be found in syslog-ng.conf(5)
# and /usr/share/doc/packages/syslog-ng/syslog-ng.txt.

options { long_hostnames(off); sync(0); perm(0640); stats(3600); };
source src {
	# include internal syslog-ng messages
	# note: the internal() soure is required!

	# the following line will be replaced by the
	# socket list generated by SuSEconfig using
	# variables from /etc/sysconfig/syslog:

	# uncomment to process log messages from network:
	udp(ip("") port(514));
tcp(ip("IPADDRESS") port(5140) keep-alive(yes));

# Other options.
destination d_mysql {
template("INSERT INTO logs (host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes));
log { source(src); destination(d_mysql);

# this is for separating out network hosts into individual log files.
destination std {
file ("/central_logs/HOSTS/$YEAR-$MONTH/$HOST/$FACILITY-$YEAR-$MONTH-$DAY" owner(root) group(root) perm(0600) dir_perm(0700) create_dirs(yes)
log {

######Addition 21/01/09
destination hostname_rsync   { file("/central_logs/HOSTS/hostname_rsync" owner(root) perm(600)); };
filter f_hostname_rsync   { level(debug) and facility(daemon) and program("rsync-"); };
filter f_hostname_rsync_stop   { not level(debug) and not facility(daemon) and not program("rsync-"); };
log { source(src); filter(f_hostname_rsync); destination(hostname_rsync); flags(final); };

#destination d_rsync_to_mysql {
#template("INSERT INTO hostname_rsyncs (host, facility, priority, level, tag, datetime, program, msg) VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n") template-escape(yes)
#log { source(src); filter(f_hostname_rsync); destination(d_rsync_to_mysql);  flags(final);
OK, thing is i'm using rsync on a server and i want the output onto my central logging server and put into MySQL as well. Thats no problem, but i added the complexity that i wanted the logs specific to only the rsync to be put into another mysql table rather than where they are going at the moment as you can see, into the "logs" with all other log data. Also as a backup all log data goes into separated out files this again no problem. The issues i'm having is that i can't seem to stop log data i've caught using the filters also being duplicated into the other log files and tables. I would like anything from "source src" thats caught by my filter "f_hostname_rsync", to be put into file "/central_logs/HOSTS/hostname_rsync" and threaded also into "destination d_rsync_to_mysql". currently commented out because i'm trying to then also not allow any other log data coming from "src" to go into these files or mysql table. As i understood it, "flags(final);" is meant to stop any duplication of data but this doesn't seem to work as my rsync log data is still making it into the "logs" table.

Any help would be great.
Old 01-23-2009, 03:53 AM   #2
Registered: Jun 2001
Location: UK
Distribution: Gentoo, RHEL, Fedora, Centos
Posts: 43,417

Rep: Reputation: 1975Reputation: 1975Reputation: 1975Reputation: 1975Reputation: 1975Reputation: 1975Reputation: 1975Reputation: 1975Reputation: 1975Reputation: 1975Reputation: 1975
Well based on the functional descriptions of the final flag that would seem to be correct. Personally I don't use that flag, instead I tend to specifically filter for each log statement, so i'd write a f_not_hostname_rsync or whatever to get the compliment of the logs there.
Old 01-23-2009, 08:55 AM   #3
Registered: Aug 2006
Location: England Somewhere
Distribution: Mandriva, PCLinuxOS, Karoshi, Suse, Redhat, Ubuntu
Posts: 518

Original Poster
Rep: Reputation: 39
Hi There,
Yes thats what i've done, although as i understood this uses a bit more on resources so i was initially trying to avoid it. I've split the source(src) out as well so that remote log entries don't go through the other parts of standard syslog install. However remote logs are still somehow making their way into the /var/log/mesages file. Not sure how that is actually happening at the moment? But as time permits i will take another look. Thanks anyway glad to know i chose a method someone else is using.



Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
syslog to central server and store logs in separate host directories steve.goldner Linux - Enterprise 11 04-07-2012 02:26 PM
Syslog-ng unable to log Cisco Logs Mohd Anis Linux - Server 3 09-03-2008 04:14 AM
Central Log Server using syslog-ng sakthi.s Linux - Security 3 12-26-2007 05:04 PM
central server log / databased . . syslog-ng maybe? sir-lancealot Linux - Server 2 08-28-2007 11:55 AM
Syslog-ng central logserver is dropping logs humbletech99 Linux - Networking 2 06-22-2006 03:09 AM > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 01:59 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration