LinuxQuestions.org
Page 1 of 2 1 2
Show 50 post(s) from this thread on one page

LinuxQuestions.org (/questions/)
-   Linux - Server (https://www.linuxquestions.org/questions/linux-server-73/)
-   -   Sending to syslog-ng server on multiple ports doesn't work (https://www.linuxquestions.org/questions/linux-server-73/sending-to-syslog-ng-server-on-multiple-ports-doesnt-work-4175416938/)

pingu 07-16-2012 05:53 AM
Sending to syslog-ng server on multiple ports doesn't work
 
We have a logserver running syslog-ng, all clients also use syslog-ng.
As of today, we use Splunk on the logserver to analyze logs.
All servers run OpenSuse 12.1
Now we need to save the logs to different files on the log-server, tried doing that by sending to different ports. I have read lots and lots of documentation, all says this should work - but it doesn't.
What happens is that logs are sent only to the first specified destination. I'm checking this with tcpdump on server, also configured Splunk to listen to ports 514 & 515 - same result.
Nothing is received on the second destination defined in senders syslog-ng.conf.
If I comment out the first destination, logs are sent fine to second destination.
I also tried using udp on one destination, tcp on the other but no difference.

Details:
Clients syslog-ng.conf, relevant parts:
Code:
source src {
        internal();
        unix-dgram("/dev/log");
};
source tripwire {
        file(/var/log/tripwire/tripwire);
};
destination loganalyzer { udp(172.16.4.114 port(514)); };
log { source(src); destination(loganalyzer); };

destination triplog { udp(172.16.4.114 port(515)); };
log { source(tripwire); destination(triplog); };
Server syslog-ng.conf, relevant parts:
Code:
source srcExt {
        udp(ip("0.0.0.0") port(514));
};
source srcExtTrip {
        udp(ip("0.0.0.0") port(515));
};

# Receive from other
destination std { file("/var/log/Hosts/$YEAR-$MONTH/$HOST-$YEAR.$MONTH.log"); };
log {
        source(srcExt);
        destination(std);
};
# Receive from other - Tripwire
destination stdTrip { file(/var/log/Hosts/tripwire.log); };
log {
        source(srcExtTrip);
        destination(stdTrip);
};
Complete config files
Code:
app10:~ # cat /etc/syslog-ng/syslog-ng.conf
#
# /etc/syslog-ng/syslog-ng.conf
#
# File format description can be found in syslog-ng.conf(5)
# and in /usr/share/doc/packages/syslog-ng/syslog-ng.txt.
#
# NOTE: The SuSEconfig script and its syslog-ng.conf.in
#      configuration template aren't used any more.
#
#      Feel free to edit this file directly.
#
#      Additional log sockets for chroot environments can
#      be declared in the /etc/sysconfig/syslog file using
#              SYSLOGD_ADDITIONAL_SOCKET<NAME>
#      variables. This way allows to define a socket from
#      RPM scripts and is used by several services, e.g.
#      bind and dhcpd.
#
#      The sockets defined in /etc/sysconfig/syslog file
#      are added by the /etc/ini.d/syslog init-script using
#      "-a path" command line options while syslog-ng is
#      started.
#
#      This syslog-ng contains an extension and appends the
#      sockets added by "-a" option to the same source group
#      and using the same socket type (unix-dgram) as the
#      "/dev/log" socket.
#      If one of the sockets added by "-a" option already
#      exists in any (other) source group in the config file,
#      then the socket added by "-a" option is ignored.
#

#
# Global options.
#
options { long_hostnames(off); sync(0); perm(0640); stats(3600); };

#
# 'src' is our main source definition. you can add
# more sources driver definitions to it, or define
# your own sources, i.e.:
#
#source my_src { .... };
#
source src {
        #
        # include internal syslog-ng messages
        # note: the internal() soure is required!
        #
        internal();

        #
        # the default log socket for local logging:
        #
        unix-dgram("/dev/log");

        #
        # uncomment to process log messages from network:
        #
        #udp(ip("0.0.0.0") port(514));
};

source tripwire {
        file(/var/log/tripwire/tripwire);
};

#
# Filter definitions
#
filter f_iptables  { facility(kern) and match("IN=") and match("OUT="); };

filter f_console    { level(warn) and facility(kern) and not filter(f_iptables)
                      or level(err) and not facility(authpriv); };

filter f_newsnotice { level(notice) and facility(news); };
filter f_newscrit  { level(crit)  and facility(news); };
filter f_newserr    { level(err)    and facility(news); };
filter f_news      { facility(news); };

filter f_mailinfo  { level(info)      and facility(mail); };
filter f_mailwarn  { level(warn)      and facility(mail); };
filter f_mailerr    { level(err, crit) and facility(mail); };
filter f_mail      { facility(mail); };

filter f_cron      { facility(cron); };

filter f_local      { facility(local0, local1, local2, local3,
                              local4, local5, local6, local7); };

#
# acpid messages
#
filter f_acpid_full { match('^acpid:'); };
filter f_acpid      { level(emerg..notice) and match('^acpid:'); };

# this is for the old acpid < 1.0.6
filter f_acpid_old  { match('^\[acpid\]:'); };

filter f_netmgm    { match('^NetworkManager:'); };

filter f_messages  { not facility(news, mail) and not filter(f_iptables); };
filter f_warn      { level(warn, err, crit) and not filter(f_iptables); };
filter f_alert      { level(alert); };

#
# Enable this and adopt IP to send log messages to a log server.
#
#destination logserver { udp("10.10.10.10" port(514)); };
#log { source(src); destination(logserver); };

#
# Enable this, if you want to keep all messages in one file:
# (don't forget to provide logrotation config)
#
#destination allmessages { file("/var/log/allmessages"); };
#log { source(src); destination(allmessages); };

#
# Most warning and errors on tty10 and on the xconsole pipe:
#
destination console  { pipe("/dev/tty10"    owner(-1) group(-1) perm(-1)); };
log { source(src); filter(f_console); destination(console); };

destination xconsole { pipe("/dev/xconsole" owner(-1) group(-1) perm(-1)); };
log { source(src); filter(f_console); destination(xconsole); };

# Enable this, if you want that root is informed immediately,
# e.g. of logins:
#
#destination root { usertty("root"); };
#log { source(src); filter(f_alert); destination(root); };


#
# News-messages in separate files:
#
destination newscrit  { file("/var/log/news/news.crit"
                              owner(news) group(news)); };
log { source(src); filter(f_newscrit); destination(newscrit); };

destination newserr    { file("/var/log/news/news.err"
                              owner(news) group(news)); };
log { source(src); filter(f_newserr); destination(newserr); };

destination newsnotice { file("/var/log/news/news.notice"
                              owner(news) group(news)); };
log { source(src); filter(f_newsnotice); destination(newsnotice); };

#
# and optionally also all in one file:
# (don't forget to provide logrotation config)
#
#destination news { file("/var/log/news.all"); };
#log { source(src); filter(f_news); destination(news); };


#
# Mail-messages in separate files:
#
destination mailinfo { file("/var/log/mail.info"); };
log { source(src); filter(f_mailinfo); destination(mailinfo); };

destination mailwarn { file("/var/log/mail.warn"); };
log { source(src); filter(f_mailwarn); destination(mailwarn); };

destination mailerr  { file("/var/log/mail.err" fsync(yes)); };
log { source(src); filter(f_mailerr);  destination(mailerr); };

#
# and also all in one file:
#
destination mail { file("/var/log/mail"); };
log { source(src); filter(f_mail); destination(mail); };


#
# acpid messages in one file:
#
destination acpid { file("/var/log/acpid"); };
destination null { };
log { source(src); filter(f_acpid); destination(acpid); flags(final); };
#
# if you want more verbose acpid logging, comment the destination(null)
# line and uncomment the destination(acpid) line
#
log { source(src); filter(f_acpid_full); destination(null); flags(final); };
# log { source(src); filter(f_acpid_full); destination(acpid); flags(final); };
#
# old acpid < 1.0.6
log { source(src); filter(f_acpid_old); destination(acpid); flags(final); };

#
# NetworkManager messages in one file:
#
destination netmgm { file("/var/log/NetworkManager"); };
log { source(src); filter(f_netmgm); destination(netmgm); flags(final); };


#
# Cron-messages in one file:
# (don't forget to provide logrotation config)
#
#destination cron { file("/var/log/cron"); };
#log { source(src); filter(f_cron); destination(cron); };


#
# Some boot scripts use/require local[1-7]:
#
destination localmessages { file("/var/log/localmessages"); };
log { source(src); filter(f_local); destination(localmessages); };


#
# All messages except iptables and the facilities news and mail:
#
destination messages { file("/var/log/messages"); };
log { source(src); filter(f_messages); destination(messages); };


#
# Firewall (iptables) messages in one file:
#
destination firewall { file("/var/log/firewall"); };
log { source(src); filter(f_iptables); destination(firewall); };


#
# Warnings (except iptables) in one file:
#
destination warn { file("/var/log/warn" fsync(yes)); };
log { source(src); filter(f_warn); destination(warn); };

# Added by PeterH 22/2 2012 for remote logging
destination logserver { udp(192.168.1.40 port(514)); };
log { source(src); destination(logserver); };
destination loganalyzer { udp(172.16.4.114 port(514)); };
log { source(src); destination(loganalyzer); };

# Testing tripwire
destination triplog { udp(172.16.4.114 port(515)); };
log { source(tripwire); destination(triplog); };

support0:~ # cat /etc/syslog-ng/syslog-ng.conf
@version:3.3
@include "scl.conf"

#
# /etc/syslog-ng/syslog-ng.conf
#
# File format description can be found in syslog-ng.conf(5)
# and in /usr/share/doc/packages/syslog-ng/syslog-ng.txt.
#
# NOTE: The SuSEconfig script and its syslog-ng.conf.in
#      configuration template aren't used any more.
#
#      Feel free to edit this file directly.
#

#
# Global options.
#
options { chain_hostnames(off); flush_lines(0); perm(0640); stats_freq(3600); threaded(yes);
# Create log directories as needed
          create_dirs (yes);

};
#
# 'src' is our main source definition. you can add
# more sources driver definitions to it, or define
# your own sources, i.e.:
#
#source my_src { .... };
#
source src {
        #
        # include internal syslog-ng messages
        # note: the internal() soure is required!
        #
        internal();

        #
        # read kernel messages directly (12.x) or
        # does klogd forward them via /dev/log?
        #
        file ("/proc/kmsg" program_override("kernel"));

        #
        # the default log socket for local logging:
        #
        unix-dgram("/dev/log");
        #
        # uncomment to process log messages from network:
        #
        #udp(ip("0.0.0.0") port(514));
};

source srcExt {
        #
        # Accept all external
        #
        udp(ip("0.0.0.0") port(514));
};

source srcExtTrip {
        #
        # Accept tripwire on 515
        #
        udp(ip("0.0.0.0") port(515));
};


include "/var/run/syslog-ng/additional-log-sockets.conf";

#
# Filter definitions
#
filter f_iptables  { facility(kern) and message("IN=") and message("OUT="); };

filter f_console    { level(warn) and facility(kern) and not filter(f_iptables)
                      or level(err) and not facility(authpriv); };

filter f_newsnotice { level(notice) and facility(news); };
filter f_newscrit  { level(crit)  and facility(news); };
filter f_newserr    { level(err)    and facility(news); };
filter f_news      { facility(news); };

filter f_mailinfo  { level(info)      and facility(mail); };
filter f_mailwarn  { level(warn)      and facility(mail); };
filter f_mailerr    { level(err, crit) and facility(mail); };
filter f_mail      { facility(mail); };

filter f_cron      { facility(cron); };

filter f_local      { facility(local0, local1, local2, local3,
                              local4, local5, local6, local7); };

#
# acpid messages
#
filter f_acpid_full { message('^acpid:'); };
filter f_acpid      { level(emerg..notice) and message('^acpid:'); };

# this is for the old acpid < 1.0.6
filter f_acpid_old  { message('^\[acpid\]:'); };

filter f_netmgm    { message('^NetworkManager:'); };

filter f_messages  { not facility(news, mail) and not filter(f_iptables); };
filter f_warn      { level(warn, err, crit) and not filter(f_iptables); };
filter f_alert      { level(alert); };


#
# Enable this and adopt IP to send log messages to a log server.
#
#destination logserver { udp("10.10.10.10" port(514)); };
#log { source(src); destination(logserver); };

#
# Enable this, if you want to keep all messages in one file:
# (don't forget to provide logrotation config)
#
#destination allmessages { file("/var/log/allmessages"); };
#log { source(src); destination(allmessages); };

#
# Most warning and errors on tty10 and on the xconsole pipe:
#
destination console  { file("/dev/tty10"    suppress(30) owner(-1) group(-1) perm(-1)); };
log { source(src); source(chroots); filter(f_console); destination(console); };

destination xconsole { pipe("/dev/xconsole" suppress(30) owner(-1) group(-1) perm(-1)); };
log { source(src); source(chroots); filter(f_console); destination(xconsole); };

# Enable this, if you want that root is informed immediately,
# e.g. of logins:
#
#destination root { usertty("root"); };
#log { source(src); source(chroots); filter(f_alert); destination(root); };


#
# News-messages in separate files:
#
destination newscrit  { file("/var/log/news/news.crit"
                              suppress(30) owner(news) group(news)); };
log { source(src); source(chroots); filter(f_newscrit); destination(newscrit); };

destination newserr    { file("/var/log/news/news.err"
                              suppress(30) owner(news) group(news)); };
log { source(src); source(chroots); filter(f_newserr); destination(newserr); };

destination newsnotice { file("/var/log/news/news.notice"
                              suppress(30) owner(news) group(news)); };
log { source(src); source(chroots); filter(f_newsnotice); destination(newsnotice); };

#
# and optionally also all in one file:
# (don't forget to provide logrotation config)
#
#destination news { file("/var/log/news.all"); };
#log { source(src); source(chroots); filter(f_news); destination(news); };


#
# Mail-messages in separate files:
#
destination mailinfo { file("/var/log/mail.info" suppress(30)); };
log { source(src); source(chroots); filter(f_mailinfo); destination(mailinfo); };

destination mailwarn { file("/var/log/mail.warn" suppress(30)); };
log { source(src); source(chroots); filter(f_mailwarn); destination(mailwarn); };

destination mailerr  { file("/var/log/mail.err"  suppress(30) fsync(yes)); };
log { source(src); source(chroots); filter(f_mailerr);  destination(mailerr); };

#
# and also all in one file:
#
destination mail { file("/var/log/mail" suppress(30)); };
log { source(src); source(chroots); filter(f_mail); destination(mail); };


#
# acpid messages in one file:
#
destination acpid { file("/var/log/acpid" suppress(30)); };
destination devnull { };
log { source(src); source(chroots); filter(f_acpid); destination(acpid); flags(final); };
#
# if you want more verbose acpid logging, comment the destination(null)
# line and uncomment the destination(acpid) line
#
log { source(src); source(chroots); filter(f_acpid_full); destination(devnull); flags(final); };
# log { source(src); source(chroots); filter(f_acpid_full); destination(acpid); flags(final); };
#
# old acpid < 1.0.6
log { source(src); source(chroots); filter(f_acpid_old); destination(acpid); flags(final); };

#
# NetworkManager messages in one file:
#
destination netmgm { file("/var/log/NetworkManager" suppress(30)); };
log { source(src); source(chroots); filter(f_netmgm); destination(netmgm); flags(final); };

#
# Cron-messages in one file:
# (don't forget to provide logrotation config)
#
#destination cron { file("/var/log/cron" suppress(30)); };
#log { source(src); source(chroots); filter(f_cron); destination(cron); };

#
# Some boot scripts use/require local[1-7]:
#
destination localmessages { file("/var/log/localmessages" suppress(30)); };
log { source(src); source(chroots); filter(f_local); destination(localmessages); };

#
# All messages except iptables and the facilities news and mail:
#
destination messages { file("/var/log/messages"); };
log { source(src); filter(f_messages); destination(messages); };

#destination messages { file("/var/log/messages" suppress(30) owner(-1) group(-1) perm(-1)); };
#log { source(src); source(chroots); filter(f_messages); destination(messages); };

#
# Firewall (iptables) messages in one file:
#
destination firewall { file("/var/log/firewall" suppress(30)); };
log { source(src); source(chroots); filter(f_iptables); destination(firewall); };

#
# Warnings (except iptables) in one file:
#
destination warn { file("/var/log/warn" suppress(30) fsync(yes)); };
log { source(src); source(chroots); filter(f_warn); destination(warn); };

destination logserver { udp("192.168.1.40" port(514)); };
log { source(src); destination(logserver); };

# Receive from other
destination std { file("/var/log/Hosts/$YEAR-$MONTH/$HOST-$YEAR.$MONTH.log"); };
log {
        source(srcExt);
        destination(std);
};

# Receive from other - Tripwire
destination stdTrip { file(/var/log/Hosts/tripwire.log); };
log {
        source(srcExtTrip);
        destination(stdTrip);
};
Why I want to do this is that I need to send Tripwire-logs to logserver from clients that must be completely isolated from the world.
I can't send tripwire logs to standard logfile, as there is no clear identification then what lines belongs to tripwire.
Any ideas what is wrong?
Or tips of other ways to achieve the same result?

acid_kewpie 07-16-2012 05:58 AM
So is data being sent by the client or not? run a tcpdump and filter on port 515

pingu 07-16-2012 06:08 AM
Nope, no data is sent on port 515 - until I comment out first destination, then data is sent on port 515.
So it must be either that syslog-ng is wrongly configured or it is not doing what it should (aka "bug").

Reuti 07-16-2012 07:01 AM
You can use:
Code:
program_override("tripwire") default-facility(daemon) default-priority(info)
in the file rule and use a filter on the target side to separate the different sources.

pingu 07-16-2012 09:27 AM
This is crazy...
I did what you suggested, changed the source to
Code:
source tripwire {
        file("/var/log/tripwire/tripwire" program_override("tripwire") flags(no-parse));
};
But now syslog-ng returns error and can't start!
Code:
app10:~ # service syslog restart
Shutting down syslog services            done
Starting syslog servicessyntax error at 64
startproc:  exit status of parent of /sbin/syslog-ng: 1
And I tried without any quotes, without 'flags()' ...
I'm beginning to think syslog-ng is severly broken, is thta possible?
Version is 2.0.9 - pretty old! But obviously the version OpenSuse uses.
Maybe syntax differs, I'll go ahead and check that!

Reuti 07-16-2012 09:34 AM
Oh, yes. I use it on 3.1. Sorry, didn’t think about it. But you can download the source and compile it. In the end it’s only to drop in the executable in /sbin. OTOH: in openSUSE 12.1 there is 3.2 included AFAICS.

Update: Did you upgrade the system? By default they install now rsyslogd, but in YaST you can still switch to syslog-ng.

acid_kewpie 07-16-2012 09:43 AM
I still can't see why this behaviour is happening at all. Maybe there's some dedeuplication going on? Doubt it... what happens if you change the log() statements around so tripwire is first?

pingu 07-16-2012 09:55 AM
First of all let me say I'm sorry I've given false information.
Thing is, we don't have OpenSuse 12.1 on all servers!

Logserver is OpenSuse 12.1 but the web-server ("app10") is actually SLES 11.
OpenSuse uses syslog-ng 3.3.5, SLES uses 2.0.9
The documentation I've followed is for 3.# so of course it doesn't work (that program_override for instance is new in 3.0, was "log_prefix" in 2.0)
Anyway, this is not good, we don't want completely different versions of system logger.

So, decision to be taken:
manually install syslog-ng 3.2 on our SLES-servers, or switch to rsyslog on all?
Switching to rsyslog means I need to spend some time learning something new, but it's almost the same version on all - OpenSuse = rsyslog 5.8.5, SLES11 = rsyslog 5.8.7

Sorry to have fooled you originally, but now that we're discussing:
What do you think, can rsyslog do everything syslog-ng can - specifically, can it solve my problem?

Reuti 07-16-2012 10:07 AM
My opinion is, that rsyslog is a competitor syslog daemon to syslog-ng, but not the successor per se. I’m happy with syslog-ng and I will continue to use it. If you are happy with the syntax and the features it offers, I would stay with it.

acid_kewpie 07-16-2012 01:36 PM
syslog-ng is great, I far prefer it to rsyslog, not that it's really something to lose sleep over.

pingu 07-17-2012 05:20 AM
So we decided to stick to syslog-ng, at least for now.
But I just can't get this to work in any way!
I have set up 2 OpenSuse 12.1 in Virtualbox, removed rsyslogd & installed syslog-ng.
Both servers now use syslog-ng 3.1.1, also no app-armor, no firewall, same subnet.

Tried several things:
1) Configuration copied in exactly as in first post
2) Used "program_override" & filter instead, as suggested by Reuti, like this:
Client:
Code:
destination loganalyzer { udp(172.16.4.19 port(514)); };
log { source(src); destination(loganalyzer); };
source tripwire { file(/var/log/tripwire/tripwire program_override("tripwire")); };
log {
        source(tripwire);
        destination(loganalyzer); };
Server:
Code:
source srcExt {
        udp(ip("0.0.0.0") port(514));
};
filter f_tripwire { program(tripwire); };
destination stdTrip { file(/var/log/Hosts/tripwire.log); };
log {
        source(srcExt); filter(f_tripwire);
        destination(stdTrip);
};
Then I write to /var/log/tripwire/tripwire on client, but nothing is written to /var/log/Hosts/tripwire.log
The "filter" method, as I understand it, whenever something is written to /var/log/tripwire/tripwire on client it should be sent to server with "program" set to "tripwire", correct?
But nothing is sent at all.

Using the method with different ports and commenting out first destination, /var/log/Hosts/tripwire.log is written to.
No other config receives anything for tripwire.

I also tried to change the log() statements around like acid_kewpie suggested, both on sender and receiver (one at a time) but to no avail.
Funny thing is, still using the "ports" method, even if I change portnumbers around so "tripwire" is first and sends to port 514, all other logs sent to port 515 - now server receives on port 515!

This is really weird, I must be doing something wrong but I just can't figure out what?
I've read & read & read examples, docs, howtos... all saying both these methods should work!

Reuti 07-17-2012 05:27 AM
By setting the program name, you don’t need two ports any longer. I only use the usual 515 to transfer all logs messages. To exclude the tripwire logs from other logs you can use a negated filter.

pingu 07-17-2012 05:45 AM
Yes, I should have mentioned that when using the "filter" method I used only one port.
But the problem is still there, the tripwire logs are not sent at all so it's obviously something on the client side (for both methods).

acid_kewpie 07-17-2012 05:51 AM
This isn't intended as a solution, and syslog-ng can totally work fine here, but you might want to generally consider using splunk as a light forwarder on the client as well as the server side. it can make things like managing host names and times better with the data passing across the network in splunk format already.

Reuti 07-17-2012 05:54 AM
Is the file continually updated? Additional options I use are follow_freq(600) flags(no-parse,no-multi-line).


All times are GMT -5. The time now is 05:18 AM.
Page 1 of 2 1 2
Show 50 post(s) from this thread on one page