LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
Search this Thread
Old 04-27-2009, 01:48 PM   #1
rustyz82
Member
 
Registered: May 2004
Posts: 49

Rep: Reputation: 15
syslog-ng broken pipe issue


So I setup a syslog-ng server per the instructions I found at this site: http://planetit.ws/linux/ubuntu/inst...interface.html

Note: i'm using Ubuntu server 9.

After the setup went fairly smoothly I am not getting any events from teh local syslog file or any remote ones i have setup. I do see the following error in t he syslog file though:

Code:
Apr 27 13:40:01 SysLog syslog-ng[3258]: I/O error occurred while writing; fd='17', error='Broken pipe (32)'

It happens every 2 or 3 lines in the syslog file from /var/log/syslog.

It seems to be logging everything to the normal log files fine but does not write anything to the mysql database that was created. Any suggestions would be appreciated.
 
Old 04-27-2009, 01:53 PM   #2
kentyler
Member
 
Registered: Dec 2008
Location: Cleveland Ohio
Distribution: CentOS
Posts: 235

Rep: Reputation: 36
Please post the config file.
 
Old 04-27-2009, 02:22 PM   #3
rustyz82
Member
 
Registered: May 2004
Posts: 49

Original Poster
Rep: Reputation: 15
Here is the config file.. its kinda long but here is all of it!

Code:
#
# Configuration file for syslog-ng under Debian
#
# attempts at reproducing default syslog behavior

# the standard syslog levels are (in descending order of priority):
# emerg alert crit err warning notice info debug
# the aliases "error", "panic", and "warn" are deprecated
# the "none" priority found in the original syslogd configuration is
# only used in internal messages created by syslogd


######
# options

options {
        # disable the chained hostname format in logs
        # (default is enabled)
        chain_hostnames(0);

        # the time to wait before a died connection is re-established
        # (default is 60)
        time_reopen(10);

        # the time to wait before an idle destination file is closed
        # (default is 60)
        time_reap(360);

        # the number of lines buffered before written to file
        # you might want to increase this if your disk isn't catching with
        # all the log messages you get or if you want less disk activity
        # (say on a laptop)
        # (default is 0)
        #sync(0);

        # the number of lines fitting in the output queue
        log_fifo_size(2048);

        # enable or disable directory creation for destination files
        create_dirs(yes);

        # default owner, group, and permissions for log files
        # (defaults are 0, 0, 0600)
        #owner(root);
        group(adm);
        perm(0640);

        # default owner, group, and permissions for created directories
        # (defaults are 0, 0, 0700)
        #dir_owner(root);
        #dir_group(root);
        dir_perm(0755);
        # enable or disable DNS usage
        # syslog-ng blocks on DNS queries, so enabling DNS may lead to
        # a Denial of Service attack
        # (default is yes)
        use_dns(yes)
   # maximum length of message in bytes
   # this is only limited by the program listening on the /dev/log Unix
   # socket, glibc can handle arbitrary length log messages, but -- for
   # example -- syslogd accepts only 1024 bytes
   # (default is 2048)
   #log_msg_size(2048);

        #Disable statistic log messages.
        stats_freq(43200);

        # Some program send log messages through a private implementation.
        # and sometimes that implementation is bad. If this happen syslog-ng
        # may recognise the program name as hostname. Whit this option
        # we tell the syslog-ng that if a hostname match this regexp than that
        # is not a real hostname.
        bad_hostname("^gconfd$");
};


######
# sources

# all known message sources
source s_all {
        # message generated by Syslog-NG
        internal();
        # standard Linux log source (this is the default place for the syslog()
        # function to send logs to)
        unix-stream("/dev/log");
        # messages from the kernel
        file("/proc/kmsg" log_prefix("kernel: "));
        # use the following line if you want to receive remote UDP logging messages
        # (this is equivalent to the "-r" syslogd flag)
        udp();
};


######
# destinations

# some standard log files
destination df_auth { file("/var/log/auth.log"); };
destination df_syslog { file("/var/log/syslog"); };
destination df_cron { file("/var/log/cron.log"); };
destination df_daemon { file("/var/log/daemon.log"); };
destination df_kern { file("/var/log/kern.log"); };
destination df_lpr { file("/var/log/lpr.log"); };
destination df_mail { file("/var/log/mail.log"); };
destination df_user { file("/var/log/user.log"); };
destination df_uucp { file("/var/log/uucp.log"); };


# these files are meant for the mail system log files
# and provide re-usable destinations for {mail,cron,...}.info,
# {mail,cron,...}.notice, etc.
destination df_facility_dot_info { file("/var/log/$FACILITY.info"); };
destination df_facility_dot_notice { file("/var/log/$FACILITY.notice"); };
destination df_facility_dot_warn { file("/var/log/$FACILITY.warn"); };
destination df_facility_dot_err { file("/var/log/$FACILITY.err"); };
destination df_facility_dot_crit { file("/var/log/$FACILITY.crit"); };

# these files are meant for the news system, and are kept separated
# because they should be owned by "news" instead of "root"
destination df_news_dot_notice { file("/var/log/news/news.notice" owner("news")); };
destination df_news_dot_err { file("/var/log/news/news.err" owner("news")); };
destination df_news_dot_crit { file("/var/log/news/news.crit" owner("news")); };

# some more classical and useful files found in standard syslog configurations
destination df_debug { file("/var/log/debug"); };
destination df_messages { file("/var/log/messages"); };


# these files are meant for the mail system log files
# and provide re-usable destinations for {mail,cron,...}.info,
# {mail,cron,...}.notice, etc.
destination df_facility_dot_info { file("/var/log/$FACILITY.info"); };
destination df_facility_dot_notice { file("/var/log/$FACILITY.notice"); };
destination df_facility_dot_warn { file("/var/log/$FACILITY.warn"); };
destination df_facility_dot_err { file("/var/log/$FACILITY.err"); };
destination df_facility_dot_crit { file("/var/log/$FACILITY.crit"); };

# these files are meant for the news system, and are kept separated
# because they should be owned by "news" instead of "root"
destination df_news_dot_notice { file("/var/log/news/news.notice" owner("news")); };
destination df_news_dot_err { file("/var/log/news/news.err" owner("news")); };
destination df_news_dot_crit { file("/var/log/news/news.crit" owner("news")); };

# some more classical and useful files found in standard syslog configurations
destination df_debug { file("/var/log/debug"); };
destination df_messages { file("/var/log/messages"); };

# pipes
# a console to view log messages under X
destination dp_xconsole { pipe("/dev/xconsole"); };

# consoles
# this will send messages to everyone logged in
destination du_all { usertty("*"); };


######
# filters

# all messages from the auth and authpriv facilities
filter f_auth { facility(auth, authpriv); };

# all messages except from the auth and authpriv facilities
filter f_syslog { not facility(auth, authpriv); };

# respectively: messages from the cron, daemon, kern, lpr, mail, news, user,
# and uucp facilities
filter f_cron { facility(cron); };
filter f_daemon { facility(daemon); };
filter f_kern { facility(kern); };
filter f_lpr { facility(lpr); };
filter f_mail { facility(mail); };
filter f_news { facility(news); };
filter f_user { facility(user); };
filter f_uucp { facility(uucp); };

# some filters to select messages of priority greater or equal to info, warn,
# and err
# (equivalents of syslogd's *.info, *.warn, and *.err)
filter f_at_least_info { level(info..emerg); };
filter f_at_least_notice { level(notice..emerg); };
filter f_at_least_warn { level(warn..emerg); };
filter f_at_least_err { level(err..emerg); };
filter f_at_least_crit { level(crit..emerg); };

# all messages of priority debug not coming from the auth, authpriv, news, and
# mail facilities
filter f_debug { level(debug) and not facility(auth, authpriv, news, mail); };

# all messages of info, notice, or warn priority not coming form the auth,
# authpriv, cron, daemon, mail, and news facilities
filter f_messages {
        level(info,notice,warn)
            and not facility(auth,authpriv,cron,daemon,mail,news);
};

# messages with priority emerg
filter f_emerg { level(emerg); };

# complex filter for messages usually sent to the xconsole
filter f_xconsole {
    facility(daemon,mail)
        or level(debug,info,notice,warn)
        or (facility(news)
                and level(crit,err,notice));
};


######
# logs
# order matters if you use "flags(final);" to mark the end of processing in a
# "log" statement

# these rules provide the same behavior as the commented original syslogd rules

# auth,authpriv.*                 /var/log/auth.log
log {
        source(s_all);
        filter(f_auth);
        destination(df_auth);
};

# *.*;auth,authpriv.none          -/var/log/syslog
log {
        source(s_all);
        filter(f_syslog);
        destination(df_syslog);
};

# this is commented out in the default syslog.conf
# cron.*                         /var/log/cron.log
#log {
#        source(s_all);
#        filter(f_cron);
#        destination(df_cron);
#};

# daemon.*                        -/var/log/daemon.log
log {
        source(s_all);
        filter(f_daemon);
        destination(df_daemon);
};

# kern.*                          -/var/log/kern.log
log {
        source(s_all);
        filter(f_kern);
        destination(df_kern);
};

# lpr.*                           -/var/log/lpr.log
log {
        source(s_all);
        filter(f_lpr);
        destination(df_lpr);
};

# mail.*                          -/var/log/mail.log
log {
        source(s_all);
        filter(f_mail);
        destination(df_mail);
};

# user.*                          -/var/log/user.log
log {
        source(s_all);
        filter(f_user);
        destination(df_user);
};

# uucp.*                          /var/log/uucp.log
log {
        source(s_all);
        filter(f_uucp);
        destination(df_uucp);
};

# mail.info                       -/var/log/mail.info
log {
        source(s_all);
        filter(f_mail);
        filter(f_at_least_info);
        destination(df_facility_dot_info);
};

# mail.warn                       -/var/log/mail.warn
log {
        source(s_all);
        filter(f_mail);
        filter(f_at_least_warn);
        destination(df_facility_dot_warn);
};

# mail.err                        /var/log/mail.err
log {
        source(s_all);
        filter(f_mail);
        filter(f_at_least_err);
        destination(df_facility_dot_err);
};

# news.crit                       /var/log/news/news.crit
log {
        source(s_all);
        filter(f_news);
        filter(f_at_least_crit);
        destination(df_news_dot_crit);
};

# news.err                        /var/log/news/news.err
log {
        source(s_all);
        filter(f_news);
        filter(f_at_least_err);
        destination(df_news_dot_err);
};

# news.notice                     /var/log/news/news.notice
log {
        source(s_all);
        filter(f_news);
        filter(f_at_least_notice);
        destination(df_news_dot_notice);
};


# *.=debug;\
#         auth,authpriv.none;\
#         news.none;mail.none     -/var/log/debug
log {
        source(s_all);
        filter(f_debug);
        destination(df_debug);
};


# *.=info;*.=notice;*.=warn;\
#         auth,authpriv.none;\
#         cron,daemon.none;\
#         mail,news.none          -/var/log/messages
log {
        source(s_all);
        filter(f_messages);
        destination(df_messages);
};

# *.emerg                         *
log {
        source(s_all);
        filter(f_emerg);
        destination(du_all);
};


# daemon.*;mail.*;\
#         news.crit;news.err;news.notice;\
#         *.=debug;*.=info;\
#         *.=notice;*.=warn       |/dev/xconsole
log {
        source(s_all);
        filter(f_xconsole);
        destination(dp_xconsole);
};

destination d_mysql {
    program("/usr/bin/mysql -usyslogadmin -psyslogadmin syslog"
    template ("INSERT INTO logs (host, facility, priority, level, tag, datetime, program, msg)
    VALUES ( '$HOST', '$FACILITY', '$PRIORITY', '$LEVEL', '$TAG', '$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC', '$PROGRAM', '$MSG' );\n")
    template-escape(yes));
};

log {
    source(s_all);
    destination(d_mysql);
};
 
Old 04-28-2009, 10:02 AM   #4
rustyz82
Member
 
Registered: May 2004
Posts: 49

Original Poster
Rep: Reputation: 15
Anyone have any ideas?
 
Old 05-03-2009, 10:47 AM   #5
jerrybell
LQ Newbie
 
Registered: May 2009
Posts: 3

Rep: Reputation: 0
Unfortunately, the program() directive doesn't work like the author of that page thinks. There is a way to get it to work. Up till recently, it required a hacked together output to a pipe, and an external script to read from that pipe and insert it into mysql. But, those days are gone.

The latest versions of syslog-ng (2 and 3 only) have native support for mysql. See the syslog-ng administrator manual for the details. (I can't post the link because this is my first post)

Edit: The admin guide link is here: http://www.balabit.com/dl/html/syslo...estination_sql

Let me know if you that doesn't work for you.

Last edited by jerrybell; 05-03-2009 at 12:11 PM. Reason: added syslog-ng admin guide link
 
Old 05-06-2009, 01:30 PM   #6
rustyz82
Member
 
Registered: May 2004
Posts: 49

Original Poster
Rep: Reputation: 15
OK I get the idea but I'm not sure how to apply that to what I have already setup. They don't show examples for mysql which is what I'm using. I'm not totaly helpless but I'm not sure how to pickup from where I am now.
 
Old 06-22-2010, 10:45 AM   #7
cdukes
LQ Newbie
 
Registered: May 2004
Location: Tampa, FL
Distribution: Gentoo
Posts: 7

Rep: Reputation: 0
Arrow Broken Pipe

Quote:
Originally Posted by rustyz82 View Post
OK I get the idea but I'm not sure how to apply that to what I have already setup. They don't show examples for mysql which is what I'm using. I'm not totaly helpless but I'm not sure how to pickup from where I am now.
Hi Rusty,
I own the tool that your syslog-ng code is pointing to.
It looks like you have an old version.
You should try downloading the latest as it uses a program call to a perl script now instead of a direct MySQL pipe.

http://code.google.com/p/php-syslog-ng/

You can also get more help in the LogZilla forums at http://forum.logzilla.info
 
  


Reply

Tags
syslog, syslogng


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
syslog not opening named pipe. fluxrad Linux - Software 5 02-10-2009 05:31 AM
Squid and Dansguardian | pipe syslog to Splunk? tekhead2 Linux - Server 1 06-28-2007 06:04 AM
Broken pipe henkelarsson12 *BSD 1 04-08-2006 11:12 AM
syslog not writing to named pipe (fc4) darbo Linux - Software 0 02-17-2006 07:35 PM


All times are GMT -5. The time now is 09:02 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration