Download your favorite Linux distribution at LQ ISO.
Go Back > Forums > Non-*NIX Forums > Programming
User Name
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.


  Search this Thread
Old 03-14-2007, 10:03 AM   #1
Registered: Jun 2003
Location: Cleveburg, OH
Distribution: mostly Fedora
Posts: 154

Rep: Reputation: 30
perl File::Tail hell

I'm using a perl script to tail an application log (the app doesn't support syslog). The script eliminates some lines through regex, and then uses Unix::Syslog to put it to the normal syslog daemon. This script is run as a daemon with daemontools, and the parsing/inserting needs to be real-time.

The problem is, when there is particularly heavy writing in the application's log (sometimes over 20 lines per second) the perl script starts parsing the file from the beginning again, giving me tons of duplicate entries.

Here is a copy of the script. Any ideas on how to fix it, or better yet, a more elegant & reliable solution?

#!/usr/bin/perl -w

use File::Tail;
my $Filename = '/var/log/application.log';
my $File = File::Tail->new(name=>$Filename, tail=>1, interval=>1.0);
if (not defined $File) {
    die "/usr/local/bin/ cannot read input \"$Filename\": $!\n";

use Unix::Syslog qw(:macros :subs);

LOOP: while (defined($_=$File->read)) {
    if ( (/blah blah blah/) or # ignore this because...
         (/blah blah blah/) or # also ignore this
         (/blah blah blah/) ) {
    elsif (/.*/) {
      openlog "taillog", LOG_PID , LOG_LOCAL6;
        syslog LOG_INFO,"$_";
Old 03-14-2007, 01:59 PM   #2
spirit receiver
Registered: May 2006
Location: Frankfurt, Germany
Distribution: SUSE 10.2
Posts: 424

Rep: Reputation: 33
Maybe you could pipe the output of "tail -f" through your script and read from standard input instead?

Something like this:
tail -f FILENAME | perl -e 'while ( <STDIN> ){ print }'

Last edited by spirit receiver; 03-14-2007 at 02:01 PM.
Old 03-14-2007, 02:44 PM   #3
Registered: Jun 2003
Location: Cleveburg, OH
Distribution: mostly Fedora
Posts: 154

Original Poster
Rep: Reputation: 30
Thanks for the suggestion!

We've been down that road before... there are process control issues relating to rolling log files, process crash recovery, etc, etc.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
I need to tail -f just a few colums o a text file ericcarlson Linux - General 3 01-30-2007 03:07 PM
scp/ssh tail(multiple file) remote copy tpreitano Linux - General 1 08-22-2005 02:17 PM
Tail Last Modified File in a Directory tyoungblood Linux - Software 3 08-12-2005 08:42 AM
dmegs tail Unchecked file sysytem -- Run e2fsck Nichole_knc Slackware 6 04-18-2004 05:10 AM
perl(Cwd) perl(File::Basename) perl(File::Copy) perl(strict)....What are those? Baldorg Linux - Software 1 11-09-2003 08:09 PM > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 06:48 AM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration