LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 12-05-2003, 08:21 AM   #1
ganninu
Member
 
Registered: Jul 2003
Distribution: RH 7.3/8.0/9.0, Debian Stable 3.0, FreeBSD 5.2, Solaris 8/9/10,HP-UX
Posts: 340

Rep: Reputation: 30
how to monitor logfiles


This thread is directly related to thread http://www.linuxquestions.org/questi...hreadid=122565

Are there any techniques which can be usedto process growing logfiles? I have this logfile and I'm trying to monitor for new entries to process them.... For some strange reason,

tail -1 | <process_entry.bash>

works perfectly well for the last entry, but:

tail -f | <process_entry.bash>

doesn't give any output at all!!!

Any techniques how to go around this?? Help very much appreciated.

Ganninu.

Last edited by ganninu; 12-05-2003 at 08:27 AM.
 
Old 12-05-2003, 11:53 AM   #2
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2401Reputation: 2401Reputation: 2401Reputation: 2401Reputation: 2401Reputation: 2401Reputation: 2401Reputation: 2401Reputation: 2401Reputation: 2401Reputation: 2401
I can explain to you why it behaves this way:

<command 1> | <command 2>

- The shell creates 2 subprocesses (fork), lets call them sp1 and sp2.
- Set up a pipe (I/O) between sp1's output and sp2's input.
- Start (exec) <command 1> in process sp1.
- Start (exec) <command 2> in process sp2.
- Wait for both prcesses to finish.

The last line is the reason why this will work with tail -1 and _not_ with tail -f. The shell will wait untill both processes are ready, tail -f will never be ready, hench: no output.

If you put a trace on your script you will get confirmation for this behavior:

rt_sigaction(SIGINT, {0x80701a0, [], 0x4000000}, {SIG_DFL}, 8) = 0
wait4(-1,

No more output after the above 2 lines. Shell is waiting for one of the processes (the tail -f in this case) to finish.

I don't know how to avoid this (restarting syslogd everytime you run the script won't be an option , but will work).

You could consider to fetch all the lines from the logfile that haven't been processesed (based on timestamps), put these in a seperate file and use that file.
 
Old 12-06-2003, 09:53 AM   #3
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2401Reputation: 2401Reputation: 2401Reputation: 2401Reputation: 2401Reputation: 2401Reputation: 2401Reputation: 2401Reputation: 2401Reputation: 2401Reputation: 2401
Had some more time to look at your problem and maybe this will work for you.
Code:
#!/bin/bash
tail -f /var/log/messages | \
while read line
do
  echo $line | grep 'session' | awk '{ print $7, $8, $9, $11 }' >> /var/log/out.file
done
The above code will look for lines containing 'session' and print the following to the out.file if it finds a hit:

session started for root,

The above example is quick and dirty, but shows that this construction will work.
 
Old 12-06-2003, 10:09 AM   #4
ganninu
Member
 
Registered: Jul 2003
Distribution: RH 7.3/8.0/9.0, Debian Stable 3.0, FreeBSD 5.2, Solaris 8/9/10,HP-UX
Posts: 340

Original Poster
Rep: Reputation: 30
thanks i'll give it a look and apply it to my needs and tell you what happened
 
Old 12-07-2003, 07:36 AM   #5
markus1982
Senior Member
 
Registered: Aug 2002
Location: Stuttgart (Germany)
Distribution: Debian/GNU Linux
Posts: 1,467

Rep: Reputation: 46
take a look at "logcheck" also :-)
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
logcheck does not read logfiles! cyberpunx Linux - Software 16 04-06-2015 03:18 AM
finding program logfiles atom Linux - Software 3 08-10-2004 05:33 PM
Any tool for analyzing Samba logfiles? mark1 Linux - Networking 2 08-07-2004 12:14 PM
What is a good logfile scanner/tool to help monitor logfiles? jdruin Linux - Security 2 11-20-2003 11:44 PM
Logfiles wonderpun Linux - General 3 09-01-2002 04:27 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 12:27 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration