LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-21-2012, 03:15 AM   #1
vaw
LQ Newbie
 
Registered: Feb 2012
Posts: 6

Rep: Reputation: Disabled
linuc shell scripting


Hi,

There are a few log files from which my shell script picks up errors messages and sends alerts. This script runs every half hour.
But the logs are created say one a day and sometimes one per week. In this case my script picks up the errors every half hour which is redundant and not preferred.

Can someone suggest me how I can pick up items(errors) from these logs that only have the timestamp less than half hour ?

Thanks,
 
Old 02-21-2012, 03:24 AM   #2
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,576
Blog Entries: 31

Rep: Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195
The shell script could write the timestamp of when it last looked at the logs to a file. Somewhere under /var/lib might be appropriate according to the Linux Filesystem Hierarchy Standard.
 
Old 02-21-2012, 03:49 AM   #3
vaw
LQ Newbie
 
Registered: Feb 2012
Posts: 6

Original Poster
Rep: Reputation: Disabled
How can we then find the errors having the timestamp greater than this stored timestamp ?
 
Old 02-21-2012, 04:15 AM   #4
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,576
Blog Entries: 31

Rep: Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195
The script could compare the saved timestamp with the timestamps in the log and only pick up errors that happened later.

What does your script look like?
 
Old 02-21-2012, 04:36 AM   #5
KatrinAlec
Member
 
Registered: Feb 2012
Posts: 67

Rep: Reputation: 8
you could maybe send the data through a pipe, so you only read it once,
or use tail -f, so you only get new output.
 
Old 02-21-2012, 06:23 AM   #6
vaw
LQ Newbie
 
Registered: Feb 2012
Posts: 6

Original Poster
Rep: Reputation: Disabled
currently the script only looks for the file modified in the last half hour, but obviously this will not work as the errors even as old as a day will get picked.

Could you tell me how we can compare the timestamps so that only the error which is not older than, say half hour is picked.
 
Old 02-21-2012, 06:30 AM   #7
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,576
Blog Entries: 31

Rep: Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195
What do the timestamps in the log look like?
 
Old 02-21-2012, 06:39 AM   #8
vaw
LQ Newbie
 
Registered: Feb 2012
Posts: 6

Original Poster
Rep: Reputation: Disabled
The timestamps are like this : 2011-01-12-02:43:52
 
Old 02-21-2012, 06:42 AM   #9
vaw
LQ Newbie
 
Registered: Feb 2012
Posts: 6

Original Poster
Rep: Reputation: Disabled
Some other I am looking for also have timestamps like : 2012-02-12 07:30:20
 
Old 02-21-2012, 08:26 AM   #10
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,576
Blog Entries: 31

Rep: Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195Reputation: 1195
Something like this (not tested):
Code:
#!/bin/bash

timestamp_file=~/last_run.timestamp
if [[ -f $timestamp_file && -r $timestamp_file ]]; then
    after_timestamp=$( cat $timestamp_file )
else
    after_timestamp=000000000000000000000
fi

while read -r line
do
    msg_timestamp=${line:0:20}
    msg_timestamp=${msg_timestamp//:}
    msg_timestamp=${msg_timestamp//-}
    msg_timestamp=${msg_timestamp// }
    if (( $msg_timestamp > $after_timestamp )); then
        echo "$line"
    fi  
done < test.log

echo "$( date '+%Y%m%d%H%M%S' )" > $timestamp_file
EDIT: it would be safer if the last line was echo "$( $msg_timestamp )" > $timestamp_file but there is still some danger of dropping some messages. A safer technique would be for the script to write the last message it sent (from each log file) in a file.

Last edited by catkin; 02-21-2012 at 09:41 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Terminal functions for shell scripting with Shell Curses LXer Syndicated Linux News 0 03-26-2008 11:50 PM
SHELL scripting/ shell functions mayaabboud Linux - Newbie 6 12-26-2007 08:18 AM
Shell Scripting: Getting a pid and killing it via a shell script topcat Programming 15 10-28-2007 02:14 AM
teaching shell scripting: cool scripting examples? fax8 Linux - General 1 04-20-2006 04:29 AM
shell interface vs shell scripting? I'm confused jcchenz Linux - Software 1 10-26-2005 03:32 PM


All times are GMT -5. The time now is 08:16 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration