LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
LinkBack Search this Thread
Old 07-23-2012, 06:29 PM   #1
blsimpson
LQ Newbie
 
Registered: Jun 2012
Posts: 15

Rep: Reputation: Disabled
Script - Find time between entries in log file


Hello all,

I have a log file that looks like this:

Quote:
280898275 [org.springframework.jms.listener.DefaultMessageListenerContainer#0-1] INFO c.v.i.c.c.s.CardGenServiceImpl - generating VirtualCard
280952720 [org.springframework.jms.listener.DefaultMessageListenerContainer#0-1] INFO c.v.i.c.c.s.CardGenServiceImpl - Found Fundingcard com.xxxxxxx.xxxxxxx.transactional.FundingCard@15ffb1bd
280952737 [org.springframework.jms.listener.DefaultMessageListenerContainer#0-1] INFO c.v.i.c.c.s.CardGenServiceImpl - Completed generating VirtualCard
280952787 [org.springframework.jms.listener.DefaultMessageListenerContainer#0-1] INFO c.v.i.c.c.s.CardGenServiceImpl - Card controls stored for Virtual Card Hanlde :19096
280953126 [org.springframework.jms.listener.DefaultMessageListenerContainer#0-1] INFO c.v.i.c.c.s.CardGenServiceImpl - generating VirtualCard
281007859 [org.springframework.jms.listener.DefaultMessageListenerContainer#0-1] INFO c.v.i.c.c.s.CardGenServiceImpl - Found Fundingcard com.xxxxxxxx.xxxxxxx.transactional.FundingCard@3ad721b1
281007874 [org.springframework.jms.listener.DefaultMessageListenerContainer#0-1] INFO c.v.i.c.c.s.CardGenServiceImpl - Completed generating VirtualCard
The first part is the time in milliseconds. (I cant change this, I dont have access to the code for that.) What I need is to grep the file for "generating VirtualCard" take the number in the beginning and make it "variable a". Then grep for "Completed generating VirtualCard", take the number in the beginning, and make it "variable b". Once thats done, I need to take variable a and subtract it from variable b, and output the result. I just need the output.

What I am looking for is the amount of time it take for the card generation. Any help would be great, as I am not familiar with this..
 
Old 07-23-2012, 06:53 PM   #2
antegallya
Member
 
Registered: Jun 2008
Location: Belgium
Distribution: Debian
Posts: 109

Rep: Reputation: 42
Hello

That should be as easy as
Code:
timeA=$(grep "generating VirtualCard" logfile | egrep -o "^[0-9]*")
timeB=$(grep "Completed generating VirtualCard" logfile | egrep -o "^[0-9]*")
echo Time elapsed : $(expr $timeB - $timeA)
 
Old 07-23-2012, 07:20 PM   #3
antegallya
Member
 
Registered: Jun 2008
Location: Belgium
Distribution: Debian
Posts: 109

Rep: Reputation: 42
Well, I saw your log file actually has multiple generations of VirtualCard.
Do you need the generation time for each log entry ? If so, you could pipe your file to the following awk script :
Code:
#!/usr/bin/awk -f
BEGIN {
	start = -1;
}

/generating VirtualCard/ {
	if (start == -1) {
		start = $1;
	} else {
		print "Generations are not sequential !";
		exit 1;
	}
}

/Completed generating VirtualCard/ {
	if (start != -1) {
		print "Generated a VirtualCard in", $1 - start, "ms";
		start = "-1";
	} else {
		print "Bad log file";
		exit 1;
	}
}
 
1 members found this post helpful.
Old 07-23-2012, 07:29 PM   #4
blsimpson
LQ Newbie
 
Registered: Jun 2012
Posts: 15

Original Poster
Rep: Reputation: Disabled
That is correct, there are multiple entries in the same log file that I need the times for. I will try both of these options, and get back to you. Thank you so much!
 
Old 07-23-2012, 07:35 PM   #5
antegallya
Member
 
Registered: Jun 2008
Location: Belgium
Distribution: Debian
Posts: 109

Rep: Reputation: 42
Okay, but note that first one won't work with multiple entries.
 
Old 07-23-2012, 07:37 PM   #6
blsimpson
LQ Newbie
 
Registered: Jun 2012
Posts: 15

Original Poster
Rep: Reputation: Disabled
Oh, Okay. Will the second one loop through the whole file and output them all?
 
Old 07-23-2012, 07:38 PM   #7
antegallya
Member
 
Registered: Jun 2008
Location: Belgium
Distribution: Debian
Posts: 109

Rep: Reputation: 42
Yes. That is what you want, right ?
 
Old 07-23-2012, 07:49 PM   #8
blsimpson
LQ Newbie
 
Registered: Jun 2012
Posts: 15

Original Poster
Rep: Reputation: Disabled
Thats correct. Thank you so much!
 
Old 07-23-2012, 09:03 PM   #9
blsimpson
LQ Newbie
 
Registered: Jun 2012
Posts: 15

Original Poster
Rep: Reputation: Disabled
Ok, I guess I dont know enough about this.. Can you point me in the direction on how I would accomplish what you said, pipe the logfile into the script..
 
Old 07-23-2012, 09:16 PM   #10
antegallya
Member
 
Registered: Jun 2008
Location: Belgium
Distribution: Debian
Posts: 109

Rep: Reputation: 42
Once you've created the script and made it executable with chmod +x, you can pipe your log file as with any command
Code:
cat logfile | ./script
 
1 members found this post helpful.
Old 07-23-2012, 09:24 PM   #11
blsimpson
LQ Newbie
 
Registered: Jun 2012
Posts: 15

Original Poster
Rep: Reputation: Disabled
Thanks. I'm getting "Generations are not sequential !"
 
Old 07-24-2012, 12:13 PM   #12
blsimpson
LQ Newbie
 
Registered: Jun 2012
Posts: 15

Original Poster
Rep: Reputation: Disabled
Thought I would check in and let you know where I got to. Using the code you gave me, I changed it a little, and came up with this:

Code:
#!/usr/bin/awk -f

BEGIN {
        start = 1;
}

/- generating VirtualCard/ {
        if (start < $1) {
                start = $1;
        } else {
                print "Generations are not sequential !";
                exit 1;
        }
}

/Completed generating VirtualCard/ {
        if (start != -1) {
                print "Generated a VirtualCard in", $1 - start, "ms";
                start = "-1";
        } else {
                print "Bad log file";
                exit 1;
        }
}
Gives me output of:

Quote:
Generated a VirtualCard in 1041 ms
Generated a VirtualCard in 1060 ms
Generated a VirtualCard in 1060 ms
Generated a VirtualCard in 1082 ms
Now to change them to seconds.
 
Old 07-24-2012, 01:14 PM   #13
blsimpson
LQ Newbie
 
Registered: Jun 2012
Posts: 15

Original Poster
Rep: Reputation: Disabled
Ok, pass on the seconds.. Maybe someone can help me with this next part. I need to be able to write a script to loop through the log directory in sequence based on date, and write the output to a file. Example:

Directory:
~/test/
logFile.2012-06-04.log
logFile.2012-06-05.log
logFile.2012-06-06.log
logFile.2012-06-07.log
script.sh
generation.log

Command I am using:
cat logFile.2012-06-04.log | ./script.sh > generation.log

I could do them all manually, by just changing the date of the log file in the command, but I would like to be able to put other files in there, and have it pick them up as well. The log file names will always be logFile.YYYY.MM.DD.log
 
Old 07-24-2012, 01:35 PM   #14
blsimpson
LQ Newbie
 
Registered: Jun 2012
Posts: 15

Original Poster
Rep: Reputation: Disabled
Using this seems to get me what I want:

Code:
#!/bin/bash

i=1
for file in /test/*.log
do
echo "$file" >> generation.log
cat $file | ./script.sh >> generation.log
# echo "File $((i++)) : $file"
done
Now to add some clean up, checking if the generation.log file exists, and if it does, move it to generation.bak.

Will continue to work through it
 
Old 07-24-2012, 01:49 PM   #15
blsimpson
LQ Newbie
 
Registered: Jun 2012
Posts: 15

Original Poster
Rep: Reputation: Disabled
That was pretty easy. Added a simple if\then:

Code:
DATE=`date '+%m-%d-%y:%H:%M:%S'`
LOG=generation.log
LOGBACK=generation.$DATE.bak

if [ -f $LOG ];
then
        mv $LOG $LOGBACK
fi
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] How to fetch entries in a log file for a particular period of time ? rhadmn Linux - Enterprise 3 03-19-2012 05:43 AM
Checking Log Entries for Specific Time Duration devUnix Programming 2 02-02-2012 04:49 PM
Need help to have a rsync script log output to a file with time stamp Thaidog Programming 5 11-15-2011 05:37 PM
a command or way to log time of iptables LOG entries? dividingbyzero Linux - Security 3 06-06-2008 01:23 AM
script to find out time taken to transfer a file aeby Linux - Networking 1 05-09-2007 05:29 AM


All times are GMT -5. The time now is 09:32 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration