LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-10-2012, 04:29 AM   #1
drnjdtj
LQ Newbie
 
Registered: Feb 2012
Posts: 9

Rep: Reputation: Disabled
grep error and lines after is until a regular expression


Hello,

I have a error logfile that looks like this:
2012 Feb 09 09:12:34:824 GMT +1 All ok
dsf
dfd
2012 Feb 09 09:12:34:824 GMT +1 Error in bla
sdff
2012 Feb 09 09:12:34:824 GMT +1 Next Ok line
sx
2012 Feb 09 09:12:34:824 GMT +1 Ok line
2012 Feb 09 09:12:34:824 GMT +1 This is an other Error
rwere
erdfsr
dsfr
dsfsdf
2012 Feb 09 09:12:34:824 GMT +1 yet an other Error
fgdfds


So I want to display all lines containing Error and the lines ather it until the next date time stamp is found.

So output should be like this:
2012 Feb 09 09:12:34:824 GMT +1 Error in bla
sdff
2012 Feb 09 09:12:34:824 GMT +1 This is an other Error
rwere
erdfsr
dsfr
dsfsdf
2012 Feb 09 09:12:34:824 GMT +1 yet an other Error
fgdfds
 
Old 02-10-2012, 04:51 AM   #2
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 14,839

Rep: Reputation: 1822Reputation: 1822Reputation: 1822Reputation: 1822Reputation: 1822Reputation: 1822Reputation: 1822Reputation: 1822Reputation: 1822Reputation: 1822Reputation: 1822
Better show us your attempts so we can comment, and maybe help.
 
Old 02-10-2012, 04:56 AM   #3
drnjdtj
LQ Newbie
 
Registered: Feb 2012
Posts: 9

Original Poster
Rep: Reputation: Disabled
I'm no grep / sed / awk guru, thats the main problem.

The best I got up until now:
grep --after-context 20 Error

This kinda works, but now I have too much lines or not enough if the error is more then 20 lines.
 
Old 02-10-2012, 05:12 AM   #4
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,252

Rep: Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685
So are the additional lines of the error ever going to start with a number? or the Year? You need to use what you know about the data to tell grep/sed/awk when to start / stop.
 
Old 02-10-2012, 05:19 AM   #5
drnjdtj
LQ Newbie
 
Registered: Feb 2012
Posts: 9

Original Poster
Rep: Reputation: Disabled
So the additional lines will never contain the date time string format like this "2012 Feb 09 09:08:59:812 GMT +1" at the beginning of the line, but it can also have no additional lines.
 
Old 02-10-2012, 06:39 AM   #6
drnjdtj
LQ Newbie
 
Registered: Feb 2012
Posts: 9

Original Poster
Rep: Reputation: Disabled
Some examples:
2012 Feb 09 09:08:59:810 GMT +1 message.log Error User [test-User] E <EventName> Name <EventID> ID:<124871.1328774907808.0> <ProcessID> 7637458 <Message> <ProcessStack> <Class> <CustomLog> HTTP Error: Webservice unvailable <EventProperties> Adapter=adatpername, Engine=endpoint, ApplicationStatus=4367
2012 Feb 09 09:08:59:812 GMT +1 message.log Error User [test-User] Event timeout. <EventName> Name <EventID> ID:<124871.1328774907808.0> <ProcessID> 7637458 <Message> Activity timed out <ProcessStack> SOAPRequestReply <Class> ActivityTimedOutException <CustomLog> Timeout
<ns0:ErrorReport xmlns:ns0="Schema.xsd"><ns0:StackTrace>Job-7637458 Error in [SOAPRequestReply]
Activity timed out
at com.tibco.pe.core.TaskImpl.eval(Unknown Source)
at com.tibco.pe.core.Job.a(Unknown Source)
at com.tibco.pe.core.Job.k(Unknown Source)
at com.tibco.pe.core.JobDispatcher$JobCourier.a(Unknown Source)
at com.tibco.pe.core.JobDispatcher$JobCourier.run(Unknown Source)
</ns0:StackTrace><ns0:Msg>Activity timed out</ns0:Msg><ns0:FullClass>com.tibco.pe.plugin.ActivityTimedOutException</ns0:FullClass><ns0:Class>ActivityTimedOutException</ns0:Class><ns0:ProcessStack>StackName</ns0:ProcessStack><ns0:MsgCode>BWENGINE-100029</ns0:MsgCode><ns0ata/></ns0:ErrorReport> <EventProperties> Adapter=adatpername, Engine=endpoint, ApplicatieStatus=5, Adapter=adatpername, Engine=endpoint
2012 Feb 09 09:08:59:813 GMT +1 message.log User [test-User] - Job-7637458 [Log]: Message not confirmed.
2012 Feb 09 09:08:59:813 GMT +1 message.log Error [BW-Core] BWENGINE-100029 Job-7637458 Error in [SOAPRequestReply]
Activity timed out
at com.tibco.pe.core.TaskImpl.eval(Unknown Source)
at com.tibco.pe.core.Job.a(Unknown Source)
at com.tibco.pe.core.Job.k(Unknown Source)
at com.tibco.pe.core.JobDispatcher$JobCourier.a(Unknown Source)
at com.tibco.pe.core.JobDispatcher$JobCourier.run(Unknown Source)
called by: [Interface]
<?xml version="1.0" encoding="UTF-8"?>
<Data>
<ns0:ActivityTimedOutException xmlns:ns0="http://schemas.tibco.com/bw/pe/plugin/5.0/exceptions">
<msg>Activity timed out</msg>
<msgCode>BWENGINE-100029</msgCode>
</ns0:ActivityTimedOutException>
</Data>
2012 Feb 09 09:08:59:816 GMT +1 All ok
2012 Feb 09 09:12:34:828 GMT +1 message.log Error [BW_Plugin] BW-HTTP-100001 Job-7637825 Error in [SOAPRequestReply]
The Http Server replied with a 5XX status code
at com.tibco.plugin.share.http.client.JakartaHttpTransportDriver$RequestExecutor.run(Unknown Source)
at com.tibco.pe.util.ThreadPool$ThreadPoolThread.run(Unknown Source)
called by: [caller]
<?xml version="1.0" encoding="UTF-8"?>

<Data>
<ns0:HttpServerException xmlns:ns0="http://schemas.tibco.com/bw/plugins/http/5.0/httpExceptions">
<msg>The Http Server replied with a 5XX status code</msg>
<msgCode>BW-HTTP-100001</msgCode>
<ns1:statusLine xmlns:ns1="http://schemas.tibco.com/bw/plugins/http/5.0/cio">
<httpVersion>HTTP/1.1</httpVersion>
<statusCode>503</statusCode>
<reasonPhrase>Service Temporarily Unavailable</reasonPhrase>
</ns1:statusLine>
<ns1:httpMessage xmlns:ns1="http://schemas.tibco.com/bw/plugins/http/5.0/cio">
<headers>
<content-type>text/html; charset=iso-8859-1</content-type>
<connection>close</connection>
<content-length>323</content-length>
<date>Thu, 09 Feb 2012 08:12:34 GMT</date>
</headers>
<binaryContent>osdfhkfkdshgfddskghfjdgsjhgsa</binaryContent>
</ns1:httpMessage>
</ns0:HttpServerException>
</Data>
 
Old 02-10-2012, 07:28 AM   #7
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,252

Rep: Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685
So based on your last example, what would be your desired output?
 
Old 02-10-2012, 07:34 AM   #8
drnjdtj
LQ Newbie
 
Registered: Feb 2012
Posts: 9

Original Poster
Rep: Reputation: Disabled
All lines, except
2012 Feb 09 09:08:59:816 GMT +1 All ok

If the line with the date time format like "2012 Feb 09 09:08:59:816 GMT +1" contains Error, include the line and all next lines until you get a line that starts with the same date time format (2012 Feb 09 09:08:59:816 GMT +1) again.
 
Old 02-10-2012, 07:55 AM   #9
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,252

Rep: Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685
So maybe something like:
Code:
awk 'BEGIN{RS = "2012"}/Error/{printf RT $0}' file
 
1 members found this post helpful.
Old 02-10-2012, 08:27 AM   #10
drnjdtj
LQ Newbie
 
Registered: Feb 2012
Posts: 9

Original Poster
Rep: Reputation: Disabled
It does exactly what I wanted!
Thanks mate!

Altered a bit:
awk 'BEGIN{RS = "[0-9][0-9][0-9][0-9] [A-Z][a-z][a-z] [0-3][0-9]"}/Error/{printf RT $0}' file

So next I want to (or al least try to) understand it.
RS = record separator, thats clear.
The rest is not so clear.
 
Old 02-10-2012, 09:01 AM   #11
drnjdtj
LQ Newbie
 
Registered: Feb 2012
Posts: 9

Original Poster
Rep: Reputation: Disabled
And how to search on "Error " Error with a space?
 
Old 02-10-2012, 10:53 AM   #12
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,252

Rep: Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685
So simply put the space in between the slashes.

RT - this matches whatever your regex for RS is equal to ... so for me it was just 2012 but yours will be what your pattern matches.

$0 - This is the currently stored record

printf - used this so no additional new line would be added to the output

And here is a reference for all things awk:

http://www.gnu.org/software/gawk/man...ode/index.html
 
Old 02-10-2012, 11:30 AM   #13
drnjdtj
LQ Newbie
 
Registered: Feb 2012
Posts: 9

Original Poster
Rep: Reputation: Disabled
Thank you very much!
 
Old 02-10-2012, 12:34 PM   #14
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,252

Rep: Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685Reputation: 2685
No problem Don't forget to mark as SOLVED once you have a solution.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] How to use regular expression in grep? sagarkha Linux - Newbie 6 01-23-2010 09:15 AM
grep regular expression casperdaghost Linux - Newbie 2 08-15-2009 12:01 PM
grep Regular Expression lamar_air Programming 2 07-24-2006 04:40 PM
Help with grep regular expression lamar_air Programming 3 07-21-2006 12:26 PM
problems with regular expression and grep bwreath Linux - General 6 03-18-2005 02:43 PM


All times are GMT -5. The time now is 08:29 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration