LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 01-22-2010, 02:22 PM   #1
jalum
LQ Newbie
 
Registered: Jan 2010
Posts: 2

Rep: Reputation: 0
sed command to extract stacktraces


Hi,
I'm trying to extract stacktraces from log files, looking for the pattern "Exception". If a line contains "Exception", a stack trace will follow in multiple lines. If the next line begins with a timestamp entry in the format "[1/13/10 23:17:00:444 CST]", the stack trace has ended in the previous line. Additionally, I want to grab the nearest line containing a timestamp entry above the original line that contained the pattern "Exception". Sometimes it's on the same line and other times it's a few lines above it. How I do write a sed command for this? Example:

Code:
[1/13/10 23:01:16:623 CST] 00000059 SystemOut     O 2010-01-13 23:01:16,623 [ORB.thread.pool : 0] INFO  
 - Exiting isAlertUpdateTimerRunning()
SystemOut     O 2010-01-13 23:01:16,623 [ORB.thread.pool : 0] INFO  com.dd.dddd.ddddddddr.ddddddr.dddddddddTriggerManager
org.hibernate.SessionException: Session is closed!
        at org.hibernate.impl.AbstractSessionImpl.errorIfClosed(AbstractSessionImpl.java:72)
        at org.hibernate.impl.SessionImpl.reconnect(SessionImpl.java:407)
        at com.ibm.ws.Transaction.JTA.RegisteredSyncs.distributeAfter(RegisteredSyncs.java:424)
        at com.ibm.ws.Transaction.JTA.TransactionImpl.distributeAfter(TransactionImpl.java:3885)
        at com.ibm.ws.Transaction.JTA.TransactionImpl.postCompletion(TransactionImpl.java:3864)
        at com.ibm.ws.Transaction.JTA.TransactionImpl.commitXAResources(TransactionImpl.java:2521)
        at com.ibm.ws.Transaction.JTA.TransactionImpl.stage1CommitProcessing(TransactionImpl.java:1647)
        at com.ibm.ws.Transaction.JTA.TransactionImpl.processCommit(TransactionImpl.java:1607)
        at com.ibm.ws.Transaction.JTA.TransactionImpl.commit(TransactionImpl.java:1542)
        at com.ibm.ws.Transaction.JTA.TranManagerImpl.commit(TranManagerImpl.java:240)
        at com.ibm.ws.Transaction.JTA.TranManagerSet.commit(TranManagerSet.java:164)
        at com.ibm.ejs.csi.TranStrategy.commit(TranStrategy.java:756)
        at com.ibm.ejs.csi.TranStrategy.postInvoke(TranStrategy.java:181)
        at com.ibm.ejs.csi.TransactionControlImpl.postInvoke(TransactionControlImpl.java:581)
        at com.ibm.ejs.container.EJSContainer.postInvoke(EJSContainer.java:3910)
        at com.ibm.ejs.container.EJSContainer.postInvoke(EJSContainer.java:3732)
[1/13/10 23:01:16:725 CST] 0000006e SystemOut     O 2010-01-13 23:01:16,725 [MessageListenerThreadPool : 12] INFO  com.ddconnector.cccccc.ccccccSession - Th
read[MessageListenerThreadPool : 12,5,main]
Here, I want to grab the first line, followed by the stack trace line starting "org.hibernate.SessionException" all the way until the last line of the stack trace.

Thanks.
 
Old 01-22-2010, 03:54 PM   #2
paulsm4
LQ Guru
 
Registered: Mar 2004
Distribution: SusE 8.2
Posts: 5,863
Blog Entries: 1

Rep: Reputation: Disabled
Hi -

The good news is that sed supports "start patterns" (along with corresponding "stop patterns"). For example:

Quote:
sed '/start/,/stop/ s/#.*//'
<= This example deletes every line beginning with a "#" for every line between the first occurrence of "start" through the next occurrence of the word "stop"
You can read more about it here:
http://www.grymoire.com/Unix/Sed.html
<= Search for "ranges by patterns"

If you've got a clearly delimited block (for example, "start" through "stop", or "Exception" through "INFO"), then you've got it made.

Otherwise, this might be a good time to start learning Perl or Python .

'Hope that helps .. PSM

PS:
Even though "awk" is a lot more powerful than "sed", I don't think it would necessarily buy you that much for this problem. You've either got a nice, straightforward text pattern (in which case "sed" should be OK), or you need procedural logic (in which case I'd recommend looking at a scripting language like Perl or Python).

Last edited by paulsm4; 01-22-2010 at 03:57 PM.
 
Old 01-22-2010, 08:02 PM   #3
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 244Reputation: 244Reputation: 244
@OP, you did not show your desired output, so here's a guess of what you want.
Code:
awk '/^\[/{f=1;g=0;o=$0}
f && /Exception/{g=1;print o}
g{print}
' file
 
Old 01-22-2010, 08:06 PM   #4
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 244Reputation: 244Reputation: 244
Quote:
Originally Posted by paulsm4 View Post
PS:
Even though "awk" is a lot more powerful than "sed", I don't think it would necessarily buy you that much for this problem. You've either got a nice, straightforward text pattern (in which case "sed" should be OK), or you need procedural logic (in which case I'd recommend looking at a scripting language like Perl or Python).
awk can do the work just fine. If you are talking about pattern ranges in sed, awk can do the same. For text parsing, awk is better/or on par with Perl/Python.

Last edited by ghostdog74; 01-22-2010 at 08:09 PM.
 
Old 01-25-2010, 02:04 AM   #5
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,999

Rep: Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190
Hey Ghostdog74 ... like the above, would it be possible to explain how it works?

I am not quite sure how the setting of your f and g variables are producing the desired result?
(Hope this isn't a stupid question)
 
Old 01-25-2010, 10:35 AM   #6
jalum
LQ Newbie
 
Registered: Jan 2010
Posts: 2

Original Poster
Rep: Reputation: 0
Yes, I have been able to make some progress -
Code:
sed -n '/Exception/,/\[.* .* CST\].*$/p' SystemOut.log
This will get me everything including the Exception line and the last line containing the timestamp entry. However this is not exactly what I need.
a) I need to exclude the last line containing timestamp entry
b) I need to get the first line containing the timestamp entry, resulting in -

Code:
[1/13/10 23:01:16:623 CST] 00000059 SystemOut     O 2010-01-13 23:01:16,623 [ORB.thread.pool : 0] INFO 
- Exiting isAlertUpdateTimerRunning()
SystemOut     O 2010-01-13 23:01:16,623 [ORB.thread.pool : 0] INFO  com.dd.dddd.ddddddddr.ddddddr.dddddddddTriggerManager
org.hibernate.SessionException: Session is closed!
        at org.hibernate.impl.AbstractSessionImpl.errorIfClosed(AbstractSessionImpl.java:72)
        at org.hibernate.impl.SessionImpl.reconnect(SessionImpl.java:407)
        at com.ibm.ws.Transaction.JTA.RegisteredSyncs.distributeAfter(RegisteredSyncs.java:424)
        at com.ibm.ws.Transaction.JTA.TransactionImpl.distributeAfter(TransactionImpl.java:3885)
        at com.ibm.ws.Transaction.JTA.TransactionImpl.postCompletion(TransactionImpl.java:3864)
        at com.ibm.ws.Transaction.JTA.TransactionImpl.commitXAResources(TransactionImpl.java:2521)
        at com.ibm.ws.Transaction.JTA.TransactionImpl.stage1CommitProcessing(TransactionImpl.java:1647)
        at com.ibm.ws.Transaction.JTA.TransactionImpl.processCommit(TransactionImpl.java:1607)
        at com.ibm.ws.Transaction.JTA.TransactionImpl.commit(TransactionImpl.java:1542)
        at com.ibm.ws.Transaction.JTA.TranManagerImpl.commit(TranManagerImpl.java:240)
        at com.ibm.ws.Transaction.JTA.TranManagerSet.commit(TranManagerSet.java:164)
        at com.ibm.ejs.csi.TranStrategy.commit(TranStrategy.java:756)
        at com.ibm.ejs.csi.TranStrategy.postInvoke(TranStrategy.java:181)
        at com.ibm.ejs.csi.TransactionControlImpl.postInvoke(TransactionControlImpl.java:581)
        at com.ibm.ejs.container.EJSContainer.postInvoke(EJSContainer.java:3910)
        at com.ibm.ejs.container.EJSContainer.postInvoke(EJSContainer.java:3732)
My knowledge of awk is limited, so I much rather use sed. Thanks.

Quote:
Originally Posted by paulsm4 View Post
Hi -

The good news is that sed supports "start patterns" (along with corresponding "stop patterns"). For example:



You can read more about it here:
http://www.grymoire.com/Unix/Sed.html
<= Search for "ranges by patterns"

If you've got a clearly delimited block (for example, "start" through "stop", or "Exception" through "INFO"), then you've got it made.

Otherwise, this might be a good time to start learning Perl or Python .

'Hope that helps .. PSM

PS:
Even though "awk" is a lot more powerful than "sed", I don't think it would necessarily buy you that much for this problem. You've either got a nice, straightforward text pattern (in which case "sed" should be OK), or you need procedural logic (in which case I'd recommend looking at a scripting language like Perl or Python).
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
sed extract parameters between sections freeindy Programming 5 07-19-2009 12:51 PM
Using sed to extract a pattern plus a number of positions after roach7711x Linux - Software 20 10-31-2008 04:37 AM
using sed ( I think ) to extract names from e-mail address rosv Programming 4 07-07-2008 02:52 AM
sed command extract contents withing body tag of html Fond_of_Opensource Linux - Newbie 6 06-04-2007 07:55 AM
How to extract a part of a line by sed? J_Szucs Programming 2 02-15-2003 06:49 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 05:52 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration