LinuxQuestions.org - sed command to extract stacktraces

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - sed command to extract stacktraces (https://www.linuxquestions.org/questions/programming-9/sed-command-to-extract-stacktraces-784082/)

sed command to extract stacktraces

Hi,
I'm trying to extract stacktraces from log files, looking for the pattern "Exception". If a line contains "Exception", a stack trace will follow in multiple lines. If the next line begins with a timestamp entry in the format "[1/13/10 23:17:00:444 CST]", the stack trace has ended in the previous line. Additionally, I want to grab the nearest line containing a timestamp entry above the original line that contained the pattern "Exception". Sometimes it's on the same line and other times it's a few lines above it. How I do write a sed command for this? Example:

Code:

[1/13/10 23:01:16:623 CST] 00000059 SystemOut    O 2010-01-13 23:01:16,623 [ORB.thread.pool : 0] INFO  

 - Exiting isAlertUpdateTimerRunning()

SystemOut    O 2010-01-13 23:01:16,623 [ORB.thread.pool : 0] INFO  com.dd.dddd.ddddddddr.ddddddr.dddddddddTriggerManager

org.hibernate.SessionException: Session is closed!

        at org.hibernate.impl.AbstractSessionImpl.errorIfClosed(AbstractSessionImpl.java:72)

        at org.hibernate.impl.SessionImpl.reconnect(SessionImpl.java:407)

        at com.ibm.ws.Transaction.JTA.RegisteredSyncs.distributeAfter(RegisteredSyncs.java:424)

        at com.ibm.ws.Transaction.JTA.TransactionImpl.distributeAfter(TransactionImpl.java:3885)

        at com.ibm.ws.Transaction.JTA.TransactionImpl.postCompletion(TransactionImpl.java:3864)

        at com.ibm.ws.Transaction.JTA.TransactionImpl.commitXAResources(TransactionImpl.java:2521)

        at com.ibm.ws.Transaction.JTA.TransactionImpl.stage1CommitProcessing(TransactionImpl.java:1647)

        at com.ibm.ws.Transaction.JTA.TransactionImpl.processCommit(TransactionImpl.java:1607)

        at com.ibm.ws.Transaction.JTA.TransactionImpl.commit(TransactionImpl.java:1542)

        at com.ibm.ws.Transaction.JTA.TranManagerImpl.commit(TranManagerImpl.java:240)

        at com.ibm.ws.Transaction.JTA.TranManagerSet.commit(TranManagerSet.java:164)

        at com.ibm.ejs.csi.TranStrategy.commit(TranStrategy.java:756)

        at com.ibm.ejs.csi.TranStrategy.postInvoke(TranStrategy.java:181)

        at com.ibm.ejs.csi.TransactionControlImpl.postInvoke(TransactionControlImpl.java:581)

        at com.ibm.ejs.container.EJSContainer.postInvoke(EJSContainer.java:3910)

        at com.ibm.ejs.container.EJSContainer.postInvoke(EJSContainer.java:3732)

[1/13/10 23:01:16:725 CST] 0000006e SystemOut    O 2010-01-13 23:01:16,725 [MessageListenerThreadPool : 12] INFO  com.ddconnector.cccccc.ccccccSession - Th

read[MessageListenerThreadPool : 12,5,main]

Here, I want to grab the first line, followed by the stack trace line starting "org.hibernate.SessionException" all the way until the last line of the stack trace.

Thanks.

Hi -

The good news is that sed supports "start patterns" (along with corresponding "stop patterns"). For example:

Quote:

sed '/start/,/stop/ s/#.*//'
<= This example deletes every line beginning with a "#" for every line between the first occurrence of "start" through the next occurrence of the word "stop"

You can read more about it here:
http://www.grymoire.com/Unix/Sed.html
<= Search for "ranges by patterns"

If you've got a clearly delimited block (for example, "start" through "stop", or "Exception" through "INFO"), then you've got it made.

Otherwise, this might be a good time to start learning Perl or Python ;).

'Hope that helps .. PSM

PS:
Even though "awk" is a lot more powerful than "sed", I don't think it would necessarily buy you that much for this problem. You've either got a nice, straightforward text pattern (in which case "sed" should be OK), or you need procedural logic (in which case I'd recommend looking at a scripting language like Perl or Python).

@OP, you did not show your desired output, so here's a guess of what you want.

Code:

awk '/^\[/{f=1;g=0;o=$0}

f && /Exception/{g=1;print o}

g{print}

' file

Quote:

Originally Posted by paulsm4 (Post 3836956)

PS:
Even though "awk" is a lot more powerful than "sed", I don't think it would necessarily buy you that much for this problem. You've either got a nice, straightforward text pattern (in which case "sed" should be OK), or you need procedural logic (in which case I'd recommend looking at a scripting language like Perl or Python).

awk can do the work just fine. If you are talking about pattern ranges in sed, awk can do the same. For text parsing, awk is better/or on par with Perl/Python.

Hey Ghostdog74 ... like the above, would it be possible to explain how it works?

I am not quite sure how the setting of your f and g variables are producing the desired result?
(Hope this isn't a stupid question)

Yes, I have been able to make some progress -

Code:

sed -n '/Exception/,/\[.* .* CST\].*$/p' SystemOut.log

This will get me everything including the Exception line and the last line containing the timestamp entry. However this is not exactly what I need.
a) I need to exclude the last line containing timestamp entry
b) I need to get the first line containing the timestamp entry, resulting in -

Code:

[1/13/10 23:01:16:623 CST] 00000059 SystemOut    O 2010-01-13 23:01:16,623 [ORB.thread.pool : 0] INFO 

- Exiting isAlertUpdateTimerRunning()

SystemOut    O 2010-01-13 23:01:16,623 [ORB.thread.pool : 0] INFO  com.dd.dddd.ddddddddr.ddddddr.dddddddddTriggerManager

org.hibernate.SessionException: Session is closed!

        at org.hibernate.impl.AbstractSessionImpl.errorIfClosed(AbstractSessionImpl.java:72)

        at org.hibernate.impl.SessionImpl.reconnect(SessionImpl.java:407)

        at com.ibm.ws.Transaction.JTA.RegisteredSyncs.distributeAfter(RegisteredSyncs.java:424)

        at com.ibm.ws.Transaction.JTA.TransactionImpl.distributeAfter(TransactionImpl.java:3885)

        at com.ibm.ws.Transaction.JTA.TransactionImpl.postCompletion(TransactionImpl.java:3864)

        at com.ibm.ws.Transaction.JTA.TransactionImpl.commitXAResources(TransactionImpl.java:2521)

        at com.ibm.ws.Transaction.JTA.TransactionImpl.stage1CommitProcessing(TransactionImpl.java:1647)

        at com.ibm.ws.Transaction.JTA.TransactionImpl.processCommit(TransactionImpl.java:1607)

        at com.ibm.ws.Transaction.JTA.TransactionImpl.commit(TransactionImpl.java:1542)

        at com.ibm.ws.Transaction.JTA.TranManagerImpl.commit(TranManagerImpl.java:240)

        at com.ibm.ws.Transaction.JTA.TranManagerSet.commit(TranManagerSet.java:164)

        at com.ibm.ejs.csi.TranStrategy.commit(TranStrategy.java:756)

        at com.ibm.ejs.csi.TranStrategy.postInvoke(TranStrategy.java:181)

        at com.ibm.ejs.csi.TransactionControlImpl.postInvoke(TransactionControlImpl.java:581)

        at com.ibm.ejs.container.EJSContainer.postInvoke(EJSContainer.java:3910)

        at com.ibm.ejs.container.EJSContainer.postInvoke(EJSContainer.java:3732)

My knowledge of awk is limited, so I much rather use sed. Thanks.

Quote:

Originally Posted by paulsm4 (Post 3836956)

Hi -

The good news is that sed supports "start patterns" (along with corresponding "stop patterns"). For example:

You can read more about it here:
http://www.grymoire.com/Unix/Sed.html
<= Search for "ranges by patterns"

If you've got a clearly delimited block (for example, "start" through "stop", or "Exception" through "INFO"), then you've got it made.

Otherwise, this might be a good time to start learning Perl or Python ;).

'Hope that helps .. PSM

PS:
Even though "awk" is a lot more powerful than "sed", I don't think it would necessarily buy you that much for this problem. You've either got a nice, straightforward text pattern (in which case "sed" should be OK), or you need procedural logic (in which case I'd recommend looking at a scripting language like Perl or Python).