ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Hi,
I'm trying to extract stacktraces from log files, looking for the pattern "Exception". If a line contains "Exception", a stack trace will follow in multiple lines. If the next line begins with a timestamp entry in the format "[1/13/10 23:17:00:444 CST]", the stack trace has ended in the previous line. Additionally, I want to grab the nearest line containing a timestamp entry above the original line that contained the pattern "Exception". Sometimes it's on the same line and other times it's a few lines above it. How I do write a sed command for this? Example:
Code:
[1/13/10 23:01:16:623 CST] 00000059 SystemOut O 2010-01-13 23:01:16,623 [ORB.thread.pool : 0] INFO
- Exiting isAlertUpdateTimerRunning()
SystemOut O 2010-01-13 23:01:16,623 [ORB.thread.pool : 0] INFO com.dd.dddd.ddddddddr.ddddddr.dddddddddTriggerManager
org.hibernate.SessionException: Session is closed!
at org.hibernate.impl.AbstractSessionImpl.errorIfClosed(AbstractSessionImpl.java:72)
at org.hibernate.impl.SessionImpl.reconnect(SessionImpl.java:407)
at com.ibm.ws.Transaction.JTA.RegisteredSyncs.distributeAfter(RegisteredSyncs.java:424)
at com.ibm.ws.Transaction.JTA.TransactionImpl.distributeAfter(TransactionImpl.java:3885)
at com.ibm.ws.Transaction.JTA.TransactionImpl.postCompletion(TransactionImpl.java:3864)
at com.ibm.ws.Transaction.JTA.TransactionImpl.commitXAResources(TransactionImpl.java:2521)
at com.ibm.ws.Transaction.JTA.TransactionImpl.stage1CommitProcessing(TransactionImpl.java:1647)
at com.ibm.ws.Transaction.JTA.TransactionImpl.processCommit(TransactionImpl.java:1607)
at com.ibm.ws.Transaction.JTA.TransactionImpl.commit(TransactionImpl.java:1542)
at com.ibm.ws.Transaction.JTA.TranManagerImpl.commit(TranManagerImpl.java:240)
at com.ibm.ws.Transaction.JTA.TranManagerSet.commit(TranManagerSet.java:164)
at com.ibm.ejs.csi.TranStrategy.commit(TranStrategy.java:756)
at com.ibm.ejs.csi.TranStrategy.postInvoke(TranStrategy.java:181)
at com.ibm.ejs.csi.TransactionControlImpl.postInvoke(TransactionControlImpl.java:581)
at com.ibm.ejs.container.EJSContainer.postInvoke(EJSContainer.java:3910)
at com.ibm.ejs.container.EJSContainer.postInvoke(EJSContainer.java:3732)
[1/13/10 23:01:16:725 CST] 0000006e SystemOut O 2010-01-13 23:01:16,725 [MessageListenerThreadPool : 12] INFO com.ddconnector.cccccc.ccccccSession - Th
read[MessageListenerThreadPool : 12,5,main]
Here, I want to grab the first line, followed by the stack trace line starting "org.hibernate.SessionException" all the way until the last line of the stack trace.
The good news is that sed supports "start patterns" (along with corresponding "stop patterns"). For example:
Quote:
sed '/start/,/stop/ s/#.*//'
<= This example deletes every line beginning with a "#" for every line between the first occurrence of "start" through the next occurrence of the word "stop"
If you've got a clearly delimited block (for example, "start" through "stop", or "Exception" through "INFO"), then you've got it made.
Otherwise, this might be a good time to start learning Perl or Python .
'Hope that helps .. PSM
PS:
Even though "awk" is a lot more powerful than "sed", I don't think it would necessarily buy you that much for this problem. You've either got a nice, straightforward text pattern (in which case "sed" should be OK), or you need procedural logic (in which case I'd recommend looking at a scripting language like Perl or Python).
PS:
Even though "awk" is a lot more powerful than "sed", I don't think it would necessarily buy you that much for this problem. You've either got a nice, straightforward text pattern (in which case "sed" should be OK), or you need procedural logic (in which case I'd recommend looking at a scripting language like Perl or Python).
awk can do the work just fine. If you are talking about pattern ranges in sed, awk can do the same. For text parsing, awk is better/or on par with Perl/Python.
Last edited by ghostdog74; 01-22-2010 at 08:09 PM.
sed -n '/Exception/,/\[.* .* CST\].*$/p' SystemOut.log
This will get me everything including the Exception line and the last line containing the timestamp entry. However this is not exactly what I need.
a) I need to exclude the last line containing timestamp entry
b) I need to get the first line containing the timestamp entry, resulting in -
Code:
[1/13/10 23:01:16:623 CST] 00000059 SystemOut O 2010-01-13 23:01:16,623 [ORB.thread.pool : 0] INFO
- Exiting isAlertUpdateTimerRunning()
SystemOut O 2010-01-13 23:01:16,623 [ORB.thread.pool : 0] INFO com.dd.dddd.ddddddddr.ddddddr.dddddddddTriggerManager
org.hibernate.SessionException: Session is closed!
at org.hibernate.impl.AbstractSessionImpl.errorIfClosed(AbstractSessionImpl.java:72)
at org.hibernate.impl.SessionImpl.reconnect(SessionImpl.java:407)
at com.ibm.ws.Transaction.JTA.RegisteredSyncs.distributeAfter(RegisteredSyncs.java:424)
at com.ibm.ws.Transaction.JTA.TransactionImpl.distributeAfter(TransactionImpl.java:3885)
at com.ibm.ws.Transaction.JTA.TransactionImpl.postCompletion(TransactionImpl.java:3864)
at com.ibm.ws.Transaction.JTA.TransactionImpl.commitXAResources(TransactionImpl.java:2521)
at com.ibm.ws.Transaction.JTA.TransactionImpl.stage1CommitProcessing(TransactionImpl.java:1647)
at com.ibm.ws.Transaction.JTA.TransactionImpl.processCommit(TransactionImpl.java:1607)
at com.ibm.ws.Transaction.JTA.TransactionImpl.commit(TransactionImpl.java:1542)
at com.ibm.ws.Transaction.JTA.TranManagerImpl.commit(TranManagerImpl.java:240)
at com.ibm.ws.Transaction.JTA.TranManagerSet.commit(TranManagerSet.java:164)
at com.ibm.ejs.csi.TranStrategy.commit(TranStrategy.java:756)
at com.ibm.ejs.csi.TranStrategy.postInvoke(TranStrategy.java:181)
at com.ibm.ejs.csi.TransactionControlImpl.postInvoke(TransactionControlImpl.java:581)
at com.ibm.ejs.container.EJSContainer.postInvoke(EJSContainer.java:3910)
at com.ibm.ejs.container.EJSContainer.postInvoke(EJSContainer.java:3732)
My knowledge of awk is limited, so I much rather use sed. Thanks.
Quote:
Originally Posted by paulsm4
Hi -
The good news is that sed supports "start patterns" (along with corresponding "stop patterns"). For example:
If you've got a clearly delimited block (for example, "start" through "stop", or "Exception" through "INFO"), then you've got it made.
Otherwise, this might be a good time to start learning Perl or Python .
'Hope that helps .. PSM
PS:
Even though "awk" is a lot more powerful than "sed", I don't think it would necessarily buy you that much for this problem. You've either got a nice, straightforward text pattern (in which case "sed" should be OK), or you need procedural logic (in which case I'd recommend looking at a scripting language like Perl or Python).
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.