LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 12-15-2013, 01:59 AM   #1
cosminel
Member
 
Registered: Sep 2013
Posts: 31

Rep: Reputation: Disabled
Display lines before and after string until blank line


Hello,

I have some date formatted in this way:
Code:
line of text
line of text

line of text
line of text
line of text
STRING
line of text
line of text

line of text
line of text
line of text

line of text
line of text
What I want:

- display lines before STRING until blank line and include it
- display lines after STRING until 2nd blank line and include it

The result I'm after is this:

Code:
 
line of text
line of text
line of text
STRING
line of text
line of text

line of text
line of text
line of text
Thank you in advance!

Last edited by cosminel; 12-15-2013 at 02:00 AM.
 
Old 12-15-2013, 04:31 AM   #2
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Try this:
Code:
awk 'BEGIN{ FS="\n" ; RS="\n\n" ; x = "0" } /STRING/ { x = $0 ; next } x !~ /0/ { printf("\n%s\n\n%s\n", x, $0) ; x="0" }' input
Example run:
Code:
 $ awk 'BEGIN{ FS="\n" ; RS="\n\n" ; x = "0" }/STRING/ { x = $0 ; next }x !~ /0/ { printf("\n%s\n\n%s\n", x, $0) ; x="0" }' input 

line of text
line of text
line of text
STRING
line of text
line of text

line of text
line of text
line of text
 
Old 12-15-2013, 04:53 AM   #3
cosminel
Member
 
Registered: Sep 2013
Posts: 31

Original Poster
Rep: Reputation: Disabled
Hi druuna, thank you for your help.

I just tried your command on the example I provided here and it worked.

But when I tried the command on my test file, it doesn't display anything.

I suspect the command doesn't work on my actual data stream maybe because the blank lines I'm seeing are not really blank lines?

I am actually trying to pipe tcpdump output to your awk command. In the output I see those blank lines and if I copy the output and paste it into a file and read it with vi, I can still see the blank lines. But when I view the file in Notepad for example, the blank lines are gone and everything is a compact block of text.

Maybe I should specify a certain parameter in tcpdump in order to inject real blank lines between each packet output.

Any thoughts?

Last edited by cosminel; 12-15-2013 at 04:54 AM.
 
Old 12-15-2013, 05:30 AM   #4
cosminel
Member
 
Registered: Sep 2013
Posts: 31

Original Poster
Rep: Reputation: Disabled
Upon further testing I found that it may have something to do with the output of the data stream, not the blank lines. I am testing a simplified version and adding to it from my original data stream in order to find at what point the awk command stops working.
 
Old 12-15-2013, 05:40 AM   #5
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
The input used is important and without the actual output it is hard to help you.

You mention tcpdump, can you post the exact command used and an example of the actual output generated? Do use code tags to preserve the layout.
 
Old 12-15-2013, 05:48 AM   #6
cosminel
Member
 
Registered: Sep 2013
Posts: 31

Original Poster
Rep: Reputation: Disabled
Found the issue. Will create another post so that druuna will be alerted by incoming new post.

Last edited by cosminel; 12-15-2013 at 06:58 AM.
 
Old 12-15-2013, 07:09 AM   #7
cosminel
Member
 
Registered: Sep 2013
Posts: 31

Original Poster
Rep: Reputation: Disabled
I just found what causes awk to stop displaying the results. Whenever a "0" is present in any of the lines: line with the STRING, line before STRING, line after STRING, awk will stop printing the whole block of lines between "before 1st blank line" and "after 2nd blank line" conditions. So, if I modify my original example by adding a 0 at any lines:

Code:
line of text
line of text

line of text
line of text
line of text 0
STRING
line of text
line of text

line of text
line of text
line of text

line of text
line of text
...and run your command, it will not work. May have something to do with
Code:
x = "0"
?

I must confess it is very hard for me to read awk syntax even if I know what it does in this case.

Last edited by cosminel; 12-15-2013 at 07:22 AM.
 
Old 12-15-2013, 07:42 AM   #8
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Troubleshooting without answers to the questions I asked in post #5 will be hard.

Without knowing any details, try this:
Code:
awk 'BEGIN{ FS="\n" ; RS="\n\n" ; x = "_Token" } /STRING/ { x = $0 ; next } x !~ /_Token/ { printf("\n%s\n\n%s\n", x, $0) ; x="_Token" }' input
Code:
$ cat input
line of text0
line of text1

line of text0
line of text1
line of text2
STRING
line of text0
line of text1

line of text0
line of text1
line of text2

line of text3
line of text0

$ $ awk 'BEGIN{ FS="\n" ; RS="\n\n" ; x = "_Token" } /STRING/ { x = $0 ; next } x !~ /_Token/ { printf("\n%s\n\n%s\n", x, $0) ; x="_Token" }' input

line of text0
line of text1
line of text2
STRING
line of text0
line of text1

line of text0
line of text1
line of text2
 
Old 12-15-2013, 08:15 AM   #9
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
BTW: Here's a breakdown of the awk command:

Code:
BEGIN{ FS="\n" ; RS="\n\n" ; x = "_Token" }
This sets the field separator (FS) to a carriage return (normally a space) and the record separator (RS) to 2 carriage returns (normally a carriage return). An empty line is now seen as the record separator and everything on 1 line is 1 field.

I also fill x with a unique token, the 0 used earlier isn't unique.
Code:
/STRING/ { x = $0 ; next }
This part fills x with the complete record when STRING is found in that record. next makes sure that the rest of the code isn't executed for the current record.

Code:
x !~ /_Token/ { printf("\n%s\n\n%s\n", x, $0) ; x="_Token" }
If x does not equal _Token then the previous record is printed (x holds the previous record) and the current record is printed. x is reset to the unique token.
 
1 members found this post helpful.
Old 12-15-2013, 08:25 AM   #10
cosminel
Member
 
Registered: Sep 2013
Posts: 31

Original Poster
Rep: Reputation: Disabled
Thank you so much, it works now.

Sorry I didn't provide the actual output but I thought I should save you from doing all the work and see why awk did not print the lines and get back directly with the answer.

Now there is one more tiny issue: it seems that if there is a 2nd blank line right after the 1st one, your awk command doesn't see it and will continue to print the lines after the blank double-lines. This happens only for the lines after STRING, it seems that the command doesn't count the 2nd blank line if it is right after the 1st.
 
Old 12-15-2013, 09:13 AM   #11
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,256

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
I feel we could be over thinking this one a little:
Code:
awk '/STRING/{printf $0 RS;getline;print}' RS="\n\n" file
And to answer your question if there are multiple blank lines, it stands to reason as currently we have the RS set to 2 consecutive newlines.

Last edited by grail; 12-15-2013 at 09:15 AM.
 
Old 12-15-2013, 09:21 AM   #12
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
@grail, nice and small but it doesn't address the double blank lines (which, as you mentioned, is handled correct in your and my previous code).

Based on my previous replies:
Code:
awk '
BEGIN{ FS="\n" ; RS="\n\n" ; x = "_Token" }
/STRING/ { x = $0 ; next }
$1 == "" { printf("%s\n", x) ;  x = "_Token" }
x !~ /_Token/ { printf("\n%s\n\n%s\n\n", x, $0) ; x="_Token" }
' input
 
Old 12-15-2013, 09:55 AM   #13
cosminel
Member
 
Registered: Sep 2013
Posts: 31

Original Poster
Rep: Reputation: Disabled
Thank you both for your input.

Is there a way to address the double blank lines issue?

Tried with grail's command, the result is the same, it only works as intended if there are no double blank lines.
 
Old 12-15-2013, 10:06 AM   #14
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,256

Rep: Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686Reputation: 2686
How about taking what you have been given and trying to come up with a solution to your new problem.
Current solutions solve for initial problem so it should not take a big change to overcome the new requirement

I would add that my version seemed to work fine by placing a second empty line after the record with STRING in it

Last edited by grail; 12-15-2013 at 10:10 AM.
 
Old 12-15-2013, 10:15 AM   #15
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Quote:
Originally Posted by cosminel View Post
Is there a way to address the double blank lines issue?
Have you tried the solution in post #12?

Input
Code:
line of text
line of text

line of text
line of text
line of text 0
STRING
line of text
line of text

line of text want
line of text want
line of text want

line of text do not want
line of text do not want

line of text 2
line of text 2

line of text 2
line of text 2
line of text 0
STRING 2
line of text 2
line of text 2


line of text do not want 2
line of text do not want 2
line of text wdo not ant 2

line of text do not want 2
line of text do not want 2
Output
Code:
line of text
line of text
line of text 0
STRING
line of text
line of text

line of text want
line of text want
line of text want

line of text 2
line of text 2
line of text 0
STRING 2
line of text 2
line of text 2
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] sed to display the pattern string, the line above it and the first line of that para rockie321 Linux - Newbie 3 04-03-2011 03:48 PM
how to read a blank line ie a string of length 0. ishandutta2007 Programming 3 04-20-2010 06:13 AM
How to display a file, omitting lines that contain a string? Thelionroars Linux - Newbie 12 01-22-2010 08:03 AM
How to grep lines containing a certain string PLUS the line following that line? kmkocot Linux - Newbie 5 09-01-2009 04:54 PM
vim: mark end-of-line? trim blank lines? prell Linux - Software 3 09-21-2004 12:04 AM


All times are GMT -5. The time now is 08:41 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration