[SOLVED] Display lines before and after string until blank line

cosminel · 12-15-2013, 12:59 AM

Hello,

I have some date formatted in this way:

Code:

line of text
line of text

line of text
line of text
line of text
STRING
line of text
line of text

line of text
line of text
line of text

line of text
line of text

What I want:

- display lines before STRING until blank line and include it
- display lines after STRING until 2nd blank line and include it

The result I'm after is this:

Code:

 
line of text
line of text
line of text
STRING
line of text
line of text

line of text
line of text
line of text

Thank you in advance!

druuna · 12-15-2013, 03:31 AM

Try this:

Code:

awk 'BEGIN{ FS="\n" ; RS="\n\n" ; x = "0" } /STRING/ { x = $0 ; next } x !~ /0/ { printf("\n%s\n\n%s\n", x, $0) ; x="0" }' input

Example run:

Code:

 $ awk 'BEGIN{ FS="\n" ; RS="\n\n" ; x = "0" }/STRING/ { x = $0 ; next }x !~ /0/ { printf("\n%s\n\n%s\n", x, $0) ; x="0" }' input 

line of text
line of text
line of text
STRING
line of text
line of text

line of text
line of text
line of text

cosminel · 12-15-2013, 03:53 AM

Hi druuna, thank you for your help.

I just tried your command on the example I provided here and it worked.

But when I tried the command on my test file, it doesn't display anything.

I suspect the command doesn't work on my actual data stream maybe because the blank lines I'm seeing are not really blank lines?

I am actually trying to pipe tcpdump output to your awk command. In the output I see those blank lines and if I copy the output and paste it into a file and read it with vi, I can still see the blank lines. But when I view the file in Notepad for example, the blank lines are gone and everything is a compact block of text.

Maybe I should specify a certain parameter in tcpdump in order to inject real blank lines between each packet output.

Any thoughts?

cosminel · 12-15-2013, 04:30 AM

Upon further testing I found that it may have something to do with the output of the data stream, not the blank lines. I am testing a simplified version and adding to it from my original data stream in order to find at what point the awk command stops working.

druuna · 12-15-2013, 04:40 AM

The input used is important and without the actual output it is hard to help you.

You mention tcpdump, can you post the exact command used and an example of the actual output generated? Do use code tags to preserve the layout.

cosminel · 12-15-2013, 04:48 AM

Found the issue. Will create another post so that druuna will be alerted by incoming new post.

cosminel · 12-15-2013, 06:09 AM

I just found what causes awk to stop displaying the results. Whenever a "0" is present in any of the lines: line with the STRING, line before STRING, line after STRING, awk will stop printing the whole block of lines between "before 1st blank line" and "after 2nd blank line" conditions. So, if I modify my original example by adding a 0 at any lines:

Code:

line of text
line of text

line of text
line of text
line of text 0
STRING
line of text
line of text

line of text
line of text
line of text

line of text
line of text

...and run your command, it will not work. May have something to do with

Code:

x = "0"

?

I must confess it is very hard for me to read awk syntax even if I know what it does in this case.

druuna · 12-15-2013, 06:42 AM

Troubleshooting without answers to the questions I asked in post #5 will be hard.

Without knowing any details, try this:

Code:

awk 'BEGIN{ FS="\n" ; RS="\n\n" ; x = "_Token" } /STRING/ { x = $0 ; next } x !~ /_Token/ { printf("\n%s\n\n%s\n", x, $0) ; x="_Token" }' input

Code:

$ cat input
line of text0
line of text1

line of text0
line of text1
line of text2
STRING
line of text0
line of text1

line of text0
line of text1
line of text2

line of text3
line of text0

$ $ awk 'BEGIN{ FS="\n" ; RS="\n\n" ; x = "_Token" } /STRING/ { x = $0 ; next } x !~ /_Token/ { printf("\n%s\n\n%s\n", x, $0) ; x="_Token" }' input

line of text0
line of text1
line of text2
STRING
line of text0
line of text1

line of text0
line of text1
line of text2

druuna · 12-15-2013, 07:15 AM

BTW: Here's a breakdown of the awk command:

Code:

BEGIN{ FS="\n" ; RS="\n\n" ; x = "_Token" }

This sets the field separator (FS) to a carriage return (normally a space) and the record separator (RS) to 2 carriage returns (normally a carriage return). An empty line is now seen as the record separator and everything on 1 line is 1 field.

I also fill x with a unique token, the 0 used earlier isn't unique.

Code:

/STRING/ { x = $0 ; next }

This part fills x with the complete record when STRING is found in that record. next makes sure that the rest of the code isn't executed for the current record.

Code:

x !~ /_Token/ { printf("\n%s\n\n%s\n", x, $0) ; x="_Token" }

If x does not equal _Token then the previous record is printed (x holds the previous record) and the current record is printed. x is reset to the unique token.

cosminel · 12-15-2013, 07:25 AM

Thank you so much, it works now.

Sorry I didn't provide the actual output but I thought I should save you from doing all the work and see why awk did not print the lines and get back directly with the answer.

Now there is one more tiny issue: it seems that if there is a 2nd blank line right after the 1st one, your awk command doesn't see it and will continue to print the lines after the blank double-lines. This happens only for the lines after STRING, it seems that the command doesn't count the 2nd blank line if it is right after the 1st.

grail · 12-15-2013, 08:13 AM

I feel we could be over thinking this one a little:

Code:

awk '/STRING/{printf $0 RS;getline;print}' RS="\n\n" file

And to answer your question if there are multiple blank lines, it stands to reason as currently we have the RS set to 2 consecutive newlines.

druuna · 12-15-2013, 08:21 AM

@grail, nice and small but it doesn't address the double blank lines (which, as you mentioned, is handled correct in your and my previous code).

Based on my previous replies:

Code:

awk '
BEGIN{ FS="\n" ; RS="\n\n" ; x = "_Token" }
/STRING/ { x = $0 ; next }
$1 == "" { printf("%s\n", x) ;  x = "_Token" }
x !~ /_Token/ { printf("\n%s\n\n%s\n\n", x, $0) ; x="_Token" }
' input

cosminel · 12-15-2013, 08:55 AM

Thank you both for your input.

Is there a way to address the double blank lines issue?

Tried with grail's command, the result is the same, it only works as intended if there are no double blank lines.

grail · 12-15-2013, 09:06 AM

How about taking what you have been given and trying to come up with a solution to your new problem.
Current solutions solve for initial problem so it should not take a big change to overcome the new requirement

I would add that my version seemed to work fine by placing a second empty line after the record with STRING in it

druuna · 12-15-2013, 09:15 AM

Quote:

Originally Posted by cosminel

Is there a way to address the double blank lines issue?

Have you tried the solution in post #12?

Input

Code:

line of text
line of text

line of text
line of text
line of text 0
STRING
line of text
line of text

line of text want
line of text want
line of text want

line of text do not want
line of text do not want

line of text 2
line of text 2

line of text 2
line of text 2
line of text 0
STRING 2
line of text 2
line of text 2


line of text do not want 2
line of text do not want 2
line of text wdo not ant 2

line of text do not want 2
line of text do not want 2

Output

Code:

line of text
line of text
line of text 0
STRING
line of text
line of text

line of text want
line of text want
line of text want

line of text 2
line of text 2
line of text 0
STRING 2
line of text 2
line of text 2