Extract last paragraph from text file

bunti01 · 07-15-2012, 02:16 PM

Hi,

From a text file like below, I want to extract the last paragraph (note paragraphs are separated by
“----------------------------------------------------------“ and not by blank lines)

So in example below I would like to extract only the paragraph in red

Source = [DATABASE1], Dest = [ConnectionALL], SessionID = []
Table name = [Connection]
Table size = [2]x[12]
[ColLabel] = [ConnectionID] [ClientID] [LoginTime] [NumOfSubscribedSymbol] [REQ_IMAGE] [REQ_IMAGE_UPDATE] [REQ_CLOSE] [REQ_SYMBOL] [REQ_MBO_IMAGE] [REQ_MBP_IMAGE] [REQ_MBO_MBP_IMAGE] [REQ_INVALID]
[User 1] = [1] [TEST01] [2012-07-06 00:00:09] [5] [2454] [5] [0] [1] [0] [0] [0] [0]
[User 2] = [2] [TEST01] [2012-07-06 00:00:11] [1928] [2454] [1928] [0] [1] [0] [0] [0] [0]
----------------------------------------------------------
Source = [DATABASE1], Dest = [ConnectionALL], SessionID = []
Table name = [Connection]
Table size = [2]x[12]
[ColLabel] = [ConnectionID] [ClientID] [LoginTime] [NumOfSubscribedSymbol] [REQ_IMAGE] [REQ_IMAGE_UPDATE] [REQ_CLOSE] [REQ_SYMBOL] [REQ_MBO_IMAGE] [REQ_MBP_IMAGE] [REQ_MBO_MBP_IMAGE] [REQ_INVALID]
[User 1] = [1] [TEST01] [2012-07-06 00:00:09] [5] [2454] [5] [0] [1] [0] [0] [0] [0]
[User 2] = [2] [TEST01] [2012-07-06 00:00:11] [1928] [2454] [1928] [0] [1] [0] [0] [0] [0]
----------------------------------------------------------
Source = [DATABASE1], Dest = [ConnectionALL], SessionID = []
Table name = [Connection]
Table size = [2]x[12]
[ColLabel] = [ConnectionID] [ClientID] [LoginTime] [NumOfSubscribedSymbol] [REQ_IMAGE] [REQ_IMAGE_UPDATE] [REQ_CLOSE] [REQ_SYMBOL] [REQ_MBO_IMAGE] [REQ_MBP_IMAGE] [REQ_MBO_MBP_IMAGE] [REQ_INVALID]
[User 1] = [1] [TEST01] [2012-07-06 00:00:09] [5] [2454] [5] [0] [1] [0] [0] [0] [0]
[User 2] = [2] [TEST01] [2012-07-06 00:00:11] [1928] [2454] [1928] [0] [1] [0] [0] [0] [0]
----------------------------------------------------------
Source = [DATABASE1], Dest = [ConnectionALL], SessionID = []
Table name = [Connection]
Table size = [2]x[12]
[ColLabel] = [ConnectionID] [ClientID] [LoginTime] [NumOfSubscribedSymbol] [REQ_IMAGE] [REQ_IMAGE_UPDATE] [REQ_CLOSE] [REQ_SYMBOL] [REQ_MBO_IMAGE] [REQ_MBP_IMAGE] [REQ_MBO_MBP_IMAGE] [REQ_INVALID]
[User 1] = [1] [TEST01] [2012-07-06 00:00:09] [5] [2454] [5] [0] [1] [0] [0] [0] [0]
[User 2] = [2] [TEST01] [2012-07-06 00:00:11] [1928] [2454] [1928] [0] [1] [0] [0] [0] [0]
----------------------------------------------------------

Any help appreciated

Many Thanks

tonyfreeman · 07-15-2012, 02:41 PM

If the paragraphs are all the same number of lines, then you can use the "tail" command:

Code:

tail -n 8 <filename>

.

bunti01 · 07-15-2012, 02:45 PM

Thanks, unfortunately the number of lines in each paragraph is unknown and always varies and therefore need a way to extract the last paragraph based on the paragraph seperator (-----------------)

Snark1994 · 07-15-2012, 02:54 PM

This awk program seems to do the job:

Code:

BEGIN { acc = ""; blank = 0; }
{
    if(blank == 1){ 
        acc = $0 "\n";
        blank = 0;
    } else {
        acc = acc $0 "\n";
    }   
}
/----------------------------------------------------------/ {
    blank = 1;
}
END   { print acc; }

Run it with:

Code:

awk -f awk_file.awk input_file.txt

bunti01 · 07-15-2012, 02:59 PM

That does the job thanks Snark1994.

If anyone has a sed/awk one liner which does the same thing that would also be great

firstfire · 07-15-2012, 03:16 PM

Hi.

Using ed:

Code:

$ printf "%s\n" \$k '?\-\{58\}?+1,$-1p' | ed -s infile

printf command simply prints its arguments, on separate lines:

Code:

$ printf "%s\n" \$k '?\-\{58\}?+1,$-1p'
$k
?\-\{58\}?+1,$-1p

These arguments are ed commands. First one $k makes ed to go to the last line in the file. Second one uses reverse search to find delimiter and then prints lines to end of file (skipping delimiters, note +1 and -1).

bunti01 · 07-15-2012, 03:21 PM

thanks firstfire, thats working nicely as well

bunti01 · 07-15-2012, 03:35 PM

Can anyone think of a way to do this so it fits into following usage:

cat inputfile.txt | "command goes here" > "required output generated"

Thank You

pixellany · 07-15-2012, 03:35 PM

Quote:

Originally Posted by bunti01

That does the job thanks Snark1994.

If anyone has a sed/awk one liner which does the same thing that would also be great

SED can do amazing things, but this one looks like a stretch.

firstfire · 07-15-2012, 03:40 PM

Here is sed solution:

Code:

$ sed -rn '{:a; /-{5,}/be; N; ba};  :e; $p' infile

Or you can do

Code:

$ cat infile | sed -rn '{:a; /-{5,}/be; N; ba};  :e; $p' > outfile

if you wish.

Funny, it was simple to invent this script, but it took a while to understand why it works..

bunti01 · 07-15-2012, 04:01 PM

Many thanks firstfire, that sed options works perfect.

If you get a spare moment would be great to understand what each part of the command is doing please

cheers

pixellany · 07-15-2012, 04:15 PM

That is pretty clever!!

If I read it right:
"read a new line, and then keep appending lines to the working register until we find 5 or more "-", then, if we are at the end of the file, print the accumulated record. Otherwise go back to the default collection of lines (without appending)."

this of course only works if there is a line of ------- at the end of the last record.

grail · 07-15-2012, 09:41 PM

Code:

awk 'NF{d=$0}END{print d}' RS='--+' file

firstfire · 07-15-2012, 11:28 PM

Hi.

Quote:

Originally Posted by pixellany

That is pretty clever!!

If I read it right:
"read a new line, and then keep appending lines to the working register until we find 5 or more "-", then, if we are at the end of the line, print the accumulated record. Otherwise go back the default collection of lines (without appending)."

this of course only works if there is a line of ------- at the end of the last record.

That's right, except for this part

Quote:

if we are at the end of the line

which should read

Quote:

if we are at the end of the file

because $ in the address position means last line of the file.

Somewhat ugly solution to the last problem you mentioned is as follows:

Code:

$ sed -rn '{:a; /-{5,}/be; $be; N; ba};  :e; $p' in

grail · 07-16-2012, 01:42 AM

Quote:

cat inputfile.txt | "command goes here" > "required output generated"

Most will not show you this form as it is a Useless use of cat