[SOLVED] sed print text between two patterns inclusive, unless a third pattern is present

Turbocapitalist · 09-14-2017, 02:10 AM

I'd like to use sed to find blocks of text that don't have a certain pattern. This script almost does it:

Code:

sed -n -E -e '/^Start-Date/,/^End-Date/{ N; }; /install/d; p; ' /var/log/apt/history.log

Unfortunately the first and last lines of the blocks containg the undesired pattern are still printed even if the main contents of those blocks are not. Thus instead of a blank line, I get lines like this when the undesired pattern is present:

Code:

Start-Date: 2017-09-08  22:14:04
End-Date: 2017-09-08  22:14:05

The following perl script does it, though perhaps in a clumsy way. How use sed to get the same results?

Code:

perl -0 -lne 'while( /^(?<start>Start-Date.*?)(?<entry>.*?)(?<end> ^End-Date.*?)$/xmgs ) { $entry=$+{entry}; if ($entry !~ m/install/mgs) { print $+{start}, $entry, $+{end}; print qq(\n\n);}}' /var/log/apt/history.log

pan64 · 09-14-2017, 03:23 AM

looks like the /install/d will be executed on all lines, not on the pattern space constructed by the previous /^Start-Date/,/^End-Date/{ N; };

your script will not work well with packages like flashplugin-installer

Turbocapitalist · 09-14-2017, 04:06 AM

Quote:

Originally Posted by pan64

your script will not work well with packages like flashplugin-installer

No problem there. That's fixed using /^Install:/ for an elimination pattern instead.

What about the main part of the script? I've tried around two dozen variations and seem to not grasp some key concept here.

Edit:
For example:

Code:

sed -n -E -e '/^Start-Date/,/^End-Date/{ N; /^End-Date/{ /^Install/d; p;}}' /var/log/apt/history.log

pan64 · 09-14-2017, 06:05 AM

Code:

sed -n -E -e '/^Start.Date:/,/^End.Date:/{ H;d; }; H;z;x; / install /d; p;' /var/log/apt/history.log

probably this is better, but this one will not print the last one, if there was no additional line afterward.

grail · 09-14-2017, 09:08 AM

Not on the same level as pan64, but this seems to work:

Code:

sed -n '/Start-Date/,/End-Date/{H};/End-Date/{x;/Install/!p;s/.*//;x}' file

I wasn't sure how else to clear the buffer

Turbocapitalist · 09-14-2017, 09:12 AM

That's it. I guess it's not surprising lot of hold swapping. Thanks.

danielbmartin · 09-15-2017, 09:21 AM

Quote:

Originally Posted by Turbocapitalist

I'd like to use sed to find blocks of text that don't have a certain pattern.

I'll rephrase the problem this way ...

Extract all blocks of text such that:
1) The block begins with "Start-Date".
2) The block ends with "End-Date".
3) The block does not contain "Install".

Is this correct?

Solutions will vary in complexity depending on how "messy" the input file may be. You didn't give a sample so consider this one ...

Code:

Chevrolet
Install
Start-Date: 2017-09-08  22:14:04
Should
print #1
End-Date: 2017-09-10  12:07:07
Install
Install
Start-Date: 2017-09-14  03:04:04
Should not
Install
print #99
End-Date: 2017-09-16  04:05:06
Honda
Start-Date: 2017-09-18  22:14:04
Should notprint
Start-Date: 2017-09-22  08:08:08
Should print #2
End-Date: 2017-09-24  09:09:09
Should not print

Please post the desired output resulting from this input file.

Daniel B. Martin

Turbocapitalist · 09-15-2017, 09:46 AM

It's solved above in #5, but parsing the input in #7 should give this output :

Code:

Start-Date: 2017-09-08  22:14:04
Should
print #1
End-Date: 2017-09-10  12:07:07
Start-Date: 2017-09-18  22:14:04
Should notprint
Start-Date: 2017-09-22  08:08:08
Should print #2
End-Date: 2017-09-24  09:09:09

I'm not sure what to do about the Start-Date with no corresponding End-Date except skip over it. It would not naturally occur. They are always paired in the actual data.

danielbmartin · 09-15-2017, 09:59 AM

Quote:

Originally Posted by Turbocapitalist

I'm not sure what to do about the Start-Date with no corresponding End-Date except skip over it. It would not naturally occur. They are always paired in the actual data.

Okay, my contrived InFile was too messy. In my (limited) real-world experience the data is not always as it should be. That leads to trouble.

Daniel B. Martin

pan64 · 09-18-2017, 03:30 AM

Why not awk?

Code:

awk 'BEGIN{RS="Start-Date"} !/Install/ { printf "Start-Date" $0 }' /var/log/apt/history.log

much more readable.

Turbocapitalist · 09-18-2017, 03:33 AM

Yes, awk is easier for this. It started simple and grew. There is a sweet spot for each tool and in looks like this use-case is getting close to the crossover between sed and awk

grep < sed < awk < perl

MadeInGermany · 09-21-2017, 03:22 PM

The following sed code procuces the output of post#8 when run on the input of post#7:

Code:

sed -n '
  /Start-Date:/ {
    :Loop
    N
    /End-Date:/!b Loop
    /Install/!p
  }
' /var/log/apt/history.log