LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 09-14-2017, 03:10 AM   #1
Turbocapitalist
Senior Member
 
Registered: Apr 2005
Distribution: Ubuntu, Devuan, OpenBSD
Posts: 2,628
Blog Entries: 3

Rep: Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159
sed print text between two patterns inclusive, unless a third pattern is present


I'd like to use sed to find blocks of text that don't have a certain pattern. This script almost does it:

Code:
sed -n -E -e '/^Start-Date/,/^End-Date/{ N; }; /install/d; p; ' /var/log/apt/history.log
Unfortunately the first and last lines of the blocks containg the undesired pattern are still printed even if the main contents of those blocks are not. Thus instead of a blank line, I get lines like this when the undesired pattern is present:

Code:
Start-Date: 2017-09-08  22:14:04
End-Date: 2017-09-08  22:14:05
The following perl script does it, though perhaps in a clumsy way. How use sed to get the same results?

Code:
perl -0 -lne 'while( /^(?<start>Start-Date.*?)(?<entry>.*?)(?<end> ^End-Date.*?)$/xmgs ) { $entry=$+{entry}; if ($entry !~ m/install/mgs) { print $+{start}, $entry, $+{end}; print qq(\n\n);}}' /var/log/apt/history.log
 
Old 09-14-2017, 04:23 AM   #2
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 9,779

Rep: Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888
looks like the /install/d will be executed on all lines, not on the pattern space constructed by the previous /^Start-Date/,/^End-Date/{ N; };

your script will not work well with packages like flashplugin-installer
 
Old 09-14-2017, 05:06 AM   #3
Turbocapitalist
Senior Member
 
Registered: Apr 2005
Distribution: Ubuntu, Devuan, OpenBSD
Posts: 2,628
Blog Entries: 3

Original Poster
Rep: Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159
Quote:
Originally Posted by pan64 View Post
your script will not work well with packages like flashplugin-installer
No problem there. That's fixed using /^Install:/ for an elimination pattern instead.

What about the main part of the script? I've tried around two dozen variations and seem to not grasp some key concept here.

Edit:
For example:

Code:
sed -n -E -e '/^Start-Date/,/^End-Date/{ N; /^End-Date/{ /^Install/d; p;}}' /var/log/apt/history.log

Last edited by Turbocapitalist; 09-14-2017 at 05:08 AM.
 
Old 09-14-2017, 07:05 AM   #4
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 9,779

Rep: Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888
Code:
sed -n -E -e '/^Start.Date:/,/^End.Date:/{ H;d; }; H;z;x; / install /d; p;' /var/log/apt/history.log
probably this is better, but this one will not print the last one, if there was no additional line afterward.
 
Old 09-14-2017, 10:08 AM   #5
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,550

Rep: Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898Reputation: 2898
Not on the same level as pan64, but this seems to work:
Code:
sed -n '/Start-Date/,/End-Date/{H};/End-Date/{x;/Install/!p;s/.*//;x}' file
I wasn't sure how else to clear the buffer
 
1 members found this post helpful.
Old 09-14-2017, 10:12 AM   #6
Turbocapitalist
Senior Member
 
Registered: Apr 2005
Distribution: Ubuntu, Devuan, OpenBSD
Posts: 2,628
Blog Entries: 3

Original Poster
Rep: Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159
That's it. I guess it's not surprising lot of hold swapping. Thanks.
 
Old 09-15-2017, 10:21 AM   #7
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,586

Rep: Reputation: 481Reputation: 481Reputation: 481Reputation: 481Reputation: 481
Quote:
Originally Posted by Turbocapitalist View Post
I'd like to use sed to find blocks of text that don't have a certain pattern.
I'll rephrase the problem this way ...

Extract all blocks of text such that:
1) The block begins with "Start-Date".
2) The block ends with "End-Date".
3) The block does not contain "Install".

Is this correct?

Solutions will vary in complexity depending on how "messy" the input file may be. You didn't give a sample so consider this one ...
Code:
Chevrolet
Install
Start-Date: 2017-09-08  22:14:04
Should
print #1
End-Date: 2017-09-10  12:07:07
Install
Install
Start-Date: 2017-09-14  03:04:04
Should not
Install
print #99
End-Date: 2017-09-16  04:05:06
Honda
Start-Date: 2017-09-18  22:14:04
Should notprint
Start-Date: 2017-09-22  08:08:08
Should print #2
End-Date: 2017-09-24  09:09:09
Should not print
Please post the desired output resulting from this input file.

Daniel B. Martin
 
Old 09-15-2017, 10:46 AM   #8
Turbocapitalist
Senior Member
 
Registered: Apr 2005
Distribution: Ubuntu, Devuan, OpenBSD
Posts: 2,628
Blog Entries: 3

Original Poster
Rep: Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159
It's solved above in #5, but parsing the input in #7 should give this output :

Code:
Start-Date: 2017-09-08  22:14:04
Should
print #1
End-Date: 2017-09-10  12:07:07
Start-Date: 2017-09-18  22:14:04
Should notprint
Start-Date: 2017-09-22  08:08:08
Should print #2
End-Date: 2017-09-24  09:09:09
I'm not sure what to do about the Start-Date with no corresponding End-Date except skip over it. It would not naturally occur. They are always paired in the actual data.
 
Old 09-15-2017, 10:59 AM   #9
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,586

Rep: Reputation: 481Reputation: 481Reputation: 481Reputation: 481Reputation: 481
Quote:
Originally Posted by Turbocapitalist View Post
I'm not sure what to do about the Start-Date with no corresponding End-Date except skip over it. It would not naturally occur. They are always paired in the actual data.
Okay, my contrived InFile was too messy. In my (limited) real-world experience the data is not always as it should be. That leads to trouble.

Daniel B. Martin
 
Old 09-18-2017, 04:30 AM   #10
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 9,779

Rep: Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888Reputation: 2888
Why not awk?
Code:
awk 'BEGIN{RS="Start-Date"} !/Install/ { printf "Start-Date" $0 }' /var/log/apt/history.log
much more readable.
 
1 members found this post helpful.
Old 09-18-2017, 04:33 AM   #11
Turbocapitalist
Senior Member
 
Registered: Apr 2005
Distribution: Ubuntu, Devuan, OpenBSD
Posts: 2,628
Blog Entries: 3

Original Poster
Rep: Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159Reputation: 1159
Yes, awk is easier for this. It started simple and grew. There is a sweet spot for each tool and in looks like this use-case is getting close to the crossover between sed and awk

grep < sed < awk < perl
 
Old 09-21-2017, 04:22 PM   #12
MadeInGermany
Member
 
Registered: Dec 2011
Location: Simplicity
Posts: 503

Rep: Reputation: 235Reputation: 235Reputation: 235
The following sed code procuces the output of post#8 when run on the input of post#7:
Code:
sed -n '
  /Start-Date:/ {
    :Loop
    N
    /End-Date:/!b Loop
    /Install/!p
  }
' /var/log/apt/history.log
 
2 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] sed print matching pattern trscookie Programming 3 03-03-2016 03:09 AM
[SOLVED] sed/awk to print every before line that pattern match niharikaananth Linux - Newbie 10 02-22-2012 11:47 PM
How do I replace the text between patterns located on separate lines? (sed, awk, etc) Quon Programming 5 02-12-2012 07:27 AM
how can i use sed to cut out all the text up until the pattern? daweefolk Linux - Newbie 4 02-15-2011 10:17 AM
Modify the only one pattern among two patterns in a text file linuxromeo Linux - Newbie 3 11-22-2010 04:43 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 11:24 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration