LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 03-27-2012, 08:50 AM   #1
vikas027
Senior Member
 
Registered: May 2007
Location: Sydney
Distribution: RHEL, CentOS, Ubuntu, Debian, OS X
Posts: 1,305

Rep: Reputation: 107Reputation: 107
Question SED/AWK - Delete all lines until empty line is found after pattern match


Dear All,

I am sitting on a CentOS 5.4 server and trying to delete all lines in a file until a blank line is found after a string is matched.

For E.g. I have file 123
Code:
$ cat 123
hi

#hello
how
are

break

hellos
one two three \
five \
six seven

eight
nine ten
eleven
hello

abcd
efgh
hello
commuted

one two three
I need to modify this file as below; such that all lines are deleted after matching string hello until a blank line is found (not delete the blank line)
Code:
hi


break


eight
nine ten
eleven


abcd
efgh


one two three
I have googled and tried to match lines after string hello until a blank line is found.
Code:
$ sed -n '/hello/,/^$/p' 123
But I need just inverse of it.

Any suggestions ? Thanks in advance.
 
Click here to see the post LQ members have rated as the most helpful post in this thread.
Old 03-27-2012, 09:03 AM   #2
vikas027
Senior Member
 
Registered: May 2007
Location: Sydney
Distribution: RHEL, CentOS, Ubuntu, Debian, OS X
Posts: 1,305

Original Poster
Rep: Reputation: 107Reputation: 107
Hi All,

I have found a partial solution.

I redirected the file and use the awk one liner but it DOES NOT preserves blank lines
Code:
$ sed -n '/hello/,/^$/p' 123 > 456
$ awk 'FNR==NR{f[$1];next}(!($1 in f)) ' 456 123
hi
break
eight
nine ten
eleven
abcd
efgh
 
Old 03-27-2012, 09:19 AM   #3
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,676

Rep: Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007
How about:
Code:
awk '/hello/{while(getline && $0 != ""){}}1' 123
 
2 members found this post helpful.
Old 03-27-2012, 09:36 AM   #4
firstfire
Member
 
Registered: Mar 2006
Location: Ekaterinburg, Russia
Distribution: Debian, Ubuntu
Posts: 709

Rep: Reputation: 427Reputation: 427Reputation: 427Reputation: 427Reputation: 427
Hi.

Code:
$ sed  '/hello/,/^$/{/^$/!d}' test.dat
hi


break


eight
nine ten
eleven

abcd
efgh

one two three
 
Old 03-27-2012, 09:37 AM   #5
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,722

Rep: Reputation: 569Reputation: 569Reputation: 569Reputation: 569Reputation: 569Reputation: 569
Quote:
Originally Posted by grail View Post
Code:
awk '/hello/{while(getline && $0 != ""){}}1' 123
I admire this concise and loop-less solution. It would help awk-learners such as me to have a few words of explanation. Thanks!

Daniel B. Martin
 
Old 03-27-2012, 09:54 AM   #6
vikas027
Senior Member
 
Registered: May 2007
Location: Sydney
Distribution: RHEL, CentOS, Ubuntu, Debian, OS X
Posts: 1,305

Original Poster
Rep: Reputation: 107Reputation: 107
Thumbs up Fantastic

Quote:
Originally Posted by grail View Post
How about:
Code:
awk '/hello/{while(getline && $0 != ""){}}1' 123
Awesome Grail.
You have been my saviour number of times. Many many thanks for this perfect one liner.
 
Old 03-27-2012, 10:00 AM   #7
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,676

Rep: Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007
@Daniel - sorry about that ... forget some times as just come up with solution and move on (my bad)

Ultimately the while loop must have an action associated so a blank set of braces will execute nothing for each loop. As the loop ends when it finds a line that is empty, when it leaves
the loop the 1 at the end will also print this line (which the user wanted)

Please let me know if you need further information and thanks again for the reminder
 
1 members found this post helpful.
Old 03-27-2012, 11:23 AM   #8
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,722

Rep: Reputation: 569Reputation: 569Reputation: 569Reputation: 569Reputation: 569Reputation: 569
Quote:
Originally Posted by grail View Post
Please let me know if you need further information and thanks again for the reminder
This is helpful. Allow me to rephrase and elaborate the operation of the awk and then you add/correct as need be.
Code:
awk '/hello/{while(getline && $0 != ""){}}1' 123
The awk spins down the "123" input file, printing each line as it goes, until it encounters a line containing the search string hello. At that point it enters a while loop inside the {braced clause}. Each iteration of the while uses a getline to read another line from "123" and tests the newly-read line with $0 != "" to see if it is *not* a null string. If not a null string, it executes a no-op (the empty braces). If a null string, it leaves the braced clause, prints the line, and resumes searching for hello.

My Dougherty & Robbins "sed & awk" book doesn't tell about getline. In context, it reads the next line. How and why is the getline and-ed with the logical result of $0 != ""?

Daniel B. Martin
 
Old 03-27-2012, 12:49 PM   #9
firstfire
Member
 
Registered: Mar 2006
Location: Ekaterinburg, Russia
Distribution: Debian, Ubuntu
Posts: 709

Rep: Reputation: 427Reputation: 427Reputation: 427Reputation: 427Reputation: 427
Hi, Daniel.

Quote from `info gawk getline':
Quote:
The `getline' command returns one if it finds a record and zero if
it encounters the end of the file. If there is some error in getting a
record, such as a file that cannot be opened, then `getline' returns
-1. In this case, `gawk' sets the variable `ERRNO' to a string
describing the error that occurred.
So the while loop stops either at end of file or at empty line.
 
1 members found this post helpful.
Old 03-27-2012, 05:31 PM   #10
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,722

Rep: Reputation: 569Reputation: 569Reputation: 569Reputation: 569Reputation: 569Reputation: 569
Quote:
Originally Posted by firstfire View Post
So the while loop stops either at end of file or at empty line.
Thank you, firstfire, that makes sense.

Please consider this input file which is the same as that posted by OP but for two lines added at the end.
Code:
how
are

break

hellos
one two three \
five \
six seven

eight
nine ten
eleven
hello

abcd
efgh
hello
commuted

one two three
hello
bogus
OP wants to suppress all lines beginning with one containing hello and up to but not including a blank line. The last record (bogus) should not appear in the output but it does. Am I being too picky?

Daniel B. Martin
 
Old 03-27-2012, 08:11 PM   #11
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 911Reputation: 911Reputation: 911Reputation: 911Reputation: 911Reputation: 911Reputation: 911Reputation: 911
Quote:
Originally Posted by danielbmartin View Post
Thank you, firstfire, that makes sense.

Please consider this input file which is the same as that posted by OP but for two lines added at the end.
Code:
how
are

break

hellos
one two three \
five \
six seven

eight
nine ten
eleven
hello

abcd
efgh
hello
commuted

one two three
hello
bogus
OP wants to suppress all lines beginning with one containing hello and up to but not including a blank line. The last record (bogus) should not appear in the output but it does. Am I being too picky?

Daniel B. Martin
But there's no blank line after bogus, which is not the OPs spec for deletion...
 
1 members found this post helpful.
Old 03-27-2012, 08:20 PM   #12
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,722

Rep: Reputation: 569Reputation: 569Reputation: 569Reputation: 569Reputation: 569Reputation: 569
Quote:
Originally Posted by Tinkster View Post
But there's no blank line after bogus, which is not the OPs spec for deletion...
You are right. I interpret the OP's requirement in an algorithmic fashion:
1) Start suppressing printing whenever you see hello.
2) Stop suppressing printing whenever you see a blank line.

There was no assurance that each and every hello had a blank line "partner."

This is another grain of evidence which shows that writing an airtight spec is more difficult than it might seem.

Daniel B. Martin
 
Old 03-28-2012, 01:42 AM   #13
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,676

Rep: Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007Reputation: 3007
Quote:
There was no assurance that each and every hello had a blank line "partner."
Actually I would disagree with your assumption, due to the following entries:
Quote:
I am sitting on a CentOS 5.4 server and trying to delete all lines in a file until a blank line is found after a string is matched.

# and

such that all lines are deleted after matching string hello until a blank line is found (not delete the blank line)
The second infers that not only is there a blank matching line but that it is not to be removed.

The other point to remember is that any programming is only ever as good as the information supplied, hence based on example data and information the current script is a worthy solution.
Of course anything could be added to the data to cause anyone one solution to fail.
 
1 members found this post helpful.
Old 03-28-2012, 08:33 AM   #14
danielbmartin
Senior Member
 
Registered: Apr 2010
Location: Apex, NC, USA
Distribution: Mint 17.3
Posts: 1,722

Rep: Reputation: 569Reputation: 569Reputation: 569Reputation: 569Reputation: 569Reputation: 569
Quote:
Originally Posted by grail View Post
... anything could be added to the data to cause anyone one solution to fail.
You are right, and I'm not trying to be annoying. Real-world data often has defects in format or content. It's impossible to code for every possible "glitch" but it's also a mistake to imagine there will be none.

Daniel B. Martin
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] sed/awk to print every before line that pattern match niharikaananth Linux - Newbie 10 02-22-2012 10:47 PM
how to delete lines according to a pattern in awk Massimo34 Programming 4 01-26-2012 03:19 PM
[SOLVED] use sed in bash to match pattern contained in 2 lines ghantauke Linux - Newbie 3 03-16-2011 10:34 AM
How to use sed to delete all lines before the first match of a pattern? C_Blade Linux - Newbie 9 05-01-2010 04:18 AM
simple pattern match with awk, sed alenD Linux - Newbie 10 03-10-2008 02:31 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 03:19 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration