LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-08-2012, 06:17 AM   #1
muzzol
LQ Newbie
 
Registered: Nov 2010
Posts: 6

Rep: Reputation: 0
Question sed: remove newline except when it's a blank line


hi,

i have a text like this:

Code:
on
two
three

four
five

six
seven
eight
i want to remove newline (\n) except when the line is blank (^$). i've tried several examples without any luck.
 
Old 02-08-2012, 08:34 AM   #2
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Arch + Xfce
Posts: 6,852

Rep: Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037
What exactly have you tried so far?

Anyway, there may be better solutions, but I got it to work this way. You use "N" to append the next line to the one in the buffer, then replace the newline between them with a space, but only if it appears between actual blocks of text. The whole thing is nested in a "t" loop in order to run the process as many times as needed.

Code:
sed -r ':a; N ;s/(.+)\n(.+)/\1 \2/; ta' file
Here are a few useful sed references.
http://www.grymoire.com/Unix/Sed.html
http://sed.sourceforge.net/grabbag/
http://sed.sourceforge.net/sedfaq.html
http://sed.sourceforge.net/sed1line.txt
 
1 members found this post helpful.
Old 02-08-2012, 09:41 AM   #3
muzzol
LQ Newbie
 
Registered: Nov 2010
Posts: 6

Original Poster
Rep: Reputation: 0
i was trying something like "if line is blank, do nothine, else remove newlines":

Code:
sed '/^$/! s/\n//g'
 
Old 02-08-2012, 09:49 AM   #4
colucix
LQ Guru
 
Registered: Sep 2003
Location: Bologna
Distribution: CentOS 6.5 OpenSuSE 12.3
Posts: 10,509

Rep: Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983Reputation: 1983
Quote:
Originally Posted by muzzol View Post
i was trying something like "if line is blank, do nothine, else remove newlines":
The newline is not inserted in the pattern space, unless you put multiple lines together with the N command, as David the H. demonstrated above.
 
Old 02-11-2012, 04:37 AM   #5
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Arch + Xfce
Posts: 6,852

Rep: Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037Reputation: 2037
As colucix said. sed commands are always applied to the current contents of the pattern buffer. One line is taken a time into this buffer (minus the delimiting newline), the commands in the expression are applied to it, and the modified contents are printed out (unless told not to). Then the contents are cleared and replaced with the next line. address ranges filter which lines get grabbed, but don't otherwise affect the basic processing sequence.

In order to operate on multiple lines at once, and the newline characters between them, you have to tell sed to store more than one line in the pattern buffer at a time. This is accomplished with the N command, and/or use of the hold buffer. "N" tells sed to append the next line of text to the current buffer, separated by a newline. This is the only time there's actually a newline in the buffer to target.

In this specific case you also need some kind of loop or hold buffer action to continue to add new lines until the blank line condition is reached, otherwise it would only process a single newline, then empty the buffer for the next line.

The hold buffer is a separate bit of swap space that can be used for temporary text storage, and there are various commands for appending to/swapping out text with the pattern buffer. sed expressions can get horribly complex with it, and trying to keep straight exactly what it's doing always makes my head hurt.

Read the grymoire link I gave earlier for more details on how it works.

Last edited by David the H.; 02-11-2012 at 04:41 AM.
 
Old 02-11-2012, 06:16 AM   #6
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,999

Rep: Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190Reputation: 3190
You could let awk at it too:
Code:
awk 'ORS=NF?"\0":RT"\n"' file
 
Old 02-12-2012, 01:52 PM   #7
muzzol
LQ Newbie
 
Registered: Nov 2010
Posts: 6

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by David the H. View Post
Read the grymoire link I gave earlier for more details on how it works.
thanks for your usefull explanation.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Sed remove trailing newline nc3b Programming 20 06-26-2015 11:02 AM
[SOLVED] remove blank line if not followed by caps dereut Linux - Software 4 11-24-2011 12:41 PM
[SOLVED] Bash; awk or sed output to variable: how keep newline at end of each output line porphyry5 Programming 3 06-10-2011 05:50 PM
Using sed to remove all but the last 17 characters on a line simplified Programming 5 06-04-2010 03:33 AM
[SOLVED] SED - how to remove blank spaces carolflb Linux - Newbie 2 01-30-2010 06:02 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 05:10 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration