Why sed is unable to remove/replace new line/'\n'

VenkatKonda · 06-26-2013, 02:26 PM

Hi there,
I am unable to replace or remove \n or new line in my source files. I will be getting 2000 files daily and I need to remove new lines from that files instantly.
Kindly suggest me workaround..

Thanks in Advance.

Venkat.

kabamaru · 06-26-2013, 03:47 PM

Although it is possible to do that with sed, you're better of with tr for the job:

Code:

tr -d '\n' < myfile > myfile.new

this will delete newlines from myfile and save the output to myfile.new.

Firerat · 06-26-2013, 04:20 PM

input

Code:

this is a line with a new line
this is a line that I would like to join
with the next line

sed

Code:

sed ':a;/would like to join$/{N;s/\n/ /;ba}' input

I cheated
http://stackoverflow.com/questions/7...lines-together
from
https://startpage.com/do/search?q=se...+match+pattern

the detail is probably in here
http://www.gnu.org/software/sed/manual/sed.html

somewhere ..

sycamorex · 06-26-2013, 05:01 PM

As much as I like all things sed, as kubamaru suggests, tr is a better tool in this situation.

Firerat · 06-26-2013, 05:19 PM

true,

but I hate googling, finding a thread that looks like what I need and is marked solved only to find the solution doesn’t quite fit what I needed.

Just trying to fix the net

Diantre · 06-26-2013, 06:39 PM

This page might be useful. Plenty of examples on how to remove newlines with several tools.

eklavya · 06-27-2013, 12:38 AM

It looks like that it should be easy with sed because we have used substitute and the replacing character so it should be
sed 's/\n/\t/g' filename or sed 's/\n/ /g' filename or sed 's/\n//g' filename
but when you will try these solutions, none of these will work because to change line you have to define all these

Code:

    :a create a label 'a'
    N append the next line to the pattern space
    $! if not the last line, ba branch (go to) label 'a'
    s substitute, /\n/ regex for new line, / / by a space, /g global match (as many times as it can)

But it does not like that if it is not done in sed, it can't be done. There are other several and much easy ways.
You have to replace new line with another character, in these examples I am replacing it with space.
If you do this using perl in a way like sed,

Code:

perl -p -e 's/\n/ /' filename.txt

with tr

Code:

tr '\n' ' ' < filename.txt

Another way with paste,
but here is some limitation, if you replace new line with one or two character(like space or tab), it is easy to use, if there are multiple charcters there can be some problem.

Code:

paste -s -d ' ' filename.txt

Madhu Desai · 06-27-2013, 03:31 AM

sed variant:

Code:

sed -i.bak ':begin;$!N;s/\n//;tbegin' file
sed -i.bak '{:q;N;s/\n//g;t q}' file

But as everybody have already mentioned, tr is easy

Code:

tr -d '\n' < file > newfile

David the H. · 06-28-2013, 07:56 AM

sed can't do this by default because of the way it processes lines. The process goes like this:

1) The first line of input is taken into the pattern buffer, with the trailing newline removed from it.

2) All command expressions are applied to the contents of the buffer, in order.

3) The modified string is printed to stdout (unless -n is used), with a new newline re-appended to it.

4) It goes on to the next line and processes that.

Since there's never a newline in the pattern buffer, there's no way to act on it.

Now, as shown above, it can be done with some of the more advanced multi-line commands. These work by appending extra lines (either the next one in the input, or the contents of the hold buffer) to the contents of the pattern buffer, with newline characters between them. Then once there are newlines in the buffer, you can operate on them as you expect.

But the code flow of sed when doing this kind of thing can be a real headache. Usually it's much easier to just use a different tool entirely.

Here are a few useful sed references:
http://www.grymoire.com/Unix/Sed.html
http://sed.sourceforge.net/grabbag/
http://sed.sourceforge.net/sedfaq.html
http://sed.sourceforge.net/sed1line.txt
http://www.catonmat.net/series/sed-one-liners-explained