LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices



Reply
 
Search this Thread
Old 09-18-2006, 10:14 AM   #1
powah
Member
 
Registered: Mar 2005
Distribution: FC, Gentoo
Posts: 276

Rep: Reputation: 30
remove \r and \n from a text file


How to remove "\r" and " \n" from a text file with a script?
e.g.
The input is:
# cat test.txt
test
it

is

$ od -txc test.txt
0000000 74736574 74690a0d 0a0d0a0d 0a0d7369
t e s t \r \n i t \r \n \r \n i s \r \n
0000020

The output should be:
$ od -txc test.txt
0000000 74736574 74690a0d 73690a0d 00000a0d
t e s t \r \n i t \r \n i s \r \n \0 \0
0000016
 
Old 09-18-2006, 10:45 AM   #2
jim mcnamara
Member
 
Registered: May 2002
Posts: 964

Rep: Reputation: 34
Your input and output are identical - \r and \n are in both.
Code:
tr -d '\r' < inputfile | tr -d '\n' > outputfile
 
Old 09-18-2006, 01:54 PM   #3
powah
Member
 
Registered: Mar 2005
Distribution: FC, Gentoo
Posts: 276

Original Poster
Rep: Reputation: 30
remove extra \r and \n from a text file

Quote:
Originally Posted by jim mcnamara
Your input and output are identical - \r and \n are in both.
Code:
tr -d '\r' < inputfile | tr -d '\n' > outputfile
To clarify, I want to remove extra \r and \n from a text file, so "\r\n\r\n" become "\r\n" but "\r\n" remain unchanged.
 
Old 09-18-2006, 02:22 PM   #4
spirit receiver
Member
 
Registered: May 2006
Location: Frankfurt, Germany
Distribution: SUSE 10.2
Posts: 424

Rep: Reputation: 33
The following command will remove all lines that contain \r\n only (and make a backup of the original file):
Code:
sed -i.bak -e '/^\r$/d' filename
 
Old 09-18-2006, 10:54 PM   #5
firstfire
Member
 
Registered: Mar 2006
Location: Ekaterinburg, Russia
Distribution: Debian, Ubuntu
Posts: 640

Rep: Reputation: 375Reputation: 375Reputation: 375Reputation: 375
Hi. Try this:
Code:
cat file | awk 'BEGIN{RS="\0";} {gsub(/(\r\n)+/,"\r\n");print}'| od -txc
 
Old 09-19-2006, 02:08 AM   #6
AnanthaP
Member
 
Registered: Jul 2004
Location: Chennai, India
Distribution: UBUNTU 5.10 since Jul-18,2006 on Intel 820 DC
Posts: 659

Rep: Reputation: 160Reputation: 160
sed s/\\r\\n\\r\\n/\\r\\n/g in a pipe line.

So a paired \r\n will become a single one.

End
 
Old 09-20-2006, 10:40 AM   #7
firstfire
Member
 
Registered: Mar 2006
Location: Ekaterinburg, Russia
Distribution: Debian, Ubuntu
Posts: 640

Rep: Reputation: 375Reputation: 375Reputation: 375Reputation: 375
Hello.
Here is an extraction of sed's info page (info sed)
Code:
3.3 Overview of Regular Expression Syntax
=========================================
 .   .   .
  `\CHAR'
     Matches CHAR, where CHAR is one of `$', `*', `.', `[', `\', or `^'.
     Note that the only C-like backslash sequences that you can
     portably assume to be interpreted are `\n' and `\\'; in particular
     `\t' is not portable, and matches a `t' under most implementations
     of `sed', rather than a tab character.
  .   .   .
 
Old 09-22-2006, 08:26 AM   #8
AnanthaP
Member
 
Registered: Jul 2004
Location: Chennai, India
Distribution: UBUNTU 5.10 since Jul-18,2006 on Intel 820 DC
Posts: 659

Rep: Reputation: 160Reputation: 160
Yes, but once ported and given that it is "visible" as \r \n (`od -c`) in a given *nix box, sed will work. That I believe is what the OP wanted.

End
 
Old 09-22-2006, 10:18 PM   #9
sundialsvcs
Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 5,455

Rep: Reputation: 1172Reputation: 1172Reputation: 1172Reputation: 1172Reputation: 1172Reputation: 1172Reputation: 1172Reputation: 1172Reputation: 1172
As usual, lots of ways to do this...

The tr command example might deserve a little closer look because it's an unusual strategy (well, not for Unix)...

In the Unix world, many programs act as filters which take some input-stream, do something to it, and write the results to an output-stream. The output-stream of one filter is often piped to another filter, becoming its input-stream. Piping is indicated by the '|' character.

In the code:
Code:
tr -d '\r' < inputfile | tr -d '\n' > outputfile
we have two instances of the tr filtering-command, separated by a pipe. So we're actually going to have two instances of the tr command running at the same time, one feeding its output as input to the other.

The first part:
Code:
tr -d '\r' < inputfile
uses the '<' operator to specify that inputfile (whatever file that is) is to be used as the input. The command writes its modified output to its standard-output stream, where it's piped to become the input to the second tr command:
Code:
tr -d '\n' > outputfile
... which uses the '>' operator to specify its output-file.

So this approach solves the problem in a very Unix-like way: by running two small, generalized programs and piping them together to solve a problem.
 
Old 10-02-2006, 07:02 AM   #10
bigearsbilly
Senior Member
 
Registered: Mar 2004
Location: england
Distribution: FreeBSD, Debian, Mint, Puppy
Posts: 3,314

Rep: Reputation: 175Reputation: 175
unix2dos, dos2unix,
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
remove some text from a file hfawzy Programming 7 07-10-2006 10:22 PM
remove text from file with script paul_mat Linux - Software 3 11-17-2005 01:21 PM
Remove odd lines from a text file Mr. Gone Programming 2 09-19-2005 12:16 PM
remove "special" text out of a file tearinox Linux - General 1 03-15-2004 09:43 PM
How to remove line of text from file netkepala Linux - General 2 05-23-2003 12:49 PM


All times are GMT -5. The time now is 10:20 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration