Welcome to the most active Linux Forum on the web.
Go Back > Forums > Linux Forums > Linux - Newbie
User Name
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!


  Search this Thread
Old 02-04-2013, 09:30 PM   #1
LQ Newbie
Registered: Jul 2012
Posts: 8

Rep: Reputation: Disabled
Remove Control Characters from a File

I want to delete all the control characters from my file using linux bash commands.

There are some control characters like EOF (0x1A) especially which are causing the problem when I load my file in another software. I want to delete these.

Here is what I have tried so far:

this will list all the control characters:

cat -v -e -t file.txt | head -n 10

^D ^_$
This will list all the control characters using grep:

$ cat file.txt | head -n 10 | grep '[[:cntrl:]]'

matches the above output of cat command.

Now, I ran the following command to show all lines not containing control characters but it is still showing the same output as above (lines with control characters)

$ cat file.txt | head -n 10 | grep '[^[:cntrl:]]'

here is the output in hex format:

$ cat file.txt | head -n 10 | grep '[[:cntrl:]]' | od -t x2
0000000 2b01 0a18 3101 0a18 2004 0a1f 2d05 0a04
0000020 2d05 0a13 3105 0a16 2506 0a1f 2d06 0a04
0000040 2e06 0a1f 2f06 0a1f
as you can see, the hex values, 0x01, 0x18 are control characters.

I tried using the tr command to delete the control characters but it deletes \r\n also:

$ cat file.txt | tr -d "[:cntrl:]" >> test.txt

$ cat test.txt | wc -l
If I delete all control characters, I will end up deleting the newline and carriage return as well which is used as the newline character on windows.

Note: I want to delete all the control characters excluding, \r\n since they are the newline characters on windows. If I delete all the control characters then everything will be on the same line.


Last edited by NeonFlash; 02-04-2013 at 09:34 PM.
Old 02-04-2013, 09:55 PM   #2
Senior Member
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,800
Blog Entries: 4

Rep: Reputation: 286Reputation: 286Reputation: 286
Can you make a try using awk?
awk '{gsub(/[:cntrl:]/,"",$0); print $0}' file.txt
Old 02-04-2013, 10:05 PM   #3
LQ Newbie
Registered: Aug 2006
Location: Chicago IL
Distribution: To many to list.
Posts: 27

Rep: Reputation: 1
Another option may be to use sed something along the lines of

sed 's/[:cntrl:]//g' file.txt
Not 100% sure that will work.
Old 02-04-2013, 10:24 PM   #4
LQ Newbie
Registered: Jul 2012
Posts: 8

Original Poster
Rep: Reputation: Disabled
thanks for the answers, I will try them out.

However, both the above command lines will remove all the control characters. How do I exclude, \r and \n from that?


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
How to remove ^L control characters from text files? bashari Linux - Newbie 2 09-12-2012 12:36 PM
supression of control characters in linux file pandunr Linux - Newbie 3 11-24-2011 11:37 AM
How to remove file with name containing only special characters abhisheknayak Linux - Newbie 5 07-04-2008 11:53 AM
Viewing control characters in a text file dmorse Linux - General 2 01-07-2007 12:10 AM
Clearing control characters from a text file LinuxLala Linux - General 1 04-07-2006 07:45 AM > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 09:09 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration