-   Linux - Newbie (
-   -   difference in cat and vi output (

keshavk 07-31-2009 04:40 AM

difference in cat and vi output
I'm copying a output to a file. This output is from a terminal.
When I'm opening this file using cat and vi I'm getting diferent output.


output of cat is like:
This is your output

while in vi editor output is like:

^M This is your output
There are a lot of unwanted character in the file like this

can anybody tell me why it is happening, and how to filter these unwanted character?
I've tried sed its unable to read vi output.

Thanks in advance.


EricTRA 07-31-2009 04:44 AM


Type at console

file yourfilename
to give you more information about the file itself. I have had these characters in files in the past also, but only when I copied a file from Windows to Linux.

dos2unix yourfilename
could solve the problem.

Kind regards,


karamarisan 07-31-2009 04:53 AM

It might be worth telling us what's producing that output. The only thing I can think of (and only because I was reading another thread about this earlier today) is if you're running ls and that's aliased to `ls --color=always` somewhere, vim won't know what to do with the escape sequences that cause text to be colored, but cat will.

Also, dos2unix probably won't help. This is a Linux program producing the output, no? And besides, I wasn't aware dos2unix did anything besides convert CR/LF line terminators to LF.

keshavk 08-01-2009 06:12 AM

Thanks for your feedback...
The thing is this is the outputs of a linux terminal rather then window, so dos2unix, not helping...

When i checked the file type, the coming filetype is:

: ASCII English text, with very long lines, with CRLF, CR line terminators, with escape sequences

Any idea how to remove these unwanted character,
Thanks in advance..


joeBuffer 08-01-2009 01:19 PM

This happens when you use the script command, the output file (default typescript) is exactly like what you're describing. I don't have any strange settings at all, that's just how it is.

file typescript
typescript: ASCII text, with CRLF, LF line terminators, with escape sequences
I don't know which programs would do that ...

keshavk 08-02-2009 12:06 PM

Any idea how to remove CRLF, CR line terminators, from a file in linux.


pwc101 08-02-2009 12:12 PM


man dos2unix

Or, if that doesn't work:

man fromdos

catkin 08-02-2009 01:31 PM

Hello keshavk :)

Your problem is impossible to solve completely. It has been asked many times before in connection with the script command and terminal spying programs. What you have is the sequence of bytes that went to the terminal; what you want to get is a text version of what appeared on the terminal. That is fundamentally impossible; the terminal is a dynamic device while your text file is static.

As long as nothing complex happened in the terminal session (switch to full screen mode followed by user interaction, command recall, command editing, title setting etc) then a "good enough" solution is possible -- filtering out the bytes that controlled colour, cursor position, rang bells etc -- which may render the text sufficiently readable for your purposes.

An alternative approach -- which has the advantage of not being fundamentally impossible -- would be to replay the captured bytes by feeding them to a terminal thus reproducing the session.



gregorian 08-02-2009 01:32 PM

If cat is working for you, just type cat filename > out.txt. You'll get your results in the text file. Hope this helps.

joeBuffer 08-02-2009 01:43 PM

Yeah, when I read catkin's post and thought about what you were saying just now, I was going to suggest cat, also.
It would work, with redirection. It works for me, when using the typescript files.

stanlo45 08-03-2009 12:34 AM

This might help with removing "^M" from a file with vi/vim
Type in the following in vi



keshavk 08-03-2009 01:05 AM

Hi Stan,
Thanks for your helped me in removing many junk...
rest i'll try to solve.


SwaJime 08-03-2009 01:14 PM

Filtering escape sequences from a file
Why are you using vi? Are you trying to edit the file? Or are you trying to view the file?

If you are only trying to view the file, then you can use "less -fr" and the color sequences will be put to use.

To remove the escape sequences, you can use this:


$ cat
# Copyright SwaJime's Cove, 2009; all rights reserved

# Based on data obtained from
                                # A control sequence starts with
CSI='\x1b\['                        # the two-character csi escape sequence Esc [, followed by
Rp='[0-9:;<=>?]'                # an optional sequence of parameter bytes in the range x30x3F,
Ri='[- !\"#$%&'\''()*+,./]'        # an optional sequence of intermediate bytes in the range x20x2F,
Rf='[]@A-Z[\\^_`a-z{|}~]'        # and a final byte in the range x40x7e.
                                # The set of standard control sequence functions are defined in Ecma-48 / ISO/IEC 6429.

ls --color=always > myFile.txt
cat myFile.txt | sed -e 's/'"$CSI$Rp"'*'"$Ri"'*'"$Rf"'//g'

karamarisan 08-03-2009 01:21 PM


Originally Posted by SwaJime (Post 3629889)

# Copyright SwaJime's Cove, 2009; all rights reserved

Good of you to provide code, but that's not really in the spirit of this place, is it? :)

kunstkopf 08-04-2009 09:24 AM

In Unix, the character transliteration tool 'tr' exists
almost from the beginning.

To remove CR=13(dec)=15(oct):

tr -d '\015' < infile.txt > outfile.txt

To remove LF=NL=10(dec)=12(oct)

tr -d '\012' < infile.txt > outfile.txt

To remove both:

tr -d '\015\012' < infile.txt > outfile.txt

... all just from the command line, no programming. You can also transliterate
other characters, of course, for instance a->d, b->e, c->f by

tr 'a-c' 'd-f' < infile.txt > outfile.txt

In Linux, there will be mnemonics for meaningful character
subsets like '[:cntrl:]' and '[: punct:]'

All times are GMT -5. The time now is 06:55 PM.