LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 07-31-2009, 04:40 AM   #1
keshavk
LQ Newbie
 
Registered: Jul 2009
Posts: 5

Rep: Reputation: 0
Post difference in cat and vi output


Hi,
I'm copying a output to a file. This output is from a terminal.
When I'm opening this file using cat and vi I'm getting diferent output.

e.g.

output of cat is like:
This is your output

while in vi editor output is like:

^M^[[3;0H
^M This is your output
^M^[[4;0H
There are a lot of unwanted character in the file like this

can anybody tell me why it is happening, and how to filter these unwanted character?
I've tried sed its unable to read vi output.

Thanks in advance.

Kesh
 
Old 07-31-2009, 04:44 AM   #2
EricTRA
Guru
 
Registered: May 2009
Location: Gibraltar, Gibraltar
Distribution: Fedora 20 with Awesome WM
Posts: 6,805
Blog Entries: 1

Rep: Reputation: 1290Reputation: 1290Reputation: 1290Reputation: 1290Reputation: 1290Reputation: 1290Reputation: 1290Reputation: 1290Reputation: 1290
Hello,

Type at console
Code:
file yourfilename
to give you more information about the file itself. I have had these characters in files in the past also, but only when I copied a file from Windows to Linux.
Executing
Code:
dos2unix yourfilename
could solve the problem.

Kind regards,

Eric
 
Old 07-31-2009, 04:53 AM   #3
karamarisan
Member
 
Registered: Jul 2009
Location: Illinois, US
Distribution: Fedora 11
Posts: 374

Rep: Reputation: 55
It might be worth telling us what's producing that output. The only thing I can think of (and only because I was reading another thread about this earlier today) is if you're running ls and that's aliased to `ls --color=always` somewhere, vim won't know what to do with the escape sequences that cause text to be colored, but cat will.

Also, dos2unix probably won't help. This is a Linux program producing the output, no? And besides, I wasn't aware dos2unix did anything besides convert CR/LF line terminators to LF.
 
Old 08-01-2009, 06:12 AM   #4
keshavk
LQ Newbie
 
Registered: Jul 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Hi,
Thanks for your feedback...
The thing is this is the outputs of a linux terminal rather then window, so dos2unix, not helping...

When i checked the file type, the coming filetype is:

: ASCII English text, with very long lines, with CRLF, CR line terminators, with escape sequences

Any idea how to remove these unwanted character,
Thanks in advance..

K.
 
Old 08-01-2009, 01:19 PM   #5
joeBuffer
Member
 
Registered: Jul 2009
Distribution: Ubuntu 9.04
Posts: 328

Rep: Reputation: 42
This happens when you use the script command, the output file (default typescript) is exactly like what you're describing. I don't have any strange settings at all, that's just how it is.
Quote:
file typescript
typescript: ASCII text, with CRLF, LF line terminators, with escape sequences
I don't know which programs would do that ...

Last edited by joeBuffer; 08-01-2009 at 01:21 PM.
 
Old 08-02-2009, 12:06 PM   #6
keshavk
LQ Newbie
 
Registered: Jul 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Hi,
Any idea how to remove CRLF, CR line terminators, from a file in linux.

K.
 
Old 08-02-2009, 12:12 PM   #7
pwc101
Senior Member
 
Registered: Oct 2005
Location: UK
Distribution: Slackware
Posts: 1,847

Rep: Reputation: 128Reputation: 128
See:

man dos2unix

Or, if that doesn't work:

man fromdos
 
Old 08-02-2009, 01:31 PM   #8
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,551
Blog Entries: 28

Rep: Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176
Hello keshavk

Your problem is impossible to solve completely. It has been asked many times before in connection with the script command and terminal spying programs. What you have is the sequence of bytes that went to the terminal; what you want to get is a text version of what appeared on the terminal. That is fundamentally impossible; the terminal is a dynamic device while your text file is static.

As long as nothing complex happened in the terminal session (switch to full screen mode followed by user interaction, command recall, command editing, title setting etc) then a "good enough" solution is possible -- filtering out the bytes that controlled colour, cursor position, rang bells etc -- which may render the text sufficiently readable for your purposes.

An alternative approach -- which has the advantage of not being fundamentally impossible -- would be to replay the captured bytes by feeding them to a terminal thus reproducing the session.

Best

Charles

Last edited by catkin; 08-02-2009 at 01:33 PM. Reason: Typo
 
Old 08-02-2009, 01:32 PM   #9
gregorian
Member
 
Registered: Apr 2006
Location: India
Posts: 473

Rep: Reputation: 33
If cat is working for you, just type cat filename > out.txt. You'll get your results in the text file. Hope this helps.
 
Old 08-02-2009, 01:43 PM   #10
joeBuffer
Member
 
Registered: Jul 2009
Distribution: Ubuntu 9.04
Posts: 328

Rep: Reputation: 42
Yeah, when I read catkin's post and thought about what you were saying just now, I was going to suggest cat, also.
It would work, with redirection. It works for me, when using the typescript files.

Last edited by joeBuffer; 08-03-2009 at 06:09 AM.
 
Old 08-03-2009, 12:34 AM   #11
stanlo45
LQ Newbie
 
Registered: Jul 2008
Location: JHB, South Africa
Distribution: Suse
Posts: 4
Blog Entries: 1

Rep: Reputation: 0
This might help with removing "^M" from a file with vi/vim
Type in the following in vi

:%s/<ctrl>v<ctrl>m//g<ENTER>

Stan
 
Old 08-03-2009, 01:05 AM   #12
keshavk
LQ Newbie
 
Registered: Jul 2009
Posts: 5

Original Poster
Rep: Reputation: 0
Hi Stan,
Thanks for your help.....it helped me in removing many junk...
rest i'll try to solve.

K.
 
Old 08-03-2009, 01:14 PM   #13
SwaJime
LQ Newbie
 
Registered: May 2009
Distribution: Ubuntu, CentOS, Redhat, Maemo
Posts: 10

Rep: Reputation: 1
Lightbulb Filtering escape sequences from a file

Why are you using vi? Are you trying to edit the file? Or are you trying to view the file?

If you are only trying to view the file, then you can use "less -fr" and the color sequences will be put to use.

To remove the escape sequences, you can use this:

Code:
$ cat dobit.sh 
#!/bin/bash
#
# Copyright SwaJime's Cove, 2009; all rights reserved
#

# Based on data obtained from http://www.gnu.org/software/teseq/manual/html_node/Escape-Sequence-Recognition.html
#
                                # A control sequence starts with
CSI='\x1b\['			# the two-character csi escape sequence Esc [, followed by
Rp='[0-9:;<=>?]'		# an optional sequence of parameter bytes in the range x30x3F,
Ri='[- !\"#$%&'\''()*+,./]'	# an optional sequence of intermediate bytes in the range x20x2F,
Rf='[]@A-Z[\\^_`a-z{|}~]'	# and a final byte in the range x40x7e.
				# The set of standard control sequence functions are defined in Ecma-48 / ISO/IEC 6429. 

ls --color=always > myFile.txt
cat myFile.txt | sed -e 's/'"$CSI$Rp"'*'"$Ri"'*'"$Rf"'//g'
 
Old 08-03-2009, 01:21 PM   #14
karamarisan
Member
 
Registered: Jul 2009
Location: Illinois, US
Distribution: Fedora 11
Posts: 374

Rep: Reputation: 55
Quote:
Originally Posted by SwaJime View Post
Code:
# Copyright SwaJime's Cove, 2009; all rights reserved
Good of you to provide code, but that's not really in the spirit of this place, is it?
 
Old 08-04-2009, 09:24 AM   #15
kunstkopf
LQ Newbie
 
Registered: Mar 2009
Posts: 4

Rep: Reputation: 0
transliterate

In Unix, the character transliteration tool 'tr' exists
almost from the beginning.

To remove CR=13(dec)=15(oct):

tr -d '\015' < infile.txt > outfile.txt

To remove LF=NL=10(dec)=12(oct)

tr -d '\012' < infile.txt > outfile.txt

To remove both:

tr -d '\015\012' < infile.txt > outfile.txt

... all just from the command line, no programming. You can also transliterate
other characters, of course, for instance a->d, b->e, c->f by

tr 'a-c' 'd-f' < infile.txt > outfile.txt

In Linux, there will be mnemonics for meaningful character
subsets like '[:cntrl:]' and '[: punct:]'

Last edited by kunstkopf; 08-04-2009 at 09:38 AM. Reason: put in some more general info on tr and stressed it is a command line tool
 
  


Reply

Tags
characters, command, line, remove, substitute, tool, tr


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
What's the difference between dd and cat > ? kimchee411 Linux - Newbie 2 02-27-2009 06:28 PM
Dar and cat: Input/output error jrdioko Linux - Software 0 03-29-2006 04:56 PM
what's the difference between cat /proc/cpuinfo and x86info jayeola Linux - Hardware 1 12-15-2005 07:45 PM
Cat input output error zaicheke Slackware 15 12-26-2004 09:08 PM
output to a file - cat? grep? Godsmacker777 Linux - Newbie 6 12-08-2004 10:06 AM


All times are GMT -5. The time now is 02:23 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration