Saving the output of "ls" command in color.

smurthygr · 11-25-2010, 07:58 AM

When I save the output of "ls" command to a "txt" or "rtf" file, the color information is lost. Is there any way to retain the color code of different files/directories obtained by the command "ls --color" while saving?

GrapefruiTgirl · 11-25-2010, 08:12 AM

You could do this, using `cat -v`:

Code:

/bin/ls --color=always | cat -v

and if you like, direct the output to a file.

colucix · 11-25-2010, 08:12 AM

Check your alias: it should be something like this:

Code:

$ alias ls
alias ls='ls --color=tty'

this means that colorized output is used only when the standard output is a terminal (tty). Hence you have simply to run an unaliased ls command. You can do either one of these:

Code:

$ \ls --color > file
$ /bin/ls --color > file

Hope this helps.

Edit: GrapefruiTgirl beat me for a bunch of seconds!

mlangdn · 11-25-2010, 08:54 AM

Didn't work for me - here's what I got:

Code:

^[[0m^[[00m#Caravan#^[[0m
^[[00m2010-11\ amended\ calendar(written).doc^[[0m
^[[00m5S_Checklist_1.pdf^[[0m
^[[00m5S_Checklist_2.pdf^[[0m
^[[00m5S_Forms.ppt^[[0m
^[[00m666Chip.pps^[[0m
^[[00mBBQInvitation.pps^[[0m
^[[00mBelkin\ Manual.pdf^[[0m
^[[00mCTSizeGuide.pdf^[[0m
^[[00mCalendar\ 2010-2011\ (7).xls^[[0m
^[[00mCamping\ List.ods^[[0m
^[[00mCaravan.odt^[[0m
^[[01;34mCheckStubs^[[0m/
^[[01;34mDLink^[[0m/
^[[01;35mDSC08259.JPG^[[0m
^[[01;34mDesktop^[[0m/

Here's what I used:

Code:

ls --color=always | cat -v > ls.txt

colucix · 11-25-2010, 08:59 AM

@mlangdn: your output is exactly what the OP was looking for. It contains the color codes as they can be interpreted by the echo statement. Look at the Bash Prompt Howto, here for an example of their usage.

GrapefruiTgirl · 11-25-2010, 09:09 AM

Hmm, yes interesting. I didn't notice that there are other ^[ control characters there which are not part of the color codes. The color codes are there, but they don't work as expected when the file is read.

This is clunky and possibly not fool-proof, but is a workaround for those extra control characters, if you want to actually see the colors when viewing the file of saved output from the command I/you used:

Code:

echo -e "$(sed 's|\^\[|\\033|g' ls.txt)"

The sed replaces the leading control chars with shell escapes (\033). Did I mention it's clunky??

GrapefruiTgirl · 11-25-2010, 09:11 AM

Quote:

Originally Posted by colucix

they can be interpreted by the echo statement.

This actually didn't work for me, leading me into the sed junk above. Does echo reproduce the original colored listing for you, from the saved file?

mlangdn · 11-25-2010, 09:34 AM

That worked. Its really nothing I need - I just did it out of boredom this morning! Happy Turkey Day!

David the H. · 11-25-2010, 09:42 AM

There's no need to use cat -v when piping output. ls color=always >file is all that's required.

For the record, colors in terminal programs are provided by ansi escape code sequences. These are non-printing characters that can tell the console how to format the output. Most programs that can provide color output are smart enough to turn it off when directed into a pipe or file, because most of the time you simply want the text itself. The existence of the embedded codes can cause havoc with further text processing tools, for example.

In addition, the program that displays the text has to be able to understand them. Your terminal can do it, as do less and some cli text editors with the proper options enabled. But gui editors like kwrite or gedit or OpenOffice generally have no ability to interpret the codes, and so they will simply appear as strings of odd characters embedded in the text.

So yes, you can save the "colors", but it may not be what you really want after all.

colucix · 11-25-2010, 09:47 AM

Quote:

Originally Posted by GrapefruiTgirl

This actually didn't work for me, leading me into the sed junk above. Does echo reproduce the original colored listing for you, from the saved file?

Nope. Indeed they have to be changed as you've shown in your previous post. Anyway, the sed statement does not actually changes the content of the file, since as you already noticed, they are hidden characters (we can easily verify this if we redirect the output to a file and see the differences). We should pipe the output of cat -v instead:

Code:

echo -e "$(cat -v ls.txt | sed 's|\^\[|\\033|g')"

The strange thing is that the ouput of the sed command alone and the output of cat without option -v is still colorized, maybe because something in the terminal causes the hidden sequences (without modifications) to be interpreted as colors:

Code:

cat ls.txt
sed -n p ls.txt
echo -e $(<ls.txt)

I cannot explain exactly why this happens.

colucix · 11-25-2010, 09:50 AM

Quote:

Originally Posted by David the H.

For the record, colors in terminal programs are provided by ansi escape code sequences. These are non-printing characters that can tell the console how to format the output.

This confirms and answers my doubt. Thank you David!

GrapefruiTgirl · 11-25-2010, 10:06 AM

Quote:

Originally Posted by colucix

The strange thing is that the ouput of the sed command alone and the output of cat without option -v is still colorized, maybe because something in the terminal causes the hidden sequences (without modifications) to be interpreted as colors:
Code:

cat ls.txt
sed -n p ls.txt
echo -e $(<ls.txt)

I cannot explain exactly why this happens.

colucix - sorry if I misunderstand what you wrote there. Can you clarify please? Do you mean that the output of all three of those commands there, for you, produce colorized output in the terminal?

colucix · 11-25-2010, 10:26 AM

Quote:

Originally Posted by GrapefruiTgirl

colucix - sorry if I misunderstand what you wrote there.

Don't worry. Actually it was me that was not clear.

Quote:

Originally Posted by GrapefruiTgirl

Can you clarify please? Do you mean that the output of all three of those commands there, for you, produce colorized output in the terminal?

Exactly. As David the H. explained the terminal (using GNOME terminal on CentOS 5.5 here) is able to interpret the ansi escape sequences as they are generated from /bin/ls --color, without further modifications. I guess it's not the same for your current terminal, is it?!

GrapefruiTgirl · 11-25-2010, 11:01 AM

That's what I thought, but no, it is not my xterm. I have discovered what is causing the difference for me - it was that I had initially included the `cat -v` in the pipline when I first saved my ls.txt file. But why it is causing the difference, is still beyond me. This is very odd, look below... I create 4 files, each containing a slightly different dump of the colored `ls` output. Two outputs have been piped through `cat -v` and dumped into a file, and two have not.:

Code:

# create 4 files:
ls -1 --color > ls1.txt
ls -1 --color=always > ls2.txt
ls -1 --color | cat -v > ls3.txt
ls -1 --color=always | cat -v > ls4.txt

# checksum shows that --color=always is the same as just using --color:
root@reactor: md5sum *.txt
e3a7cae6f12c66b7bae2dfd7c515cd21  ls1.txt
e3a7cae6f12c66b7bae2dfd7c515cd21  ls2.txt
58c07d70b9792a030ea2bde319d908a6  ls3.txt
58c07d70b9792a030ea2bde319d908a6  ls4.txt

# weird part #1: when I `cat` the files, the first two, created without the `cat -v`,
appear as colorized output in my xterm (simulated below) but the second two files appear
with control characters & stuff:
root@reactor: cat *.txt
directory
ls1.txt
ls2.txt
ls3.txt
ls4.txt
symlink
directory
ls1.txt
ls2.txt
ls3.txt
ls4.txt
symlink
^[[0m^[[04;30mdirectory^[[0m
ls1.txt
ls2.txt
ls3.txt
ls4.txt
^[[01;36msymlink^[[0m
^[[0m^[[04;30mdirectory^[[0m
ls1.txt
ls2.txt
ls3.txt
ls4.txt
^[[01;36msymlink^[[0m
root@reactor:

# weird part #2: But now look at the listing of `cat -v` on all 4 files;
I inserted a blank line between each listing for readability; notice that
the contents of the files *appear* identical:
root@reactor: cat -v *.txt

^[[0m^[[04;30mdirectory^[[0m
ls1.txt
ls2.txt
ls3.txt
ls4.txt
^[[01;36msymlink^[[0m

^[[0m^[[04;30mdirectory^[[0m
ls1.txt
ls2.txt
ls3.txt
ls4.txt
^[[01;36msymlink^[[0m

^[[0m^[[04;30mdirectory^[[0m
ls1.txt
ls2.txt
ls3.txt
ls4.txt
^[[01;36msymlink^[[0m

^[[0m^[[04;30mdirectory^[[0m
ls1.txt
ls2.txt
ls3.txt
ls4.txt
^[[01;36msymlink^[[0m
root@reactor:

So I am still a little stumped.. I see in ls output that the two files that had been `cat -v`'d have 5 more bytes than the other two:

Code:

root@reactor: ls -l
total 20
-rw-r--r-- 1 root root   78 Nov 25 12:36 ls1.txt
-rw-r--r-- 1 root root   78 Nov 25 12:36 ls2.txt
-rw-r--r-- 1 root root   83 Nov 25 12:36 ls3.txt
-rw-r--r-- 1 root root   83 Nov 25 12:36 ls4.txt
root@reactor:

but contrary to what I thought would be the case, look at `file` output:

Code:

root@reactor: file ls1.txt
ls1.txt: ASCII text, with escape sequences
root@reactor: file ls2.txt
ls2.txt: ASCII text, with escape sequences
root@reactor: file ls3.txt
ls3.txt: ASCII text
root@reactor: file ls4.txt
ls4.txt: ASCII text
root@reactor

The files that are allegedly with escape sequences, are smaller than those that are allegedly without.

What have I gotten into here?! BTW - sorry to the OP - I hope I am not dragging this off-course too much, and/or that you are also finding this educational (or at the least, interesting!)

colucix · 11-25-2010, 02:45 PM

He he... just to add more confusion, here is my output (similar but not identical) with escape sequences along with the output of od -c:

Code:

$ ls -l *.txt                                                           
-rw-r--r-- 1 alex users 120 2010-11-25 21:36 ls1.txt
-rw-r--r-- 1 alex users 120 2010-11-25 21:36 ls2.txt
-rw-r--r-- 1 alex users 134 2010-11-25 21:36 ls3.txt
-rw-r--r-- 1 alex users 134 2010-11-25 21:36 ls4.txt

$ cat -v ls1.txt
^[[0m^[[01;34mdirectory^[[0m
^[[00mls1.txt^[[0m
^[[00mls2.txt^[[0m
^[[00mls3.txt^[[0m
^[[00mls4.txt^[[0m
^[[41;33;01msymlink^[[0m
^[[m

$ cat -v ls3.txt                                                                                                
^[[0m^[[01;34mdirectory^[[0m
^[[00mls1.txt^[[0m
^[[00mls2.txt^[[0m
^[[00mls3.txt^[[0m
^[[00mls4.txt^[[0m
^[[41;33;01msymlink^[[0m
^[[m

$ od -c ls1.txt
od -c ls1.txt                                                                                                  
0000000 033   [   0   m 033   [   0   1   ;   3   4   m   d   i   r   e
0000020   c   t   o   r   y 033   [   0   m  \n 033   [   0   0   m   l
0000040   s   1   .   t   x   t 033   [   0   m  \n 033   [   0   0   m
0000060   l   s   2   .   t   x   t 033   [   0   m  \n 033   [   0   0
0000100   m   l   s   3   .   t   x   t 033   [   0   m  \n 033   [   0
0000120   0   m   l   s   4   .   t   x   t 033   [   0   m  \n 033   [
0000140   4   1   ;   3   3   ;   0   1   m   s   y   m   l   i   n   k
0000160 033   [   0   m  \n 033   [   m
0000170

$ od -c ls3.txt
0000000   ^   [   [   0   m   ^   [   [   0   1   ;   3   4   m   d   i
0000020   r   e   c   t   o   r   y   ^   [   [   0   m  \n   ^   [   [
0000040   0   0   m   l   s   1   .   t   x   t   ^   [   [   0   m  \n
0000060   ^   [   [   0   0   m   l   s   2   .   t   x   t   ^   [   [
0000100   0   m  \n   ^   [   [   0   0   m   l   s   3   .   t   x   t
0000120   ^   [   [   0   m  \n   ^   [   [   0   0   m   l   s   4   .
0000140   t   x   t   ^   [   [   0   m  \n   ^   [   [   4   1   ;   3
0000160   3   ;   0   1   m   s   y   m   l   i   n   k   ^   [   [   0
0000200   m  \n   ^   [   [   m
0000206

Unfortunately I'm too tired now to try to understand. Maybe tomorrow.... good night... zzzzzzzzzz......