LinuxQuestions.org
Help answer threads with 0 replies.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 11-27-2009, 05:29 PM   #1
the4thamigo_uk
LQ Newbie
 
Registered: Nov 2009
Posts: 6

Rep: Reputation: 0
xargs and filenames with quotes


I wanted to take the output of 'ls' and pipe it into xargs to execute a command for each filename. It doesnt much matter which command I wanted to run so for example here I will just 'echo' the filenames to serve as a trivial example :

Create a test directory with a filename containing a single quote :

$mkdir amigo\'s_folder

Now run the example command for all files in the folder

$ls | xargs 'echo {}'
xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option

As you can see I get the error message shown above. This is because the filename containing a single ' character confuses xargs which treats it as a special character. Using the '-0' option does not appear to solve the problem.

Is there a command in linux that will take an input string and escape any special characters found within it. e.g a command called 'esc' that I could use as follows :

$ls | esc | xargs 'echo {}'

Or is there a better solution?
 
Old 11-27-2009, 10:49 PM   #2
GrapefruiTgirl
LQ Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 556Reputation: 556Reputation: 556Reputation: 556Reputation: 556Reputation: 556
xargs with single quote in filename

Using `find` and -exec seem to work. I figured this out based on http://www.linuxquestions.org/questi...e-quote-99830/ which was the second result Google gave me when I searched for "single quote with xargs". Here's an example:

Code:
sasha@reactor:~/test$ ls | xargs 'echo {}'
xargs: unmatched single quote; by default quotes are special to xargs unless you use the -0 option
sasha@reactor:~/test$ find ./ -type d -exec echo "{}" \;   
./
./amigo's_folder
sasha@reactor:~/test$
Hope this helps you and welcome to LQ!

Sasha

Last edited by GrapefruiTgirl; 11-27-2009 at 10:51 PM. Reason: clarity
 
Old 11-28-2009, 09:45 AM   #3
H_TeXMeX_H
LQ Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301
To pipe find to xargs without having problems do:

Code:
find . -print0 | xargs -0 echo
The '-print0' and '-0' options allow null terminating characters to delimit input and output, this avoiding any problems with whitespace or special characters.
 
Old 11-29-2009, 04:52 AM   #4
the4thamigo_uk
LQ Newbie
 
Registered: Nov 2009
Posts: 6

Original Poster
Rep: Reputation: 0
Solution?

Thanks for your replies but I think ive got an answer to my question :

First of all, the output of 'ls' is delimited by new line characters i.e. '\n'. Its a bit confusing because it seems that bash, by default, ignores them for ls and displays the results left to right. To prove this do the following :

$ls > ls.txt
$cat ls.txt

You will see that cat interprets the \n's as new lines

Is this valid reasoning?


So, given this we can use the delimiter option '-d' of xargs :

$ls | xargs -d\\n echo

Here note that since bash itself would escape '\n' as a literal 'n' character we need to make sure we pass the literal string '\n' to the command, therefore we must escape the backslash.
 
Old 11-29-2009, 05:02 AM   #5
GrapefruiTgirl
LQ Guru
 
Registered: Dec 2006
Location: underground
Distribution: Slackware64
Posts: 7,594

Rep: Reputation: 556Reputation: 556Reputation: 556Reputation: 556Reputation: 556Reputation: 556
Hi!

Not sure I understand your reasoning in the first half of your post. Pretty much everything is terminated line-by-line with \n characters. Bash of course displays stuff L to R (unless you're using a R to L language), and, bash isn't ignoring the characters, it's interpreting them, as newlines. So this is normal behavior.

For the second half of your post: The real answer is, "Does it work? Using the -d option?", and if so, then you've got s solution that involves less typing than the previous suggestions

xargs is something I don't use much at all; I should try to use it more, is it's pretty handy.

Sasha
 
Old 11-30-2009, 04:03 AM   #6
the4thamigo_uk
LQ Newbie
 
Registered: Nov 2009
Posts: 6

Original Poster
Rep: Reputation: 0
ls

Well, you can see what I mean by doing a hex dump of the output of ls :

For the usual row and column listing using 'ls' you get :

andy@esprimo-ubuntu:~$ ls
Desktop Downloads Music projects Templates Videos
Documents examples.desktop Pictures Public test wxsmith

..and you can display this is a single column using 'ls -1' ... i.e.

andy@esprimo-ubuntu:~$ ls -1
Desktop
Documents
Downloads
examples.desktop
Music
Pictures
projects
Public
Templates
test
Videos
wxsmith


However, when you inspect the output for control characters both versions are byte-by-byte identical :

00000000 44 65 73 6b 74 6f 70 0a 44 6f 63 75 6d 65 6e 74 |Desktop.Document|
00000010 73 0a 44 6f 77 6e 6c 6f 61 64 73 0a 65 78 61 6d |s.Downloads.exam|
00000020 70 6c 65 73 2e 64 65 73 6b 74 6f 70 0a 4d 75 73 |ples.desktop.Mus|
00000030 69 63 0a 50 69 63 74 75 72 65 73 0a 70 72 6f 6a |ic.Pictures.proj|
00000040 65 63 74 73 0a 50 75 62 6c 69 63 0a 54 65 6d 70 |ects.Public.Temp|
00000050 6c 61 74 65 73 0a 74 65 73 74 0a 56 69 64 65 6f |lates.test.Video|
00000060 73 0a 77 78 73 6d 69 74 68 0a |s.wxsmith.|
0000006a
andy@esprimo-ubuntu:~$ ls -1 | hd
00000000 44 65 73 6b 74 6f 70 0a 44 6f 63 75 6d 65 6e 74 |Desktop.Document|
00000010 73 0a 44 6f 77 6e 6c 6f 61 64 73 0a 65 78 61 6d |s.Downloads.exam|
00000020 70 6c 65 73 2e 64 65 73 6b 74 6f 70 0a 4d 75 73 |ples.desktop.Mus|
00000030 69 63 0a 50 69 63 74 75 72 65 73 0a 70 72 6f 6a |ic.Pictures.proj|
00000040 65 63 74 73 0a 50 75 62 6c 69 63 0a 54 65 6d 70 |ects.Public.Temp|
00000050 6c 61 74 65 73 0a 74 65 73 74 0a 56 69 64 65 6f |lates.test.Video|
00000060 73 0a 77 78 73 6d 69 74 68 0a |s.wxsmith.|
0000006a

Therefore strange as it sounds, it must be bash that is interpreting the output?
 
Old 11-30-2009, 05:27 AM   #7
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Debian
Posts: 8,578
Blog Entries: 31

Rep: Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208Reputation: 1208
Quote:
Originally Posted by the4thamigo_uk View Post
Therefore strange as it sounds, it must be bash that is interpreting the output?
How intiguing!
Code:
c:~$ ls | od > /tmp/trash
c:~$ ls -1 | od > /tmp/trash1
c:~$ diff /tmp/trash /tmp/trash1
[no output]
Alternatively ls may behave differently depending on the output device. Here's a couple of excerpts from the ls man page suggesting that ls does adapt to the output device
Code:
-w, --width=COLS
    assume screen width instead of current value

...

With --color=auto, color codes are output only if standard output is connected to a terminal (tty)

Last edited by catkin; 11-30-2009 at 05:27 AM. Reason: Cleaning, polishing, keeping the sub-editors happy ....
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Problems with quotes and double quotes Andruha Slackware 6 01-02-2010 04:44 PM
xargs removes single quotes after awk Poki Linux - Newbie 4 06-26-2009 09:25 AM
Using xargs SBN Linux - General 2 04-07-2008 07:44 PM
`wc`, `xargs` h/w Programming 11 09-30-2007 04:22 PM
Using single quotes vs double quotes in PHP strings vharishankar Programming 6 07-11-2005 11:41 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 07:33 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration