how to handle file names with space characters in a shell script?

Sammy_G · 10-10-2008, 08:16 PM

I'm a complete newbie to *nix.
I'm trying to write a script to symlink my windows files, but my script doesn't work if there're space characters in their names:

Code:

#!/bin/bash

#replace all space characters with '\ '
string=${@// /\\ }

#strip ''
string=${string%'}
string=${string#'}

echo $string > ~/linkfilename_log
ln -fs $string /tmp/temp_target

(This code is written from memory in windows, as I can't get my network working in BSD, yet)

I saved this script as 'linkfilename' in my scripts folder, and I linked my data files with the following command in Thunar :

Code:

linkfilename "%F"

if my file name is 'filename_with_space.cpt', the script works, but
if it is 'filename with space.cpt', it doesn't.

I'm lost. If I type in a xterm shell:

Code:

$ ln -fs filename\ with\ space.cpt /tmp/temp_target

,
the file is linked. And 'filename\ with\ space.cpt' is exactly what I see in my 'linkfilename_log' file.

FYI, I have to load my files in a windowz application running in Wine, but I can't load files with space characters in thier names if I don't change their names or link them as in the script.
I know that it's very easy to convert all space characters with '_' in Thunar, but I'd like to keep them as they are if possible.(this is not my personal taste, there are good reasons.)

Can someone help me, please?

ErV · 10-10-2008, 08:34 PM

Quote:

Originally Posted by Sammy_G

Can someone help me, please?

Try

Code:

ln -fs "$string" /tmp/temp_target

instead of

Code:

ln -fs $string /tmp/temp_target

Notice quotes.

Sammy_G · 10-10-2008, 08:41 PM

Quote:

Originally Posted by ErV

Try

Code:

ln -fs "$string" /tmp/temp_target

instead of

Code:

ln -fs $string /tmp/temp_target

Notice quotes.

Thanks, I'm in windows now, so I'll answer later after i've tried your solution in BSD..

John VV · 10-10-2008, 08:55 PM

for the most part you want to use double " " quotes and not single ' ' and not back quotes ` `

ErV · 10-10-2008, 08:58 PM

Quote:

Originally Posted by John VV

for the most part you want to use double " " quotes and not single ' ' and not back quotes ` `

It depends on what do you want to do.

John VV · 10-10-2008, 10:03 PM

true there are subtle differences in "" '' ``

Sammy_G · 10-11-2008, 12:13 AM

It works now.
Thanks, guys.

jschiwal · 10-11-2008, 12:30 AM

You can still run into a problem if you need to run the names through a pipe. Such as a directory with 50k entries in it where you can't use file globbing (due to memory restrictions). In this case a common technique is to use the find command to print the filenames and use '-print0' to separate the names with the NULL character. On the other side of the pipe, xargs uses the -0 argument to use NULL as a delimiter. It also has arguments you can use to limit the number of arguments handled at once.

You also need to watch out for other characters as well such as (,),{,} which have special meanings to the shell. The worst character is "!" when working in an interactive shell. A filename beginning with a hyphen can cause problems as well.

If you have a list of filenames, perhaps the results of a sed command, you can use "tr '\n' '\0'" to produce the same nulls as find's "-print0" argument.
Example finding and deleting duplicate files:

Code:

find directory/ -type f -exec md5sum '{}' \; |   # prints out all of the files, one per line
sort |                                           # sorts out the results
uniq -d -w32 |                                   # compares the md5sums and prints out duplicates
tr '\n' '\0' |                                   # converts newlines to nulls to eliminate problems with spaces in filenames
xargs -0 -L 50 rm                                # removes the duplicates

jlinkels · 10-11-2008, 08:53 AM

Quote:

Originally Posted by jschiwal

You also need to watch out for other characters as well such as (,),{,} which have special meanings to the shell. The worst character is "!" when working in an interactive shell. A filename beginning with a hyphen can cause problems as well.

.. which indicates that when Unix was designed, assumptions were made about the knowledge level of users, in the sense that certain characters in a file name should not be used. Still I would *never* use a file name which contains other characters than [A-z0-9._\-], not on either platform.

But.. MAC is even worse. Once I had to view a CD with picture files on a Windows machine which was created on the MAC and contained "?" in the file names. No way Windows would read that, but in Linux I was able to convert the file names (with rename and friends) to something usable in Windows.

jlinkels

archtoad6 · 10-11-2008, 09:13 AM

More radical, change the spaces to underscores & solve the problem permanently:

Code:

ls |grep \ |while read F; do mv \"$F\" \"`echo $F|tr \  _`"; done

Edit: Oops, that code doesn't work -- I should have removed the backslashes in the mv statement (they are an artifact of wrapping that mv statement in an echo during testing):
Here is the correct code:

Code:

ls |grep \ |while read F; do mv "$F" "`echo $F|tr \  _`"; done

jschiwal · 10-11-2008, 02:56 PM

Quote:

Originally Posted by jlinkels

.. which indicates that when Unix was designed, assumptions were made about the knowledge level of users, in the sense that certain characters in a file name should not be used. Still I would *never* use a file name which contains other characters than [A-z0-9._\-], not on either platform.

But.. MAC is even worse. Once I had to view a CD with picture files on a Windows machine which was created on the MAC and contained "?" in the file names. No way Windows would read that, but in Linux I was able to convert the file names (with rename and friends) to something usable in Windows.

jlinkels

The font encoding is probably different between the Mac and Windows. I think a highlighted ? is printed if a character is used that isn't in your character set.

Imagine if you have a filesystem from a PPC machine using UTF-16 and then mount it on a X86 machine. The byte order of the characters will be different.

Mountain · 04-26-2009, 11:05 PM

Quote:

Originally Posted by archtoad6

More radical, change the spaces to underscores & solve the problem permanently:

Code:

ls |grep \ |while read F; do mv \"$F\" \"`echo $F|tr \  _`\"; done

How can this be modified to work on directories?

David the H. · 04-26-2009, 11:48 PM

In my opinion, the best way to handle spaces in a script is to change the internal field separator. I get headaches trying to get quoting to work correctly, and I often just can't seem to get them right. But changing the IFS so that it doesn't recognize spaces as separators generally does the trick.

Just do the following:

Code:

#!/bin/bash

OLDIFS=$IFS
IFS="
"
<your script>

IFS=$OLDIFS

Note that the new IFS is simply a newline. The original is newline+tab+space. Actually, you probably won't need to save the original IFS, because the script runs in its own subshell and doesn't affect anything else, but it's a good idea if you need to change it back while inside the script for some reason, such as if you have to do a loop with a space-separated list of items.

PS: Not to sound too critical, but this topic comes up quite often, so you would've saved us all some effort if you'd searched the forums first.

archtoad6 · 04-27-2009, 05:44 AM

Mountain,

I suppose I should thank you for pointing me at an error my code above;
but given that it wouldn't work even on files,
I have to wonder if you even tried it before asking if would work on directories.

The answer is that in *nix directories are files, so the corrected code works on either, as is.

If you want a demonstration, try this:

Code:

mkdir -p test/test
cd test/test
touch tfile\ {1..9}
mkdir tdir\ {1..9}
ls; sleep 5   # verify test corpus
ls |grep \ |while read F; do mv "$F" "`echo $F|tr \  _`"; done
ls; sleep 5   # verify results
cd ..
rm -rf test   # remember how dangerous this is

Note: I did test this -- inside a parent directory also named "test" -- but you still want to be very careful around any rm -rf

Mountain · 04-27-2009, 10:40 AM

Quote:

Originally Posted by archtoad6

Mountain,

I suppose I should thank you for pointing me at an error my code above;
but given that it wouldn't work even on files,
I have to wonder if you even tried it before asking if would work on directories.

The answer is that in *nix directories are files, so the corrected code works on either, as is.

Hmmm, you're welcome???

I did in fact test your code on directories and it failed. That's why I asked my question. I did not know directories are files.