LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
LinkBack Search this Thread
Old 03-15-2009, 04:51 PM   #1
xp_newbie
Member
 
Registered: Nov 2006
Posts: 86

Rep: Reputation: 16
Question list filenames with spaces in a shell script


I am trying to iterate on all files in the current directory, using a bash script:

Code:
LSFILELIST=`ls -1`

for EACHFILE in "${LSFILELIST}"; do
  echo "${EACHFILE}"
done
The problem: ${EACHFILE} "spits" the entire list of files, thinking it is one long filename (with spaces).

Trying to troubleshoot the problem, I came to the conclusion that the difficulty in using the for loop in such manner lies in the fact that I can current use $LSFILELIST in either:

1) bare (no quotes) $LSFILELIST:
Quote:
==> $LSFILELIST="file name1 file name2 file name3"
2) quoted "${LSFILELIST}"
Quote:
==> $LSFILELIST="file name1
file name2
file name3"
Neither of the above satisfies bash's 'for'!

What I really need instead is some mechanism that will generate for me $LSFILELIST like this:
Quote:
$LSFILELIST="file name1"
"file name2"
"file name3"
However, I don't know to accomplish this.

Clues, tips, hints or ideas would be greatly appreciated.

Thanks!
 
Old 03-15-2009, 05:03 PM   #2
billymayday
Guru
 
Registered: Mar 2006
Location: Sydney, Australia
Distribution: Fedora, CentOS, OpenSuse, Slack, Gentoo, Debian, Arch, PCBSD
Posts: 6,678

Rep: Reputation: 120Reputation: 120
This will find them for you
Code:
find ./ -type f -name '* *'
 
Old 03-15-2009, 06:14 PM   #3
Hko
Senior Member
 
Registered: Aug 2002
Location: Groningen, The Netherlands
Distribution: ubuntu
Posts: 2,530

Rep: Reputation: 108Reputation: 108
Two way of looping over file names from "ls" command, handling files names with spaces correctly:
  • Pipe output of "ls" directly to loop. Disadvantage is that if some variable is assigned to or changed inside the loop, the value will be lost after the loop. (TESTVAR here used to show this effect).
    Code:
    #!/bin/bash
    
    TESTVAR="Initial test value"
    
    ls -1 | while read FILE; do
        echo "== FIRST LINE OF: $FILE"
        head -1 "$FILE"  # double quotes needed for filename with spaces!
        echo 
        TESTVAR="Changed!"
    done
    
    echo "Value of TESTVAR after te loop = \"$TESTVAR\""
    exit 0
  • One way to solve the issue with changes to a variable inside the loop it to save the output of ls to a temp file.
    Code:
    #!/bin/bash
    
    TESTVAR="Initial test value"
    
    TMP=`mktemp`  # Create a temp file, path stored in var $TMP
    ls -1 >"$TMP" # Write list of files to temp file
    
    while read FILE; do
        echo "== FIRST LINE OF: $FILE"
        head -1 "$FILE"  # double quotes needed for filename with spaces!
        echo 
        TESTVAR="Changed!"
    done <$TMP # Feed the loop from temp file
    
    echo "Value of TESTVAR after te loop = \"$TESTVAR\""
    rm $TMP # When done with the file list, clean up temp file
    exit 0
Note: When using a file name from a variable ($FILE in the examples) in a command, you should double-quote it to have it work with file names with spaces in them.
 
Old 03-15-2009, 06:35 PM   #4
xp_newbie
Member
 
Registered: Nov 2006
Posts: 86

Original Poster
Rep: Reputation: 16
Hko, you seem to have found a solution to my problem although I may have not explained it very well: I don't need to read the files - I only need to have their filenames for various operations.

Thus, the following script is enough and seems to work:
Code:
#!/bin/bash

ls -1 | while read FILE; do
    echo "$FILE"  # double quotes needed for filename with spaces!
done

exit 0
However, I am not sure what the 'read' above does: Does it really open the file and attempts to read from it?

If so, then this may not be very efficient. I just want to be able to check whether the file exists in another directory - I don't need to read it or even open it.

Is there a way to accomplish the above without the 'read'?

Thank you!
 
Old 03-15-2009, 07:07 PM   #5
Hko
Senior Member
 
Registered: Aug 2002
Location: Groningen, The Netherlands
Distribution: ubuntu
Posts: 2,530

Rep: Reputation: 108Reputation: 108
Quote:
Originally Posted by xp_newbie View Post
Code:
#!/bin/bash

ls -1 | while read FILE; do
    echo "$FILE"  # double quotes needed for filename with spaces!
done

exit 0
However, I am not sure what the 'read' above does: Does it really open the file and attempts to read from it?
No. it does not read or open the files.

while read SOME_VAR; do just loops over each line it get from the standard input ("stdin"), assigning the line read to $SOME_VAR each time.

If you use it this way:
Code:
while read SOME_VAR; do
    echo "==> $SOME_VAR"
done
This script will just wait for you to enter some lines until you press ctrl-d to signal "end of file (EOF)". The loop will echo each line you typed with "==>" before each line.

If you pipe the contents of a file with "cat" to the standard input of such a while-read-loop:
Code:
cat /path/to/some/file.txt | while read LINE; do
    echo "==> $LINE"
done
Then "cat" reads the file and feeds it to the standard input (stdin) of the while-read-loop. So, the loop runs for each line in the file.

What I did in the post above is feed the output of "ls -1" to the while-read-loop the same way.

You said you need the list of file names "for various operations". If you need to loop more than one time over the same list of files, I'd recommend using the second example I posted (storing the list in a temp-file).

It is probably possible to do without a read-loop and do it with a for-loop instead. But I don't know one from the top of my head. Would need to fiddle with file seperators and some extra comands I suppose. But IMO there's no need to explicitly using a for-loop, and it would be probably be less efficient than while-read especially when the list of files is long.

Last edited by Hko; 03-15-2009 at 07:09 PM.
 
Old 03-15-2009, 07:34 PM   #6
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946
I've always found spaces to be a big headache too, and things like quoting variables and whatnot often don't work very well. I've found that the least troublesome way to handle spaces is to change the internal field separator in bas so that it doesn't recognize them.

Simply add the following to your script, or run them in the terminal:

Code:
OLDIFS=$IFS

IFS="
"        #this is a line break

{the block of commands you want to run}

IFS=$OLDIFS
Now if everything goes as planned, spaces will be ignored in filenames.
Note that it isn't really necessary to back up and reset the IFS in a script, since variables set in scripts are only local in scope.

The default IFS is space, tab, and newline, BTW.
 
Old 03-15-2009, 07:46 PM   #7
xp_newbie
Member
 
Registered: Nov 2006
Posts: 86

Original Poster
Rep: Reputation: 16
Quote:
Originally Posted by Hko View Post
If you need to loop more than one time over the same list of files, I'd recommend using the second example I posted (storing the list in a temp-file).
Thank you very much for your great help. No, I don't need to loop more than once, so your original solution is 100% PERFECT for me. Thank you!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
script to remove spaces from multiple filenames jeffreybluml Linux - Newbie 36 07-31-2013 02:10 AM
script help - add absolute path to list of filenames greengrocer Linux - Newbie 7 10-12-2008 04:23 AM
script to list the filenames which are in lower case naveensankineni Programming 2 03-12-2008 07:09 AM
shell script - truncating filenames mr_scary Programming 3 09-04-2006 02:50 PM
Shell script: Delete filenames containing a substring? Drack Linux - General 4 02-12-2006 03:11 PM


All times are GMT -5. The time now is 03:25 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration