LinuxQuestions.org - BASH: Reading long filenames into an array using a loop

- Programming (https://www.linuxquestions.org/questions/programming-9/)

- - BASH: Reading long filenames into an array using a loop (https://www.linuxquestions.org/questions/programming-9/bash-reading-long-filenames-into-an-array-using-a-loop-455279/)

BASH: Reading long filenames into an array using a loop

Hello, everybody.

I'm trying to write a BASH script to recursively change the permissions of files and directories, respectively. The reason for this is so that, having removed about 50GB of from a vfat partition with umask=000 set onto a reiserfs partition, I want to change the permissions to something sane, as opposed to -rwxrwxrwx and drwxrwxrwx.

Here's the code I'm having problems with (just an excerpt from my rather messy work-in-progress):

Code:

#!/bin/bash



num=0



for i in `ls -a1 --color=none | grep -v ^'[.]\|[..]'$ | sed s/' '/'\\ '/g` ; do

        testarray[$num]=$i

        echo ${testarray[$num]}

        num=$(($num + 1))

done

What it's supposed to do: List all files in a directory, one per line. Remove "." and ".." entries. Insert backslashes ("\") to escape spaces in filenames. Feed each filename into the array. Echo the value just entered into the array (to make sure it did it right; I'll remove this later). Do it all over again until all filenames are stored in array.

What it's doing wrong: when a file or directory such as, "filename with spaces in it" is put into the array, it all comes out in separate members of the array, such as, testarray[0]=filename, testarray[1]=with, testarray[2]=spaces, etc. This is bad :-).

Here's a sample of the output of just the ls/grep/sed command (run in a test directory):

Code:

[dane@Orchestrator fubar]$ ls -a1 --color=none | grep -v ^'[.]\|[..]'$ | sed s/' '/'\\ '/g

directory\ with\ spaces\ in\ it

files.log

foo

morestuff

stuff

As you can see, the output *seems* to be listing one file at a time, with each filename intact. Why isn't it going into the array that way?

Thanks for your time.

--Dane

I have tried for ages to do something to get "for i in" to work with spaces, but with no luck. I have tried to put "\" in the file names, put them in quotes, put them in quotes and backslash spaces... But it seems that it treats spaces as separators regardless. I think that the find command can be used to run commands on files and dirs separately with:
find -type [f|d] -exec (command) {}
but I only remember it form a script I was looking at a while ago so you may have to look at the man page of find.

Hope that helps.

P.S. if you use "ls -Al" insted of "ls -al" it will not display . and ..

why not using "chmod --recursive"?

Well I was under the impression that files and dirs were to be treated differently in the script.
But if not then chmod -R would work, yes.

why not
find -type d | xargs chmod

find -type f -exec chmod {} \;

Quote:

Originally Posted by binary_y2k2

I have tried for ages to do something to get "for i in" to work with spaces

The following will do the trick:

Code:

#! /bin/bash

IFS=$'\n'

for NAME in $(find -type f)

do

  echo "I've found a file named \"$NAME\""

done

IFS is Bash's internal field separator, and here we set it to consist of a newline character only.

if only I knew that a while ago,
ah well, live and learn :p
I'll have to rwite that down :)

Thanks, everybody, for the replies!

I didn't understand the bit about xargs (I'm pretty new to BASH scripting), but inserting "IFS=$'\n'" before the for loop did the trick!

I do, however have another question about this script, if you all don't mind.

I was wondering how to get it to output an error message and exit with status 1 if anything at all gets written to stderr.

Here is the script in its mostly complete state:

Code:

#!/bin/bash

#chmodr.sh: recursively change the permissions of files and directories, respectively.



# Set some variables

dirperms=$1

fileperms=$2

target=$3

rootdir=`pwd`



# Sanity checks and help message.

if [ $# -ne 3 ] || [ $1 == "-h" -o $1 == "--help" ]

        then

                echo -e "chmodr.sh: Recursively change the permissions of files and directories,"

                echo -e "          respectively.\n"

                echo -e "Usage: chmodr.sh directorypermissions filepermissions target\n"

                echo -e "Note: The target MUST be a directory that's executable to you!\n"

                echo -e "Also note: This script will not work if any of the directories"

                echo -e "          or files in the path to be changed end in \":\".\n"

                echo -e "Further note: The script will fail with unexpected results if"

                echo -e "              you select non-executable directory permissions!\n"

                echo -e "-h --help    Display this message."

                exit 0

fi



# Is the first argument a directory?  If so, change its permissions; else exit with error.

if [ -d "$target" ]

        then echo "chmod $dirperms $target"

        chmod $dirperms $target

        else echo "The last argument must be a directory!"

        echo "For single files, use chmod."

        exit 1

fi



# Format the ls output and get a list of directories to cd into.

#for d in "`ls -aR1 --color=none | grep \:$ | sed -e s/:$/''/ | grep -v ^'[.]\|[..]'$`" #| sed s/' '/'\\\\ '/g`"



IFS=$'\n'

dnum=0

for d in `ls -AR1 --color=none $target | grep \:$ | sed s/:$/''/ | grep -v '^[.]$\|^[..]$' | sed s/' '/'\\ '/g` ; do

#        echo "\$d='$d'"

        dir[$dnum]="$d"

#        echo "dir[\$dnum]=${dir[$dnum]}"

#        echo "pwd=`pwd`"

#        echo "changing to directory: ${dir[$dnum]}"

#        echo "\$target='$target'"

        # cd into each directory in turn and change the permissions of everything in it.

        if cd "${dir[$dnum]}"

                then

#                echo "past cd command"

                num=0

                for i in `ls -A1 --color=none | grep -v '^[.]$\|^[..]$' | sed s/' '/'\\ '/g` ; do

                        file[$num]="$i"

#                        echo ${file[$num]}

                        if [ -d "${file[$num]}" ]

                                then echo "chmod $dirperms ${dir[$dnum]}/${file[$num]}"

                                chmod $dirperms ${file[$num]}

                                else echo "chmod $fileperms ${dir[$dnum]}/${file[$num]}"

                                chmod $fileperms ${file[$num]}

                        fi

                        num=$(($num + 1))

                done

                cd "$rootdir"

                else cd "$rootdir"

                echo "Error! Could not cd into directory '${dir[$dnum]}'!"

                echo "Make sure that you own the directory in question."

                exit 1

        fi

        dnum=$(($dnum + 1))

done

exit 0

Thanks for your help!

--Dane

I'd suggest using something like the following. It won't catch stderr, but you can query the exit status of each command:

Code:

# first option: simply check for errors and keep track of them in a variable

cat this-file-does-not-exist || (echo "an error occured"; let "ERROR+=1")

if(( $ERROR ))

then

  echo "something went wrong"

  exit 1

fi



# second option: query the previous command's exit status

cat this-file-does-not-exist

echo "The cat command returned $?".

If you really need stderr, you could probably use redirection and "exec". Have a look at the bash reference.

Thanks, spirit receiver, for the advice.

I ended up using option #2 like this:

Code:

# Create a function for testing the exit status of a command.

# If something goes wrong, call error function.

exitstat ()

{

        if [ $? != 0 ]

        then

                error

        fi

}



# Create a function for what happens when something goes wrong.

error ()

{

        echo "Something went wrong!  Are you sure you own ${dir[$dnum]}/${file[$num]}?"

        echo "You might want to use 'chown username:group -R $rootdir/$target' (as root)."

        echo "Exiting now."

        exit 1

}





[...]





# Is the first argument a directory?  If so, change its permissions; else exit with error.

if [ -d "$target" ]

then

        echo "chmod $dirperms $target"

        chmod $dirperms $target

        if [ $? != 0 ]

        then

                echo "Something went wrong!  Are you sure you own $rootdir/$target?"

                echo "You might want to use 'chown username:group -R $rootdir/$target' (as root)."

                echo "Exiting now."

                exit 1

        fi

else

        echo "The last argument must be a directory!"

        echo "For single files, use chmod."

        exit 1

fi





[...]





                        if [ -d "${file[$num]}" ]

                        then

                                echo "chmod $dirperms ${dir[$dnum]}/${file[$num]}"

                                chmod $dirperms ${file[$num]}

                                exitstat

                        else

                                echo "chmod $fileperms ${dir[$dnum]}/${file[$num]}"

                                chmod $fileperms ${file[$num]}

                                exitstat

                        fi

Hopefully this will prove useful to someone else, as it has proved useful to me.

Cheers!

--Dane

Here is a solution that takes into count the $IFS environment variable

Here is a solution that takes into count the $IFS
environment variable:

Code:

# Here is a solution that takes into count the $IFS

# environment variable

DIR="/path/to/dir"

 

# Save and change IFS

OLDIFS=$IFS

IFS=$'\n' 

 

# Read all directory names into an array

DirsArray=($(find $DIR -maxdepth 1 -type d))



# For files, do the following 

# Read all file names into an array

# DirsArray=($(find $DIR -maxdepth 1 -type f))

 

# Restore it

IFS=$OLDIFS

 

# Get length of an array

DirsIndex=${#DirsArray[@]}

 

# Use for loop read all directory names

for (( i=0; i<${DirsIndex}; i++ ));

do

  echo "${DirsArray[$i]}"

done



# Print again using seq

for j in $(seq 0 $((${#DirsArray[@]} - 1))) ; do

  echo $j ":" ${DirsArray[$j]}

done



exit 0;

Thank-you for the helpful reply. I appreciate you posting even though it's been a few years since I wrote that; the info is still useful!

--Dane

a little tip as regards playing with IFS
put it in subshell parenthesis

Code:

( 

IFS=blah

blah blah blah 

)

# when you get here IFS is back to normal

a particular favourite of mine,
on the command line, list all executables:

Code:

(IFS=:;ls -1 $PATH)