LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 06-16-2006, 12:22 AM   #1
DaneM
Member
 
Registered: Oct 2003
Location: Chico, CA, USA
Distribution: Linux Mint
Posts: 881

Rep: Reputation: 130Reputation: 130
BASH: Reading long filenames into an array using a loop


Hello, everybody.

I'm trying to write a BASH script to recursively change the permissions of files and directories, respectively. The reason for this is so that, having removed about 50GB of from a vfat partition with umask=000 set onto a reiserfs partition, I want to change the permissions to something sane, as opposed to -rwxrwxrwx and drwxrwxrwx.

Here's the code I'm having problems with (just an excerpt from my rather messy work-in-progress):

Code:
#!/bin/bash

num=0

for i in `ls -a1 --color=none | grep -v ^'[.]\|[..]'$ | sed s/' '/'\\ '/g` ; do
        testarray[$num]=$i
        echo ${testarray[$num]}
        num=$(($num + 1))
done
What it's supposed to do: List all files in a directory, one per line. Remove "." and ".." entries. Insert backslashes ("\") to escape spaces in filenames. Feed each filename into the array. Echo the value just entered into the array (to make sure it did it right; I'll remove this later). Do it all over again until all filenames are stored in array.

What it's doing wrong: when a file or directory such as, "filename with spaces in it" is put into the array, it all comes out in separate members of the array, such as, testarray[0]=filename, testarray[1]=with, testarray[2]=spaces, etc. This is bad :-).

Here's a sample of the output of just the ls/grep/sed command (run in a test directory):

Code:
[dane@Orchestrator fubar]$ ls -a1 --color=none | grep -v ^'[.]\|[..]'$ | sed s/' '/'\\ '/g
directory\ with\ spaces\ in\ it
files.log
foo
morestuff
stuff
As you can see, the output *seems* to be listing one file at a time, with each filename intact. Why isn't it going into the array that way?

Thanks for your time.

--Dane
 
Old 06-16-2006, 01:03 AM   #2
binary_y2k2
Member
 
Registered: Jul 2005
Location: England, UK
Distribution: Ubuntu 8.04 Server, Kubuntu 12.04
Posts: 698
Blog Entries: 1

Rep: Reputation: 31
I have tried for ages to do something to get "for i in" to work with spaces, but with no luck. I have tried to put "\" in the file names, put them in quotes, put them in quotes and backslash spaces... But it seems that it treats spaces as separators regardless. I think that the find command can be used to run commands on files and dirs separately with:
find -type [f|d] -exec (command) {}
but I only remember it form a script I was looking at a while ago so you may have to look at the man page of find.

Hope that helps.


P.S. if you use "ls -Al" insted of "ls -al" it will not display . and ..

Last edited by binary_y2k2; 06-16-2006 at 01:08 AM.
 
Old 06-16-2006, 01:08 AM   #3
scoban
Member
 
Registered: Nov 2004
Location: Turkey
Distribution: Slackware
Posts: 145

Rep: Reputation: 16
why not using "chmod --recursive"?
 
Old 06-16-2006, 01:27 AM   #4
binary_y2k2
Member
 
Registered: Jul 2005
Location: England, UK
Distribution: Ubuntu 8.04 Server, Kubuntu 12.04
Posts: 698
Blog Entries: 1

Rep: Reputation: 31
Well I was under the impression that files and dirs were to be treated differently in the script.
But if not then chmod -R would work, yes.
 
Old 06-16-2006, 03:38 AM   #5
bigearsbilly
Senior Member
 
Registered: Mar 2004
Location: england
Distribution: Mint, Armbian, NetBSD, Puppy, Raspbian
Posts: 3,515

Rep: Reputation: 239Reputation: 239Reputation: 239
why not
find -type d | xargs chmod

find -type f -exec chmod {} \;
 
Old 06-16-2006, 04:12 AM   #6
spirit receiver
Member
 
Registered: May 2006
Location: Frankfurt, Germany
Distribution: SUSE 10.2
Posts: 424

Rep: Reputation: 33
Quote:
Originally Posted by binary_y2k2
I have tried for ages to do something to get "for i in" to work with spaces
The following will do the trick:
Code:
#! /bin/bash
IFS=$'\n'
for NAME in $(find -type f)
do
  echo "I've found a file named \"$NAME\""
done
IFS is Bash's internal field separator, and here we set it to consist of a newline character only.

Last edited by spirit receiver; 06-17-2006 at 04:07 AM.
 
Old 06-16-2006, 04:39 AM   #7
binary_y2k2
Member
 
Registered: Jul 2005
Location: England, UK
Distribution: Ubuntu 8.04 Server, Kubuntu 12.04
Posts: 698
Blog Entries: 1

Rep: Reputation: 31
if only I knew that a while ago,
ah well, live and learn
I'll have to rwite that down
 
Old 06-16-2006, 02:46 PM   #8
DaneM
Member
 
Registered: Oct 2003
Location: Chico, CA, USA
Distribution: Linux Mint
Posts: 881

Original Poster
Rep: Reputation: 130Reputation: 130
Thanks, everybody, for the replies!

I didn't understand the bit about xargs (I'm pretty new to BASH scripting), but inserting "IFS=$'\n'" before the for loop did the trick!

I do, however have another question about this script, if you all don't mind.

I was wondering how to get it to output an error message and exit with status 1 if anything at all gets written to stderr.

Here is the script in its mostly complete state:

Code:
#!/bin/bash
#chmodr.sh: recursively change the permissions of files and directories, respectively.

# Set some variables
dirperms=$1
fileperms=$2
target=$3
rootdir=`pwd`

# Sanity checks and help message.
if [ $# -ne 3 ] || [ $1 == "-h" -o $1 == "--help" ]
	then
		echo -e "chmodr.sh: Recursively change the permissions of files and directories,"
		echo -e "           respectively.\n"
		echo -e "Usage: chmodr.sh directorypermissions filepermissions target\n"
		echo -e "Note: The target MUST be a directory that's executable to you!\n"
		echo -e "Also note: This script will not work if any of the directories"
		echo -e "           or files in the path to be changed end in \":\".\n"
		echo -e "Further note: The script will fail with unexpected results if"
		echo -e "              you select non-executable directory permissions!\n"
		echo -e "-h --help     Display this message."
		exit 0
fi

# Is the first argument a directory?  If so, change its permissions; else exit with error.
if [ -d "$target" ]
	then echo "chmod $dirperms $target"
	chmod $dirperms $target
	else echo "The last argument must be a directory!"
	echo "For single files, use chmod."
	exit 1
fi

# Format the ls output and get a list of directories to cd into.
#for d in "`ls -aR1 --color=none | grep \:$ | sed -e s/:$/''/ | grep -v ^'[.]\|[..]'$`" #| sed s/' '/'\\\\ '/g`"

IFS=$'\n'
dnum=0
for d in `ls -AR1 --color=none $target | grep \:$ | sed s/:$/''/ | grep -v '^[.]$\|^[..]$' | sed s/' '/'\\ '/g` ; do
#	echo "\$d='$d'"
	dir[$dnum]="$d"
#	echo "dir[\$dnum]=${dir[$dnum]}"
#	echo "pwd=`pwd`"
#	echo "changing to directory: ${dir[$dnum]}"
#	echo "\$target='$target'"
	# cd into each directory in turn and change the permissions of everything in it.
	if cd "${dir[$dnum]}"
		then
#		echo "past cd command"
		num=0
		for i in `ls -A1 --color=none | grep -v '^[.]$\|^[..]$' | sed s/' '/'\\ '/g` ; do
	        	file[$num]="$i"
#			echo ${file[$num]}
			if [ -d "${file[$num]}" ]
				then echo "chmod $dirperms ${dir[$dnum]}/${file[$num]}"
				chmod $dirperms ${file[$num]}
		        	else echo "chmod $fileperms ${dir[$dnum]}/${file[$num]}"
				chmod $fileperms ${file[$num]}
			fi
			num=$(($num + 1))
		done
		cd "$rootdir"
		else cd "$rootdir"
		echo "Error! Could not cd into directory '${dir[$dnum]}'!"
		echo "Make sure that you own the directory in question."
		exit 1
	fi
	dnum=$(($dnum + 1))
done
exit 0
Thanks for your help!

--Dane
 
Old 06-16-2006, 03:50 PM   #9
spirit receiver
Member
 
Registered: May 2006
Location: Frankfurt, Germany
Distribution: SUSE 10.2
Posts: 424

Rep: Reputation: 33
I'd suggest using something like the following. It won't catch stderr, but you can query the exit status of each command:
Code:
# first option: simply check for errors and keep track of them in a variable
cat this-file-does-not-exist || (echo "an error occured"; let "ERROR+=1")
if(( $ERROR ))
then
   echo "something went wrong"
   exit 1
fi

# second option: query the previous command's exit status
cat this-file-does-not-exist
echo "The cat command returned $?".
If you really need stderr, you could probably use redirection and "exec". Have a look at the bash reference.
 
Old 06-16-2006, 06:32 PM   #10
DaneM
Member
 
Registered: Oct 2003
Location: Chico, CA, USA
Distribution: Linux Mint
Posts: 881

Original Poster
Rep: Reputation: 130Reputation: 130
Thanks, spirit receiver, for the advice.

I ended up using option #2 like this:

Code:
# Create a function for testing the exit status of a command.
# If something goes wrong, call error function.
exitstat ()
{
        if [ $? != 0 ]
        then
                error
        fi
}

# Create a function for what happens when something goes wrong.
error ()
{
        echo "Something went wrong!  Are you sure you own ${dir[$dnum]}/${file[$num]}?"
        echo "You might want to use 'chown username:group -R $rootdir/$target' (as root)."
        echo "Exiting now."
        exit 1
}


[...]


# Is the first argument a directory?  If so, change its permissions; else exit with error.
if [ -d "$target" ]
then
        echo "chmod $dirperms $target"
        chmod $dirperms $target
        if [ $? != 0 ]
        then
                echo "Something went wrong!  Are you sure you own $rootdir/$target?"
                echo "You might want to use 'chown username:group -R $rootdir/$target' (as root)."
                echo "Exiting now."
                exit 1
        fi
else
        echo "The last argument must be a directory!"
        echo "For single files, use chmod."
        exit 1
fi


[...]


                        if [ -d "${file[$num]}" ]
                        then
                                echo "chmod $dirperms ${dir[$dnum]}/${file[$num]}"
                                chmod $dirperms ${file[$num]}
                                exitstat
                        else
                                echo "chmod $fileperms ${dir[$dnum]}/${file[$num]}"
                                chmod $fileperms ${file[$num]}
                                exitstat
                        fi
Hopefully this will prove useful to someone else, as it has proved useful to me.

Cheers!

--Dane
 
Old 09-08-2009, 04:05 PM   #11
dtoader
LQ Newbie
 
Registered: Sep 2009
Posts: 3

Rep: Reputation: 1
Here is a solution that takes into count the $IFS environment variable

Here is a solution that takes into count the $IFS
environment variable:

Code:
# Here is a solution that takes into count the $IFS
# environment variable
DIR="/path/to/dir"
 
# Save and change IFS
OLDIFS=$IFS
IFS=$'\n' 
 
# Read all directory names into an array
DirsArray=($(find $DIR -maxdepth 1 -type d))

# For files, do the following 
# Read all file names into an array
# DirsArray=($(find $DIR -maxdepth 1 -type f))
 
# Restore it
IFS=$OLDIFS
 
# Get length of an array
DirsIndex=${#DirsArray[@]}
 
# Use for loop read all directory names
for (( i=0; i<${DirsIndex}; i++ ));
do
  echo "${DirsArray[$i]}"
done

# Print again using seq
for j in $(seq 0 $((${#DirsArray[@]} - 1))) ; do
  echo $j ":" ${DirsArray[$j]}
done

exit 0;
 
Old 09-08-2009, 05:07 PM   #12
DaneM
Member
 
Registered: Oct 2003
Location: Chico, CA, USA
Distribution: Linux Mint
Posts: 881

Original Poster
Rep: Reputation: 130Reputation: 130
Thanks!

Thank-you for the helpful reply. I appreciate you posting even though it's been a few years since I wrote that; the info is still useful!

--Dane
 
Old 09-11-2009, 07:24 AM   #13
bigearsbilly
Senior Member
 
Registered: Mar 2004
Location: england
Distribution: Mint, Armbian, NetBSD, Puppy, Raspbian
Posts: 3,515

Rep: Reputation: 239Reputation: 239Reputation: 239
a little tip as regards playing with IFS
put it in subshell parenthesis
Code:
( 
IFS=blah
blah blah blah 
)
# when you get here IFS is back to normal
a particular favourite of mine,
on the command line, list all executables:
Code:
(IFS=:;ls -1 $PATH)
 
  


Reply

Tags
array, bash, exit, filenames, ifs, spaces, status



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How do you make long filenames with mkisofs? Huddlebum Linux - Newbie 2 11-11-2012 06:03 PM
Reading crontab into bash array nutthick Programming 2 05-22-2006 12:44 PM
Can't see long filenames starionwolf Linux - Newbie 5 11-12-2004 03:13 PM
Long filenames in OpenOffice Trying to Learn Linux - Newbie 3 05-22-2003 12:36 PM
Cannot print files with long filenames esael Linux - Networking 6 11-15-2002 09:48 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 11:48 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration