bash how to pass value to variable in this function?

babag · 04-02-2008, 04:53 PM

i use this function to delete contents of directory
with too much stuff for bash.

Code:

cd /home/xfers/directory_with_lots_of_files

function delete_all ()
     {
          while read line1; do

            echo $line1
            rm /home/xfers/directory_with_lots_of_files/$line1

          done
     }

ls -l /home/xfers/directory_with_lots_of_files | delete_all

i want to use it to pass the filenames of the first
and last files now in this directory to two variables.
find errors out as there are many thousands of files
in the directory. this is what i've changed it to for
starters:

Code:

cd /home/xfers/directory_with_lots_of_files

function find_first ()
     {
          while read line1; do

            echo $line1

          done
     }

ls -l /home/xfers/directory_with_lots_of_files | find_first

this does fine at printing a directory listing and
i can change the final line to 'ls -rl' to do a
reverse listing, but i can't seem to retain the
first or last value as a variable. how would i pass
the value of the last file listed by this function
using either the 'ls -l' or 'ls -rl' form? knowing
those, i could pass the first and last filenames in
this large directory on for further processing but
i seem to be stuck.

thanks,
BabaG

chrism01 · 04-02-2008, 06:36 PM

Don't understand why you want 'first' & 'last' filenames.
There are 2 options I'd use to 'empty' a large dir.

Code:

# option 1 delete dir & replace it
rm -rf $targetdir
mkdir $targetdir

# option 2: (slower) use a simple loop
cd $targetdir
for file in `ls`
do
    rm $file
done

HTH

babag · 04-02-2008, 08:21 PM

thanks chris.

i'm just using the delete function as a starting point
to build a larger function for another purpose. the
'delete_all' function is working fine as it is for
emptying my large directory. the other function i have
to create is to split the directory's contents into
six equal (roughly), parts and copy each of the six
parts to a different directory.

my main directory has many thousands of numbered files.
the filenames are in a convention of:

name_of_project-01234.tga

i need to find the total number of files, divide it by
six. i'll then find the first file in the directory,
find its number (the 01234 part) and add number_of_files/6
to it. that will give me the first and last file for part
one of six. i can then do something like that for the next
five parts. that should give me an articulation of which
files belong to which of the six parts. then i can begin
copying the six parts to the six directories.

i just need to get the first and last filenames in the
directory into a variable so i can begin to do the simple
math and filename parsing. i can't use something like
the 'find' command because there are too many files in the
directory for bash to process without an error. that's why
i'm using my 'delete_all' function as a starting point. it
avoids the bash error.

but how do i pass a variable value for the first and last
filenames from within the function so that i can do my
parsing and math?

thanks again,
BabaG

unSpawn · 04-02-2008, 08:41 PM

I'm not convinced the amount of files is the problem. For example if I build a BaSH array like: 'array=($(find /home/xfers/directory_with_lots_of_files -type f))'it holds 130K worth of filenames easily. I loathe using 'ls' but if it really is too much, and if sort criterium can be time then something like 'ls -t|head -1' and 'ls -tr|head -1' could work?

konsolebox · 04-02-2008, 08:42 PM

you can use sed, tail, head or a for loop there but since you want to use ls, you can do

Code:

FIRSTFILE=$(ls -1 | while read LINE; do echo "${LINE}"; break; done)
LASTFILE=$(ls -1r | while read LINE; do echo "${LINE}"; break; done)

note that you can't directly do

Code:

ls -1 | while read FIRSTFILE; do break; done

you'll only have the variable saved in the subprocess

konsolebox · 04-02-2008, 08:52 PM

.. but i still don't get it .. do you mean something like this:

Code:

ls -1 | {
	read FIRSTFILE
	# process first file

	while read FILE; do
		# process second to last file
		LASTFILE=${FILE}
	done
}

or

Code:

process() {
	read FIRSTFILE
	# process first file	

	while read FILE; do
		# process second to last file
		LASTFILE=${FILE}
	done
}
ls -1 | process

?

babag · 04-02-2008, 09:27 PM

unSpawn:

the directory has something like 25000 files all with
long filenames. each file is about 1 megabyte. whenever i try
using simple bash commands i get an error. i read somewhere about
bash limitations and that i should try using functions in this kind
of circumstance. that has made it possible to do things like my 'delete_all' function. 'rm' errored in this instance but the 'delete_all' function works so i can only assume the limitations i read about are
coming into play here.

konsolebox:

tried the first suggestion:

Code:

FIRSTFILE=$(ls -1 | while read LINE; do echo "${LINE}"; break; done)
LASTFILE=$(ls -1r | while read LINE; do echo "${LINE}"; break; done)

but it just sat for a while and then returned to the command prompt.

added an 'echo $FIRSTFILE' and 'echo $LASTFILE' at the end and that
returned:

total 23500705
total 23500705

still no luck in passing the first and last filenames in this large
directory to variables.

thanks again,
BabaG

babag · 04-02-2008, 11:02 PM

this is seeming to be a scope issue. i'm using mandriva
2007.1. i'm finding that variables defined inside the
function are lost when the function completes. variables
defined outside the function are taken into the function
and manipulated, but revert to their original,
outside-the-function, value when the function finishes.

this is contrary to what i've seen online where i've looked
at more than one n00b example page that says the behaviour
i'm seeing should be the reverse, that, unless declared as
'local' within the function, the variables should carry over
as global variables.

thanks,
BabaG

konsolebox · 04-02-2008, 11:07 PM

use number 1 not el: 'ls -1' and 'ls -1r'

babag · 04-02-2008, 11:18 PM

DOH!

that did it. thanks konsolebox!

BabaG

konsolebox · 04-02-2008, 11:42 PM

you're welcome.. btw .. this should make things easier:

Code:

FIRSTFILE=$(ls -1 | ( read LINE; echo "${LINE}"; ))
LASTFILE=$(ls -1r | ( read LINE; echo "${LINE}"; ))