[SOLVED] problem wiht find command

prasunjit · 03-19-2013, 07:28 AM

Hi All,

I am using the below find command to find out all the directories and storing in an array. I want to handle the directory name with spaces also(LINUX FINAL)
find /home/myadmin/ -maxdepth 1 -type d
/home/myadmin/
/home/myadmin/LINUX FINAL
/home/myadmin/Downloads

I have modifies the above command as to handle the blank spaces in names
find /home/myadmin/ -maxdepth 1 -type d -print0
But the output is here different
/home/myadmin//home/myadmin/LINUX FINAL /home/myadmin/Downloads/

how do i print the output as before and also the blank spaces in directory names is also handled

thanks in advance

TB0ne · 03-19-2013, 09:21 AM

Quote:

Originally Posted by prasunjit

Hi All,
I am using the below find command to find out all the directories and storing in an array. I want to handle the directory name with spaces also(LINUX FINAL)
find /home/myadmin/ -maxdepth 1 -type d
/home/myadmin/
/home/myadmin/LINUX FINAL
/home/myadmin/Downloads

I have modifies the above command as to handle the blank spaces in names
find /home/myadmin/ -maxdepth 1 -type d -print0
But the output is here different
/home/myadmin//home/myadmin/LINUX FINAL /home/myadmin/Downloads/

how do i print the output as before and also the blank spaces in directory names is also handled

Well, we can only assume you're doing this in some sort of script, since you mention an array. However, you don't say what kind of script, or post the code, so there's no way we can tell you how to fix it.

Post your code, and we can try to help.

shivaa · 03-19-2013, 09:46 AM

Better try -exec option, as:

Code:

~$ find /home/myadmin/ -maxdepth 1 -type d -exec ls -la {} \;

whizje · 03-19-2013, 02:42 PM

Find with spaces

Code:

find /home/myadmin/ -maxdepth 1 -type d -exec echo \"{}\" \;

Or if you want to proces the result use

Code:

find /home/myadmin/ -maxdepth 1 -type d -print0 |xargs -0 command

David the H. · 03-20-2013, 07:10 AM

Please use ***[code][/code]*** tags around your code and data, to preserve the original formatting and to improve readability. Do not use quote tags, bolding, colors, "start/end" lines, or other creative techniques.

See here for how to safely process the output of a command and store it in an array:

How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
http://mywiki.wooledge.org/BashFAQ/001

How can I use array variables?
http://mywiki.wooledge.org/BashFAQ/005/

How can I find and deal with file names containing newlines, spaces or both?
http://mywiki.wooledge.org/BashFAQ/020

Note that the -print0 option separates each entry by an ascii null character. Since shell parameters cannot contain nulls, this makes them the safest delimiter to use in processing them. As non-printing characters, they won't show up visibly in the direct output of find, but they are there, and subsequent commands can use them to safely split the text.

Code:

# This will load all regular files in a directory into an array:
while IFS='' read -rd '' ; do
   array+=( "$REPLY" )
done < <( find . -depth -type f -print0 )

# Now check the output
printf 'listing file: [%s]\n' "${array[@]}"

grail · 03-20-2013, 09:58 AM

@David - Whilst the while loop makes perfect sense, is there a reason you know of why the following does not work?

Code:

IFS= read -r -d '' -a array <<<$(find -maxdepth 1 -type d -print0)

Could be, "it just doesn't", but my head seems to think it should

prasunjit · 03-25-2013, 12:26 AM

Thanks a lot to all of you.
while the codes given by all of you were working fine but it seems those were not fitting to my requirement in my script. Here is what i want to do:
find all the sub directories/ in a given directory and display their memory usage

Code:

arr_dir=(`find "/home/myadmin" -maxdepth 1 -type d`) #storing the values in array
for i in "${arr_dir[@]}" ; do 
mem_usag=`du -sh "$i" | cut -f 1`
echo -e "${mem_usag}\t/home/myadmin"
done
fi

/home/myadmin contains below directories
/home/myadmin/LINUX FINAL
/home/myadmin/Downloads
though i am able to store the values in the array but when du comes to LINUX FINAL it assumes two diff values due to bad naming convention(space).Is possible to store values in the arry in sucha way that du also handles these kind of folders ?
Please help..couldnt find a sol after tweaking my code in many ways

prasunjit · 03-25-2013, 12:45 AM

I have also tried to take the input to array as the below codes:
@David and @whizje

Code:

while IFS='' read -rd '' ; do
  arr_dir+=( "$REPLY" )
done < <(`find "/home/myadmin" -maxdepth 1 -type d -print0`)
for i in "${arr_dir[@]}" ; do 
mem_usag=`du -sh "$i" | cut -f 1`
echo -e "${mem_usag}\t/home/myadmin"
done
OR
arr_dir=(`find /home/myadmin/ -maxdepth 1 -type d -exec echo \"{}\" \;`)

but du gets starngled when it comes to folder names with spaces

shivaa · 03-25-2013, 12:48 AM

Removing space character will change the directory name too, so it's not a good option. Instead you can leave the array and use process substitution, like:

Code:

while read -r dir; do 
space_usag=$(du -sh "$i" | awk '{print $1})  # It's not memory usege, but space usege
# space_usag=$(du -sh "$i" | awk -F"\t" '{print $1,$2})       # Prints space usage + dirname both
echo -e "$space_usag\t/home/myadmin"         # Leave this line. It's of no use. 
# echo -e "$space_usag\t$i"                  # Prints space usage + dirname both
done < <(find "/home/myadmin" -maxdepth 1 -type d` -print)

prasunjit · 03-25-2013, 01:11 AM

Hi All,
I just found shell is adding a front slash automatically to LINUX FINAL after pressing tab key as shown below

Code:

du -sh LINUX FINAL 
du: cannot access `LINUX': No such file or directory
du: cannot access `FINAL ': No such file or directory
du -sh LINUX\ FINAL\ 
8.0K	LINUX FINAL

Can this be incoraporated to my code ?. No idea how to do this. Please let me know if anyone has other solution

shivaa · 03-25-2013, 01:27 AM

You can use double quote around file name, as:

Code:

du -sh \""$i"\"

prasunjit · 03-25-2013, 02:00 AM

Quote:

Originally Posted by shivaa

Removing space character will change the directory name too, so it's not a good option. Instead you can leave the array and use process substitution, like:

Code:

while read -r dir; do 
space_usag=$(du -sh "$i" | awk '{print $1})  # It's not memory usege, but space usege
# space_usag=$(du -sh "$i" | awk -F"\t" '{print $1,$2})       # Prints space usage + dirname both
echo -e "$space_usag\t/home/myadmin"         # Leave this line. It's of no use. 
# echo -e "$space_usag\t$i"                  # Prints space usage + dirname both
done < <(find "/home/myadmin" -maxdepth 1 -type d` -print)

unfortunately i am not aware of awk programming.could you please check whether my code is good to go which u gve

Code:

while read -r dir; do 
  mem_usag=`du -sh "$dir" | cut -f 1`     
  echo -e "$mem_usag\t$dir"                  
done <<`find "home/myadmin" -maxdepth 1 -type d -print`

the code

Code:

done <<`find "/home/myadmin" -maxdepth 1 -type d -print`

is throwing error.Pls bear me as i am a novice in unix progrm

shivaa · 03-25-2013, 02:09 AM

Awk is not necessory at all. You can use cut also. Also don't modify the original syntax. Following should work.

Code:

while read -r dir; do 
mem_usag=$(du -sh \""$dir"\" | cut -d" " -f1)     
echo -e "$mem_usag\t$dir"                  
done < <(find "$j" -maxdepth 1 -type d -print)

You can also use sed to insert double quote at beginning and end of the directory names.

Also follow the post#5, and go through the links suggested by David. Most of your doubts will clear by following those links.

David the H. · 03-26-2013, 09:35 AM

Quote:

Originally Posted by grail

@David - Whilst the while loop makes perfect sense, is there a reason you know of why the following does not work?

Code:

IFS= read -r -d '' -a array <<<$(find -maxdepth 1 -type d -print0)

Could be, "it just doesn't", but my head seems to think it should

It does seem like it at first glance, doesn't it? But with read's -d delimiter set to null, it will only process up to the first null in the input text, and so there's nothing in the string for IFS to split on.

The same thing would happen any time IFS and -d are set to the same thing.

Incidentally, when I use the here string+command substitution as above I get the entire output as a single string, but when I use a process substitution for the input I get the first file only. It appears that the command substitution is acting as a parameter and silently dropping the nulls from the input. A bit of testing with cat -A bears this out.

Code:

$ touch file{1..5}

$ cat -A <<<"$( find . -type f -print0 )"
./file5./file1./file3./file2./file4$

$ cat -A < <( find . -type f -print0 )
./file5^@./file1^@./file3^@./file2^@./file4^@

Actually, what I can't figure out is why this isn't working:

Code:

$ IFS='' read -ra array < <( find . -type f -print0 ) ; printf '%s\n' "${array[@]}"
./file5

Since the delimiter is now the default newline, I believe it should be processing the entire input and splitting it into the array on the nulls, but it's still only seeing the first value.

David the H. · 03-26-2013, 10:38 AM

Quote:

Originally Posted by prasunjit

unfortunately i am not aware of awk programming.could you please check whether my code is good to go which u gve

Code:

while read -r dir; do 
  mem_usag=`du -sh "$dir" | cut -f 1`     
  echo -e "$mem_usag\t$dir"                  
done <<`find "home/myadmin" -maxdepth 1 -type d -print`

1) $(..) is highly recommended over `..`. Please don't use backticks anymore unless you have to deal with a very old shell.

2) Once you have a string stored in a variable, instead of cut or other external commands, you can almost always use parameter substitution or another built-in string manipulation technique to process substrings from them.

The fewer external processes you use, the faster your script will generally be, unless you are doing bulk modification of large blocks of text, in which case something like sed or awk may be faster.

3) Your here string syntax is wrong. It uses three arrows, not two. But actually, you really should be using a process substitution instead, at least when using bash.

4) I'd probably use printf instead of echo -e for the output.

Code:

while IFS='' read -rd '' dir; do

    mem_usag=$( du -sh "$dir" )
    printf '%s\t%s\n' "${mem_usag%%[[:space:]]*}" "$dir"

done < <( find "home/myadmin" -maxdepth 1 -type d -print0 )

Beware of the space between the arrows in the '< <(..)' pattern. '<(..)' is the process substitution itself, and the other arrow is a standard shell redirection. Unlike with a normal file, however, they need to be separated into individual tokens.

Although why you want to go to the trouble to extract the first column from the du output only to print it out again in exactly the same format (right down to the tab separating the columns) is beyond me.

Speaking of which, instead of the above loop, you could just use find on its own:

Code:

find "home/myadmin" -maxdepth 1 -type d -exec du -sh '{}' \;