LinuxQuestions.org - [SOLVED] problem wiht find command

- Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)

- - problem wiht find command (https://www.linuxquestions.org/questions/linux-newbie-8/problem-wiht-find-command-4175454678/)

problem wiht find command

Hi All,

I am using the below find command to find out all the directories and storing in an array. I want to handle the directory name with spaces also(LINUX FINAL)
find /home/myadmin/ -maxdepth 1 -type d
/home/myadmin/
/home/myadmin/LINUX FINAL
/home/myadmin/Downloads

I have modifies the above command as to handle the blank spaces in names
find /home/myadmin/ -maxdepth 1 -type d -print0
But the output is here different
/home/myadmin//home/myadmin/LINUX FINAL /home/myadmin/Downloads/

how do i print the output as before and also the blank spaces in directory names is also handled

thanks in advance

Quote:

Originally Posted by prasunjit (Post 4914441)

Hi All,
I am using the below find command to find out all the directories and storing in an array. I want to handle the directory name with spaces also(LINUX FINAL)
find /home/myadmin/ -maxdepth 1 -type d
/home/myadmin/
/home/myadmin/LINUX FINAL
/home/myadmin/Downloads

I have modifies the above command as to handle the blank spaces in names
find /home/myadmin/ -maxdepth 1 -type d -print0
But the output is here different
/home/myadmin//home/myadmin/LINUX FINAL /home/myadmin/Downloads/

how do i print the output as before and also the blank spaces in directory names is also handled

Well, we can only assume you're doing this in some sort of script, since you mention an array. However, you don't say what kind of script, or post the code, so there's no way we can tell you how to fix it.

Post your code, and we can try to help.

Better try -exec option, as:

Code:

~$ find /home/myadmin/ -maxdepth 1 -type d -exec ls -la {} \;

Find with spaces

Code:

find /home/myadmin/ -maxdepth 1 -type d -exec echo \"{}\" \;

Or if you want to proces the result use

Code:

find /home/myadmin/ -maxdepth 1 -type d -print0 |xargs -0 command

Please use ***[code][/code]*** tags around your code and data, to preserve the original formatting and to improve readability. Do not use quote tags, bolding, colors, "start/end" lines, or other creative techniques.

See here for how to safely process the output of a command and store it in an array:

How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
http://mywiki.wooledge.org/BashFAQ/001

How can I use array variables?
http://mywiki.wooledge.org/BashFAQ/005/

How can I find and deal with file names containing newlines, spaces or both?
http://mywiki.wooledge.org/BashFAQ/020

Note that the -print0 option separates each entry by an ascii null character. Since shell parameters cannot contain nulls, this makes them the safest delimiter to use in processing them. As non-printing characters, they won't show up visibly in the direct output of find, but they are there, and subsequent commands can use them to safely split the text.

Code:

# This will load all regular files in a directory into an array:

while IFS='' read -rd '' ; do

  array+=( "$REPLY" )

done < <( find . -depth -type f -print0 )



# Now check the output

printf 'listing file: [%s]\n' "${array[@]}"

@David - Whilst the while loop makes perfect sense, is there a reason you know of why the following does not work?

Code:

IFS= read -r -d '' -a array <<<$(find -maxdepth 1 -type d -print0)

Could be, "it just doesn't", but my head seems to think it should :(

Thanks a lot to all of you.
while the codes given by all of you were working fine but it seems those were not fitting to my requirement in my script. Here is what i want to do:
find all the sub directories/ in a given directory and display their memory usage

Code:

arr_dir=(`find "/home/myadmin" -maxdepth 1 -type d`) #storing the values in array

for i in "${arr_dir[@]}" ; do 

mem_usag=`du -sh "$i" | cut -f 1`

echo -e "${mem_usag}\t/home/myadmin"

done

fi

/home/myadmin contains below directories
/home/myadmin/LINUX FINAL
/home/myadmin/Downloads
though i am able to store the values in the array but when du comes to LINUX FINAL it assumes two diff values due to bad naming convention(space).Is possible to store values in the arry in sucha way that du also handles these kind of folders ?
Please help..couldnt find a sol after tweaking my code in many ways

I have also tried to take the input to array as the below codes:
@David and @whizje

Code:

while IFS='' read -rd '' ; do

  arr_dir+=( "$REPLY" )

done < <(`find "/home/myadmin" -maxdepth 1 -type d -print0`)

for i in "${arr_dir[@]}" ; do 

mem_usag=`du -sh "$i" | cut -f 1`

echo -e "${mem_usag}\t/home/myadmin"

done

OR

arr_dir=(`find /home/myadmin/ -maxdepth 1 -type d -exec echo \"{}\" \;`)

but du gets starngled when it comes to folder names with spaces

Removing space character will change the directory name too, so it's not a good option. Instead you can leave the array and use process substitution, like:

Code:

while read -r dir; do 

space_usag=$(du -sh "$i" | awk '{print $1})  # It's not memory usege, but space usege

# space_usag=$(du -sh "$i" | awk -F"\t" '{print $1,$2})      # Prints space usage + dirname both

echo -e "$space_usag\t/home/myadmin"        # Leave this line. It's of no use. 

# echo -e "$space_usag\t$i"                  # Prints space usage + dirname both

done < <(find "/home/myadmin" -maxdepth 1 -type d` -print)

Hi All,
I just found shell is adding a front slash automatically to LINUX FINAL after pressing tab key as shown below

Code:

du -sh LINUX FINAL 

du: cannot access `LINUX': No such file or directory

du: cannot access `FINAL ': No such file or directory

du -sh LINUX\ FINAL\ 

8.0K        LINUX FINAL

Can this be incoraporated to my code ?. No idea how to do this. Please let me know if anyone has other solution

You can use double quote around file name, as:

Code:

du -sh \""$i"\"

Quote:

Originally Posted by shivaa (Post 4918114)

Removing space character will change the directory name too, so it's not a good option. Instead you can leave the array and use process substitution, like:

Code:

while read -r dir; do 

space_usag=$(du -sh "$i" | awk '{print $1})  # It's not memory usege, but space usege

# space_usag=$(du -sh "$i" | awk -F"\t" '{print $1,$2})      # Prints space usage + dirname both

echo -e "$space_usag\t/home/myadmin"        # Leave this line. It's of no use. 

# echo -e "$space_usag\t$i"                  # Prints space usage + dirname both

done < <(find "/home/myadmin" -maxdepth 1 -type d` -print)

unfortunately i am not aware of awk programming.could you please check whether my code is good to go which u gve

Code:

while read -r dir; do 

  mem_usag=`du -sh "$dir" | cut -f 1`    

  echo -e "$mem_usag\t$dir"                  

done <<`find "home/myadmin" -maxdepth 1 -type d -print`

the code

Code:

done <<`find "/home/myadmin" -maxdepth 1 -type d -print`

is throwing error.Pls bear me as i am a novice in unix progrm

Awk is not necessory at all. You can use cut also. Also don't modify the original syntax. Following should work.

Code:

while read -r dir; do 

mem_usag=$(du -sh \""$dir"\" | cut -d" " -f1)    

echo -e "$mem_usag\t$dir"                  

done < <(find "$j" -maxdepth 1 -type d -print)

You can also use sed to insert double quote at beginning and end of the directory names.

Also follow the post#5, and go through the links suggested by David. Most of your doubts will clear by following those links.

Quote:

Originally Posted by grail (Post 4915247)

@David - Whilst the while loop makes perfect sense, is there a reason you know of why the following does not work?

Code:

IFS= read -r -d '' -a array <<<$(find -maxdepth 1 -type d -print0)

Could be, "it just doesn't", but my head seems to think it should :(

It does seem like it at first glance, doesn't it? But with read's -d delimiter set to null, it will only process up to the first null in the input text, and so there's nothing in the string for IFS to split on.

The same thing would happen any time IFS and -d are set to the same thing.

Incidentally, when I use the here string+command substitution as above I get the entire output as a single string, but when I use a process substitution for the input I get the first file only. It appears that the command substitution is acting as a parameter and silently dropping the nulls from the input. A bit of testing with cat -A bears this out.

Code:

$ touch file{1..5}



$ cat -A <<<"$( find . -type f -print0 )"

./file5./file1./file3./file2./file4$



$ cat -A < <( find . -type f -print0 )

./file5^@./file1^@./file3^@./file2^@./file4^@

Actually, what I can't figure out is why this isn't working:

Code:

$ IFS='' read -ra array < <( find . -type f -print0 ) ; printf '%s\n' "${array[@]}"

./file5

Since the delimiter is now the default newline, I believe it should be processing the entire input and splitting it into the array on the nulls, but it's still only seeing the first value. :scratch:

Quote:

Originally Posted by prasunjit (Post 4918146)

unfortunately i am not aware of awk programming.could you please check whether my code is good to go which u gve

Code:

while read -r dir; do 

  mem_usag=`du -sh "$dir" | cut -f 1`    

  echo -e "$mem_usag\t$dir"                  

done <<`find "home/myadmin" -maxdepth 1 -type d -print`

1) $(..) is highly recommended over `..`. Please don't use backticks anymore unless you have to deal with a very old shell.

2) Once you have a string stored in a variable, instead of cut or other external commands, you can almost always use parameter substitution or another built-in string manipulation technique to process substrings from them.

The fewer external processes you use, the faster your script will generally be, unless you are doing bulk modification of large blocks of text, in which case something like sed or awk may be faster.

3) Your here string syntax is wrong. It uses three arrows, not two. But actually, you really should be using a process substitution instead, at least when using bash.

4) I'd probably use printf instead of echo -e for the output.

Code:

while IFS='' read -rd '' dir; do



    mem_usag=$( du -sh "$dir" )

    printf '%s\t%s\n' "${mem_usag%%[[:space:]]*}" "$dir"



done < <( find "home/myadmin" -maxdepth 1 -type d -print0 )

Beware of the space between the arrows in the '< <(..)' pattern. '<(..)' is the process substitution itself, and the other arrow is a standard shell redirection. Unlike with a normal file, however, they need to be separated into individual tokens.

Although why you want to go to the trouble to extract the first column from the du output only to print it out again in exactly the same format (right down to the tab separating the columns) is beyond me.

Speaking of which, instead of the above loop, you could just use find on its own:

Code:

find "home/myadmin" -maxdepth 1 -type d -exec du -sh '{}' \;

Quote:

Originally Posted by shivaa (Post 4918125)

You can use double quote around file name, as:

Code:

du -sh \""$i"\"

No, this will not work. The only purpose of quoting is to tell the shell what is and isn't part of an argument (a "token"), i.e, which characters to treat specially, and which ones not to. They are among the first things processed in the shell parsing order, and are removed from the line before the command is executed. Everything left over after processing is considered a literal value, part of the argument itself.

So by escaping the quotes in the above command, you are in effect telling the shell (and subsequently du) that they are part of the filename itself. All you will get is a "file not found" error from the above.

There are almost no cases where you can, or would want to, try to embed syntactical quotes like this. About the only time it would work at all is if some sort of double-parsing were being done, such as with eval. And double-parsing is something you generally want to avoid doing at all costs, for security reasons.

I'm trying to put a command in a variable, but the complex cases always fail!
http://mywiki.wooledge.org/BashFAQ/050

Eval command and security issues
http://mywiki.wooledge.org/BashFAQ/048

Quote:

Originally Posted by David the H. (Post 4919241)

Code:

while IFS='' read -rd '' dir; do



    mem_usag=$( du -sh "$dir" )

    printf '%s\t%s\n' "${mem_usag%%[[:space:]]*}" "$dir"



done < <( find "home/myadmin" -maxdepth 1 -type d -print0 )

Code:

find "home/myadmin" -maxdepth 1 -type d -exec du -sh '{}' \;

Finally i got my code working by inserting above changes.The IFS has handled the culprits and i am able to force du to do its work.
I have one doubt here-By doing this am i not making any changes to my IFS value ?

The below code really ashamed me as i was going in a very complex way and i totally forgot about -exec which could do my job easily

Code:

find "home/myadmin" -maxdepth 1 -type d -exec du -sh '{}' \;

Thanks for all the tips and tricks

@David..

I would be thankful to you if you can please explain the below. i tried to dig but i didnt get any satisfactory explanation.

i was of the view that -print0 option in find command allows file names that contain newlines or other types of white space to be correctly interpreted by programs that process the find output
so i thpught du command which processes the ouput of find command in my script should be able to handle the file names with spaces (ex:LINUX FINAL). So why i need to use IFS again ?

Code:

while IFS='' read -rd '' dir; do



    mem_usag=$( du -sh "$dir" )

    printf '%s\t%s\n' "${mem_usag%%[[:space:]]*}" "$dir"



done < <( find "home/myadmin" -maxdepth 1 -type d -print0 )

Also the below code doesnt use any -print0 or IFS kind of stuff and still it is giving me result

Code:

find "home/myadmin" -maxdepth 1 -type d -exec du -sh '{}' \;

I am afraid i am looking foolish by asking these but i tried my best to undersatnd
the IFS but couldn't relate to the above queries.

Don't feel too embarrassed about doing it the hard way. We all experience that kind of thing sometimes. That's what helps you learn. Besides, I didn't really know myself whether you really intended your final output to be like that, or if you were just keeping it simple for this discussion. That's why I discussed the loop on detail first before mentioning it.

You don't need -print0 when using find alone because you aren't sending the output anywhere for another program to read. Instead you're executing du directly from within find, ignoring the shell entirely.

The basic find syntax is this:

Code:

find <starting dirs> <global options> <matching expressions> <actions>

Things like -print and -exec are actions; they do something using the files matched by the previous expressions. the various print options send the list to stdout, while the -exec option runs an external command on them. In this case -exec runs one instance of du for each file found, and its du that's printing its output to the screen according to its usual behavior.

http://mywiki.wooledge.org/UsingFind
http://www.grymoire.com/Unix/Find.html

Re: your question about changing IFS, no you are not changing the global setting this way. Any simple command directly preceded by a variable setting (with only whitespace between them) will be launched with that variable in its environment, while not affecting the parent environment.

Code:

IFS='' read -rd '' dir <input

In this case the IFS setting only affects the action of read. You could instead change the global environment IFS and it would work too, but then you have to remember to set it back to the default afterwards. This is much cleaner.

( Edit: There's one final thing about IFS+read that I need to mention. When IFS includes whitespace, it will remove any of those whitespace characters found at the beginning or end of the string, even if there's no other word-splitting being done to the input (i.e. for setting a multiple variable list or array). Giving read an IFS set to null disables this so that you're guaranteed to get the entire input unaltered. )

As for your previous problems, when referring to post #7, the key issue is here:

Code:

arr_dir=(`find "/home/myadmin" -maxdepth 1 -type d`)

It's the same old word-splitting trouble again. The "`..`" command substitution first executes find and inserts its output into the array-setting command. THEN word-splitting occurs on that output, and the array ends up setting one index per word, rather than the desired one-per-file.

There is no easy way to work around this kind of situation. When expanding variables and command substitutions you can only have it treated as a single word (by quoting it), or have it word-split according to the IFS value. There's no way to tell it which spaces are inside the filenames and which are separating them. That's why you have to use null separators and a loop.

Note that if the delimiter in the ouptput was a non-whitespace character, such as a colon, we could use IFS to split it safely in this fashion. But null separators don't work here, as I discovered in post #14.

As for post #8, it's mostly ok actually, except for this:

Code:

done < <(`find "/home/myadmin" -maxdepth 1 -type d -print0`)

You have a command substitution inside a process substitution. As far as I can tell this should not work at all. The p.s. subshell would attempt to treat the expanded file list as a command and try to execute it directly, which would no doubt fail. Remove the backticks and it will probably work.

I'd also point out that this is a good example of why "$(..)" is better than backticks too. It would be easier to see the improper nesting.

For a better understanding of IFS, start with the first three links to get a better understanding of how the shell processes arguments and whitespace internaly, and then the fourth one for more on how IFS works specifically.

http://mywiki.wooledge.org/Arguments
http://mywiki.wooledge.org/WordSplitting
http://mywiki.wooledge.org/Quotes

http://mywiki.wooledge.org/IFS

Finally, having a good understanding of the shell's parsing order, what happens before or after what, will help you to avoid many mistakes like this.

http://mywiki.wooledge.org/BashParser