LinuxQuestions.org
Visit the LQ Articles and Editorials section
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 03-26-2013, 11:15 AM   #16
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946

Quote:
Originally Posted by shivaa View Post
You can use double quote around file name, as:
Code:
du -sh \""$i"\"
No, this will not work. The only purpose of quoting is to tell the shell what is and isn't part of an argument (a "token"), i.e, which characters to treat specially, and which ones not to. They are among the first things processed in the shell parsing order, and are removed from the line before the command is executed. Everything left over after processing is considered a literal value, part of the argument itself.

So by escaping the quotes in the above command, you are in effect telling the shell (and subsequently du) that they are part of the filename itself. All you will get is a "file not found" error from the above.

There are almost no cases where you can, or would want to, try to embed syntactical quotes like this. About the only time it would work at all is if some sort of double-parsing were being done, such as with eval. And double-parsing is something you generally want to avoid doing at all costs, for security reasons.

I'm trying to put a command in a variable, but the complex cases always fail!
http://mywiki.wooledge.org/BashFAQ/050

Eval command and security issues
http://mywiki.wooledge.org/BashFAQ/048
 
1 members found this post helpful.
Old 03-27-2013, 12:06 AM   #17
prasunjit
LQ Newbie
 
Registered: Dec 2012
Posts: 17

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by David the H. View Post
1) $(..) is highly recommended over `..`. Please don't use backticks anymore unless you have to deal with a very old shell.

2) Once you have a string stored in a variable, instead of cut or other external commands, you can almost always use parameter substitution or another built-in string manipulation technique to process substrings from them.

The fewer external processes you use, the faster your script will generally be, unless you are doing bulk modification of large blocks of text, in which case something like sed or awk may be faster.

3) Your here string syntax is wrong. It uses three arrows, not two. But actually, you really should be using a process substitution instead, at least when using bash.

4) I'd probably use printf instead of echo -e for the output.

Code:
while IFS='' read -rd '' dir; do

    mem_usag=$( du -sh "$dir" )
    printf '%s\t%s\n' "${mem_usag%%[[:space:]]*}" "$dir"

done < <( find "home/myadmin" -maxdepth 1 -type d -print0 )
Beware of the space between the arrows in the '< <(..)' pattern. '<(..)' is the process substitution itself, and the other arrow is a standard shell redirection. Unlike with a normal file, however, they need to be separated into individual tokens.

Although why you want to go to the trouble to extract the first column from the du output only to print it out again in exactly the same format (right down to the tab separating the columns) is beyond me.

Speaking of which, instead of the above loop, you could just use find on its own:
Code:
find "home/myadmin" -maxdepth 1 -type d -exec du -sh '{}' \;
Finally i got my code working by inserting above changes.The IFS has handled the culprits and i am able to force du to do its work.
I have one doubt here-By doing this am i not making any changes to my IFS value ?

The below code really ashamed me as i was going in a very complex way and i totally forgot about -exec which could do my job easily
Code:
find "home/myadmin" -maxdepth 1 -type d -exec du -sh '{}' \;
Thanks for all the tips and tricks
 
Old 03-27-2013, 01:50 AM   #18
prasunjit
LQ Newbie
 
Registered: Dec 2012
Posts: 17

Original Poster
Rep: Reputation: Disabled
@David..

I would be thankful to you if you can please explain the below. i tried to dig but i didnt get any satisfactory explanation.

i was of the view that -print0 option in find command allows file names that contain newlines or other types of white space to be correctly interpreted by programs that process the find output
so i thpught du command which processes the ouput of find command in my script should be able to handle the file names with spaces (ex:LINUX FINAL). So why i need to use IFS again ?
Code:
while IFS='' read -rd '' dir; do

    mem_usag=$( du -sh "$dir" )
    printf '%s\t%s\n' "${mem_usag%%[[:space:]]*}" "$dir"

done < <( find "home/myadmin" -maxdepth 1 -type d -print0 )
Also the below code doesnt use any -print0 or IFS kind of stuff and still it is giving me result
Code:
find "home/myadmin" -maxdepth 1 -type d -exec du -sh '{}' \;
I am afraid i am looking foolish by asking these but i tried my best to undersatnd
the IFS but couldn't relate to the above queries.
 
Old 03-28-2013, 03:28 AM   #19
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946Reputation: 1946
Don't feel too embarrassed about doing it the hard way. We all experience that kind of thing sometimes. That's what helps you learn. Besides, I didn't really know myself whether you really intended your final output to be like that, or if you were just keeping it simple for this discussion. That's why I discussed the loop on detail first before mentioning it.


You don't need -print0 when using find alone because you aren't sending the output anywhere for another program to read. Instead you're executing du directly from within find, ignoring the shell entirely.

The basic find syntax is this:
Code:
find <starting dirs> <global options> <matching expressions> <actions>
Things like -print and -exec are actions; they do something using the files matched by the previous expressions. the various print options send the list to stdout, while the -exec option runs an external command on them. In this case -exec runs one instance of du for each file found, and its du that's printing its output to the screen according to its usual behavior.

http://mywiki.wooledge.org/UsingFind
http://www.grymoire.com/Unix/Find.html


Re: your question about changing IFS, no you are not changing the global setting this way. Any simple command directly preceded by a variable setting (with only whitespace between them) will be launched with that variable in its environment, while not affecting the parent environment.

Code:
IFS='' read -rd '' dir <input
In this case the IFS setting only affects the action of read. You could instead change the global environment IFS and it would work too, but then you have to remember to set it back to the default afterwards. This is much cleaner.

( Edit: There's one final thing about IFS+read that I need to mention. When IFS includes whitespace, it will remove any of those whitespace characters found at the beginning or end of the string, even if there's no other word-splitting being done to the input (i.e. for setting a multiple variable list or array). Giving read an IFS set to null disables this so that you're guaranteed to get the entire input unaltered. )


As for your previous problems, when referring to post #7, the key issue is here:

Code:
arr_dir=(`find "/home/myadmin" -maxdepth 1 -type d`)
It's the same old word-splitting trouble again. The "`..`" command substitution first executes find and inserts its output into the array-setting command. THEN word-splitting occurs on that output, and the array ends up setting one index per word, rather than the desired one-per-file.

There is no easy way to work around this kind of situation. When expanding variables and command substitutions you can only have it treated as a single word (by quoting it), or have it word-split according to the IFS value. There's no way to tell it which spaces are inside the filenames and which are separating them. That's why you have to use null separators and a loop.

Note that if the delimiter in the ouptput was a non-whitespace character, such as a colon, we could use IFS to split it safely in this fashion. But null separators don't work here, as I discovered in post #14.


As for post #8, it's mostly ok actually, except for this:
Code:
done < <(`find "/home/myadmin" -maxdepth 1 -type d -print0`)
You have a command substitution inside a process substitution. As far as I can tell this should not work at all. The p.s. subshell would attempt to treat the expanded file list as a command and try to execute it directly, which would no doubt fail. Remove the backticks and it will probably work.

I'd also point out that this is a good example of why "$(..)" is better than backticks too. It would be easier to see the improper nesting.


For a better understanding of IFS, start with the first three links to get a better understanding of how the shell processes arguments and whitespace internaly, and then the fourth one for more on how IFS works specifically.

http://mywiki.wooledge.org/Arguments
http://mywiki.wooledge.org/WordSplitting
http://mywiki.wooledge.org/Quotes

http://mywiki.wooledge.org/IFS


Finally, having a good understanding of the shell's parsing order, what happens before or after what, will help you to avoid many mistakes like this.

http://mywiki.wooledge.org/BashParser

Last edited by David the H.; 03-28-2013 at 03:53 AM. Reason: wording mistake + stated edit
 
1 members found this post helpful.
  


Reply

Tags
find


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Problem wiht openser. mythiliram Linux - Newbie 2 08-10-2009 03:39 PM
problem wiht 5.1 chanel surround on slackware boneff Slackware 2 06-07-2004 08:02 PM
problem wiht wget true_atlantis Linux - Software 5 01-13-2004 07:52 PM
Problem wiht Redhat 8 and new kernel lafuma Linux - Distributions 2 06-20-2003 04:29 PM


All times are GMT -5. The time now is 02:33 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration