LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-26-2013, 01:31 PM   #16
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,668

Original Poster
Rep: Reputation: 46

druuna, I do have one other question, more for my knowledge rather than this exact problem.

If I have a variable
Quote:
file = "/path/*D_2*Stuff*.pdf"
for an if then statement in a .sh, how would I apply this better way to get the file I really need to assign it to that file variable? we have some if thens in other stuff we do, and i'm just wondering in case this new way to get what I really want may prove to be a better setup than what we currently have.
 
Old 07-26-2013, 01:33 PM   #17
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Quote:
Originally Posted by David the H. View Post
Assuming that the files sort naturally
Isn't that a bit of a dangerous assumption?

Too my knowledge this will not be the case if there would be files like these:

- Junk_D_2_07-26-2013_K10_Stuff_MoreJunk.pdf
- Junk_D_2_07-26-2013_K11_Stuff_MoreJunk.pdf

Sorting would be:

- Junk_D_2_07-26-2013_K1_Stuff_MoreJunk.pdf
- Junk_D_2_07-26-2013_K10_Stuff_MoreJunk.pdf
- Junk_D_2_07-26-2013_K11_Stuff_MoreJunk.pdf
- Junk_D_2_07-26-2013_K2_Stuff_MoreJunk.pdf

The ls command, for example, does have the -v switch, but how do I set this for bash when using something like files=( *D_2*Stuff*.pdf )? Setting a different locale [tried: C,POSIX,nl_NL.utf8 and en_US.utf8] doesn't seem to do the trick.

Last edited by druuna; 07-26-2013 at 01:40 PM.
 
Old 07-26-2013, 01:39 PM   #18
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Quote:
Originally Posted by rjo98 View Post
druuna, I do have one other question, more for my knowledge rather than this exact problem.

If I have a variable for an if then statement in a .sh, how would I apply this better way to get the file I really need to assign it to that file variable? we have some if thens in other stuff we do, and i'm just wondering in case this new way to get what I really want may prove to be a better setup than what we currently have.
Code:
file="`ls -tr /folder/*D_2*Stuff*.pdf | tail -1`"
No spaces before or after the = sign.

BTW: Do keep an eye out for David the H.'s posts. Although I think your are safe using ls in this particular case, he has a point about using ls's output as input for other commands. I do, however, have to point out that ls has a lot of options that can overcome most, if not all the Using ls as input is bad, m'kay... replies....

Last edited by druuna; 07-26-2013 at 02:23 PM. Reason: Changed comment on using ls
 
Old 07-26-2013, 02:46 PM   #19
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,668

Original Poster
Rep: Reputation: 46
ah ok, so you can just use it the same way kinda to assign it to a variable then, that's cool.

I'll definitely keep up on this thread, see what he says.
 
Old 07-26-2013, 03:12 PM   #20
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Quote:
Originally Posted by rjo98 View Post
ah ok, so you can just use it the same way kinda to assign it to a variable then, that's cool.
Exactly the same way.

This is how you assign a variable:
Code:
variable="<content>"
<content> can for example be:
- a string ( variable="foo" )
- a variable ( variable="$bar" / variable="${bar}" )
- output of a command ( variable="$(command)" / variable="`command`" )

It is very important to know and understand how the shell parses/processes the command line. Do have a good look at this link: The Bash Parser

And another one on that subject: (POSIX) Shell Command Line Processing Outline
 
Old 07-26-2013, 03:21 PM   #21
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,668

Original Poster
Rep: Reputation: 46
cool, thanks man
 
Old 08-02-2013, 01:27 PM   #22
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Quote:
Originally Posted by druuna View Post
Isn't that a bit of a dangerous assumption?
Actually it's not so much an assumption as a limitation. If the patterns don't sort naturally, then you just have to work harder to match them exactly. It all comes down to how much you can trust the input.

Quote:
Too my knowledge this will not be the case if there would be files like these:

- Junk_D_2_07-26-2013_K10_Stuff_MoreJunk.pdf
- Junk_D_2_07-26-2013_K11_Stuff_MoreJunk.pdf

Sorting would be:

- Junk_D_2_07-26-2013_K1_Stuff_MoreJunk.pdf
- Junk_D_2_07-26-2013_K10_Stuff_MoreJunk.pdf
- Junk_D_2_07-26-2013_K11_Stuff_MoreJunk.pdf
- Junk_D_2_07-26-2013_K2_Stuff_MoreJunk.pdf

The ls command, for example, does have the -v switch, but how do I set this for bash when using something like files=( *D_2*Stuff*.pdf )? Setting a different locale [tried: C,POSIX,nl_NL.utf8 and en_US.utf8] doesn't seem to do the trick.
Yes, this is the main limitation. Lexographical sorting doesn't work properly when you have different numbers of digits. Locale has no effect on that, since the strings are being sorted by individual character-column.

That's why I recommend always zero-padding numbers in your filenames, so the necessary columns always line up. Dates, too, should be in ISO-8601 standard format (YYYY-MM-DDTHH:MM:SS, hyphens optional), which will also fall out in natural sorting order.

It's still possible to use multiple globs and/or extended globbing patterns on non-zero-padded data, as I mentioned at the end of my last post, although it can become increasingly cumbersome to handle the larger the range you need to match.

Code:
files=( *_[0-9]_Stuff* *_k[0-9]_Stuff* *_k[0-9][0-9]_Stuff* )
Conveniently, though, it's possible to combine the globs with brace expansion to shorten things up. The braces will expand first into the individual patterns, which will then match (or not) at the end of the parsing sequence. You'll probably need to enable the nullglob option too, however, to ignore non-matching patterns.

Code:
shopt -s nullglob
files=( *_{,k,k[0-9]}[0-9]_Stuff* )

By the way, just to be clear, I'm not completely against using ls in all situations. If you know what you're doing then it can be a convenient quick-and-dirty method for sorting. The files just need to not contain embedded newlines, or depending on how you're parsing it, other whitespace.

But in general I prefer to keep to the safest methods as much as possible, even when not strictly necessary. It's the most likely to avoid potential problems, usually the most efficient, and it keeps me from getting lazy. In most cases it's also not all that harder to use.
 
Old 08-02-2013, 01:43 PM   #23
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Quote:
Originally Posted by rjo98 View Post
druuna, I do have one other question, more for my knowledge rather than this exact problem.

If I have a variable
Code:
file = "/path/*D_2*Stuff*.pdf"
for an if then statement in a .sh, how would I apply this better way to get the file I really need to assign it to that file variable? we have some if thens in other stuff we do, and i'm just wondering in case this new way to get what I really want may prove to be a better setup than what we currently have.
Any time you have a pattern that can potentially expand into multiple strings, you should always use an array (assuming the shell supports them). That's what they're designed for. Scalar variables are for single strings, while arrays are for lists of multiple, related strings.

I've already shown how to set an array from a globbing pattern. You can always reassign a single array entry to a scalar variable afterward if needed.


On the other hand, if you want to store a globbing pattern in a variable and use that later, all you have to do is make sure to quote it when setting it.

Code:
file_match="/path/*D_2*Stuff*.pdf"
But do not quote it when expanding it, or else the globbing pattern won't work. This is an exception to the usual advice to always quote variables.

Code:
for fname in $file_match ; do
   echo "This is file: $fname"
done
It all comes down to how arguments are handled, and to the command parsing order. See these three links, and the one to the parsing order that druuna posted. It's a vital concept in scripting.

http://mywiki.wooledge.org/Arguments
http://mywiki.wooledge.org/WordSplitting
http://mywiki.wooledge.org/Quotes
 
Old 08-03-2013, 03:07 AM   #24
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Quote:
Originally Posted by David the H. View Post
Actually it's not so much an assumption as a limitation. If the patterns don't sort naturally, then you just have to work harder to match them exactly. It all comes down to how much you can trust the input.

...<snip>....

By the way, just to be clear, I'm not completely against using ls in all situations. If you know what you're doing then it can be a convenient quick-and-dirty method for sorting. The files just need to not contain embedded newlines, or depending on how you're parsing it, other whitespace.

But in general I prefer to keep to the safest methods as much as possible, even when not strictly necessary. It's the most likely to avoid potential problems, usually the most efficient, and it keeps me from getting lazy. In most cases it's also not all that harder to use.
Looking at your answers (not limited to this thread) it is clear that you approach things from a teaching (theoretical?) point of view. I, on the other hand, approach things from a practical point of view (don't work harder, work smarter / get the job done). In both cases one should know the limitations and dangers that one might come across.

I'm not saying that your approach is "wrong" in any way, on the contrary; LQ is mainly used by beginners and your elaborate posts are, in general, a fountain of information for those that are willing to take the time to study/try them. I am, however, not too fond of valid answers being discarded as Uggh/unsafe/sloppy/etc.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Pick out contents of a file andre3050 Linux - Newbie 16 07-03-2013 05:07 PM
How to search for a string in a text file using wildcards(*,?) tnjones Programming 2 08-23-2008 08:04 AM
how to pick random file name from a list of filenames in a text file. pdklinux79 Linux - Newbie 9 06-20-2008 03:46 PM
I need to pick out the first, last and how many from a file...?!!? vous Programming 3 03-22-2005 02:44 PM
Using wildcards with PHP file functions patpawlowski Programming 3 03-18-2004 10:20 AM


All times are GMT -5. The time now is 02:32 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration