LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-26-2013, 09:07 AM   #1
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,674

Rep: Reputation: 47
Need help with wildcards to pick correct file


Hi guys. Wildcards always give me troubles, so I'm hoping someone can help me. Below is a directory listing, and the current wildcard pattern I was using. Only problem with this is that is doesn't account for the _0_ or the _K1_ part that I just realized I need to account for. and with that part, a K1 would be the one I want to find over the 0, a K2 over a K1 and 0, and K3 over a K2, K1, 0, etc etc, that I would want to find and copy to another folder. Right now I just have a cp with that wildcard below, but realize i now need to account for this other part and I have no idea how in this .sh script I have that was just some simple cp's with wildcard patterns. Any ideas?

Code:
ll *D_2*Stuff*.pdf
-rw-rw-rw- 1 user group 7218255 Jul 25 23:32 Junk_D_2_07-26-2013_0_Stuff_MoreJunk.pdf
-rw-rw-rw- 1 user group 7650126 Jul 26 00:01 Junk_D_2_07-26-2013_K1_Stuff_MoreJunk.pdf
 
Old 07-26-2013, 09:18 AM   #2
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
This part is unclear to me:
Quote:
a K1 would be the one I want to find over the 0, a K2 over a K1 and 0, and K3 over a K2, K1, 0, etc etc, that I would want to find and copy to another folder.
Are you saying that if a K1, K0 and 0 file exists, you only want to copy the K1 file? I.e.: Only copy the highest numbered K file (or the 0 file if no K file exists)?

If so: You cannot do this with cp and wild-cards alone, you need other commands as well.

BTW: Is a higher K version always the newest version?
 
1 members found this post helpful.
Old 07-26-2013, 09:24 AM   #3
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,674

Original Poster
Rep: Reputation: 47
Hi druuna. Yeah, you have it right. The highest numbered K file or the 0 file if no K's exist (while still matching the other stuff in my ll) is the one I need to yank out of that folder.

That was my fear that I can't do this with cp and wildcards alone. Putting simple commands like cp and mv in a .sh are about the extent of my scripting skills haha.

Higher K should in theory always be the newest, but in the system generating those files it's user input so someone could get them out of order I suppose, but the timestamp would probably be ok to key off of, as I truly do want the last version (0 or K#) of that file.
 
Old 07-26-2013, 09:37 AM   #4
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
This should do what you want:
Code:
cp "$(ls -tr *D_2*Stuff*.pdf | tail -1)" /some/dir/
The above consist of (roughly) 2 parts:

- 1: the ls -tr ... | tail -1 looks for files that match your wildcard, newest one last. the tail -1 makes sure that only the last one is shown. The $( .... ) construct makes sure that this is done before the cp command executes.

- 2: the output of the first part is used in the cp command. Do _NOT_ forget the double quotes. They are there to make sure that cp can handle possible special characters (a space in the filename being one of those).

BTW: These might come in handy:

Last edited by druuna; 07-26-2013 at 09:38 AM.
 
1 members found this post helpful.
Old 07-26-2013, 09:40 AM   #5
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,674

Original Poster
Rep: Reputation: 47
Very cool! I will try that out when I get in today. Thanks druuna.

And I'll take a peek at those links too. I wish I could do all this kind of fancy stuff.
 
Old 07-26-2013, 09:41 AM   #6
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Quote:
Originally Posted by rjo98 View Post
Very cool! I will try that out when I get in today. Thanks druuna.
You're welcome

Quote:
I wish I could do all this kind of fancy stuff.
Give it time. I'm still learning new things after all those years behind a keyboard......
 
1 members found this post helpful.
Old 07-26-2013, 09:42 AM   #7
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,674

Original Poster
Rep: Reputation: 47
I hear you there, not a day goes by that I don't learn/stumble upon something new haha
 
Old 07-26-2013, 11:08 AM   #8
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,674

Original Poster
Rep: Reputation: 47
If I try to run a cp "$(...)" /some/folder/ manually i get Illegal variable name. is that because that line will only work within a .sh?

doing the ls part works fine, its just when I put it all together I get that error.

Last edited by rjo98; 07-26-2013 at 11:12 AM.
 
Old 07-26-2013, 11:23 AM   #9
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Are you actually using a variable in that command? Maybe show the exact command and the error it throws?

A test shows it works on my side:
Code:
~/Schuur/Tmp $ ls -l
total 0
-rw-r----- 1 druuna druuna 0 jul 26 15:24 Aunk_D_2_07-26-2013_K0_Stuff_MoreJunk.pdf
-rw-r----- 1 druuna druuna 0 jul 26 15:30 Cunk_D_2_07-26-2013_K7 _Stuff _MoreJunk.pdf
-rw-r----- 1 druuna druuna 0 jul 26 15:22 Junk_D_2_07-26-2013_0_Stuff_MoreJunk.pdf
-rw-r----- 1 druuna druuna 0 jul 26 15:22 Junk_D_2_07-26-2013_K1_Stuff_MoreJunk.pdf
-rw-r----- 1 druuna druuna 0 jul 26 15:23 Junk_D_2_07-26-2013_K3_Stuff_MoreJunk.pdf
-rw-r----- 1 druuna druuna 0 jul 26 15:24 Lunk_D_2_07-26-2013_K2_Stuff_MoreJunk.pdf

~/Schuur/Tmp $ ls -tr *D_2*Stuff*.pdf | tail -1
Cunk_D_2_07-26-2013_K7 _Stuff _MoreJunk.pdf

~/Schuur/Tmp $ cp "$(ls -tr *D_2*Stuff*.pdf | tail -1)" /tmp/

~/Schuur/Tmp $ ls -l /tmp/*D_2*Stuff*.pdf
-rw-r----- 1 druuna druuna 0 jul 26 15:30 /tmp/Cunk_D_2_07-26-2013_K7 _Stuff _MoreJunk.pdf
 
Old 07-26-2013, 11:32 AM   #10
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,674

Original Poster
Rep: Reputation: 47
I'm guessing I did something wrong


cp "$(ls -tr /folder/*D_2*Stuff*.pdf | tail -1)" /home/rj/tmp/
Illegal variable name.
ls -tr /folder/*D_2*Stuff*.pdf | tail -1
/folder/Test_D_2_07-26-2013_K1_Stuff_001.pdf
 
Old 07-26-2013, 11:40 AM   #11
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Strange, which bash version are you using (bash --version):
Code:
$ bash --version
GNU bash, version 4.1.5(1)-release (x86_64-pc-linux-gnu)
If you have an older bash version the $( ... ) construct might not work, try using the "old way", using back-ticks, instead:
Code:
cp "`ls -tr /folder/*D_2*Stuff*.pdf | tail -1`" /home/rj/tmp/
This is being done on a Linux box, right?
 
Old 07-26-2013, 11:45 AM   #12
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,674

Original Poster
Rep: Reputation: 47
based on that command, not a recent one haha

GNU bash, version 3.2.25(1)-release (x86_64-redhat-linux-gnu)


I just tried removing the $ and substituting apostrophes for the ( and ) but now I get a cannot stat everything in the apostrophes.

are backticks different than apostrophes?
 
Old 07-26-2013, 11:50 AM   #13
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Quote:
Originally Posted by rjo98 View Post
based on that command, not a recent one haha

GNU bash, version 3.2.25(1)-release (x86_64-redhat-linux-gnu)


I just tried removing the $ and substituting apostrophes for the ( and ) but now I get a cannot stat everything in the apostrophes.

are backticks different than apostrophes?
Back-ticks (or Grave_accents) differ from single quotes (Apostrophes). You need to use back-ticks.

Last edited by druuna; 07-26-2013 at 11:51 AM. Reason: Typo: singel -> single
 
Old 07-26-2013, 12:53 PM   #14
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958Reputation: 1958
Uggh, All this stuff with ls and command substitutions and whatnot. Quite sloppy and unsafe, IMO.

Assuming that the files sort naturally, and you're using a shell with arrays, it shouldn't be difficult. Use globbing to match the best subset of files you can, and copy the last one matched.

Code:
files=( *D_2*Stuff*.pdf )
cp "${files[@]:(-1)}" targetdir
If that isn't accurate enough, you can try using bash/ksh's extended globbing patterns. Or if you need something in the middle of the list, run a simple for loop over the array and match the filename you want with a case construct or something.

And finally, there's always find.

How can I use array variables?
http://mywiki.wooledge.org/BashFAQ/005/

How can I find the latest (newest, earliest, oldest) file in a directory?
http://mywiki.wooledge.org/BashFAQ/003

http://mywiki.wooledge.org/UsingFind
http://www.grymoire.com/Unix/Find.html


Edit: BTW, for even better control, you can also use multiple globs together. To better ensure matching the desired OP pattern exactly, try something like this:

Code:
files=( *D_2*_0_Stuff*.pdf *D_2*_k[0-9]_Stuff*.pdf )
This ensures that the "0" files will always come earlier in the array than the "k" files, however they might otherwise sort out.

Last edited by David the H.; 07-26-2013 at 01:07 PM. Reason: Altered array entry for bash v.3 compatibility
 
Old 07-26-2013, 01:21 PM   #15
rjo98
Senior Member
 
Registered: Jun 2009
Location: US
Distribution: RHEL, CentOS
Posts: 1,674

Original Poster
Rep: Reputation: 47
OHHH, back-ticks, to the left of the 1 on the keyboard. I think I've only ever used that key for ~ so totally forgot about those haha. It worked with the back-ticks!!

Thanks David. I understand the general idea of an array, and I have seen where people put ranges of stuff in square brackets before too, not really knowing how that worked. I'll look at your examples though and try to figure out why those would work too, as I'd like to learn about that way as well.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Pick out contents of a file andre3050 Linux - Newbie 16 07-03-2013 05:07 PM
How to search for a string in a text file using wildcards(*,?) tnjones Programming 2 08-23-2008 08:04 AM
how to pick random file name from a list of filenames in a text file. pdklinux79 Linux - Newbie 9 06-20-2008 03:46 PM
I need to pick out the first, last and how many from a file...?!!? vous Programming 3 03-22-2005 02:44 PM
Using wildcards with PHP file functions patpawlowski Programming 3 03-18-2004 10:20 AM


All times are GMT -5. The time now is 05:01 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration