Need help with wildcards to pick correct file
 

Hi guys. Wildcards always give me troubles, so I'm hoping someone can help me. Below is a directory listing, and the current wildcard pattern I was using. Only problem with this is that is doesn't account for the _0_ or the _K1_ part that I just realized I need to account for. and with that part, a K1 would be the one I want to find over the 0, a K2 over a K1 and 0, and K3 over a K2, K1, 0, etc etc, that I would want to find and copy to another folder. Right now I just have a cp with that wildcard below, but realize i now need to account for this other part and I have no idea how in this .sh script I have that was just some simple cp's with wildcard patterns. Any ideas?

This part is unclear to me:Are you saying that if a K1, K0 and 0 file exists, you only want to copy the K1 file? I.e.: Only copy the highest numbered K file (or the 0 file if no K file exists)?

If so: You cannot do this with cp and wild-cards alone, you need other commands as well.

BTW: Is a higher K version always the newest version?

Hi druuna. Yeah, you have it right. The highest numbered K file or the 0 file if no K's exist (while still matching the other stuff in my ll) is the one I need to yank out of that folder.

That was my fear that I can't do this with cp and wildcards alone. Putting simple commands like cp and mv in a .sh are about the extent of my scripting skills haha.

Higher K should in theory always be the newest, but in the system generating those files it's user input so someone could get them out of order I suppose, but the timestamp would probably be ok to key off of, as I truly do want the last version (0 or K#) of that file.

This should do what you want:The above consist of (roughly) 2 parts:

- 1: the ls -tr ... | tail -1 looks for files that match your wildcard, newest one last. the tail -1 makes sure that only the last one is shown. The $( .... ) construct makes sure that this is done before the cp command executes.

- 2: the output of the first part is used in the cp command. Do _NOT_ forget the double quotes. They are there to make sure that cp can handle possible special characters (a space in the filename being one of those).

BTW: These might come in handy:

• BASH Programming - Introduction HOW-TO
• Bash Guide for Beginners
• Advanced Bash-Scripting Guide

• Bash Pitfalls
• Bash Frequently Asked Questions (BashFAQ)
• Parameter expansion

Very cool! I will try that out when I get in today. Thanks druuna.

And I'll take a peek at those links too. I wish I could do all this kind of fancy stuff.

You're welcome :)

Give it time. I'm still learning new things after all those years behind a keyboard......

I hear you there, not a day goes by that I don't learn/stumble upon something new haha

If I try to run a cp "$(...)" /some/folder/ manually i get Illegal variable name. is that because that line will only work within a .sh?

doing the ls part works fine, its just when I put it all together I get that error.

Are you actually using a variable in that command? Maybe show the exact command and the error it throws?

A test shows it works on my side:

I'm guessing I did something wrong :(


cp "$(ls -tr /folder/*D_2*Stuff*.pdf | tail -1)" /home/rj/tmp/
Illegal variable name.
ls -tr /folder/*D_2*Stuff*.pdf | tail -1
/folder/Test_D_2_07-26-2013_K1_Stuff_001.pdf

Strange, which bash version are you using (bash --version):If you have an older bash version the $( ... ) construct might not work, try using the "old way", using back-ticks, instead: This is being done on a Linux box, right?

based on that command, not a recent one haha

GNU bash, version 3.2.25(1)-release (x86_64-redhat-linux-gnu)


I just tried removing the $ and substituting apostrophes for the ( and ) but now I get a cannot stat everything in the apostrophes.

are backticks different than apostrophes?

Back-ticks (or Grave_accents) differ from single quotes (Apostrophes). You need to use back-ticks.

Uggh, All this stuff with ls and command substitutions and whatnot. Quite sloppy and unsafe, IMO.

Assuming that the files sort naturally, and you're using a shell with arrays, it shouldn't be difficult. Use globbing to match the best subset of files you can, and copy the last one matched.

If that isn't accurate enough, you can try using bash/ksh's extended globbing patterns. Or if you need something in the middle of the list, run a simple for loop over the array and match the filename you want with a case construct or something.

And finally, there's always find.

How can I use array variables?
http://mywiki.wooledge.org/BashFAQ/005/

How can I find the latest (newest, earliest, oldest) file in a directory?
http://mywiki.wooledge.org/BashFAQ/003

http://mywiki.wooledge.org/UsingFind
http://www.grymoire.com/Unix/Find.html


Edit: BTW, for even better control, you can also use multiple globs together. To better ensure matching the desired OP pattern exactly, try something like this:

This ensures that the "0" files will always come earlier in the array than the "k" files, however they might otherwise sort out.

OHHH, back-ticks, to the left of the 1 on the keyboard. I think I've only ever used that key for ~ so totally forgot about those haha. It worked with the back-ticks!!

Thanks David. I understand the general idea of an array, and I have seen where people put ranges of stuff in square brackets before too, not really knowing how that worked. I'll look at your examples though and try to figure out why those would work too, as I'd like to learn about that way as well.