I'm trying to copy files using wildcards, but can't get it to work:

cp -f /Src/FULL_\^RRRR\^_IP_2013-03* /Dest/

When I do the same wildcard with find, I see all the files I am after. The file names and the directory in which the files rest have spaces in them. Could this be the problem. Thanks.

Certainly. Enclose all filename parameters to cp in single or double quotes and try again.

Problem is with this line. Either remove escape sequence or share name of any one of the file that you want to copy to Dest.

I'm confused. Could you show us exactly what the filenames look like? Also point out which ones you want to copy and which ones to exclude.

Spaces in filenames aren't a problem when using globbing like this, btw, since the filename expansion happens after the word-splitting step is complete. This is almost certainly due to incorrectly escaping the characters.

Figured out the problem, but to answer the questions to my question, an actual file name example follows:


FULL_^RRRR^_IP_2013-03-01 someotherstuff.someextension

So, I figured that if I executed the following command that I would be able to copy all files with a March 2013 timestamp:

cp -f /Src/FULL_^RRRR^_IP_2013-03* /Dest/

Turns out that this works, but not with 6,000 plus files. I recall that I encountered this "too many files to copy" problem in the past. I suppose I could simply use a for loop with find to get what I want. It bothers me that there is a limit to the number of files I can copy at onces AND the fact that there is no error message indicating this. So, here is my attempt at the for loop, but it is having problems with spaces. Anyone know how to copy multiple files with for loop, taking into acount spaces in both the file name and at least one directory along the path. Nothing seems to be working.

Thanks.

It is not directly a limit on cp, but a limit on the memory allowed for use in parameters to a process - hence the "too many files..." due to the bash parameter expansion. You can expand the number of files by shortening the name... Do a cd into /Src - and this will free up four bytes per file name...

The reason find works is that the search expression is evaluated by the find utility itself, and does not pass the entire list as a parameter list.

Use find instead of the for loop - it does the same thing, and is optimized for doing it. The disadvantage for both methods is that you have to spawn off a "cp" process for every file.

jpollard - helpful advise, thanks. When you write to use find, how would I do that? Pipe the find results to cp? How would that look?

find / -name *** | cp something?

Thanks.

Figured it out:

find . -name filename -exec cp {} Destination \;

It's great because it works with spaces in files!

Is that 6000 files just for March?

You could run a loop and processing it by each day individually.

Notice also how you can quote-protect the fixed parts of the string while leaving the globbing characters open for expansion. It's not really needed here, however, since there are no reserved characters in the string. But it never hurts.

You may have to get a bit fancier to avoid the unnecessary days on short months, however. Perhaps we should use an intermediate array instead.

But that's getting to be a lot of work, and there's still no guarantee that you won't hit ARG_MAX, so I suppose find might still be best choice.

I recommend modifying it to this, however:

The '+' ending for the -exec option enables xargs-style batch processing mode. It will run only as many instances of the command as necessary to avoid the ARG_MAX limit. The main limitation is that the '{}' brackets must then come at the end of the command (it expands to the full list of filenames), so for cp or mv you need to use the -t target directory option to move it to the front.

It's also recommended to avoid hard-coding data strings into your commands. Set variables at the top of your script (or pass them to it from outside), and use those in the actual commands.

thanks for the follow-up information. Everything works great now.