"argument list too long" - why am I getting this with the find command
Hey all,
I'm running into this issue on OSX 10.3.5, but since these are basic linux commands I thought I would check with you all to see if I am doing anything wrong. I am trying to split up a huge directory based on the modified date of the files within. Due to the "Argument list too long" issue with mv and cp and such, I googled around and found that to get around this it was recommended to use find | xargs. Here was the command to just get files older than 30 days out of the directory: Code:
find . -ctime +30 -print | xargs -J % mv % /destination/directory/ Code:
find . -name 200703* -print | xargs -J % mv % /desination/directory/ Thanks. |
It's the first command (find) which breaks, 200703* expands to something larger than what your environment can handle.
Just use Code:
find . -name "200703*" -print | ... By the way, if you have a posix compliant find, you can avoid xargs and use the better "+" termination feature. eg.: Code:
find . -name "200703*" -exec mv {} /destination/directory + |
You need quotation marks to stop the shell from expanding the wildcard.
Btw, find is a Unix command, it wasn't invented with Linux ;) Joel |
Quote:
If any of files found might have whitespace in the name, consider using: Code:
find . -name "200703*" -print0 | xargs -0 -L 1000 mv -t /destination/directory After copying the "target.md5" file to the other machine you can verify the files with "md5sum --check target.md5 >checked". Imagine that the files that checked out OK on the source machine can be deleted. The lines for the files OK to delete end with ": OK". So if you copy the checked file to the source directory you can run: "sed -n '/: OK$/s/: OK//p' checked | tr '\n' '\000' | xargs -0 -L 1000 rm -v" The "tr" commands converts newline characters to null characters allowing for filenames with spaces or other "evil" filename characters. |
Fantastic...thanks all...I had thought I tried putting the wildcard in quotes, but I guess not. But, once I did, I had no issues...I couldn't get the find without xargs to work, but I'm not sure if it's an issue with my implementation or OSX's find command.
Either way...thanks again. |
If found myself using the form "| tr '\n' '\000' | xargs -0" in one-liners very useful. It allows you to process a list of files, such as finding the the files unique to one of the lists, and treating the output as if it came from the "find" command.
Another example is extracting the XML file from a saved K3B job, extracting the filenames, and using the results to delete the files which were backed up. Having intermediate files and also be useful in debugging, readability of your script, or in allowing a larger number files to be processed. |
Quote:
|
Code:
-exec command {} + <update>The info file says that it is done so that the maximum command line length isn't exceeded. So the manpage could be worded better, or that phrase from the info file included.</update> I had a problem getting it to work until I found the part in the info manual that explains that the + character is used in place of the ; character to terminate the command. The manpage could have said that the '+' character was a command terminator. example: find ./ -maxdepth 1 -name "*.txt" -exec '{}' \+ The info manual does say that using xargs may be faster. Quote:
|
The gnu find manual page is indeed misleading.
By the way, you do not need to escape the plus like you do with the semicolon, as '+' isn't a special character to the shell. |
I think the one example in the info manual did escape it in case you have regular expression globbing turned on. In that case '+' means at least one of the previous character.
|
FInd - "Argument list too long"
I am also running into similar issue when trying to archive files on directory older than 90 days. It worked very well with a small amount of files.
find * -mtime +90 > file.out I have tried narrowing down on file name but the results were similar. Anyone have any idea on how I can construct this directive without causing argument list too long? |
Try that one.
Code:
find . -mtime +90 -type f | sed -e 's/^\.\///' -e '/^\./d' > file.out |
All times are GMT -5. The time now is 03:03 PM. |