[SOLVED] I need a shell script to delete files wich location should be read from a log file
ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
xargs should also work (once you take care of the word separation issue), as I believe it will process the input in suitably-sized chunks if necessary.
Hi,
how would you propose to solve the space issue in this case? The only way xargs can cope with this, afaik, is to use the -0 option. However, grep has only the -Z option which according to the man page
Quote:
-Z, --null
Output a zero byte (the ASCII NUL character) instead of the character that normally follows a file name. For example, grep -lZ outputs a zero byte after each file name instead of the usual newline. This option makes the output unambiguous, even in the presence of file names containing unusual characters like newlines. This option can be used with commands like find -print0, perl -0, sort -z, and xargs -0 to process arbitrary file names, even those that contain newline characters.
So no option to zero-terminate an "ordinary" match, unless I missed something.
[EDIT]
As for the chunk-sized processing, yes I can acknowledge this behaviour. I recently had such a case with a huge amount of input data. It would output a couple of screens, pause for a few seconds and then continue processing.
To tell the truth, I'm still a novice myself when it comes to the intricacies of xargs. But it looks like you at least have the option to set the xargs delimiter to newline (-d "\n"). If that's not enough you process the input with sed or something and add some other delimiter to the end of each entry that xargs could use.
By the way, I see that xargs --show-limits will tell you how big your command buffer is, and there are several options for controlling how much input is processed at any one time.
Couldn't the white spaces just be escaped? e.g. /media/My\ Book/...
This could be accomplished by simply replacing " " (space) with "\ " (backslash-space) in the logfile. Any decent text editor could do that in a few seconds.
To tell the truth, I'm still a novice myself when it comes to the intricacies of xargs. But it looks like you at least have the option to set the xargs delimiter to newline (-d "\n"). If that's not enough you process the input with sed or something and add some other delimiter to the end of each entry that xargs could use.
By the way, I see that xargs --show-limits will tell you how big your command buffer is, and there are several options for controlling how much input is processed at any one time.
Hi,
tried it with the -d option and it worked fine. Seems that it can be done with a one liner after all.
Couldn't the white spaces just be escaped? e.g. /media/My\ Book/...
This could be accomplished by simply replacing " " (space) with "\ " (backslash-space) in the logfile. Any decent text editor could do that in a few seconds.
Yes, that's a possibility. But it seems to me like an unnecessary brute-force approach when xargs can handle it internally with the proper delimiter configured.
Also, filenames are generally ok, but let's say instead that you want to grab and process all the lines containing the word "alias" from a file like this (a sample bashrc).
Code:
#!/bin/bash
# This is a sample bashrc
# User specific aliases
alias ll='ls -l --color=auto'
alias la='ls -A --color=auto'
other stuff, etc...
Look at what happens when I try to escape the spaces in the input:
Code:
sed -n '/alias/ s/ /\\ /gp' sample_bashrc | xargs printf "[%s]\n"
[# User specific aliases and functions]
[alias ll=ls\ -l\ --color=auto]
[alias la=ls\ -A\ --color=auto]
Admittedly this is a useless example, but imagine that it's being sent to a script or something instead. First of all, I needed to escape it twice just so that one backslash went through to the final command. But also notice how the quotation marks in the input have protected some of the escapes, so that xargs doesn't recognize them. And the quotes themselves have disappeared.
Setting the -d delimiter, however, disables xargs' default processing, so that everything else is ignored as part of the string and only that delimiter means anything special.
Code:
grep 'alias' sample_bashrc | xargs -d "\n" printf "[%s]\n"
[# User specific aliases and functions]
[alias ll='ls -l --color=auto']
[alias la='ls -A --color=auto']
Now everything works as intended.
Last edited by David the H.; 06-29-2010 at 09:12 AM.
Reason: minor change for readability
Neat. I hadn't heard of parallel before. Some of the options look very useful.
Strangely though, it doesn't seem to be in the Debian repositories. So while it may be more convenient to use in cases like this, it seems that it's not something that you can depend on to be available everywhere, unlike like xargs, which is a standard core utility.
That is probably because it hasn't been around that long, if this archive (http://ftp.gnu.org/gnu/parallel/) contains the whole release history.
Scroll down to the maintainer on this (http://www.gnu.org/software/parallel/) site. It appears that the poster is also the author/maintainer of the program. From the linked youtube video I'd say that the main purpose is to enhance performance for computers using a multicore cpu. Since this is seemingly a rather "young" program I would also recommend to stick with xargs, especially because the development focus was probably not on replacing xargs. So substituting xargs for parallel might be a bit premature at this point.
Parallel has been around since 2002 and had been in production at least since 2005. The reason for the apparent short history is that Parallel quite recently was accepted as a GNU tool and thus renamed to GNU Parallel.
GNU parallel did not have one but two main objectives: Replace xargs and run commands in parallel.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
First of all, I needed to escape it twice just so that one backslash went through to the final command. But also notice how the quotation marks in the input have protected some of the escapes, so that xargs doesn't recognize them. And the quotes themselves have disappeared.
Well Done
