Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place! |
Notices |
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Are you new to LinuxQuestions.org? Visit the following links:
Site Howto |
Site FAQ |
Sitemap |
Register Now
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
|
 |
02-09-2017, 10:03 AM
|
#1
|
Member
Registered: Dec 2012
Location: UK
Distribution: Ubuntu 16 & 17
Posts: 131
Rep: 
|
Delete files containing a certain bit of text.
I can use the code below to search any file in my status folder that contains the words ftp://entrop12.
Code:
ls /home/tim/Documents/p5e/status/ | grep "ftp://entrop12" *
What I need to do is delete any files that contain that text. How can I change my line of code to achieve this goal?
Many thanks
Tim
|
|
|
02-09-2017, 10:09 AM
|
#2
|
Senior Member
Registered: Mar 2004
Location: UK
Distribution: CentOS 6/7
Posts: 1,375
|
Are you after something like: grep -ri "<text to delete>" <directory> | awk -F":" '{print $1}' | xargs -r -L 1 rm -f
Code:
# echo "abc" > test/test1
# echo "abc" > test/test2
# echo "def" > test/test3
# grep -ri "abc" test
test/test1:abc
test/test2:abc
# grep -ri "abc" test | awk -F":" '{print $1}'
test/test1
test/test2
# grep -ri "abc" test | awk -F":" '{print $1}' | xargs -r -L 1 rm -f
# grep -ri "abc" test
# ls test
aaabb test3
WARNING: I'd advise testing this for yourself before using this, there is some inherent danger whenever using rm, I won't take any responsibility if you use it wrongly and deleted unintentional things.
Last edited by r3sistance; 02-09-2017 at 10:15 AM.
Reason: warning and better example
|
|
|
02-09-2017, 10:13 AM
|
#3
|
Member
Registered: Dec 2012
Location: UK
Distribution: Ubuntu 16 & 17
Posts: 131
Original Poster
Rep: 
|
Quote:
Originally Posted by r3sistance
Are you after something like: grep -ri "<text to delete>" <directory> | awk -F":" '{print $1}' | xargs -r -L 1 rm -f
Code:
# export PS1="# "
# grep -ri "abc" test
test/test1:abc
test/test2:abc
# grep -ri "abc" test | awk -F":" '{print $1}'
test/test1
test/test2
# grep -ri "abc" test | awk -F":" '{print $1}' | xargs -r -L 1 rm -f
# grep -ri "abc" test
WARNING: I'd advise testing this for yourself before using this, there is some inherent danger whenever using rm, I won't take any responsibility if you use it wrongly and deleted unintentional things.
|
Possibly. I only understand the first part of the command. What does the awk and xargs do?
Cheers
|
|
|
02-09-2017, 10:18 AM
|
#4
|
LQ Guru
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,756
|
You can also use find with -exec. The following find files containing the string "bash" anywhere and then prints the first line of said file. You can modify it to delete the file, if that is your goal.
Code:
find . -maxdepth 1 -type f -exec grep -q bash {} \; -exec head -n 1 {} \;
The option -exec passes the exit code from the executed program(s) back to find where it is used within the expression.
|
|
2 members found this post helpful.
|
02-09-2017, 10:20 AM
|
#5
|
Senior Member
Registered: Mar 2004
Location: UK
Distribution: CentOS 6/7
Posts: 1,375
|
I improved the example a bit.
awk is an extremely powerful tool, as the man page says: gawk - pattern scanning and processing language
in this example I am using awk to split the text using a delimiter of a colon and printing the first output parameter which in this case would be test/test1 for test, in other words formatting it ready to pass to xargs
xargs is used for running a command repeatedly against the passed paramters or as the man page states: xargs - build and execute command lines from standard input
the -r here is to stop it running if it receives no input, since we have an rm it is very prudent we don't let it run randomly. The -L 1 specifies that the parameters are split into single lines (rather than delimited by space, tabs, etc). following this is the command to run, in this case it is "rm -f". So it will delete any hits, since it lacks the recursive flag then we aren't in danger of deleting entire directories but rm is still dangerous.
iirc recursive greps don't work on some *nix distros, so the find example given by Turbocapitalist is going to be more portable.
Last edited by r3sistance; 02-09-2017 at 10:25 AM.
|
|
|
02-09-2017, 10:25 AM
|
#6
|
Member
Registered: Dec 2012
Location: UK
Distribution: Ubuntu 16 & 17
Posts: 131
Original Poster
Rep: 
|
Quote:
Originally Posted by Turbocapitalist
You can also use find with -exec. The following find files containing the string "bash" anywhere and then prints the first line of said file. You can modify it to delete the file, if that is your goal.
Code:
find . -maxdepth 1 -type f -exec grep -q bash {} \; -exec head -n 1 {} \;
The option -exec passes the exit code from the executed program(s) back to find where it is used within the expression.
|
Yes I do want to delete the file that contains that text. So would the code below be valid?
Code:
find /home/tim/Documents/p5e/status/ -maxdepth 1 -type f -exec grep -q ftp://entrop12 {} \; -exec rm {} \;
|
|
|
02-09-2017, 10:36 AM
|
#7
|
LQ Guru
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,756
|
You can usually test it by putting an echo in before the actual utility:
Code:
find /home/tim/Documents/p5e/status/ -maxdepth 1 -type f -exec grep -q 'ftp://entrop12' {} \; -exec echo rm {} \;
That will show you what it would run, if the echo were removed.
|
|
2 members found this post helpful.
|
02-09-2017, 10:44 AM
|
#8
|
Member
Registered: Dec 2012
Location: UK
Distribution: Ubuntu 16 & 17
Posts: 131
Original Poster
Rep: 
|
Quote:
Originally Posted by Turbocapitalist
You can usually test it by putting an echo in before the actual utility:
Code:
find /home/tim/Documents/p5e/status/ -maxdepth 1 -type f -exec grep -q 'ftp://entrop12' {} \; -exec echo rm {} \;
That will show you what it would run, if the echo were removed.
|
That's a great tip. Many thanks
|
|
|
02-09-2017, 11:15 AM
|
#9
|
Member
Registered: May 2016
Posts: 222
Rep: 
|
Code:
for i in /path/to/folder/*; do
if grep pattern "$i"; then
rm "$i";
fi
done
In real life i would make sure "$i" is a file and do something in case rm fails (rm "$i" || do_something_bout_the_failure ).
Ups: and probably make path/to/folder and pattern a variable.
Last edited by nodir; 02-09-2017 at 11:20 AM.
|
|
1 members found this post helpful.
|
02-10-2017, 12:55 AM
|
#10
|
LQ 5k Club
Registered: Oct 2003
Location: Melbourne
Distribution: Slackware64-15.0
Posts: 6,575
|
Grep has the -l option for for listing filenames containing matches, -Z for outputting zero byte delimited filenames suitable for xargs and -r for recursive searching.
Code:
grep -lZr "ftp://entrop12" /home/tim/Documents/p5e/status/ | xargs -0 echo "rm -f "
If happy, remove the 'echo'
|
|
1 members found this post helpful.
|
02-10-2017, 11:00 AM
|
#11
|
Senior Member
Registered: Feb 2003
Distribution: debian
Posts: 4,137
|
I tend to take a more batch processing POV. Because you can audit your work. And start or stop the process at any stage. But it's certainly not a method for unsupervised automation.
$ egrep -r -i entrop12 /home/time/Documents/p5e/status/ | tee pre_delete_log_of_files.txt
(find the suspected files targeted for deletion)
$ nano pre_delete_log_of_files.txt
(manually edit the file in case there's things you should keep like .bash_history since you're searching for it)
$ cat pre_delete_log_of_files.txt | while read $FILE; do echo $FILE; rm $FILE; done
(do the deletes)
$ cat pre_delete_log_of_files.txt | while read FILE; do if [ -r $FILE ]; then echo $FILE" --- not deleted "; fi; done
(and verify that what you told it to do happened)
Optionally delete the temp file, depending on your needs. Or move the files to another location, instead of delete if you might want to "UNDO" the process someday. Which is when the temp file is useful, to know what to "UNDO".
|
|
|
02-10-2017, 12:11 PM
|
#12
|
Member
Registered: May 2016
Posts: 222
Rep: 
|
Quote:
Originally Posted by Shadow_7
$ cat pre_delete_log_of_files.txt | while read $FILE; do echo $FILE; rm $FILE; done
(do the deletes)
$ cat pre_delete_log_of_files.txt | while read FILE; do if [ -r $FILE ]; then echo $FILE" --- not deleted "; fi; done
(and verify that what you told it to do happened)
|
using cat and a pipe to read seems at least odd to me.
http://mywiki.wooledge.org/BashFAQ/001
so :
Code:
while IFS= read -r line; do
printf '%s\n' "$line"
done < "$file"
|
|
|
02-10-2017, 09:04 PM
|
#13
|
LQ Veteran
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,399
|
I love these threads that toss up so many ways to skin what appeared a simple q.
|
|
|
All times are GMT -5. The time now is 10:17 AM.
|
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
|
Latest Threads
LQ News
|
|