LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 02-09-2017, 10:03 AM   #1
Entropy1024
Member
 
Registered: Dec 2012
Location: UK
Distribution: Ubuntu 16 & 17
Posts: 131

Rep: Reputation: Disabled
Delete files containing a certain bit of text.


I can use the code below to search any file in my status folder that contains the words ftp://entrop12.

Code:
ls /home/tim/Documents/p5e/status/ | grep "ftp://entrop12" *
What I need to do is delete any files that contain that text. How can I change my line of code to achieve this goal?

Many thanks
Tim
 
Old 02-09-2017, 10:09 AM   #2
r3sistance
Senior Member
 
Registered: Mar 2004
Location: UK
Distribution: CentOS 6/7
Posts: 1,375

Rep: Reputation: 217Reputation: 217Reputation: 217
Are you after something like: grep -ri "<text to delete>" <directory> | awk -F":" '{print $1}' | xargs -r -L 1 rm -f

Code:
# echo "abc" > test/test1
# echo "abc" > test/test2
# echo "def" > test/test3
# grep -ri "abc" test
test/test1:abc
test/test2:abc
# grep -ri "abc" test | awk -F":" '{print $1}'
test/test1
test/test2
# grep -ri "abc" test | awk -F":" '{print $1}' | xargs -r -L 1 rm -f
# grep -ri "abc" test
# ls test
aaabb  test3
WARNING: I'd advise testing this for yourself before using this, there is some inherent danger whenever using rm, I won't take any responsibility if you use it wrongly and deleted unintentional things.

Last edited by r3sistance; 02-09-2017 at 10:15 AM. Reason: warning and better example
 
Old 02-09-2017, 10:13 AM   #3
Entropy1024
Member
 
Registered: Dec 2012
Location: UK
Distribution: Ubuntu 16 & 17
Posts: 131

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by r3sistance View Post
Are you after something like: grep -ri "<text to delete>" <directory> | awk -F":" '{print $1}' | xargs -r -L 1 rm -f

Code:
# export PS1="# "
# grep -ri "abc" test
test/test1:abc
test/test2:abc
# grep -ri "abc" test | awk -F":" '{print $1}'
test/test1
test/test2
# grep -ri "abc" test | awk -F":" '{print $1}' | xargs -r -L 1 rm -f
# grep -ri "abc" test
WARNING: I'd advise testing this for yourself before using this, there is some inherent danger whenever using rm, I won't take any responsibility if you use it wrongly and deleted unintentional things.
Possibly. I only understand the first part of the command. What does the awk and xargs do?
Cheers
 
Old 02-09-2017, 10:18 AM   #4
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,000
Blog Entries: 3

Rep: Reputation: 3632Reputation: 3632Reputation: 3632Reputation: 3632Reputation: 3632Reputation: 3632Reputation: 3632Reputation: 3632Reputation: 3632Reputation: 3632Reputation: 3632
You can also use find with -exec. The following find files containing the string "bash" anywhere and then prints the first line of said file. You can modify it to delete the file, if that is your goal.

Code:
find . -maxdepth 1 -type f -exec grep -q bash {} \; -exec head -n 1 {} \;
The option -exec passes the exit code from the executed program(s) back to find where it is used within the expression.
 
2 members found this post helpful.
Old 02-09-2017, 10:20 AM   #5
r3sistance
Senior Member
 
Registered: Mar 2004
Location: UK
Distribution: CentOS 6/7
Posts: 1,375

Rep: Reputation: 217Reputation: 217Reputation: 217
I improved the example a bit.

awk is an extremely powerful tool, as the man page says: gawk - pattern scanning and processing language
in this example I am using awk to split the text using a delimiter of a colon and printing the first output parameter which in this case would be test/test1 for test, in other words formatting it ready to pass to xargs

xargs is used for running a command repeatedly against the passed paramters or as the man page states: xargs - build and execute command lines from standard input
the -r here is to stop it running if it receives no input, since we have an rm it is very prudent we don't let it run randomly. The -L 1 specifies that the parameters are split into single lines (rather than delimited by space, tabs, etc). following this is the command to run, in this case it is "rm -f". So it will delete any hits, since it lacks the recursive flag then we aren't in danger of deleting entire directories but rm is still dangerous.

iirc recursive greps don't work on some *nix distros, so the find example given by Turbocapitalist is going to be more portable.

Last edited by r3sistance; 02-09-2017 at 10:25 AM.
 
Old 02-09-2017, 10:25 AM   #6
Entropy1024
Member
 
Registered: Dec 2012
Location: UK
Distribution: Ubuntu 16 & 17
Posts: 131

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Turbocapitalist View Post
You can also use find with -exec. The following find files containing the string "bash" anywhere and then prints the first line of said file. You can modify it to delete the file, if that is your goal.

Code:
find . -maxdepth 1 -type f -exec grep -q bash {} \; -exec head -n 1 {} \;
The option -exec passes the exit code from the executed program(s) back to find where it is used within the expression.
Yes I do want to delete the file that contains that text. So would the code below be valid?

Code:
find /home/tim/Documents/p5e/status/ -maxdepth 1 -type f -exec grep -q ftp://entrop12 {} \; -exec rm {} \;
 
Old 02-09-2017, 10:36 AM   #7
Turbocapitalist
LQ Guru
 
Registered: Apr 2005
Distribution: Linux Mint, Devuan, OpenBSD
Posts: 7,000
Blog Entries: 3

Rep: Reputation: 3632Reputation: 3632Reputation: 3632Reputation: 3632Reputation: 3632Reputation: 3632Reputation: 3632Reputation: 3632Reputation: 3632Reputation: 3632Reputation: 3632
You can usually test it by putting an echo in before the actual utility:

Code:
find /home/tim/Documents/p5e/status/ -maxdepth 1 -type f -exec grep -q 'ftp://entrop12' {} \; -exec echo rm {} \;
That will show you what it would run, if the echo were removed.
 
2 members found this post helpful.
Old 02-09-2017, 10:44 AM   #8
Entropy1024
Member
 
Registered: Dec 2012
Location: UK
Distribution: Ubuntu 16 & 17
Posts: 131

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Turbocapitalist View Post
You can usually test it by putting an echo in before the actual utility:

Code:
find /home/tim/Documents/p5e/status/ -maxdepth 1 -type f -exec grep -q 'ftp://entrop12' {} \; -exec echo rm {} \;
That will show you what it would run, if the echo were removed.
That's a great tip. Many thanks
 
Old 02-09-2017, 11:15 AM   #9
nodir
Member
 
Registered: May 2016
Posts: 222

Rep: Reputation: Disabled
Code:
for i in /path/to/folder/*; do
  if grep pattern "$i"; then
    rm "$i"; 
  fi 
done
In real life i would make sure "$i" is a file and do something in case rm fails (rm "$i" || do_something_bout_the_failure ).
Ups: and probably make path/to/folder and pattern a variable.

Last edited by nodir; 02-09-2017 at 11:20 AM.
 
1 members found this post helpful.
Old 02-10-2017, 12:55 AM   #10
allend
LQ 5k Club
 
Registered: Oct 2003
Location: Melbourne
Distribution: Slackware64-15.0
Posts: 6,213

Rep: Reputation: 2679Reputation: 2679Reputation: 2679Reputation: 2679Reputation: 2679Reputation: 2679Reputation: 2679Reputation: 2679Reputation: 2679Reputation: 2679Reputation: 2679
Grep has the -l option for for listing filenames containing matches, -Z for outputting zero byte delimited filenames suitable for xargs and -r for recursive searching.
Code:
grep -lZr "ftp://entrop12" /home/tim/Documents/p5e/status/ | xargs -0 echo "rm -f "
If happy, remove the 'echo'
 
1 members found this post helpful.
Old 02-10-2017, 11:00 AM   #11
Shadow_7
Senior Member
 
Registered: Feb 2003
Distribution: debian
Posts: 4,137
Blog Entries: 1

Rep: Reputation: 874Reputation: 874Reputation: 874Reputation: 874Reputation: 874Reputation: 874Reputation: 874
I tend to take a more batch processing POV. Because you can audit your work. And start or stop the process at any stage. But it's certainly not a method for unsupervised automation.

$ egrep -r -i entrop12 /home/time/Documents/p5e/status/ | tee pre_delete_log_of_files.txt
(find the suspected files targeted for deletion)

$ nano pre_delete_log_of_files.txt
(manually edit the file in case there's things you should keep like .bash_history since you're searching for it)

$ cat pre_delete_log_of_files.txt | while read $FILE; do echo $FILE; rm $FILE; done
(do the deletes)

$ cat pre_delete_log_of_files.txt | while read FILE; do if [ -r $FILE ]; then echo $FILE" --- not deleted "; fi; done
(and verify that what you told it to do happened)

Optionally delete the temp file, depending on your needs. Or move the files to another location, instead of delete if you might want to "UNDO" the process someday. Which is when the temp file is useful, to know what to "UNDO".
 
Old 02-10-2017, 12:11 PM   #12
nodir
Member
 
Registered: May 2016
Posts: 222

Rep: Reputation: Disabled
Quote:
Originally Posted by Shadow_7 View Post

$ cat pre_delete_log_of_files.txt | while read $FILE; do echo $FILE; rm $FILE; done
(do the deletes)

$ cat pre_delete_log_of_files.txt | while read FILE; do if [ -r $FILE ]; then echo $FILE" --- not deleted "; fi; done
(and verify that what you told it to do happened)
using cat and a pipe to read seems at least odd to me.
http://mywiki.wooledge.org/BashFAQ/001
so :
Code:
while IFS= read -r line; do
 printf '%s\n' "$line"
done < "$file"
 
Old 02-10-2017, 09:04 PM   #13
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 20,944

Rep: Reputation: 4069Reputation: 4069Reputation: 4069Reputation: 4069Reputation: 4069Reputation: 4069Reputation: 4069Reputation: 4069Reputation: 4069Reputation: 4069Reputation: 4069
I love these threads that toss up so many ways to skin what appeared a simple q.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Delete files using a text file Fixit7 Linux - General 6 06-26-2016 03:17 PM
Rm use text file to delete files Fixit7 Puppy 4 01-18-2016 10:10 PM
How to delete files keeping the files listed in a text file -urgent jeesun Linux - General 4 10-21-2011 11:28 AM
Delete files containing matching text hessodreamy Linux - Newbie 11 05-21-2008 09:05 PM
delete all files with certain text dougp23 Linux - General 3 04-21-2008 01:17 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 06:06 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration