Delete files containing a certain bit of text.

Entropy1024 · 02-09-2017, 10:03 AM

I can use the code below to search any file in my status folder that contains the words ftp://entrop12.

Code:

ls /home/tim/Documents/p5e/status/ | grep "ftp://entrop12" *

What I need to do is delete any files that contain that text. How can I change my line of code to achieve this goal?

Many thanks
Tim

r3sistance · 02-09-2017, 10:09 AM

Are you after something like: grep -ri "<text to delete>" <directory> | awk -F":" '{print $1}' | xargs -r -L 1 rm -f

Code:

# echo "abc" > test/test1
# echo "abc" > test/test2
# echo "def" > test/test3
# grep -ri "abc" test
test/test1:abc
test/test2:abc
# grep -ri "abc" test | awk -F":" '{print $1}'
test/test1
test/test2
# grep -ri "abc" test | awk -F":" '{print $1}' | xargs -r -L 1 rm -f
# grep -ri "abc" test
# ls test
aaabb  test3

WARNING: I'd advise testing this for yourself before using this, there is some inherent danger whenever using rm, I won't take any responsibility if you use it wrongly and deleted unintentional things.

Entropy1024 · 02-09-2017, 10:13 AM

Quote:

Originally Posted by r3sistance

Are you after something like: grep -ri "<text to delete>" <directory> | awk -F":" '{print $1}' | xargs -r -L 1 rm -f

Code:

# export PS1="# "
# grep -ri "abc" test
test/test1:abc
test/test2:abc
# grep -ri "abc" test | awk -F":" '{print $1}'
test/test1
test/test2
# grep -ri "abc" test | awk -F":" '{print $1}' | xargs -r -L 1 rm -f
# grep -ri "abc" test

WARNING: I'd advise testing this for yourself before using this, there is some inherent danger whenever using rm, I won't take any responsibility if you use it wrongly and deleted unintentional things.

Possibly. I only understand the first part of the command. What does the awk and xargs do?
Cheers

kakistocrat · 02-09-2017, 10:18 AM

You can also use find with -exec. The following find files containing the string "bash" anywhere and then prints the first line of said file. You can modify it to delete the file, if that is your goal.

Code:

find . -maxdepth 1 -type f -exec grep -q bash {} \; -exec head -n 1 {} \;

The option -exec passes the exit code from the executed program(s) back to find where it is used within the expression.

r3sistance · 02-09-2017, 10:20 AM

I improved the example a bit.

awk is an extremely powerful tool, as the man page says: gawk - pattern scanning and processing language
in this example I am using awk to split the text using a delimiter of a colon and printing the first output parameter which in this case would be test/test1 for test, in other words formatting it ready to pass to xargs

xargs is used for running a command repeatedly against the passed paramters or as the man page states: xargs - build and execute command lines from standard input
the -r here is to stop it running if it receives no input, since we have an rm it is very prudent we don't let it run randomly. The -L 1 specifies that the parameters are split into single lines (rather than delimited by space, tabs, etc). following this is the command to run, in this case it is "rm -f". So it will delete any hits, since it lacks the recursive flag then we aren't in danger of deleting entire directories but rm is still dangerous.

iirc recursive greps don't work on some *nix distros, so the find example given by Turbocapitalist is going to be more portable.

Entropy1024 · 02-09-2017, 10:25 AM

Quote:

Originally Posted by Turbocapitalist

You can also use find with -exec. The following find files containing the string "bash" anywhere and then prints the first line of said file. You can modify it to delete the file, if that is your goal.

Code:

find . -maxdepth 1 -type f -exec grep -q bash {} \; -exec head -n 1 {} \;

The option -exec passes the exit code from the executed program(s) back to find where it is used within the expression.

Yes I do want to delete the file that contains that text. So would the code below be valid?

Code:

find /home/tim/Documents/p5e/status/ -maxdepth 1 -type f -exec grep -q ftp://entrop12 {} \; -exec rm {} \;

kakistocrat · 02-09-2017, 10:36 AM

You can usually test it by putting an echo in before the actual utility:

Code:

find /home/tim/Documents/p5e/status/ -maxdepth 1 -type f -exec grep -q 'ftp://entrop12' {} \; -exec echo rm {} \;

That will show you what it would run, if the echo were removed.

Entropy1024 · 02-09-2017, 10:44 AM

Quote:

Originally Posted by Turbocapitalist

You can usually test it by putting an echo in before the actual utility:

Code:

find /home/tim/Documents/p5e/status/ -maxdepth 1 -type f -exec grep -q 'ftp://entrop12' {} \; -exec echo rm {} \;

That will show you what it would run, if the echo were removed.

That's a great tip. Many thanks

nodir · 02-09-2017, 11:15 AM

Code:

for i in /path/to/folder/*; do
  if grep pattern "$i"; then
    rm "$i"; 
  fi 
done

In real life i would make sure "$i" is a file and do something in case rm fails (rm "$i" || do_something_bout_the_failure ).
Ups: and probably make path/to/folder and pattern a variable.

allend · 02-10-2017, 12:55 AM

Grep has the -l option for for listing filenames containing matches, -Z for outputting zero byte delimited filenames suitable for xargs and -r for recursive searching.

Code:

grep -lZr "ftp://entrop12" /home/tim/Documents/p5e/status/ | xargs -0 echo "rm -f "

If happy, remove the 'echo'

Shadow_7 · 02-10-2017, 11:00 AM

I tend to take a more batch processing POV. Because you can audit your work. And start or stop the process at any stage. But it's certainly not a method for unsupervised automation.

$ egrep -r -i entrop12 /home/time/Documents/p5e/status/ | tee pre_delete_log_of_files.txt
(find the suspected files targeted for deletion)

$ nano pre_delete_log_of_files.txt
(manually edit the file in case there's things you should keep like .bash_history since you're searching for it)

$ cat pre_delete_log_of_files.txt | while read $FILE; do echo $FILE; rm $FILE; done
(do the deletes)

$ cat pre_delete_log_of_files.txt | while read FILE; do if [ -r $FILE ]; then echo $FILE" --- not deleted "; fi; done
(and verify that what you told it to do happened)

Optionally delete the temp file, depending on your needs. Or move the files to another location, instead of delete if you might want to "UNDO" the process someday. Which is when the temp file is useful, to know what to "UNDO".

nodir · 02-10-2017, 12:11 PM

Quote:

Originally Posted by Shadow_7

$ cat pre_delete_log_of_files.txt | while read $FILE; do echo $FILE; rm $FILE; done
(do the deletes)

$ cat pre_delete_log_of_files.txt | while read FILE; do if [ -r $FILE ]; then echo $FILE" --- not deleted "; fi; done
(and verify that what you told it to do happened)

using cat and a pipe to read seems at least odd to me.
http://mywiki.wooledge.org/BashFAQ/001
so :

Code:

while IFS= read -r line; do
 printf '%s\n' "$line"
done < "$file"

syg00 · 02-10-2017, 09:04 PM

I love these threads that toss up so many ways to skin what appeared a simple q.