[SOLVED] How to use 'grep' for files found from 'find' ..

susja · 06-22-2016, 08:05 PM

find . -iname '*.zip' -print0 | while read -d '' -r file; do
mkdir temp
unzip -qq "$file" -d temp
grep -irl DNA0000005619_H002 temp/*
rm -fr temp
done
--
./test2.sh
warning: ./Backup files 1.zip appears to use backslashes as path separators
fchmod (file attributes) error: Permission denied
fchmod (file attributes) error: Permission denied
fchmod (file attributes) error: Permission denied
fchmod (file attributes) error: Permission denied
fchmod (file attributes) error: Permission denied
temp/C/ProgramData/Tecan/EVOware/output/Sample Tracking/Archive/RePf000000001.csv
[TBD]$
--

Great! It found correct file. The last thing left is somehow get rid of 'warning' and other 'irrelevant' stuff. I need to see only file and nothing more ..

Thanks

grail · 06-22-2016, 09:37 PM

Seeing as no one else has mentioned it, please use [code][/code] tags around your code so it is more readable and maintains formatting.

"irrelevant" is a stretch .. it is giving you information that may be necessary, but if you really want to get rid of it you can redirect the stderr from your loop to go to /dev/null if you feel the
information is not warranted.

susja · 06-22-2016, 10:04 PM

Code:

find . -iname '*.zip' -print0 | while read -d '' -r file; do
mkdir temp
unzip -qq "$file" -d temp
grep -irl DNA0000005619_H002 temp/*
rm -fr temp
done

I'm going to 'SOLVE' it .. but could you point me which place should I redirect stderr to /dev/null ? like 2> /dev/null where to put it?
Thanks

grail · 06-22-2016, 11:05 PM

After 'done'

pan64 · 06-23-2016, 12:58 AM

try to put it after done:

Code:

find . .... | while read ... do
...
...
...
done 2>/dev/null

susja · 06-23-2016, 09:36 AM

Folks,
Now everything works as expected. Thanks for the help.
The last thing

still missing is the 'path' to the file
I have an output like this:
temp/C/ProgramData/Tecan/EVOware/output/..../file.x
But this does not help me. I need 'absolute' path to the file like
ABC.zip/C/ProgramData...
or
DEF.zip/C/....
How script could be modified to get the whole path?
Thanks again ..

suicidaleggroll · 06-23-2016, 10:13 AM

Try this:

Code:

#!/bin/bash

find . -iname '*.zip' -print0 | while read -d '' -r file; do
  mkdir temp
  unzip -qq "$file" -d temp
  grep -irl DNA0000005619_H002 temp/* | awk -F\n '{print gensub("^temp","'"$file"'",1,$1)}'
  rm -fr temp
done

susja · 06-23-2016, 11:20 AM

- suicidaleggroll

It did not make any difference.
I still get this:
/search.sh DNA0000005631
temp/C/ProgramData/Tecan/EVOware/output/Sample Tracking/.../file.x
temp/C/ProgramData/Tecan/EVOware/output/Sample Tracking/.../file.c

suicidaleggroll · 06-23-2016, 11:25 AM

Are you sure? That shouldn't be possible and I tested it on my own system before posting.

Please copy-paste the code and output from running it.

susja · 06-23-2016, 11:40 AM

cat ./search.sh
#!/bin/bash

## This script will search for provided PATTERN in 'zipped' files and list files
## For better performance go to the 'lowest' directory
## Usage: ./search.sh <PATTERN> | sort | uniq

/usr/bin/find . -iname '*.zip' -print0 | while read -d '' -r file; do
mkdir temp
/usr/bin/unzip -qq "$file" -d temp | awk -F\n '{print gensub("^temp","'"$file"'",1,$1)'
grep -irl $1 temp/*
rm -fr temp
done 2>/dev/null
---
./search.sh DNA0000005631|sort|uniq
temp/C/ProgramData/Tecan/EVOware/output/Sample Tracking/Archive/DVALABB002419_04282016_131616.csv
temp/C/ProgramData/Tecan/EVOware/output/Sample Tracking/Archive/DVALABB002420_04282016_131616.csv
temp/C/ProgramData/Tecan/EVOware/output/Sample Tracking/Reports/NormalizeReport04282016_131615.csv
sh-4.3$

Thanks again

suicidaleggroll · 06-23-2016, 11:48 AM

You added the awk to the output of unzip instead of grep and you missed the '}'.

susja · 06-23-2016, 12:57 PM

Well ... sorry for my typo
Now it produce correct path but it's truncated and does not give me the name of the file I'm interested in ..
I have this:
./search.sh DNA0000005631
./Backup Files 2016-05-26 142101/Backup files 1.zip/C/ProgramData/Teca
While the path is:
temp/C/ProgramData/Tecan/EVOware/output/Sample Tracking/Archive/DVALABB002419_04282016_131616.csv
Having truncated path does not allow me to locate the file

suicidaleggroll · 06-23-2016, 01:24 PM

heh, my test case apparently didn't have any N's in the path to the file!

Try this awk instead:

Code:

awk '{print gensub("^temp","'"$file"'",1)}'

keefaz · 06-23-2016, 01:47 PM

Maybe terminal output is seen through some sort of pager that truncates long lines? Just a thought

suicidaleggroll · 06-23-2016, 02:03 PM

Quote:

Originally Posted by keefaz

Maybe terminal output is seen through some sort of pager that truncates long lines? Just a thought

Nah, it's the field separator I used in the awk. I was trying to effectively disable it by setting it to a new line, but apparently "-F\n" just sets it to a literal "n", so the first "n" it ran into it split the line. I just switched the gensub to use the entire input "$0" and disabled field splitting, which should take care of it.