[SOLVED] Find files with $ and spaces in name

BeachHead · 01-21-2014, 08:01 PM

Hallo,
i want to parse all files in a folder recursively. The 'sed' command works properly if i pass a literal name but it fails if i run it in this 'for' loop.

I tried many variations with quotes and double quotes as well as -print0/'xargs -0' examples but it looks like i'm just poking in the dark...

Anyone know what's wrong here?

GNU bash, version 4.1.10(4)-release (i686-pc-cygwin)

Code:

#!/bin/bash

FILES="myfolder" 
for i in "$(find "$FILES" -name '*.*')"; do
	#echo "$i"
	sed -n '/invoke/ {
		s/.*->\(.*\)/\1/
		s/(.*//
		p
	}' < "$i" >> output.txt
done

Code:

$ ls -R
file$1.txt  new folder

./new folder:
file$2.txt

Error is:

Code:

./sed-invokes.sh: line 6: myfolder/file$1.txt
myfolder/new folder/file$2.txt: No such file or directory

If i run 'echo "$i"' in the loop it shows the files properly.

Code:

myfolder/file$1.txt
myfolder/new folder/file$2.txt

grail · 01-21-2014, 10:47 PM

Try using the following echo instead and see what happens:

Code:

echo "|$i|"

Also, whilst '*.*' may work in this limited case, should any directory or other file type have a period in it, this code will quickly unravel.

BeachHead · 01-22-2014, 10:13 AM

Thanks. I changed 'find' to '-type f'.

Code:

#!/bin/bash

FILES="myfolder"
for i in "$(find "$FILES" -type f)"; do
	echo "|$i|"
done

The output seems one solid string only.

Code:

|myfolder/file$1.txt
myfolder/new folder/file$2.txt|

If i remove the double quotes surrounding $(find...) i get separate entries now but suffering whitespace problem (myfolder/new folder/file$2.txt).

Code:

for i in $(find "$FILES" -type f); do

Code:

|myfolder/file$1.txt|
|myfolder/new|
|folder/file$2.txt|

Beryllos · 01-22-2014, 10:53 AM

for + find = problems with whitespace and other special characters like * and names like --help

Try "while read" instead of "for":

Code:

#!/bin/bash

FILES="myfolder"
find "$FILES" -type f | while read -r i
do
	echo "|$i|"
done

That should handle just about anything except filenames containing a newline character, since the read command expects items to be separated by newline characters.

Edit: I also tested it with your sed command. It seems to work.

BeachHead · 01-22-2014, 11:23 AM

Yep, looks good. Thanks.
I slightly changed it to use \0 as separator.

This is the current script:

Code:

#!/bin/bash

function do_sed {
	sed -n '/invoke/ {
		s/.*->\(.*\)/\1/
		s/(.*//
		p
	}' < "$1" >> output.txt
}

FILES="myfolder"
if [ -f output.txt ]; then rm output.txt; fi
time find "$FILES" -type f -print0 | while read -d $'\0' i
do
	echo "$i"
	do_sed "$i"
done

Beryllos · 01-22-2014, 11:54 AM

Quote:

Originally Posted by BeachHead

Yep, looks good. Thanks.
I slightly changed it to use \0 as separator.

Nice. Does that mean it can handle any character in the filename (except \0 which is never allowed anyway)? Thanks for pointing that out.

BeachHead · 01-22-2014, 12:05 PM

That's at least the intention. I cannot test it atm though. Newlines on NTFS/Cygwin are probably not allowed i think (Edit: It does work in cygwin. Probably abstracted).

Edit: Tested, seems to work. I created files with 'echo aaa > "$(echo -e "aaa\012bbb")"' and they were processed properly. Newlines on 'echo' feedback looks funny though.

grail · 01-23-2014, 07:02 AM

As a final note, remember that the current method used for feeding the while loop through a pipe will mean any data changed within the loop will be lost once you have left
the loop due to it being in a sub-shell. I will let you investigate the alternatives