[SOLVED] Bash, find : How to avoid [...] pattern matching in file names expanded from "$var"?
ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Bash, find : How to avoid [...] pattern matching in file names expanded from "$var"?
I have two directories, dir1 and dir2. There are about 450 files in each directory. The files in dir2 should have exactly the same names as the files in dir1.
Some of the file names have [...] in them. I can't change the names of the files due to existing naming conventions.
My goal is to find files in dir2 with the same name as files in dir1 and execute commands on them. What follows is an abbreviated example of my failure until now.
Code:
foo$ ls -R
.:
dir1 dir2
./dir1:
file1 file2 file[a1]
./dir2:
file1 file2 file[a1]
foo$ cd dir1
dir1$ for src in * ; do find ../dir2/ -name "$src" ; done
../dir2/file1
../dir2/file2
../dir2/file1
dir1$
The file ../dir2/file[a1] was not listed, and ../dir2/file1 was listed twice!
The expansions of $src are as follows.
Code:
dir1$ for src in * ; do echo "$src" ; done
file1
file2
file[a1]
dir1$
find -name obeys shell pattern matching rules. When it sees the pattern file[a1] it is looking for filea and file1.
I want to stop the pattern matching and treat the expansion of "$src" as a literal. How?
Edit
The reason I want to use find is because some of the files in dir2 are nested in subdirectories which don't exist in dir1. So I'm trying to match files like this:
for src in * ; do [ -f "../dir2/$src" ] && echo "$src" ; done
That would solve the problem in the example. The reason I want to use find is because the files in dir2 are nested within subdirectories which don't exist in dir1.
if you need the full path in ../dir2 then this instead
perl -MFile::Find -e 'map { $files{$_} = $_ } @ARGV; find({ wanted => sub {print "$File::Find::name\n" if exists $files{$_}}}, "../dir2")' *
Last edited by estabroo; 04-19-2011 at 08:31 PM.
Reason: full path option
I'd much rather have a solution I can understand. I'd prefer one which doesn't require me to abandon Bash or Gnu Find. Most importantly, I'd really like to know an answer to the question posed in the OP.
Thanks for your replies so far, BTW. I do appreciate it
I'd much rather have a solution I can understand. I'd prefer one which doesn't require me to abandon Bash or Gnu Find. Most importantly, I'd really like to know an answer to the question posed in the OP.
Thanks for your replies so far, BTW. I do appreciate it
LOL, I can understand your reluctance, but perl users are like zombies, they want everyone else to be a zombie too (and eat braaaaiiinnns).
Having said that (join us), it doesn't require you to abandon bash, using a perl script or one-liner is the same as using find, sed, awk, whatever. You can just run it in a subshell and capture/use the output (really just join us).
Code:
#!/bin/bash
other_dir="../dir2"
for f in $(perl -MFile::Find -e 'map { $files{$_} = $_ } @ARGV; find({ wanted => sub {print "$File::Find::name\n" if exists $files{$_}}}, '"'$other_dir'"')' *) ; do
some_command $f
done
This doesn't work because the files listed are simply the contents of dir2 (and its subdirectories). The globbing is now being done by find instead of bash. I did a sample run with a unique file name in dir2 to show this.
Code:
foo$ ls -R
.:
dir1 dir2
./dir1:
file1 file2 file[a1]
./dir2:
file[a1] ONLY_IN_DIR2 subdir1 subdir2
./dir2/subdir1:
file1
./dir2/subdir2:
file2
foo$ cd dir1/
dir1$ set -f
dir1$ for i in * ; do find ../dir2/ -type f -name $i ; done
../dir2/file[a1]
../dir2/subdir2/file2
../dir2/ONLY_IN_DIR2
../dir2/subdir1/file1
dir1$ set +f
dir1$
set -f turns globbing off
for i in * opens a loop which, on its first and only iteration, places an unaltered* character into variable i
find ../dir2/ -type f -name $i expands to find ../dir2/ -type f -name *
findinternally performs the * pattern match against all file names from dir2 and prints the matching file names
done
I want to find files in dir2 (and its subdirectories) with the same names as files in dir1 (with no subdirectories) and then perform commands on them.
This doesn't work either.
Code:
dir1$ for i in * ; do set -f ; find ../dir2/ -type f -name "$i" ; done ; set +f
../dir2/subdir1/file1
../dir2/subdir2/file2
../dir2/subdir1/file1
dir1$
It is clear that even though bash globbing is disabled, find still performs pattern matching internally. This leads me to believe that I need to rethink the way I'm using find. I don't think there is any way to disable pattern matching with the name tests in find.
Thank you for showing me how to disable globbing in Bash though
#!/bin/bash
for src in *; do
# no need to worry about * or ? in my file names
src=${src//\[/\\\[}
src=${src//\]/\\\]}
find ../dir2/ -type f -name "$src"
done
Your solution directly addresses the problem as explained in my OP. I have to give you props for reading carefully and understanding my help request even though I should have explained better.
Code:
foo$ ls -R
.:
dir1 dir2
./dir1:
file1 file2 file[a1] messed[[up]name ONLY_IN_DIR1
./dir2:
ONLY_IN_DIR2 subdir1 subdir2
./dir2/subdir1:
file1
./dir2/subdir2:
deepdir file2 file[a1]
./dir2/subdir2/deepdir:
messed[[up]name
foo$ cd dir1
dir1$ for src in *; do
> src=${src//\[/\\\[}
> src=${src//\]/\\\]}
> find ../dir2/ -type f -name "$src"
> done
../dir2/subdir1/file1
../dir2/subdir2/file2
../dir2/subdir2/file[a1]
../dir2/subdir2/deepdir/messed[[up]name
dir1$
Seems to work perfectly! (I'm excited because this is the first time I've found a real use for this form of parameter expansion.) I think I'd like to use fewer backslashes though.
Code:
dir1$ for src in * ; do src="${src//[/\[}" ; find ../dir2/ -type f -name "${src//]/\]}" -exec echo '{}' \; ; done
../dir2/subdir1/file1
../dir2/subdir2/file2
../dir2/subdir2/file[a1]
../dir2/subdir2/deepdir/messed[[up]name
dir1$
I have also learned that it is only necessary to escape the [ (opening bracket), because without it the ] (closing bracket) is non-special.
Code:
dir1$ for src in * ; do find ../dir2/ -type f -name "${src//[/\[}" -exec echo '{}' \; ; done
../dir2/subdir1/file1
../dir2/subdir2/file2
../dir2/subdir2/file[a1]
../dir2/subdir2/deepdir/messed[[up]name
dir1$
cd dir1
find /full/path/to/dir2 -type f -exec basename {} \; |
while read f; do
[ -f "$f" ] && command "$f"
done
This would work fine, except that it executes commands on files in dir1.
Code:
dir1$ find /tmp/foo/dir2 -type f -exec basename {} \; | while read f; do [ -f "$f" ] && echo $(dirname "$f")/$(basename "$f"); done
./file2
./file1
./messed[[up]name
./file1
dir1$
I want to perform commands on files in the dir2 hierarchy.
Code:
dir1$ find /tmp/foo/dir2 -type f |
> while read f ; do
> [ -f "$(basename "$f")" ] &&
> echo $(dirname "$f")/$(basename "$f")
> done
/tmp/foo/dir2/subdir2/file2
/tmp/foo/dir2/subdir2/file[a1]
/tmp/foo/dir2/subdir2/deepdir/messed[[up]name
/tmp/foo/dir2/subdir1/file1
dir1$
That's not a bad solution at all. It is exactly the kind of re-thinking I referred to in my reply to grail's post. It seems pretty obvious now, and I don't know why I didn't think of it.
I wonder what the limits or downside might be of piping the output of find into a loop?
I'm marking this thread solved now. If anyone else has more interesting or more graceful solutions, then please feel free to add them here.
for f in $(perl -MFile::Find -e 'map { $files{$_} = $_ } @ARGV; find({ wanted => sub {print "$File::Find::name\n" if exists $files{$_}}}, '"'$other_dir'"')' *) ; do
some_command $f
done
OMG, I think I just went permanently cross-eyed!
I wrote a few perl scripts in college. Still have the book in the basement somewhere. Maybe some day I'll dig it out and give it another go. Most likely not, though.
Wonder if the Ruby and Python camps will be chipping in too?
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.