how does recursive grep work?

serutan · 07-09-2008, 02:03 AM

According to many sources (example: The Linux Cookbook 2nd Ed. p349) the command

grep -r sometext ~/doc/*.txt

should search for the string "sometext" in .txt files in the directory ~/doc and all its subdirectories. Yet on my system (Ubuntu heron) it only searches files in ~/doc, ignoring subdirectories. Neither -r nor -R seem to have any effect at all. What am I doing wrong?

serutan

Mr. C. · 07-09-2008, 02:17 AM

You are thinking that grep "sees" the argument ~/doc/*.txt. It doesn't. The shell first does file name expansion before it ever starts grep running. So, grep sees whatever the wildcard pattern expands to. Add echo in front of grep, and you'll see what grep sees. Or add set -x one line before the grep.

matthewg42 · 07-09-2008, 02:33 AM

You can do

Code:

grep -r pattern ~/doc

Which will grep all files under ~/doc. If you need to search in only some files, then you can employ find to location the files and xargs to call grep on them:

Code:

find ~/doc -type f -name '*.txt' -print0 | xargs -0 grep pattern

The -print0 option to find and the -0 option to xargs are not necessary if you have sensibly named files (without spaces and other weird characters), but it's generally a good idea to use them, despite the extra typing.

serutan · 07-11-2008, 09:41 AM

Thanks for the help. I understand that "grep -r pattern ~/doc" will search ~/doc and its subdirectories for files containing the pattern. I also see that as Mr. C. says, "~/doc/*.txt" expands to a list of files ending in .txt in the ~/doc folder only. I'm trying to figure out how to search a subset of files, for example *.txt, within a directory and all its subdirectories. The reason is that I have a subdirectory tree containing many very large media files and a few .txt files, and I want to search only the .txt files.

After more research I believe this has to be a 2-step process: find the files by matching name pattern, then search the found files for the string pattern. The following works for me:

find ~/doc -name "*.txt" | xargs grep -H "test"

If there is a simpler equivalent command I would still like to know. Back in my VMS days I'm pretty sure this type of search was easy.

matthewg42 · 07-11-2008, 10:10 AM

The xargs approach is the tried and tested way. If you find it too long-winded, you can easily write a little script, put it in your PATH. An alternative is to write a shell function and put it in your .bashrc file

Code:

rgrep () {
        if [ $# -ne 2 ]; then
                echo "Usage: rgrep pattern directory/[filepattern]"
                return 1
        fi

        file_pattern=""
        dir="."

        if [ -d "$2" ]; then
                dir="$2"
        else
                case "$2" in
                */*)
                        # looks like we have a pattern in a directory
                        file_pattern="${2##*/}"
                        dir="${2%/*}"
                        ;;
                *)
                        # looks like we only have a pattern (no directory)
                        file_pattern="$2"
                        ;;
                esac
        fi

        if [ "$file_pattern" != "" ]; then
                file_pattern="-name \"$file_pattern\""
        fi

        find "$dir" -type f $file_pattern -print0 |xargs -0 grep -H "$1"
}

Which you can use like this:

Code:

rgrep pattern /some/existing/dir

Which will searh for pattern in all files under /some/existing/dir and it's sub-directories.
Or like this:

Code:

rgrep pattern /some/place/*.txt

Which will search for pattern in all files whose name ends with ".txt" in /some/place or any of it's sub-directories.

Mr. C. · 07-11-2008, 01:00 PM

Perhaps what you want is a slight change:

find ~/doc -name "*.txt" | xargs grep -l test | other_commands

eg.

find ~/doc -name "*.txt" | xargs grep -l test | sort -u