LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 04-19-2011, 11:49 AM   #1
sneakyimp
Senior Member
 
Registered: Dec 2004
Posts: 1,027

Rep: Reputation: 67
grep aliases in .bashrc prevent use of extra flags in Ubuntu


I'm trying to limit a recursive grep search to just PHP files by doing this:
Code:
sneakyimp@Ubuntu-64bit:/var/www/site$ grep -irl --include='*.php' 'car_images' *
2011_02_11_dump.sql
car_images_pending.sql
client/search_results.php
client/ext_setup.php
client/bobs/ext_setup.php
As you can see, it's also searching sql files for some reason despite the --include parameter. What the heck?

I think it may have something to do with the .bashrc alias for grep:
Code:
# enable color support of ls and also add handy aliases
if [ -x /usr/bin/dircolors ]; then
    test -r ~/.dircolors && eval "$(dircolors -b ~/.dircolors)" || eval "$(dircolors -b)"
    alias ls='ls --color=auto'
    #alias dir='dir --color=auto'
    #alias vdir='vdir --color=auto'

    alias grep='grep --color=auto'
    alias fgrep='fgrep --color=auto'
    alias egrep='egrep --color=auto'
fi
Can anyone help me fix this problem? I really need to search only my php files.
 
Old 04-19-2011, 12:08 PM   #2
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian + kde 4 / 5
Posts: 6,837

Rep: Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981
You can temporarily bypass an alias in several ways. Backslash-escape it, quote it, or use the command built-in.

Code:
\grep -irl --include='*.php' 'car_images' *
"grep" -irl --include='*.php' 'car_images' *
command grep -irl --include='*.php' 'car_images' *
Edit: Oh yeah, or use the full path to the command.
Code:
/bin/grep  -irl --include='*.php' 'car_images' *

Last edited by David the H.; 04-19-2011 at 12:11 PM. Reason: as stated.
 
Old 04-19-2011, 12:14 PM   #3
sneakyimp
Senior Member
 
Registered: Dec 2004
Posts: 1,027

Original Poster
Rep: Reputation: 67
Thanks for the helpful info. I may just remove the alias altogether if I can't fix it.

Any thoughts on why that alias would prevent the --include flag from working?
 
Old 04-19-2011, 12:16 PM   #4
sneakyimp
Senior Member
 
Registered: Dec 2004
Posts: 1,027

Original Poster
Rep: Reputation: 67
Hm. I've tried all four of those solutions that you suggested and the sql file is still showing up? WTF? Problem must be somewhere else.
 
Old 04-19-2011, 12:26 PM   #5
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian + kde 4 / 5
Posts: 6,837

Rep: Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981
Actually, now that I look at it, the alias only adds --color=auto to the command, which shouldn't affect the way the files are read at all.

I'm testing it out myself now, and so far I see pretty much the same effect. The --include option isn't really narrowing down the search. I wonder if it has anything to do with the recursive option or shell globbing?
 
Old 04-19-2011, 12:29 PM   #6
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,564

Rep: Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901
The issue is not the command or the aliases. This is partly born out by the fact that none of the solutions in post #2 make a difference.

The issue you have is that the asterix is expanding to all the files in the current directory.
You should find that once on any level lower than current directory, the only files being returned will be php files (as per your example)

Not sure if grep has a hard and fast solution for this, but of course you could use find to restrict to php files and then use your grep in the -exec.
 
Old 04-19-2011, 12:47 PM   #7
sneakyimp
Senior Member
 
Registered: Dec 2004
Posts: 1,027

Original Poster
Rep: Reputation: 67
This command works for me on other machines (e.g., centOS or debian machines). That it doesn't work here is really confusing. I've also posted the ubuntu forums.

grail, which asterisk do you mean? also, those sql files are the only ones in the file hierarchy so this may also happen in nested directories too.
 
Old 04-19-2011, 12:59 PM   #8
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian + kde 4 / 5
Posts: 6,837

Rep: Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981
It certainly does seem to be a globbing problem or something, but I'm not getting any consistent results from any of my tests. Nothing I do seems to have any logical relationship to what the man page or my previous experience tells me it should. In particular:
Code:
--include=GLOB   Search only files whose base name matches GLOB (using
                 wildcard matching as described under --exclude).
This description doesn't give any indication that globbing would override it. I've also just gone through the entire info file and found nothing about it. The closest it says about it is that this:
Code:
grep -rH --include='*.c' 'hello' /home/gigi
..is similar to this:
Code:
find /home/gigi -name '*.c' -print0 | xargs -0r grep -H 'hello'
But building a similar grep myself doesn't seem to do anything of the sort.

Even your statement that "You should find that once on any level lower than current directory, the only files being returned will be php files" doesn't seem to hold true in my tests.

This has got me thoroughly confused.
 
Old 04-19-2011, 12:59 PM   #9
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,564

Rep: Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901
The last asterix is the issue. Easy enough test though, simply copy one of the sql files into the client directory and run again.
Actually try the following as it seems to cure the ills:
Code:
grep -rI --include=*.php 'car_images' .
 
Old 04-19-2011, 01:11 PM   #10
sneakyimp
Senior Member
 
Registered: Dec 2004
Posts: 1,027

Original Poster
Rep: Reputation: 67
nice work, grail. That command does indeed seem to produce the results I'm after (it returns 55 results instead of 57, the difference being the two sql files).

However, this doesn't explain why the --include flag fails to exclude the sql files. From the man page:
Code:
       --exclude=GLOB
              Skip  files  whose  base  name  matches  GLOB  (using  wildcard  matching).  A file-name glob can use *, ?, and [...]  as wildcards, and \ to quote a wildcard or backslash character
              literally.
       --include=GLOB
              Search only files whose base name matches GLOB (using wildcard matching as described under --exclude).
Unless I'm missing something, 2011_02_11_dump.sql does not match a GLOB of *.php.

What's more irritating (and more confusing) is that Ubuntu behaves differently than every other linux distro (and Mac OSX 10.5) that I've ever used. Bizarre.
 
Old 04-19-2011, 01:16 PM   #11
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,564

Rep: Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901
Well I am not familiar with these 2 options before today, but searching around I found reference that indicated that they are only in effect when used with a recursive
search and then only affect directories not files, hence again the fact that * expands to all files in the current directory they are getting searched.

I have not found collaboration of this information in the man pages.
 
1 members found this post helpful.
Old 04-19-2011, 01:19 PM   #12
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian + kde 4 / 5
Posts: 6,837

Rep: Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981
Ok, I don't know what was going on before, but I just built a clean file tree (as opposed to working with existing files) and things are looking more rational.
Code:
$ tree
.
├── foofile-lvl1.sh
├── foofile-lvl1.txt
└── lvl2
    ├── foofile-lvl2.sh
    ├── foofile-lvl2.txt
    └── lvl3
        ├── foofile-lvl3.sh
        └── foofile-lvl3.txt

2 directories, 6 files

#Each file contains the single word "foo"

$ grep -rli --include='*.txt' "foo" *
foofile-lvl1.sh
lvl2/lvl3/foofile-lvl3.txt
lvl2/foofile-lvl2.txt

$ grep -rli --include='*.txt' "foo" .
./lvl2/lvl3/foofile-lvl3.txt
./lvl2/foofile-lvl2.txt
./foofile-lvl1.txt
So grail seems to have been right all along. Don't use globbing and --include at the same time.
 
Old 04-19-2011, 01:41 PM   #13
sneakyimp
Senior Member
 
Registered: Dec 2004
Posts: 1,027

Original Poster
Rep: Reputation: 67
grail, let me first say I'm grateful for your knowledge. that command is going to work for me i think. I can get over the fact that the man pages specify FILE as the last arg and "." is a directory.

However, the --include flag does in fact filter all files to be searched and NOT directories when used in a recursive search. At least this has been my experience on numerous other platforms including fedora, centos, and debian. I believe you are also correct that --include and --exclude also only apply in a recursive search. The use of --include in a non-recursive search appears to have the opposite effect. Note that the --include flag below results in no search results:
Code:
sneakyimp@Ubuntu-64bit:/var/www/site$ grep -il 'version' *.php
CHANGELOG.php
COPYRIGHT.php
CREDITS.php
index2.php
index.php
INSTALL.php
LICENSES.php
sneakyimp@Ubuntu-64bit:/var/www/site$ grep --include=*.php -il 'version' *.php
sneakyimp@Ubuntu-64bit:/var/www/site$
Having just typed that, I'm starting to understand what you are saying. That last asterisk tells grep to search not just the current directory (.) but also every file in the current directory and the --include param has a reversed effect on the current directory and works properly on all subdirs. I'm not certain, but I believe other distros don't work this way.

David, thanks also for your insights here -- and also for bearing witness to the confusing behavior -- at least inasmuch as the documentation doesn't adequately describe how this function works.
 
Old 04-19-2011, 07:38 PM   #14
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,564

Rep: Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901Reputation: 2901
Well now it is getting a little weird. I just did a commensurate test on my machine (CLFS based) and with or without the --include I return the
same responses???
Code:
$ ls *.sh
d.sh
d2.sh
$ grep -il 'echo' *
d.sh
d2.sh
grub.cfg
$ grep -il 'echo' *.sh
d.sh
d2.sh
$ grep --include=*.sh -il 'echo' *.sh
d.sh
d2.sh
But personally this does agree with my findings as the globbing is expanded and all files in current directory are searched.

So not sure why yours is doing an exclude?? I will try on Ubuntu when I get home to see if I get the same affect as here.
 
Old 04-20-2011, 12:34 AM   #15
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian + kde 4 / 5
Posts: 6,837

Rep: Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981Reputation: 1981
I think we have to first make sure we understand exactly what globbing does. Globbing is pattern matching using the special characters *,?, and [].

http://tldp.org/LDP/Bash-Beginners-G...ect_04_03.html

We also have to recognize that there are two levels of globbing going on here. The * in --include="*.txt" is globbing done by grep, while the * at the end of the command is globbing done by bash.

In bash, * expands to a list of all files in the current directory (or whatever directory is specified by a partial match). After globbing expansion is done, bash runs the command on the expanded filelist.

grep 'foo' * in a directory containing two files and one subdirectory really means grep 'foo' file1 file2 dir1.

grep then performs its globbing on the files/directories passed to it by bash.

I think the following output in my last post is telling.
Code:
$ grep -rli --include='*.txt' "foo" *
foofile-lvl1.sh
lvl2/lvl3/foofile-lvl3.txt
lvl2/foofile-lvl2.txt
Notice how the lvl1 file appears first, then it jumps down to lvl3 for the next match. That means that grep is processing the current directory first, then working its way recursively through the subdirectory tree.

So I'm guessing that grep is processing everything passed to it by the shell globbing in order, and if that something is a directory, it also processes that recursively.

Although I notice now that my test also failed to match foofile-lvl1.txt. It certainly does seem like the --include is acting like an --exclude on the top level.

I added a second subtree to my test, and came up with this.
Code:
$ tree
.
├── foofile-lvl1.sh
├── foofile-lvl1.txt
├── lvl2
│** ├── foofile-lvl2.sh
│** ├── foofile-lvl2.txt
│** └── lvl3
│**     ├── foofile-lvl3.sh
│**     └── foofile-lvl3.txt
└── lvl2a
    ├── foofile-lvl2a.sh
    ├── foofile-lvl2a.txt
    └── lvl3a
        ├── foofile-lvl3a.sh
        └── foofile-lvl3a.txt

$ grep -rli 'foo' *
foofile-lvl1.sh
foofile-lvl1.txt
lvl2/lvl3/foofile-lvl3.sh
lvl2/lvl3/foofile-lvl3.txt
lvl2/foofile-lvl2.sh
lvl2/foofile-lvl2.txt
lvl2a/foofile-lvl2a.txt
lvl2a/lvl3a/foofile-lvl3a.txt
lvl2a/lvl3a/foofile-lvl3a.sh
lvl2a/foofile-lvl2a.sh

$ grep -rli 'foo' .
./lvl2/lvl3/foofile-lvl3.sh
./lvl2/lvl3/foofile-lvl3.txt
./lvl2/foofile-lvl2.sh
./lvl2/foofile-lvl2.txt
./lvl2a/foofile-lvl2a.txt
./lvl2a/lvl3a/foofile-lvl3a.txt
./lvl2a/lvl3a/foofile-lvl3a.sh
./lvl2a/foofile-lvl2a.sh
./foofile-lvl1.sh
./foofile-lvl1.txt

$ grep -rli --include='*.txt' 'foo' *
foofile-lvl1.sh
lvl2/lvl3/foofile-lvl3.txt
lvl2/foofile-lvl2.txt
lvl2a/foofile-lvl2a.txt
lvl2a/lvl3a/foofile-lvl3a.txt

$ grep -rli --include='*.txt' 'foo' .
./lvl2/lvl3/foofile-lvl3.txt
./lvl2/foofile-lvl2.txt
./lvl2a/foofile-lvl2a.txt
./lvl2a/lvl3a/foofile-lvl3a.txt
./foofile-lvl1.txt

$ grep -rli --include='*.txt' 'foo' . ./lvl2/lvl3/foofile-lvl3.sh
./lvl2/lvl3/foofile-lvl3.txt
./lvl2/foofile-lvl2.txt
./lvl2a/foofile-lvl2a.txt
./lvl2a/lvl3a/foofile-lvl3a.txt
./foofile-lvl1.txt
./lvl2/lvl3/foofile-lvl3.sh
I think the last one tells the story.

It looks like grep only filters files if it processes the directory itself during recursion. If you pass it a filename directly, through globbing or otherwise, it will accept that as input, whether or not it matches the --include or --exclude globs.

Except for that bizarre top-level exclusion, of course.

Last edited by David the H.; 04-20-2011 at 12:36 AM. Reason: minor rewording
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to force sudo to use ~/.bashrc aliases? bigrigdriver Linux - General 10 02-24-2008 06:14 PM
Eterm doesn't import aliases from bashrc oldmanofthesea Linux - General 1 05-16-2005 08:17 AM
bashrc Aliases don't work soren625 Linux - Newbie 10 01-27-2005 09:59 AM
aliases in .bashrc file davalos Linux - Newbie 7 08-21-2003 02:03 PM
.bashrc aliases Crashed_Again Linux - General 8 01-30-2003 07:42 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 11:07 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration