LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 11-24-2005, 09:07 AM   #1
logicalfuzz
Member
 
Registered: Aug 2005
Distribution: Arch Linux
Posts: 291

Rep: Reputation: 41
using grep with expressions ANDed


is there a way to use grep ina way such that it matches expression1 AND expression2 AND expression3. I know with the 'e' option you can do an OR operation, so that it matches all 3 or only 1 expression
 
Old 11-24-2005, 10:06 AM   #2
otoomet
Member
 
Registered: Oct 2004
Location: Tartu, Ċrhus,Nürnberg, Europe
Distribution: Debian, Ubuntu, Puppy
Posts: 588

Rep: Reputation: 45
What do you want to do?
If the multiple patterns are on the same line, you may use multiple greps:

$ cat file |grep pat1 |grep pat2 ...

Perhaps it helps
Ott
 
Old 11-24-2005, 10:07 AM   #3
Samsara
Member
 
Registered: May 2003
Distribution: Ubuntu, Mac OS X Tiger
Posts: 481

Rep: Reputation: 32
By piping:

grep expr1 file | grep expr2

HTH,

Samsara
.
 
Old 11-24-2005, 12:51 PM   #4
logicalfuzz
Member
 
Registered: Aug 2005
Distribution: Arch Linux
Posts: 291

Original Poster
Rep: Reputation: 41
Quote:
What do you want to do?
i have
expr1 in file1
expr2 in file2 and
both expr1&expr2 in file3

so i want to use grep such that i grep ( expr1 AND expr2 ) to get the result as file3.
Well, in reality i have more than 2 expressions to match and more than 3 files to search. To cut things short, its a kind of database search thing.
 
Old 11-24-2005, 01:19 PM   #5
titopoquito
Senior Member
 
Registered: Jul 2004
Location: Ruhr Area, Germany
Distribution: Slackware64 14.0
Posts: 1,525

Rep: Reputation: 94
I tried this myself and found no way how to do this easily. So I wrote a script that will search for TWO search strings. It's not a good script I think, but it worked as far as I tried it. You will see that it creates several files to store the results. To use three search terms you will have to use even more files, but just look at the code and you will see what I am trying to say :

Code:
#!/bin/sh
# this script requires two arguments.
# all files in the current directory will
# be searched for the occurence of each argument
# (keyword) with grep. The files used:
# found-1: lists files that contain keyword A = $1
# found-2: lists files that contain keyword B = $2
# found-3: lists all lines (= file names) that differ 
#          between found-1 and found-2
#          i. e. files that don't contain both keywords
# found-4: the same like found-3, but cleaned up from 
#          additional information that is not needed
# found-5: contains (found-1 minus found-5), so only
#          files in which both keywords appear, remain
#
SEARCHDIR=/path/to/files/to/be/searched
#
# check, if two arguments are given,
# else display user how to use it
if [ $# -ne 2 ]; then echo "usage: command <keyword A> <keyword B>"; echo "Be sure to enclose the keywords in double quotes"; exit; fi
grep -i $1 -l $SEARCHDIR/* > found-1
grep -i $2 -l $SEARCHDIR/* > found-2
# now take only the lines with the file names from the 
# search results. they are marked with an "> " or an "< " 
# at the beginning of the line
diff found-1 found-2 | grep \> > found-3
diff found-1 found-2 | grep \< >> found-3
# remove the greater and lesser signs at the beginning of the lines
sed 's/> //' found-3 | sed 's/< //' > found-4 
# output the search result for keyword A - if any of these lines is not 
# in found-4 (i. e. the diff command reported it is the same in both
# searches, i. e. both keywords can be found in this file) put an entry
# for it in found-5
# clean found-5 first
rm -f found-5
touch found-5
cat found-1 | while read line; do if [ $(grep -c "$line"  found-4) -eq 0 ]; then echo "$line" >> found-5; fi; done
echo "The search results for \"$1\" and \"$2\" are stored in found-5"
 
Old 11-24-2005, 01:25 PM   #6
otoomet
Member
 
Registered: Oct 2004
Location: Tartu, Ċrhus,Nürnberg, Europe
Distribution: Debian, Ubuntu, Puppy
Posts: 588

Rep: Reputation: 45
If the patterns are not in the same line, I would probably use for-loop. Something like (not tested):

Code:
for file in $(ls) ; do
  if $(grep -q pat1 $file) ; then
    if $(grep -q pat2 $file) ; then
       ...
       echo "YESS!"
    fi
fi
...
done
If the files are small, you can also look at the context lines.

best,
Ott
 
Old 11-24-2005, 02:42 PM   #7
logicalfuzz
Member
 
Registered: Aug 2005
Distribution: Arch Linux
Posts: 291

Original Poster
Rep: Reputation: 41
well otoomet, i did something similar and it did not impress me. I had four searh strings, all to match at the same time in the same file. Remember, they may occur individually in different files, but i eliminate those results. Here's a part of the script i wrote. sombody please suggest a smarter solution. The large piece of code sucks.. i know.
Code:
#Get strings to be searched from user and
#assume * if user does not enter anything.
echo "Enter string to be found: "
read find_string
[ -z $find_string ] && find_string=.*
echo "Enter the Title: "
read find_title
[ -z $find_title ] && find_title=.*
echo "Enter the Content type: "
read find_content
[ -z $find_content ] && find_content=.*
echo "Enter the part of decription: "
read find_desc
[ -z $find_desc ] && find_desc=.*

#searching
for file_string in `grep -il $find_string $dbpath/*` ; do
        grep -il $find_title $file_string >> /dev/null
        if [ $? -eq 0 ] ; then
                for file_title in `grep -il $find_title $file_string` ; do
                        grep -il $find_content $file_title >> /dev/null
                        if [ $? -eq 0 ] ; then
                                for file_content in `grep -il $find_content $file_title` ; do
                                        grep -il $find_desc $file_content >> /dev/null
                                        if [ $? -eq 0 ] ; then
                                                for file_desc in `grep -il $find_desc $file_content
` ; do
                                                        grep -iH $find_string $file_desc >> /dev/tt
y
                                                done
                                        else
                                        echo Cannot find file with string $find_string with descrip
tion $find_desc... ; exit 0
                                        fi
                                done
                        else
                        echo Cannot find file with string $find_string with content $find_content..
. ; exit 0
                        fi
                done
        else
        echo Cannot find file with string $find_string under title $find_title... ; exit 0
        fi
echo Finished finding string...
done
edit:
What i did basically was.. searched $file_string in all files in the database directory using grep and used only the filenames of the result for a similar grep in the next stage. So you'll see 4 stages of grep for four strings.

Last edited by logicalfuzz; 11-24-2005 at 02:47 PM.
 
Old 11-24-2005, 03:04 PM   #8
logicalfuzz
Member
 
Registered: Aug 2005
Distribution: Arch Linux
Posts: 291

Original Poster
Rep: Reputation: 41
thanks titopoquito for the code. Seems you are looking for long filenames by giving 2 keywords. So doesn't that look for keyword1 and keyword2 in the same line? i have keyword1 and keyword 2 in the same _file_ . so the case maybe a bit different. To elaborate things, I have made a plain text cd database by listing the contents of /dev/cdrom into a file. This file begins with the lines..
TITLE:
TYPE::
DESCRIPTION:
So the search goes with 4 criterias in mind, the search string, the title,type and description, all in a single file.
 
Old 11-24-2005, 03:21 PM   #9
titopoquito
Senior Member
 
Registered: Jul 2004
Location: Ruhr Area, Germany
Distribution: Slackware64 14.0
Posts: 1,525

Rep: Reputation: 94
No, I'm searching for the occurance of the search patterns not in a line but in a whole file.

Code:
grep -i $1 -l $SEARCHDIR/* > found-1
grep -i $2 -l $SEARCHDIR/* > found-2
For each time the search pattern is met in * (any of the files) in $SEARCHDIR, a line with the filename is appended to found-1 or found-2 respectively.
 
Old 11-24-2005, 08:28 PM   #10
Samsara
Member
 
Registered: May 2003
Distribution: Ubuntu, Mac OS X Tiger
Posts: 481

Rep: Reputation: 32
Quote:
Originally posted by titopoquito
I tried this myself and found no way how to do this easily. So I wrote a script that will search for TWO search strings. It's not a good script I think, but it worked as far as I tried it. You will see that it creates several files to store the results. To use three search terms you will have to use even more files, but just look at the code and you will see what I am trying to say :
Code:
#!/bin/sh
# this script requires two arguments.
# all files in the current directory will
# be searched for the occurence of each argument
# (keyword) with grep. The files used:
# found-1: lists files that contain keyword A = $1
# found-2: lists files that contain keyword B = $2
# found-3: lists all lines (= file names) that differ 
#          between found-1 and found-2
#          i. e. files that don't contain both keywords
# found-4: the same like found-3, but cleaned up from 
#          additional information that is not needed
# found-5: contains (found-1 minus found-5), so only
#          files in which both keywords appear, remain
#
SEARCHDIR=/path/to/files/to/be/searched
#
# check, if two arguments are given,
# else display user how to use it
if [ $# -ne 2 ]; then echo "usage: command <keyword A> <keyword B>"; echo "Be sure to enclose the keywords in double quotes"; exit; fi
grep -i $1 -l $SEARCHDIR/* > found-1
grep -i $2 -l $SEARCHDIR/* > found-2

cat found-1 found -2 | sort | uniq -d > found-5
@titopoquito:

Does this do what you want?

It can be condensed to this:
Code:
#!/bin/sh
grep -i $1 -l $SEARCHDIR/* > found-1
grep -i $2 -l $SEARCHDIR/* >> found-1

sort found-1 | uniq -d > found-5
Samsara
.

Last edited by Samsara; 11-25-2005 at 06:29 PM.
 
Old 11-24-2005, 09:29 PM   #11
Samsara
Member
 
Registered: May 2003
Distribution: Ubuntu, Mac OS X Tiger
Posts: 481

Rep: Reputation: 32
@logicalfuzz:

I think this is what you want:
Code:
#!/usr/bin/perl
chomp (my @list = `ls`);
foreach (@list) {
        my $execute = "";
        for (my $i = 0; $i < @ARGV; $i++) {
                $execute .= "grep -i $ARGV[$i] -l $_ > /dev/null && ";
        }
        $execute .= " echo $_\n";
        system $execute;
}
Regards,

Samsara
.

Last edited by Samsara; 11-24-2005 at 09:33 PM.
 
Old 11-24-2005, 09:52 PM   #12
Samsara
Member
 
Registered: May 2003
Distribution: Ubuntu, Mac OS X Tiger
Posts: 481

Rep: Reputation: 32
Solution in bash

This should do the same trick, in pure bash:
Code:
#!/bin/sh
for regexp in $@;do
        grep -i $regexp -l * >> found-1
done
sort found-1 > found-2
for file in `uniq found-2`;do
        if [ $# -eq `grep $file found-2 | wc -l` ] ; then
                echo $file
        fi
done
rm found-1
rm found-2
You should check that found-1 and found-2 do not exist before writing to them! I've done enough work for free for one night!

btw you want to read:
http://www.tldp.org/LDP/abs/html/

Samsara
.

Last edited by Samsara; 12-11-2005 at 07:12 AM.
 
Old 11-25-2005, 02:51 AM   #13
titopoquito
Senior Member
 
Registered: Jul 2004
Location: Ruhr Area, Germany
Distribution: Slackware64 14.0
Posts: 1,525

Rep: Reputation: 94
Re: Solution in bash

Quote:
Originally posted by Samsara
This should do the same trick, in pure bash:
Code:
#!/bin/sh
touch found-1
for regexp in $@;do
        grep -i $regexp -l * >> found-1
done
sort found-1 > found-2
for file in `uniq found-2`;do
        if [ $# -eq `grep $file found-2 | wc -l` ] ; then
                echo $file
        fi
done
rm found-1
rm found-2
You should check that found-1 and found-2 do not exist before writing to them! I've done enough work for free for one night!

btw you want to read:
http://www.tldp.org/LDP/abs/html/

Samsara
.
Maybe it would stay in my memory if I used it more often. :P

I can't test it right now, had two spontanous reboots due to power failures. I will test it later and write again if it works :-/
 
Old 11-25-2005, 03:59 AM   #14
theYinYeti
Senior Member
 
Registered: Jul 2004
Location: France
Distribution: Arch Linux
Posts: 1,897

Rep: Reputation: 61
I wrote this a long time ago. I don't remember if this works, but it should probably
Code:
#!/bin/bash

# icase: -i for ignoring case, nothing for case independant.
icase=
if [ "$1" == "-i" ]; then
  icase="-i"
  shift
fi

# typeOp: AND|and or OR|or (default: AND)
typeOp="$1"
shift

# build the filtering command
CMD=
while [ $# -gt 1 ]; do
  regexp="${1//\//\\\/}"
  shift
  if [ -n "$regexp" ]; then
    if [ "$typeOp" == "OR" -o "$typeOp" == "or" ]; then
      CMD="$CMD /${regexp}/ {print FILENAME \":\" FNR \":\" \$0; next}"
    else
      CMD="$CMD ! /${regexp}/ {next}"
    fi
  fi
done
[ -n "$icase" ] && CMD="BEGIN {IGNORECASE=1} $CMD"
if [ "$typeOp" != "OR" -a "$typeOp" != "or" ]; then
  CMD="awk '${CMD} {print FILENAME \":\" FNR \":\" \$0}'"
else
  CMD="awk '${CMD}'"
fi

# run command
find "${1:-.}" -type f -print | eval xargs $CMD
This is for finding AND'd or OR'd regular expressions recursively in a directory. It is run like this:
Code:
mygrep.sh [-i] (AND|and|OR|or) "regexp1" ... "regexpN" "/path/to/dir"

eg:
$ mygrep.sh -i AND "cooking" "chocolate cake" .
Yves.
 
Old 11-25-2005, 06:56 PM   #15
titopoquito
Senior Member
 
Registered: Jul 2004
Location: Ruhr Area, Germany
Distribution: Slackware64 14.0
Posts: 1,525

Rep: Reputation: 94
Re: Solution in bash

Quote:
Originally posted by Samsara
This should do the same trick, in pure bash:
Code:
#!/bin/sh
touch found-1
for regexp in $@;do
        grep -i $regexp -l * >> found-1
done
sort found-1 > found-2
for file in `uniq found-2`;do
        if [ $# -eq `grep $file found-2 | wc -l` ] ; then
                echo $file
        fi
done
rm found-1
rm found-2
[snipped rest]
.
It works very well. The only problem I had is that it didn't handle long file names with spaces in between. So I made some changes to handle this correctly and tried a even shorter notation (I didn't try if the cut command always is correct to assume nine characters to cut):

Code:
#!/bin/sh
touch found-1
for regexp in $@;do
        grep -i $regexp -l * >> found-1
done
sort found-1 > found-2

uniq -c found-2 | egrep "^[ ]*$#" | cut -c 9-

rm found-1
rm found-2
EDIT: removed typo
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Regular expressions using grep linuxmandrake Programming 3 11-16-2005 05:29 PM
grep ?? can grep us variables? DaFrEQ Linux - Software 4 09-14-2005 01:22 PM
What does rpm -qa |grep th* (as compared to rpm -qa |grep th) display? davidas Linux - Newbie 2 03-18-2004 02:35 AM
"Undeleting" data using grep, but get "grep: memory exhausted" error SammyK Linux - Software 2 03-13-2004 04:11 PM
ps -ef|grep -v root|grep apache<<result maelstrombob Linux - Newbie 1 09-24-2003 12:38 PM


All times are GMT -5. The time now is 12:24 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration