grep this and this2 and that and that2

ezekieldas · 05-22-2012, 04:20 PM

I'm probably overlooking something obvious here. I've found a number of pages which discuss
grep and logical AND but none seem to accomplish what I need.

I have a script (which I cannot modify). It outputs (something like) the following:

Code:

ViewAsString=[1.2.3.1:11|3] [1.2.3.1:11, 1.2.3.2:122, 1.2.3.8:123, 1.2.3.4:124]

I want to grab an exit code of 0 if my list of ipaddrs prints, 1 if it doesn't. I need a one liner (not another script) so grep seems to be a good choice here. The following ipaddrs _must_ always be present, even though one of them repeats: 1.2.3.1, 1.2.3.2, 1.2.3.8, 1.2.3.4

So I can do this

Code:

'foo.sh | grep 1.2.3.1 | grep 1.2.3.2 | grep 1.2.3.4 | grep 1.2.3.8'

But that seems too elaborate. There's a previous discussion here that was close but it didn't actually work:

Code:

foo.sh | awk '/1.2.3.1/ && /6.6.6.6/'
echo $?
0

How can I use grep or awk (or some other common tool) to get all 4 ipaddrs (no more, no less)?

bigrigdriver · 05-22-2012, 05:15 PM

I can't help you with grabbing exit codes, but the grep for number patterns works with the -e option.

Your code:

Code:

'foo.sh | grep 1.2.3.1 | grep 1.2.3.2 | grep 1.2.3.4 | grep 1.2.3.8'

My edit:

Code:

'foo.sh | grep -e 1.2.3.1 -e 1.2.3.2 -e 1.2.3.4 -e 1.2.3.8'

I made a simple list of number patterns running from 1.2.3.1 to 1.2.3.9, then ran grep with the -e options as shown above, and it returned only the desired numbers from the list.

colucix · 05-22-2012, 05:25 PM

You have to force awk to return an exit code if the expression is not true:

Code:

foo.sh | awk '!(/1.2.3.1/ && /1.2.3.2/ && /1.2.3.4/ && /1.2.3.8/){exit 1}'

In addition, maybe you have to refine the regular expressions, since /1.2.3.1/ matches 1.2.3.10, 1.2.3.11 and so on. Maybe adding a colon at the end of the addresses, as in your example?

Tinkster · 05-22-2012, 05:48 PM

Quote:

Originally Posted by colucix

You have to force awk to return an exit code if the expression is not true:

Code:

foo.sh | awk '!(/1.2.3.1/ && /1.2.3.2/ && /1.2.3.4/ && /1.2.3.8/){exit 1}'

In addition, maybe you have to refine the regular expressions, since /1.2.3.1/ matches 1.2.3.10, 1.2.3.11 and so on. Maybe adding a colon at the end of the addresses, as in your example?

And let's not forget the special meaning of the dot ... ;}

His string would also match e.g.: 1.213.1 ... back-slashes are our friends!

Cheers,
Tink

chrism01 · 05-22-2012, 05:57 PM

I tried these test

Code:

$ echo '1 x'|grep -e 1 -e 2
1 x

$ echo $?
0

$ echo '1 x'|grep -e g -e z
$ echo $?
1

$ echo 1|grep -e 1 -e x
1
$ echo $?
0

which to me implies -e does a logical OR, not a logical AND

whizje · 05-22-2012, 07:08 PM

Code:

grep -o  "[1][.][2][.][3][.][1248]"|uniq | wc -w

Search for ip string pipe through uniq and count if count is 4 you got your string.
for uniq the repeated lines need to be adjacent else you can use

Code:

grep -o  "[1][.][2][.][3][.][1248]"|sort -u | wc -w
a possible test
grep -o  "[1][.][2][.][3][.][1248]"|sort -u |[ "$(wc -w)" -eq "4" ];echo $?

David the H. · 05-23-2012, 09:51 AM

Yes, grep's (and sed's) "-e" expressions (including regex combinations) act like OR, not AND.

Think about how it works. Each line is read in in sequence, and compared to all the expressions given. If it matches any of them, it returns it as output. Then the next line is read.

So no, if the file must contain ALL of the given expressions, then a single grep can't handle it. You have to use awk, or another tool like perl or even shell scripting, that can keep track of what has and hasn't been found as it iterates through the file.

It would generally also take something more complex than a short one-liner as well. Indeed Colucix's offering:

Code:

awk '!(/1.2.3.1/ && /1.2.3.2/ && /1.2.3.4/ && /1.2.3.8/){exit 1}'

...won't really do, as it stands, because the test is checking each record (i.e. line) at a time for the existence of all entries, not the entire file as a whole.

It can be made to work, however, if the file is small enough to sit in memory, and you're using a compatible version of awk (i.e. gnu). Just redefine the record separator to null, which should make it treat the whole file as a single record. In which case I'd probably go with something more like this:

Code:

awk -v RS='' '{ if (/1[.]2[.]3[.]1/ && /1[.]2[.]3[.]2/ && /1[.]2[.]3[.]4/ && /1[.]2[.]3[.]8/ ) { print "all entries found"; exit 0 } else { print "something not found" ; exit 1 }}' infile.txt

The print commands are optional, of course I just put them there to clearly illustrate the output.

But if the file is too large for the above, you'd have to a more complex run-through of each line in turn, setting a variable for each matched string, say, then testing those for final compliance at the end.

Here's a proof of concept I just whipped up that appears to work, although it could undoubtedly be made much cleaner with a bit of effort.

Code:

awk '/1[.]2[.]3[.]1/ { a=1 } ; /1[.]2[.]3[.]2/ { b=1 } ; /1[.]2[.]3[.]4/ { c=1 } ; /1[.]2[.]3[.]8/ { d=1 } END{ x=a+b+c+d ; if ( x == 4 ){ print "all 4 found" ; exit 0 } else print "something not found" ; exit 1 }' infile.txt

whizje · 05-23-2012, 01:00 PM

Quote:

Originally Posted by David the H.

So no, if the file must contain ALL of the given expressions, then a single grep can't handle it. You have to use awk, or another tool like perl or even shell scripting, that can keep track of what has and hasn't been found as it iterates through the file.

You can get all the expressions with a single grep, but you need to verify if the result contains the four different ip's.

Code:

grep -o  "[1][.][2][.][3][.][1248]"|sort -u |[ "$(wc -w)" -eq "4" ];echo $?