[SOLVED] grep / egrep negation in an OR expression

wandering dog · 01-04-2014, 10:38 PM

Hi there, I couldn't find a solution so far, although I've been googling quite a bit.

My goal is to grep through a file and match
° either lines that contain no alphabetic letters
° OR lines that contain word

the closest I came to was

Code:

cat file | egrep "(-v [[:alpha:]]|word)"

but that will only find word and not the numbers-only lines.

So is there a solution that goes like

Code:

cat file | egrep (![[:alpha:]]|word)
        EITHER    ↑no alpha  OR word

(pseudocode, ! like the bash negation)

I guess I'm blind or too tired, but I can't figure it out...

Yes, egrep (this|that) file would be nicer than cat file |, but whenever I have read the file first, I just leave the cat and apply the grep ;-)

syg00 · 01-04-2014, 11:59 PM

You can get close with grep, but you have to know too much about the structure of the data - use something with regex smarts; awk or perl say.

rknichols · 01-05-2014, 10:28 AM

This appears to do what you want:

Code:

egrep '^[^[:alpha:]]*$|word'

wandering dog · 01-05-2014, 10:31 PM

Quote:

Originally Posted by rknichols

This appears to do what you want:

Code:

egrep '^[^[:alpha:]]*$|word'

Wow! Thanks! It does. I admit I can't see the negation here, i.e. where exactly this expression does it. I read like

Code:

^[^[:alpha:]]*$
^^^    ^     ^^
|||    |     ||
|||    |     ||
|||    |     ||
|||    |     |------------------------------------------------------------------------------the line ends
|||    |     ---------------------------------------------------------------------------until
|||    --------------------------------------------------then, any alphabetic character
||----------------------------------term begins again (??) 
| --------------a square bracket (?)
term begins with

OK, i tried to google it once again, and I notice I had already skimmed a site where negation is explained.

If ever anyone faces the same question and comes here, here's another link:

Be careful, because [^a-z] inside a character class indicates negation

thanks rk and syg

syg00 · 01-05-2014, 11:12 PM

What about if there is whitespace in a numeric line - or a alphanumeric line ?. This is what I meant about having to know the data intimately.
grep (and sed) can do this well, but it can be difficult to cover all corner cases.

Glad you picked up some pointers.

rknichols · 01-06-2014, 10:50 AM

Quote:

Originally Posted by syg00

What about if there is whitespace in a numeric line - or a alphanumeric line ?.

Whitespace, punctuation, or control characters are not alphabetic characters and would not block the match. The presence of any alphabetic characters would block the first branch of the RE, but might still match word in the second branch.

The given RE doesn't really address the "negation in an OR expression" posed in the thread title (a RE can't do that), but restates the problem as either of two positive matches. The first branch of the RE parses as:

Code:

^[^[:alpha:]]*$
^^^\       /^^^
||| \     / |||
|||  \   /  |||
|||   \ /   |||
|||    ^    ||+---------------------------------------------------end of line
|||    |    |+---------------------------------------zero or more matches of the preceding atom
|||    |    +----------------------------end of bracket expression
|||    +-------------------a list of all alphabetic characters
||+--------------match any character not in the list
|+-------start of bracket expression
beginning of line

druuna · 01-06-2014, 11:36 AM

@rknichols: Nice solution! However

You might have overlooked something:

Code:

egrep '^[^[:alpha:]]*$|word'

The second part looks for word, which would include xword, wordy and xwordy, which might not be wanted.

I just noticed that word boundaries aren't picked up when you do this:

Code:

# invalid example
egrep '^[^[:alpha:]]*$|\bword\b' infile

This does work on my side:

Code:

grep -P '^[^[:alpha:]]*$|\bword\b' infile

rknichols · 01-06-2014, 12:33 PM

Quote:

Originally Posted by druuna

The second part looks for word, which would include xword, wordy and xwordy, which might not be wanted.

Actually I thought of that at the time, but also realized that always delimiting the word with "\b" would prevent strings like "123word" from matching, which might or might not be desired behavior. And really, that part of the expression is just like any common use of grep.

wandering dog · 01-06-2014, 12:42 PM

Actually the common grep behavior was the one I wanted to have, I just wanted to filter either numerical ip-adresses OR a specific domain from a large list

.