Using Grep with Pattern File and PCRE
 

I would like to write a newline delimeted rules file using PCREs for use with the grep command. Grep has the option -f to obtain the search pattern from a file, and option -P to search using PCREs. However, these two options do not work together. The -f option only seems to work with fixed string rules.

A friend previously helped me get around this limitation somehow, but I can't remember how he did it. I also would like the ability to add comments at the end of each rule in the file.

Example Rules File-
Here is what I've tried so far-
Thanks for any help!

I'm not getting what yer tryin to do.

One thing (and I know this is not addressing your question), you can lose the 'cat' part before the pipe and just run:

grep -P -f rules.file data.file


Per the grep man page, -P is experimental....

I believe what the original poster is trying to do is to have one call to grep work for multiple patterns. So if I had a file that had two lines in it, one "Hello" and the other "World", I could do something akin to "grep -f HelloWorld.txt *", and have every line containing EITHER hello or world show in the results.

I don't know how to do this with grep. This is possible with awk and perl scripting, but that's probably more time consuming than the poster wants.


One alternative might be, since you're using regular expressions, to wrap the whole thing in a series of OR groups. So for "Hello" and "World" you could use "(Hello|World)"

The '-e' switch allows for multiple expressions so you can do a 'grep -e <item> -e <item>' over and over again....can also do the multiple grep's seperated by pipes but that is so lame.

I used awk to find any lines with IMAGE and print column 6 and find any lines with FRAG and print column 9:
awk '{ if($1=="IMAGE") print $6; if($1=="FRAG") print $9}'


Not sure if any of this is helpful.

You could iterate over all patterns in your file, and run grep with individual patterns.
--- rod.

You could look at egrep for more advanced options.

Thank you for the responses and that's an interesting solution theNbomr. Unfortunately my Linux computer is not working so well right now so I am forced to use the Windows machine to post. However, I did get in touch with my friend and he reminded me about how we did this before. This isn't going to be 100% correct but I will edit it later when I am able to test it.

So basically we cat the rules file out, remove all spaces before the #, the # itself, and everything after the #. Then the newlines of the rules.file are replaced with pipes. Finally the last newline in the rules file (which will screw up grep) is removed.

Grep *should* interpret this command as:

Phew! When running this yesterday it seemed to work pretty well. The only thing I couldn't get to work was a "grep -vP '^$' " statement to remove all blank lines. I put that in after the cat statement, but kept getting some weird variable error when the whole thing was run. I know that would work on the command line, so I'm not sure what the problem is.

So ultimately it appears you could replace all of:
with something like (untested):

A better way is to concat the regex pattern first, then pass the pattern to grep instead of calling grep for every pattern iterated.

Thank you for the responses and the awesome awk solution! Here is what ended up working (with the awk command as well):
For some reason the double quotes were needed around the entire grep statement. Something to do with how bash interpreted the command. Single quotes would not work.

Also added the grep command to remove blank lines.

Well I am glad you have a solution although I am not sure why you need so many calls to all the different apps. Calling cat is definitely not required

I agree and I'm just not familiar with awk yet. I will use your example to see if I can use that instead.