Grep-like filter exists?

carl.waldbieser · 08-31-2005, 08:22 PM

Often, when I am writing a shell script, especially quick one liners, I find myself wishing for a filters similar to grep but, a little different than the standard options provide for.

Basically, what I would want this filter to do is to activate/deactivate on predicates (sometimes regular expression matching like grep, sometimes on particular line numbers) and to pass data or not pass data through the pipe until it is deactivated/activated. I know this sounds weird, but maybe an example can illustrate what I'm talking about.

I a really simple case, say I want to do something with the output of the "route" command. Now nowmally, route shows something like:

Code:

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.0.0     *               255.255.255.0   U     0      0        0 eth0
default         192.168.0.1     0.0.0.0         UG    0      0        0 eth0

Notice how there are a couple lines of header info before the actual route lines. If I had a pipeline that processes each record uniformly, the two header lines throw a wrench in the works. What I'd like to do is something like:

Code:

$ route | nfilter --pass 3 | res-of-pipeline

What this would do is only write the third and successive rows to standard output.

Or consider a config file with stanzas like:

Code:

[irrelevant-section-1]
blah
blah
blah
[relevant-section]
interesting
interesting
interesting
[irrelevant-section-2]
blah
blah
blah

If I want to pass the interesting parts to the rest of the pipe:

Code:

$ cat config | filter --pass '\[relevant' '\['

The idea here being that once a line matched the first expression, successive lines would pass until a line matched the second expression.

Anyway, what I want to know if some standard command line tools like that exist, and if they do, what are they called. If not, is it because it is just considered easy enough to do with the existing tools?

ilikejam · 08-31-2005, 09:10 PM

Hi.

'sed' is what you're after.

For the first case (miss the first two lines)

Code:

sed -n '3,$p' inputFile

the 3 matches the third line (the first line to print), the ',' means do the command for every line until the next match, the '$' always matches the last line in the file, and the 'p' means print. The -n flag suppresses the normal behaviour of sed to print every line it comes across.

For the second case, you can use the same syntax, but with regular expressions instead of absolute line numbers

Code:

sed -n '/^[relevant-section]/,/^[irrelevant-section]/p' inputFile

will print every line between (and including) [relevant-section] and [irrelevant-section]. The '^' matches the start of a line.

Dave

N.B. I believe this syntax is particular to GNU sed. It'll be fine if you're doing this on Linux, but might not work on other Unixen.

Matir · 08-31-2005, 09:11 PM

Awk is probably your best option for this. It's a very versatile, though complex, parsing engine.

carl.waldbieser · 08-31-2005, 11:34 PM

I know enough awk to have done this, or I could have done a python script to do it, too. I always thought sed was just for doing find & replace, but the syntax ilikejam demonstrated seems very close to what I described. I'll have to look into the portability issues, but it is basically what I was looking for.

Thanks, everyone!