Remove 38 lines after finding match

0.o · 03-21-2008, 07:02 AM

Hello,

I need to edit a MRTG config file that has about 5000 targets in it. Needless to say i don't want to go through and remove one at a time. I need to look for a match on the word 'Target' and then on the word Bundle. Once that combination is found, remove exactly 38 lines from there to the bottom. The only way I can think of doing it is with perl and a counter. I am sure there is a way with awk, i just don't know it well enough to complete this. Do any of you know of a way to do this?

--Thanks

jschiwal · 03-21-2008, 09:07 AM

Maybe not the best solution. The range '1,/Target.*bundle/' selects from the beginning of the file to the "Target" line. The range /Target.*bundle/,+38 selects the "Target" line and the next 38 lines.

Code:

sed -n '1,/Target.*Bundle/p;/Target.*Bundle/,+38{ /Target.*Bundle/!p }' testc

I hope your version of SED can use ranges like this.

osor · 03-21-2008, 07:40 PM

Either I don’t understand or I don’t see what’s wrong with this:

Code:

sed '/Target.*Bundle/,+38{/Target.*Bundle/!d }'

syg00 · 03-21-2008, 08:13 PM

I don't know MRTG, but is it always the case that "Target" and "bundle" will be on the same line ???.

jschiwal · 03-22-2008, 03:42 AM

Quote:

Originally Posted by osor

Either I don’t understand or I don’t see what’s wrong with this:

Code:

sed '/Target.*bundle/,+38{/Target.*bundle/!d }'

This will delete the line with "Target.*bundle" in it. It will print the lines after the 38th line however.

Code:

sed -n '1,/Target.*Bundle/p;/Target.*Bundle/,+38{ /Target.*Bundle/!p }' testc

Note that the "testc" file was just the name of the file I used to test the sed command. That part would be changes to something like mtrg.conf >mtrg.conf.new

The "-n" part causes sed to not print out lines unless you use the "p" print command. This is often used if you want to extract information from the original instead of editing parts of a file.

Note the semicolon in the middle of the command. This is one way of using more than one sed command on a oneliner. Another is the form "sed -e 'command1' -e 'command2'.
The first command prints all of the lines from 1 to the line with the words Target and Bundle in it.
The second command first matches all lines from the "Target.*Bundle" line and the next 38 lines. The sub-range matches the same lines that don't match the "Target.*Bundle" line. This prevents the "Target.*Bundle" line from being printed twice.

I will often use sed to print out a section of the output of commands like "lspci -v" which separate segments with blank lines:
sed -n '/NVidia/,/^$/p'

This is a common type of output that is used to make it easy to process with sed or awk. In awk you can have one field on each line with a blank line separating records.

syg00 · 03-22-2008, 04:21 AM

Quote:

Originally Posted by jschiwal

This will delete the line with "Target.*bundle" in it. It will print the lines after the 38th line however.

Not on my system.
It deletes (precisely) the 38 lines following the address line.

As was requested.

jschiwal · 03-22-2008, 04:51 AM

You are right. I missed the negation. However, I think the user wants to keep the next 38 lines but delete the 39th to the bottom of the the file.

Maybe the phrase "to the bottom" threw me off from what the OP wants.

The phrase "from there" does tend to imply "the next 38 lines"; but along with "to the bottom" I don't think the OP wants from the matching line to the end of the file, because the 38 lines would be meaningless.

A fragment of a real sample along with the desired result would help explain what is wanted and make testing the answer easier.

osor · 03-22-2008, 09:34 PM

Quote:

Originally Posted by jschiwal

Code:

sed -n '1,/Target.*Bundle/p;/Target.*Bundle/,+38{ /Target.*Bundle/!p }' testc

This will print all lines from the first line down to the first line containing the RE in question. Then, it will print the 38 lines following each other instance of the same RE (not including the line containing the RE itself). I don’t understand why you chose to keep the lines following the first occurrence of the RE, regardless of how you read the OP. In my reading of the OP, it specifically asked to “remove exactly 38 lines from there” (but in yours, you remove everything but the lines).

Anyhow, both our solutions make use of GNU sed features, and 0.o lists FreeBSD and Solaris in the profile. So if you are forced to use POSIX versions of the standard tools, you can use a counter in awk similar to how you would in Perl. For example,

Code:

awk 'BEGIN{a=38};a++>37;/Target.*Bundle/{a=0}'

is the equivalent of my GNU sed solution (except it has different behavior when two instances of the RE appear within 38 lines of each other).

jschiwal · 03-23-2008, 10:48 AM

Without the sample input it is hard to tell what is wanted. I took "to the bottom" to mean "to the end of the file".

0.o · 03-24-2008, 08:16 AM

Sorry for the late response. The input will look like this:

Code:

Target[172.22.9.149_Bundle50]: \Bundle50:community@172.22.9.149:::3:2:2
SnmpOptions[172.22.9.149_Bundle50]:  
nohc[172.22.9.149_Bundle50]: yes
SetEnv[172.22.9.149_Bundle50]: MRTG_INT_IP="<SOME PUB IP>" MRTG_INT_DESCR="Bundle50"
MaxBytes[172.22.9.149_Bundle50]: 75000000
Title[172.22.9.149_Bundle50]: Bundle50  
PageTop[172.22.9.149_Bundle50]: <h1>Traffic Analysis for Bundle50</h1>
                <div id="sysdetails"> 
                        <table>
                                <tr>
                                        <td>System:</td>
                                        <td>CHSPCMTK01HR.coxfiber.net in CHSPCMTK01</td>
                                </tr>         
                                <tr>  
                                        <td>Maintainer:</td>
                                        <td>CHSPCMTK01</td>
                                </tr>         
                                <tr>  
                                        <td>Description:</td>
                                        <td>Bundle50  </td>
                                </tr>         
                                <tr>  
                                        <td>ifType:</td>
                                        <td>CATV MAC Layer (127)</td>
                                </tr>         
                                <tr>  
                                        <td>ifName:</td>
                                        <td>Bu50</td>
                                </tr>         
                                <tr>  
                                        <td>Max Speed:</td>
                                        <td>600.0 Mbits/s</td>
                                </tr>         
                                <tr>  
                                        <td>Ip:</td>
                                        <td>68.228.136.1 ()</td>
                                </tr>         
                        </table>      
                </div>

That entry is exactly 38 lines long, and there are quite a few of them. What i want to do is find 'Target' then find 'Bundle', if they both exists, remove 38 lines following them.

osor · 03-24-2008, 12:25 PM

Quote:

Originally Posted by 0.o

What i want to do is find 'Target' then find 'Bundle', if they both exists, remove 38 lines following them.

If they both exist on the same line? So for the above input, the correct output would be the following.

Code:

Target[172.22.9.149_Bundle50]: \Bundle50:community@172.22.9.149:::3:2:2

Is that correct?

If so, did you try my two solutions (originally, I had a lowercase ‘b’ but I changed them now to match an uppercase ‘B’)?

0.o · 03-24-2008, 01:20 PM

Yep. It got it to work. I had to change a few things, but it works. I really appreciate your help!

jschiwal · 03-28-2008, 02:15 AM

Would the line count be accurate in the future?

You might want to use something like
sed '/^Target.*Bundle/,/<\/div>/{ /Target.*Bundle/!d }'
instead in case after an update or for different targets there are other fields added or changed. This solution depends on the <div id> ..</div> section appearing last.

If there is a blank line following target entries, that would make things easier:
sed '/^Target.*Bundle/,/^$/{ /Target.*Bundle/!d }'

sed '/PATTERN/,/^$/ ... is a common idiomatic pattern for selecting sections of an output, such as the output of lspci.

In awk, you can change FS to "\n" and RS to "" ('BEGIN {FS="\n"; RS=""} ...) to handle this type of input where each line is a filed and a blank line separates records.

If every record starts with the Target.*Bundle line and each record is separated by blank lines, then you simply need to print the first line of each record:
awk 'BEGIN {FS="\n"; RS=""} { print $1 }' file >newfile

When using sed, awk or perl to extract the information you want, the structure of the file is the most important information. It is what regex patterns depend on.