Bash remove part of a file based on contents of another file

bhepdogg · 01-31-2007, 11:14 AM

I have file x and file y. File y contains a lot of text, including the contents of file x (at some point, the command cat x >> y had been issued). Is there a quick and easy way to reverse this? Basically this means removing from file y, the part that matches the exact contents of file x?

For example, if file y contains:
a
b
c
d
e
f
d
c

and file x contains:
c
d

then the desired output would be file y containing:
a
b
e
f
d
c

wjevans_7d1@yahoo.co · 01-31-2007, 12:18 PM

I'm not aware of how you can do this by scripting only in bash. I have three questions for you.

1. Are you willing to bite the bullet and try Perl? It could be rather simple in Perl.

2. Is either of the two input files likely to be huge huge huge, or are they of reasonably short length?

3. Suppose the content of the file x is found more than once in file y. Do you want to remove all occurrences, or just the first one?

ygloo · 01-31-2007, 12:24 PM

file "x" contents -

first line
...
...
....
..
last line

sed 's/first line/last line/d' y

this deletes all lines between 1st and last line
check if it works and then run "sed" with "-i" option

sed -i.bak 's/....
(".bak" creates backup file )

bhepdogg · 01-31-2007, 02:19 PM

Thanks a bunch. That was exactly what I was looking for. Here is my script that works in case anyone else would like it:

#!/bin/bash

FIRSTLINE=`head -1 x`
LASTLINE=`tail -1 x`

sed -i "/^$FIRSTLINE$/,/^$LASTLINE$/d" y

ygloo · 01-31-2007, 03:13 PM

with "^" sed will search for pattern at begging of line

sed -i "/$FIRSTLINE$/,/$LASTLINE$/d" y

found info here:
http://www-128.ibm.com/developerwork...-textedit.html