Remove lines in a text file based on another text file

asiandude · 01-28-2009, 05:18 AM

I have a text file called file1.txt containing many lines

eg.

line1
line2
line3
line4
line5
line6

Then i have another text file called file2.txt contains

3
5
6

Is there a command to remove the lines in file1.txt based on the keywords in file2.txt? note: It should remove line3,line5,line6 based on 3,5,6

Thanks.

burschik · 01-28-2009, 05:55 AM

This question is asked so often it sounds like homework.

Hko · 01-28-2009, 06:31 AM

Code:

sed -ni $(xargs -I: echo -n :d\;<file2.txt;echo p) file1.txt

PTrenholme · 01-28-2009, 06:50 AM

Or grep -vf file2.txt file1.txt might be a solution. (See info grep for details.)

Echoing burschik's question, is this, perhaps, a "qualifying question" for a job or certification? Please tell us why you're asking this question?

asiandude · 01-28-2009, 09:33 AM

Thanks guys.. it works for my examples

however it failed to run when file2.txt contains some full keywords.

colucix · 01-28-2009, 10:28 AM

Quote:

Originally Posted by asiandude

however it failed to run when file2.txt contains some full keywords.

So please, post a REAL example.

PTrenholme · 01-28-2009, 11:06 AM

http://www.linuxquestions.org/questi...AAAElFTkSuQmCC

Quote:

Originally Posted by asiandude

Thanks guys.. it works for my examples

however it failed to run when file2.txt contains some full keywords.

Yes, please explain what you mean by "full keywords."

Code:

$ #Sample data from first post

$ cat file1.txt
line1
line2
line3
line4
line5
line6

$ cat file2.txt
3
5
6

$ # Output from proposed solution

$ grep -vf file2.txt file1.txt
line1
line2
line4

$ # Longer test strings.  Is "line3" a "full keyword?"

$ cat file3.txt
line3
ne5
6

$ Output from proposed solution

$ grep -vf file3.txt file1.txt
line1
line2
line4

asiandude · 01-28-2009, 08:08 PM

I just checked my remove-words list and found a . in it. I want to remove lines that contain a '.' as well.

PTrenholme · 01-28-2009, 09:42 PM

When you read info grep, you should have noted the the contents of the file referenced by the -f argument are regular expressions, not simple strings. Thus "." is the wild character.

So, obviously, you need to "escape" the period before you can use it:

Code:

$ cat file1.txt 
line1
line2
line3
line4
line5
line6
Line.1
Line.7

$ cat file2.txt 
3
5
6
\.

$ grep -vf file2.txt file1.txt
line1
line2
line4

asiandude · 01-29-2009, 12:45 AM

Thanks PTrenholme. You have been very helpful.

I will go read up more.

makyo · 01-29-2009, 10:59 AM

Hi.

If you don't need (or want) the regular expression processing, consider fgrep (or grep -F in some versions), which does not treat characters as having the magic of regular expressions ... cheers, makyo