[SOLVED] Grep question

freeroute · 10-26-2016, 11:07 AM

I have 2 files, contains only rows. I would like to remove files from left.txt beginning lines from found.txt (separator

My command was: cut -d: -f1 found.txt | grep -vf- left.txt > remain.txt
But it doesn't work (remain.txt is an empty file). Is it a syntax error? I could not find the solution.

Could you help me, please?

found.txt contains:

00305aa421cda76cc07228111a5650a8:test1
011bbcf1eb8f65e326ebdc4f9e349c94:test2
0127f1eeffb32ed2d6001c0a3ce365f1:test3

left.txt:

4a37a393387cfef1dc8d8ee82c763003
8f55b05e6f40e6f2fb38f1a76308df64
6a8eba4925beeaf2ffa5e498f0f22c35
ae067c5c1b846ff420d94462421197c5
8d03eb07d810d9f486ca82ce2b2df106
00305aa421cda76cc07228111a5650a8
011bbcf1eb8f65e326ebdc4f9e349c94
0127f1eeffb32ed2d6001c0a3ce365f1

keefaz · 10-26-2016, 11:18 AM

It works here...

Check output from ' cut -d: -f1 found.txt '

freeroute · 10-26-2016, 11:33 AM

This command working:

cut -d: -f1 found.txt | less
00305aa421cda76cc07228111a5650a8
011bbcf1eb8f65e326ebdc4f9e349c94
0127f1eeffb32ed2d6001c0a3ce365f1

and this command not:
cut -d: -f1 found.txt | grep -vf- left.txt | less

So I don't understand what the problem....
Maybe very simple thing...

grail · 10-26-2016, 12:39 PM

Is there a reason for the '-' after the 'f' in the grep?

keefaz · 10-26-2016, 12:48 PM

Quote:

Originally Posted by freeroute

This command working:

cut -d: -f1 found.txt | less
00305aa421cda76cc07228111a5650a8
011bbcf1eb8f65e326ebdc4f9e349c94
0127f1eeffb32ed2d6001c0a3ce365f1

and this command not:
cut -d: -f1 found.txt | grep -vf- left.txt | less

So I don't understand what the problem....
Maybe very simple thing...

You use 'less', that means there are more lines than those you quoted, no?
Maybe grep doesn't find exact match for all the text coming from ' cut -d: -f1 found.txt '

With the lines you gave as example, that works

keefaz · 10-26-2016, 12:50 PM

Quote:

Originally Posted by grail

Is there a reason for the '-' after the 'f' in the grep?

I thinks it tells grep to use standard input for -f option (obtain patterns from file)

grail · 10-26-2016, 01:36 PM

OP you need to think about the order in which you are doing things.

Starting with:

Code:

00305aa421cda76cc07228111a5650a8:test1
011bbcf1eb8f65e326ebdc4f9e349c94:test2
0127f1eeffb32ed2d6001c0a3ce365f1:test3

Your cut will return:

Code:

00305aa421cda76cc07228111a5650a8
011bbcf1eb8f65e326ebdc4f9e349c94
0127f1eeffb32ed2d6001c0a3ce365f1

Then you tell grep to use the data contained in another file to exclude all lines of the above that are not in the file ... problem is, all the lines are in the file

What you actually want is to store the data from fist file and output lines in second file that are not mentioned ... something like:

Code:

awk -F: 'NR==FNR{_[$1];next}!($1 in a)' found.txt left.txt

Boucane · 10-26-2016, 01:55 PM

comm -12 left.txt found.txt

grail · 10-26-2016, 02:07 PM

Quote:

Originally Posted by Boucane

comm -12 left.txt found.txt

Would seem to be a small problem with that option:

Code:

$ comm -12 left.txt found.txt
comm: file 1 is not in sorted order

freeroute · 10-26-2016, 02:22 PM

I solved. Still I don't know what was the problem.

I opened and saved all the 2 files with nano. Voalaaaa everything works with the original command.
Still not understand what was the problem. The original left.txt and found.txt not my files (contains more than 10k lines). I guess maybe made by windows notepad, workpad....

I will try "comm" "awk" command too. Thank you very much for your advices.

grail · 10-26-2016, 02:29 PM

ahhh ... I took of the dash thinking it was an error. It does indeed work

You are correct that if the files were created in Windows it would mess with your results.