File reading line by line and compare

Goni · 09-19-2005, 08:01 AM

Hi, I am quite confused here trying to implement a scenerio in bash where I have to read a file line by line and to search that line in from another file. Basically I want to compare two files, two files as seperate inputs read each line from src_file and try to search that line in from dst_file. Case sensativity is not a problem. We can use grep -i <..> anyway, if a match is found it displays line blah = found else not found. Each line from src_file should be parsed till EOF on dst_file.

If anyone can help me out in bash, I would really appreciate.

Goni

homey · 09-19-2005, 11:00 AM

Are you talking about something like diff?
diff src_file dst_file

Goni · 09-19-2005, 11:05 AM

To some extent yes, diff works but I can't parse its output. Infact I am doing this to automate some router audits for a set of commands in my SOP which may or maynot be present in the router. The I got the router current config file in here and the SOP in the other file, just want to see the commands in SOP are present in router config or not. That all.

Goni

Goni · 09-20-2005, 02:10 AM

no replies ... anyone ??

scuzzman · 09-20-2005, 02:15 AM

Is there any reason you can't just do this:

Code:

for line in `cat src_file`; do grep -i $line dst_file; done

And BTW: Please wait a full 24 hours before bumping your thread in accordance with the LQ Rules.

Goni · 09-20-2005, 02:48 AM

I can do that but you see clearly it shows, for each line cat file, show the match. Means, if even a single match is from src_file, it will loop till the EOF of src_file. And the list goes on for multiple matches.

Sorry about the bumping, infact I was expecting replies a bit quicker but anyway, I'll try to follow the rules ..

.. thankx

Goni

scuzzman · 09-20-2005, 02:52 AM

Do you not want it to go until EOF of src_file? By this I mean, do you want it to stop the first time it finds a match?

Goni · 09-20-2005, 03:03 AM

nop, I want it for ALL possible matches in src_file (compared in from dst_file) till EOF.

scuzzman · 09-20-2005, 03:08 AM

That script I gave you should do that. An example may help. Let's assume you have 2 files: src_file and dst_file.
Contents of src_file:

Code:

orange
apple
kiwi
grapefruit

That script will get the first item in src_file ('orange' in this case) and compare it to every line in dst_file and display any matches. It will then move on to the second, third, etc doing the same.
What that is saying, essentially, is this:
Take each item from 'cat src_file' and store it in $line. Then, compare this to the entire contents of dst_file, line for line. Then, move onto the next line in src_file.

Goni · 09-20-2005, 03:13 AM

Logically you are correct but the script ain't doing so. I got 1 matche in both files, total 5 lines in src_file and the script is showing me the match 5 times. As you said, is should move on but its not doing so. BTW: I am prety sure the messy output does not have to do anything weather if I run it on command or script?

Snowbat · 09-20-2005, 03:41 AM

src_file contains duplicate lines? Remove them and try again:

Code:

cat src_file | sort | uniq > src_file_2
for line in `cat src_file_2`; do grep -i $line dst_file; done

scuzzman · 09-20-2005, 05:28 AM

Quote:

Originally posted by Snowbat
src_file contains duplicate lines? Remove them and try again:

Code:

cat src_file | sort | uniq > src_file_2 for line in `cat src_file_2`; do grep -i $line dst_file; done

You can actually eliminate a full step by doing this:

Code:

for line in `cat src_file | sort | uniq`; do grep -i $line dst_file; done

Goni · 09-20-2005, 05:34 AM

No, src_file does not contains any duplicate, not atleast at the moment. There are just 5 lines,

Line1
This is lane2
Mismatch3
Someone here4
and this is line 5

Just like this, and dst_file contains some other different commands/lines but one or more than one from src_file for example "This is lane2". When I run the script, I get the output correct but it shows me 5 lines of the same output. Says "This is lane2"

What will make difference sorting out the file?

Snowbat · 09-20-2005, 09:29 PM

So dst_file contains (or can contain) duplicates? You can get grep to stop after the first matching line:

Code:

for line in `cat src_file`; do grep -i -m 1 $line dst_file; done

Quote:

Originally posted by scuzzman
You can actually eliminate a full step by doing this:

Code:

for line in `cat src_file | sort | uniq`; do grep -i $line dst_file; done

Good point, though it will sort and uniq the complete file for every line... potentially slower for a large file?

scuzzman · 09-21-2005, 12:24 AM

Quote:

Good point, though it will sort and uniq the complete file for every line...

It shouldn't. It's taking every line of 'cat src_file | sort | uniq' then running the loop on that, not the original file. Once it cat's sort's and uniq's the file, it forgets about it.