Visit Jeremy's Blog.
Go Back > Forums > Non-*NIX Forums > Programming
User Name
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.


  Search this Thread
Old 08-28-2007, 11:09 PM   #1
LQ Newbie
Registered: Jul 2007
Posts: 27

Rep: Reputation: 15
shell script: compare 2 files

I have 2 files. Now I want write 1 shell script which check one bye one row in file1. If the row of file1 is found in the file2, we won't do anything. Otherwise, we will save the row of file1 in a new file ( file3).
I also wrote a shell script. But it is so slow because there are a lot of rows in file1 and file2.

cat file1 | while read line;
echo $line > /tmp/check
x=`cut -f1 -d" " /tmp/check`
y=`grep -c "$x" file2
if [ "$y" -eq 1 ];then
echo $x >> file3

Help me !
Old 08-28-2007, 11:12 PM   #2
Senior Member
Registered: Dec 2005
Location: Brisbane, Australia
Distribution: Slackware64 14.0
Posts: 4,124

Rep: Reputation: 164Reputation: 164
It's probably easier to use one of the sdiff, diff or cmp commands. Have you tried them before?
Old 08-28-2007, 11:41 PM   #3
LQ Guru
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.7, Centos 5.10
Posts: 16,673

Rep: Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158Reputation: 2158
for line in `cat t1.t`
        rslt=`grep $line t2.t`
        if [[ $? -ne 0 ]]
                echo $line >>t3.t
Old 08-29-2007, 01:49 AM   #4
Senior Member
Registered: Aug 2006
Posts: 2,697
Blog Entries: 5

Rep: Reputation: 242Reputation: 242Reputation: 242
awk 'FNR==NR{ arr[$0];next}
     {if ($0 in arr) print "found"
      else print "not found"   
     }' "file1" "file2"
Old 08-29-2007, 02:04 AM   #5
Registered: Oct 2006
Distribution: usually use arch
Posts: 103

Rep: Reputation: 15
Originally Posted by gilead View Post
It's probably easier to use one of the sdiff, diff or cmp commands. Have you tried them before?
I agree with this, sdiff will mark all changes with a "<", ">", or a "|". use the equivalent of a grep -v for those three things and then cut out one of the columns and you have exactly what you want. unfortunately right now I am too tired for code thats any better than stupid slow and inefficient at the moment so you'll need to figure out the details on your own.
Old 08-29-2007, 02:33 AM   #6
Senior Member
Registered: Jul 2004
Location: France
Distribution: Arch Linux
Posts: 1,897

Rep: Reputation: 61
If the order does not matter, and all rows in each file are unique (no two identical rows in the same file), then the fastest is probably:
cat file1 file2 | sort | uniq -d | cat file1 - | sort | uniq -u
which would only display lines from file1 which are not in common between file1 and file2. If you want the reverse (lines in common), then it is simpler:
cat file1 file2 | sort | uniq -d
If the above conditions are false (identical rows or important order), then to some extent (some thousands of lines), this should be a good solution:
grep -vxf <(sed 's/[]\.*^$[]/\\\0/g' file2) file1
which gives lines in file1 that are not in file2.

To output the result to a "file3" file, just append
to any of the above commands.


[edit:]I had forgotten the 'f' option in grep. Now it is ok.[/edit]

Last edited by theYinYeti; 08-29-2007 at 02:47 AM.
Old 08-29-2007, 02:39 AM   #7
Amigo developer
Registered: Dec 2003
Location: Germany
Distribution: Slackware
Posts: 4,805

Rep: Reputation: 489Reputation: 489Reputation: 489Reputation: 489Reputation: 489
comm is the program you want -does exactly what you want.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
How can I compare floating number in a shell script? abefroman Programming 29 09-10-2012 11:29 AM
shell script to recursively "compare" all files in a directory... silex_88 Programming 3 05-12-2007 04:24 AM
shell script to compare filese b/w local and remote and delete files from local serve dsids Linux - Networking 9 08-23-2006 07:20 AM
compare files in C shell shashwat.gupta Programming 8 05-24-2006 02:13 AM
Shell script to compare blocks of strings? bruno buys Programming 10 04-15-2006 02:16 PM

All times are GMT -5. The time now is 02:16 AM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration