LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 08-28-2007, 11:09 PM   #1
anhtt
LQ Newbie
 
Registered: Jul 2007
Posts: 27

Rep: Reputation: 15
shell script: compare 2 files


I have 2 files. Now I want write 1 shell script which check one bye one row in file1. If the row of file1 is found in the file2, we won't do anything. Otherwise, we will save the row of file1 in a new file ( file3).
I also wrote a shell script. But it is so slow because there are a lot of rows in file1 and file2.

cat file1 | while read line;
do
echo $line > /tmp/check
x=`cut -f1 -d" " /tmp/check`
y=`grep -c "$x" file2
if [ "$y" -eq 1 ];then
continue
else
echo $x >> file3
fi
done


Help me !
 
Old 08-28-2007, 11:12 PM   #2
gilead
Senior Member
 
Registered: Dec 2005
Location: Brisbane, Australia
Distribution: Slackware64 14.0
Posts: 4,123

Rep: Reputation: 151Reputation: 151
It's probably easier to use one of the sdiff, diff or cmp commands. Have you tried them before?
 
Old 08-28-2007, 11:41 PM   #3
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,269

Rep: Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028
Code:
for line in `cat t1.t`
do
        rslt=`grep $line t2.t`
        if [[ $? -ne 0 ]]
        then
                echo $line >>t3.t
        fi
done
 
Old 08-29-2007, 01:49 AM   #4
ghostdog74
Senior Member
 
Registered: Aug 2006
Posts: 2,695
Blog Entries: 5

Rep: Reputation: 241Reputation: 241Reputation: 241
Code:
awk 'FNR==NR{ arr[$0];next}
     {if ($0 in arr) print "found"
      else print "not found"   
     }' "file1" "file2"
 
Old 08-29-2007, 02:04 AM   #5
farkus888
Member
 
Registered: Oct 2006
Distribution: usually use arch
Posts: 103

Rep: Reputation: 15
Quote:
Originally Posted by gilead View Post
It's probably easier to use one of the sdiff, diff or cmp commands. Have you tried them before?
I agree with this, sdiff will mark all changes with a "<", ">", or a "|". use the equivalent of a grep -v for those three things and then cut out one of the columns and you have exactly what you want. unfortunately right now I am too tired for code thats any better than stupid slow and inefficient at the moment so you'll need to figure out the details on your own.
 
Old 08-29-2007, 02:33 AM   #6
theYinYeti
Senior Member
 
Registered: Jul 2004
Location: France
Distribution: Arch Linux
Posts: 1,897

Rep: Reputation: 61
If the order does not matter, and all rows in each file are unique (no two identical rows in the same file), then the fastest is probably:
Code:
cat file1 file2 | sort | uniq -d | cat file1 - | sort | uniq -u
which would only display lines from file1 which are not in common between file1 and file2. If you want the reverse (lines in common), then it is simpler:
Code:
cat file1 file2 | sort | uniq -d
If the above conditions are false (identical rows or important order), then to some extent (some thousands of lines), this should be a good solution:
Code:
grep -vxf <(sed 's/[]\.*^$[]/\\\0/g' file2) file1
which gives lines in file1 that are not in file2.

To output the result to a "file3" file, just append
Code:
 >file3
to any of the above commands.

Yves.

[edit:]I had forgotten the 'f' option in grep. Now it is ok.[/edit]

Last edited by theYinYeti; 08-29-2007 at 02:47 AM.
 
Old 08-29-2007, 02:39 AM   #7
gnashley
Amigo developer
 
Registered: Dec 2003
Location: Germany
Distribution: Slackware
Posts: 4,755

Rep: Reputation: 466Reputation: 466Reputation: 466Reputation: 466Reputation: 466
comm is the program you want -does exactly what you want.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
How can I compare floating number in a shell script? abefroman Programming 29 09-10-2012 11:29 AM
shell script to recursively "compare" all files in a directory... silex_88 Programming 3 05-12-2007 04:24 AM
shell script to compare filese b/w local and remote and delete files from local serve dsids Linux - Networking 9 08-23-2006 07:20 AM
compare files in C shell shashwat.gupta Programming 8 05-24-2006 02:13 AM
Shell script to compare blocks of strings? bruno buys Programming 10 04-15-2006 02:16 PM


All times are GMT -5. The time now is 05:56 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration