LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-02-2010, 03:49 AM   #1
brainlesseinstein
LQ Newbie
 
Registered: Jul 2009
Location: Pakistan
Posts: 7

Rep: Reputation: 0
Unhappy Comparing files in bash script


Hello,

I want to compare two files and display values unique to file1. I tried using comm but it did not give me useful outputs.

comm -2 -3 file1 file2 does not work

Similar threads provide matched content. What I am looking for is unique content

Last edited by brainlesseinstein; 09-02-2010 at 03:50 AM.
 
Old 09-02-2010, 03:55 AM   #2
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,552

Rep: Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899Reputation: 2899
How about diff?
 
Old 09-02-2010, 04:02 AM   #3
brainlesseinstein
LQ Newbie
 
Registered: Jul 2009
Location: Pakistan
Posts: 7

Original Poster
Rep: Reputation: 0
Diff performs a line by line check whereas situation can be as following:

file1
1
2
3
4
5

file2
4
3
2
5


I need the output to be 1 because it is unique to file1. Whereas diff performs a line by line search and gives the following result

diff file1 file2
1,4d0
< 1
< 2
< 3
< 4
5a2,4
> 4
> 3
> 2
 
Old 09-02-2010, 04:10 AM   #4
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390
Hi,

Both comm and diff only work on sorted files.

If you sort file 2 then comm -23 file1 file2 will give you the answer you are looking for.

Hope this clears things up a bit.
 
1 members found this post helpful.
Old 09-02-2010, 04:19 AM   #5
repo
LQ 5k Club
 
Registered: May 2001
Location: Belgium
Distribution: Arch
Posts: 8,528

Rep: Reputation: 899Reputation: 899Reputation: 899Reputation: 899Reputation: 899Reputation: 899Reputation: 899
How about using the option
Code:
comm file-1 file-2 outputs three columns:
column 1 = lines unique to file-1
column 2 = lines unique to file-2
column 3 = lines common to both.

-1 suppresses column 1
-2 suppresses column 2
-3 suppresses column 3
-12 suppresses both columns 1 and 2, etc.

Code:
comm -23 file1 file2
See man com
Kind regards

Last edited by repo; 09-02-2010 at 04:23 AM.
 
Old 09-02-2010, 04:26 AM   #6
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390
@repo: The OP's command given in the initial post works _if_ both files are sorted. I don't see how your comment solves the unsorted input file(s), I do think s/he looked at the man page to come up with: comm -2 -3 file1 file2
 
Old 09-02-2010, 04:29 AM   #7
repo
LQ 5k Club
 
Registered: May 2001
Location: Belgium
Distribution: Arch
Posts: 8,528

Rep: Reputation: 899Reputation: 899Reputation: 899Reputation: 899Reputation: 899Reputation: 899Reputation: 899
Quote:
Originally Posted by druuna View Post
@repo: The OP's command given in the initial post works _if_ both files are sorted. I don't see how your comment solves the unsorted input file(s), I do think s/he looked at the man page to come up with: comm -2 -3 file1 file2
Correct
 
Old 09-02-2010, 04:45 AM   #8
brainlesseinstein
LQ Newbie
 
Registered: Jul 2009
Location: Pakistan
Posts: 7

Original Poster
Rep: Reputation: 0
Thanks druuna ... this was the missing link I was using it on unsorted files
 
Old 09-02-2010, 04:45 AM   #9
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390Reputation: 2390
You're welcome
 
Old 09-02-2010, 01:41 PM   #10
makyo
Member
 
Registered: Aug 2006
Location: Saint Paul, MN, USA
Distribution: {Free,Open}BSD, CentOS, Debian, Fedora, Solaris, SuSE
Posts: 730

Rep: Reputation: 75
Hi.

I often find it irritating to need to sort files for comm, join, etc. One can write scripts to that, of course. However, occasionally, there exists a utility that is designed to handle such situations. In this case command combine can do this. For your data on files data1 and data2:
Code:
combine data1 not data2
produces:
Code:
1
The command combine was found in package moreutils in the Debian 5 ("lenny") repository, and the latter can also be found at
http://kitenet.net/~joey/code/moreutils/

A quick glance with zypper on openSUSE 11.3 "Teal" did not find it, but yum on Fedora release 13 (Goddard) did find it. Don't confuse it with perl module List::MoreUtils.

Best wishes ... cheers, makyo

The environment was:
Code:
Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0 
GNU bash 3.2.39
combine - ( /usr/bin/combine Jun 28 2008 )

Last edited by makyo; 09-02-2010 at 01:49 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Trouble with making a bash script to read in different files and rename output files. rystke Linux - Software 1 05-07-2009 09:00 AM
Comparing two files to get matched contents in another file using shell script pooppp Linux - Networking 3 08-05-2008 01:11 AM
To rename files in a directory should I use Bash script or a Perl Script ? jamtech Programming 7 01-23-2008 12:25 AM
Shell script for comparing certain lines in two files mou5e Linux - Newbie 9 06-06-2007 02:40 PM
Comparing file sizes using a bash script. IanChristie Programming 5 12-19-2003 11:14 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 01:55 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration