LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-16-2009, 10:45 PM   #1
novice82
LQ Newbie
 
Registered: Oct 2009
Posts: 13

Rep: Reputation: 0
Compare semicolon seperated data in 2 files using shell script


hello members,

I have some data ( seperated by semicolon ) with close to 240 rows in a text file temp1.
temp2.txt stores 204 rows of data ( seperated by semicolon ).
I want to :
Sort the data in both files by field1.i.e first data field in every row.
compare the data in both files and print out the rows that are not equal in seperate files.

I was trying to do this with excel using vlookup, without a great deal of success. hence, i'm exploring the shell script option.

Code:
temp1.txt
1000xyz400100xyzA00680xyz0;19722.83;19565.7;157.13;11;2.74;11.00
1000xyz400100xyzA00682xyz0;7210.68;4111.53;3099.15;216.95;1.21;216.94
1000xyz430200xyzA00651xyz0;146.70;0.00;0.00;0.00;0.00;0.00

temp2.txt
1000xyz400100xyzA00680xyz0;19722.83;19565.7;157.13;11;2.74;11.00
1000xyz400100xyzA00682xyz0;7210.68;4111.53;3099.15;216.95;1.21;216.94
Appreciate if you can get me started. I have a solaris machine, where i intend to run the scripts.

regards,

kris
 
Old 11-16-2009, 11:16 PM   #2
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Rocky 9.2
Posts: 18,308

Rep: Reputation: 2744Reputation: 2744Reputation: 2744Reputation: 2744Reputation: 2744Reputation: 2744Reputation: 2744Reputation: 2744Reputation: 2744Reputation: 2744Reputation: 2744
You'll need 'sort' with the -t option http://linux.die.net/man/1/sort, then comm http://linux.die.net/man/1/comm
 
Old 11-17-2009, 02:10 AM   #3
novice82
LQ Newbie
 
Registered: Oct 2009
Posts: 13

Original Poster
Rep: Reputation: 0
Thanks for the tip.

But how do i re-direct the missing / un-matched rows to different files, so i can trace which file they belong to ?
 
Old 11-17-2009, 03:50 AM   #4
Guttorm
Senior Member
 
Registered: Dec 2003
Location: Trondheim, Norway
Distribution: Debian and Ubuntu
Posts: 1,437

Rep: Reputation: 440Reputation: 440Reputation: 440Reputation: 440Reputation: 440
Hi

I don't have a real solution, but maybe some ideas you can work on.

First sort the files. You should not need any options if you want it sorted on the first column.

sort temp1.txt >temp1.sorted.txt
sort temp2.txt >temp2.sorted.txt

Then use diff to compare the files.

diff -y temp1.sorted.txt temp2.sorted.txt

I don't know if the report is good enough to be understood. You can read "man diff" to see options you can use. For example, if you are not interested in lines that are equal, you can add --suppress-common-lines.

Last edited by Guttorm; 11-17-2009 at 03:51 AM.
 
Old 11-17-2009, 05:26 PM   #5
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Rocky 9.2
Posts: 18,308

Rep: Reputation: 2744Reputation: 2744Reputation: 2744Reputation: 2744Reputation: 2744Reputation: 2744Reputation: 2744Reputation: 2744Reputation: 2744Reputation: 2744Reputation: 2744
If you look at the man page link I gave for comm, it shows you what args to use to extract which set of files.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to Compare Two files using shell script pooppp Linux - Networking 14 08-05-2008 03:35 AM
Need a shell script to turn a lined file into a comma seperated file Thaidog Linux - General 8 03-08-2008 07:19 PM
shell script: compare 2 files anhtt Programming 6 08-29-2007 02:39 AM
shell script to recursively "compare" all files in a directory... silex_88 Programming 3 05-12-2007 04:24 AM
shell script to compare filese b/w local and remote and delete files from local serve dsids Linux - Networking 9 08-23-2006 07:20 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 11:41 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration