LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 09-07-2006, 11:04 PM   #1
gruffy
LQ Newbie
 
Registered: Oct 2004
Distribution: RHE4-WS
Posts: 7

Rep: Reputation: 0
Can diff display the differences/uniques for just one of the 2 files?


I'm keeping a text file of line data that is about 1.2 GB in size.. lets call it "FileA"

I generate a "FileB" daily (about 25MB in size) which contains about 80% duplicate entries that are already contained in "FileA".


I need the unique lines only from "FileB" sent to the output. Is there a way to have diff do this?

Last edited by gruffy; 09-07-2006 at 11:06 PM.
 
Old 09-07-2006, 11:47 PM   #2
gilead
Senior Member
 
Registered: Dec 2005
Location: Brisbane, Australia
Distribution: Slackware64 14.0
Posts: 4,123

Rep: Reputation: 151Reputation: 151
Running the following produces a standard diff of the files. The grep limits the output to lines added (a '> ' at the start of line) and the sed trims the leading '> '. Does this do what you want?
Code:
diff file1 file2 | grep '^>' | sed -e 's/^> //'
 
Old 09-08-2006, 12:34 PM   #3
gruffy
LQ Newbie
 
Registered: Oct 2004
Distribution: RHE4-WS
Posts: 7

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by gilead
Running the following produces a standard diff of the files. The grep limits the output to lines added (a '> ' at the start of line) and the sed trims the leading '> '. Does this do what you want?
Code:
diff file1 file2 | grep '^>' | sed -e 's/^> //'

Actually I get "diff: memory exhausted" before it does anything. The -H switch doesn't help either... I'm guessing diff tries to load the entire file into memory (1.2GB into 1GB). Think I'll have to go find some bigger dimm's to put in this machine.
 
Old 09-10-2006, 12:51 PM   #4
gruffy
LQ Newbie
 
Registered: Oct 2004
Distribution: RHE4-WS
Posts: 7

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by gilead
Running the following produces a standard diff of the files. The grep limits the output to lines added (a '> ' at the start of line) and the sed trims the leading '> '. Does this do what you want?
Code:
diff file1 file2 | grep '^>' | sed -e 's/^> //'

I actually got this working and doing what I need - thanks a lot!
 
Old 09-10-2006, 02:17 PM   #5
gilead
Senior Member
 
Registered: Dec 2005
Location: Brisbane, Australia
Distribution: Slackware64 14.0
Posts: 4,123

Rep: Reputation: 151Reputation: 151
No problem - glad to hear it's working
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
two aplication in one pc in diff display casol Linux - Software 1 02-04-2006 01:12 PM
How does "diff" only show differences? daYz Linux - Software 11 01-05-2006 10:24 PM
How can I sort out the first and last of a list of uniques?!?!! vous Programming 8 03-22-2005 09:05 AM


All times are GMT -5. The time now is 09:43 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration