LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-05-2011, 01:01 PM   #1
dinakumar12
Member
 
Registered: Mar 2010
Location: INDIA (chennai)
Distribution: centos
Posts: 271
Blog Entries: 7

Rep: Reputation: 18
merging files in linux


Hi all,

Assume that i am having two files file1 and file2.

these were the contents of file1

Name=Dinesh
Age=24
sex=male
character=good
profession=admin

these were the contents of file2

Name=vino
Age=22
sex=female

Now i want to copy only the contents from file1 which were not in file2.

i.e. after copying my file2 content should look like this.

Name=vino
Age=22
sex=female
character=good
profession=admin

i.e. only the content which is not in file2 needs to be copied, remaining content should be undisturbed.

In linux is there a way to do this. If possible kindly post your suggestions.

Thanks in advance,
Dinesh.

Last edited by dinakumar12; 03-05-2011 at 01:03 PM.
 
Old 03-05-2011, 01:55 PM   #2
Ignotum Per Ignotius
Member
 
Registered: Sep 2009
Location: Wales, UK
Distribution: Slackware
Posts: 69
Blog Entries: 3

Rep: Reputation: 42
A quick 'n dirty method:

Code:
diff -y file[12] | sed '/< $/!s/=[^=]*=/=/g'
...Provided none of the fields end in the character < followed by a space!
 
Old 03-05-2011, 02:12 PM   #3
PTrenholme
Senior Member
 
Registered: Dec 2004
Location: Olympia, WA, USA
Distribution: Fedora, (K)Ubuntu
Posts: 4,186

Rep: Reputation: 346Reputation: 346Reputation: 346Reputation: 346
Note that the solution proposed by Ignotum per Ignoius assumes that your two files were literally named "file1" and "file2." If your actual names are different from those, the diff command should be written with the two file names following the -y option.

You can look at the output of man diff for a description of the diff command. (And, of course, man sed for a description of the stream editor into which the diff output is piped.)
 
Old 03-05-2011, 02:30 PM   #4
Ignotum Per Ignotius
Member
 
Registered: Sep 2009
Location: Wales, UK
Distribution: Slackware
Posts: 69
Blog Entries: 3

Rep: Reputation: 42
Quote:
Note that the solution proposed by Ignotum per Ignotius assumes that your two files were literally named "file1" and "file2." If your actual names are different from those, the diff command should be written with the two file names following the -y option.
Thanks for pointing that out: I'm a lazy typist...
 
Old 03-05-2011, 03:14 PM   #5
dinakumar12
Member
 
Registered: Mar 2010
Location: INDIA (chennai)
Distribution: centos
Posts: 271
Blog Entries: 7

Original Poster
Rep: Reputation: 18
Hi Ignotum,

I used your code,this is the output.

root@dinesh-laptop:~# diff -y file1 file2 | sed '/< $/!s/=[^=]*=/=/g'
Name=vino
Age=22
sex=female
character=good <
profession=admin <

But the contents is not copied to file2. it remains the same.
 
Old 03-05-2011, 06:52 PM   #6
Ignotum Per Ignotius
Member
 
Registered: Sep 2009
Location: Wales, UK
Distribution: Slackware
Posts: 69
Blog Entries: 3

Rep: Reputation: 42
Quote:
Hi Ignotum

I used your code,this is the output.

Code:
Name=vino
Age=22
sex=female
character=good <
profession=admin <
...As I said, quick 'n dirty...

Filter out the extraneous angle brackets with another application of sed.

Quote:
But the contents is not copied to file2. it remains the same.
The alteration is pretty simple --- just redirect to a temporary file and then move that over the original file.

Here's a one-liner that should do what you want:

Code:
diff -y file[12] | sed '/<$/!s/=[^=]*=/=/g' | sed 's/<$//g' > .tmpfile2 && mv .tmpfile2 file2
Test it first on some backup copies of your files, though!
 
Old 03-06-2011, 10:45 AM   #7
dinakumar12
Member
 
Registered: Mar 2010
Location: INDIA (chennai)
Distribution: centos
Posts: 271
Blog Entries: 7

Original Poster
Rep: Reputation: 18
Hi,

Thank you that works fine. But i have one doubt is it necessary to redirect the output to a hidden file.

if thats the case can you please explain me why we cant redirect to a regular file.
 
Old 03-06-2011, 12:06 PM   #8
Ignotum Per Ignotius
Member
 
Registered: Sep 2009
Location: Wales, UK
Distribution: Slackware
Posts: 69
Blog Entries: 3

Rep: Reputation: 42
Quote:
Thank you that works fine.
Good show.

Quote:
But i have one doubt is it necessary to redirect the output to a hidden file.
The step of creating a temporary file is necessary --- you can't simply redirect the output of diff/sed to file2, as file2 is still being read line-by-line. However, it's not necessary to make the file hidden --- it's just my usual practice with temporary files.

Quote:
if thats the case can you please explain me why we cant redirect to a regular file.
You can redirect to a "regular" file: really, a hidden file is "regular" in every sense except that it's not (normally) listed by ls --- it doesn't have any special attributes in and of itself. The reason why (or at least one of the reasons why) I generally choose to hide temporary files containing redirected output is to reflect the fact that they exist only fleetingly as a stop-gap. You could just as easily have used an unhidden file. ...So yes, if it's bothering you, feel free to remove the period!

Last edited by Ignotum Per Ignotius; 03-06-2011 at 12:22 PM. Reason: Excising pleonasm.
 
1 members found this post helpful.
Old 03-06-2011, 07:20 PM   #9
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,425

Rep: Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826
Alternative:
Code:
awk -F"=" 'NR == FNR{_[$1]=$0;next}!($1 in _){print _[$1]}' file1 file2 > file3; mv file3 file2
 
Old 03-07-2011, 01:39 AM   #10
dinakumar12
Member
 
Registered: Mar 2010
Location: INDIA (chennai)
Distribution: centos
Posts: 271
Blog Entries: 7

Original Poster
Rep: Reputation: 18
Hi grail,

Thanks for your reply,

i executed your code without redirecting to a file.and this is the output i got.

[root@localhost ~]# awk -F"=" 'NR == FNR{_[$1]=$0;next}!($1 in _){print _[$1]}' file1 file2



[root@localhost ~]#


Actually it looks empty.


Your suggestions please.
 
Old 03-07-2011, 01:47 AM   #11
TheIndependentAquarius
Senior Member
 
Registered: Dec 2008
Posts: 4,674
Blog Entries: 29

Rep: Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917Reputation: 917
You can also try kdiff3, it is more user friendly! If you are using Kde, it will be
installed by default.
 
1 members found this post helpful.
Old 03-07-2011, 02:06 AM   #12
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,425

Rep: Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826
Sorry about that ... appears I was on drugs with my thinking
This seems to do what you want:
Code:
awk -F"=" 'NR == FNR{_[$1]=$0;next}$1 in _{delete _[$1];print}END{for(x in _)print _[x]}' file1 file2
 
Old 03-07-2011, 02:27 AM   #13
grail
LQ Guru
 
Registered: Sep 2009
Location: Perth
Distribution: Manjaro
Posts: 9,425

Rep: Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826Reputation: 2826
Another alternative using the diffs and sed earlier, I am just not sure if appending straight away might cause issues:
Code:
diff -y file[12] | sed -n '/</s/ .*$//p' >> file2
It seems to work for me but I am sure there is some kind of danger ... unless of course diff closes the file after outputting information??

I am sure some guru will let us know
 
Old 03-07-2011, 03:44 AM   #14
dinakumar12
Member
 
Registered: Mar 2010
Location: INDIA (chennai)
Distribution: centos
Posts: 271
Blog Entries: 7

Original Poster
Rep: Reputation: 18
Hi,


Thanks that works.Thanks to all especially to Ignotum.
 
Old 03-07-2011, 06:10 PM   #15
Ignotum Per Ignotius
Member
 
Registered: Sep 2009
Location: Wales, UK
Distribution: Slackware
Posts: 69
Blog Entries: 3

Rep: Reputation: 42
...Happy to have helped.

All the best.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Merging Data Files toshibalaptoplinux Linux - Newbie 6 12-15-2008 12:32 AM
Merging Two Files using C++ ckoniecny Programming 5 09-26-2006 10:00 AM
merging mpg files karhu Linux - Software 4 07-28-2005 06:25 AM
merging files using perl pantera Programming 1 06-03-2004 01:56 PM
merging log files help please digitalgravy Linux - Newbie 3 12-10-2003 03:26 PM


All times are GMT -5. The time now is 10:41 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration