Help answer threads with 0 replies.
Go Back > Forums > Linux Forums > Linux - Software
User Name
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.


  Search this Thread
Old 12-29-2018, 09:50 PM   #1
Registered: Aug 2010
Posts: 44

Rep: Reputation: 0
Question Diff two directories to a tarball?

I would like to create a tarball containing only the differences between two nearly identical file systems. Basically, I'm adding some new programs, configuration files, web pages, etc. to a development file system and I want to compare it to a previous setup then put all the new and changed files into a tarball so I can extract it on other machines to bring them up to date.

Is there an app that does this or is there a magic incantation of diff and tar that would accomplish this?
Old 12-29-2018, 09:58 PM   #2
Senior Member
Registered: Nov 2013
Location: Tokyo
Distribution: A few
Posts: 3,986

Rep: Reputation: 1113Reputation: 1113Reputation: 1113Reputation: 1113Reputation: 1113Reputation: 1113Reputation: 1113Reputation: 1113Reputation: 1113
You should look into rsync. This stackexchange discussion seems to be relevant to your case, especially the answer starting with Just for those less familiar with rsync.
1 members found this post helpful.
Old 12-29-2018, 10:04 PM   #3
Registered: May 2016
Posts: 222

Rep: Reputation: Disabled
I thought of rsync too. I would use --dry-run, at stackoverflow one comment uses -n which does the same, because it isn't that obvious what rsync -u considers to be "newer".
This webpage also mentions --ignore-existing (but i for one don't know it).
Old 12-30-2018, 10:44 AM   #4
Registered: Aug 2010
Posts: 44

Original Poster
Rep: Reputation: 0
Okay, but then how do you feed that to tar? Can you do this with a pipe?
rsync outputs some informational messages that are irrelevant to the tar half of the question.

"sending incremental file list"

...list of files...

"sent 9,128 bytes received 762 bytes 6,593.33 bytes/sec
total size is 25,150,426 speedup is 2,543.02 (DRY RUN)"

Bear in mind that my goal is to generate a tarball that I can upload to different machines to install only the new or modified files. There will be no connection between the development machine and the production machine(s) because they might be set up elsewhere in the world.
Old 12-30-2018, 07:39 PM   #5
Registered: Jul 2018
Location: Paris
Distribution: Debian
Posts: 887

Rep: Reputation: 288Reputation: 288Reputation: 288
Maybe could be inspiring to you?
Old 01-03-2019, 01:50 PM   #6
Registered: Jul 2017
Distribution: korora
Posts: 72

Rep: Reputation: Disabled
This sounds like a very interesting solution to an age-old problem. You want to deploy systems that have a particular set of applications, configs, etc. You are expecting an exact input each time, which helps.

Have you considered using something like ansible to help configure the targets the way you expect?

If you really want a tarball of the differences, here's a small sample to get you started.
Original directory: test
Modified directory: final
Output tarball: mydifferences.tgz
rsync -avzn final/ test/ | grep -viE '^$|sending.*file list|^(sent |total size is )[0-9,]+' | xargs tar -zc -C final -f mydifferences.tgz
rsync -avzn: the -n is for dry-run. Basically, only list the files that are different. The order of directories is important here, as well as the trailing slash on the first directory listed.
grep: exclude these strings. I couldn't come up with a flag for rsync for this in my 3 minutes of research.
xargs: take the lines from the pipe, and use them as parameters at the end of the rest of this command.
-z use gzip compression
-c create new file, named with value from -f
-C final: change directory to this first, so that all relative directories in the tarball are like you expect.

My testing showed some duplicated filenames in the resulting tarball, but it expanded to what I expected. So please test before you use in prod!

Last edited by bgstack15; 01-03-2019 at 01:55 PM. Reason: more detail
Old 01-04-2019, 02:30 AM   #7
Registered: Jul 2018
Location: Paris
Distribution: Debian
Posts: 887

Rep: Reputation: 288Reputation: 288Reputation: 288
Originally Posted by bgstack15 View Post
rsync -avzn: the -n is for dry-run. Basically, only list the files that are different.
Are you really sure of that? From my side rsync in dry-run lists all source contents.
$ ls doss*
fic1	fic2	fic8
fic1	fic2	fic5

$ rsync -avzn doss1/ doss2
building file list ... done
sent 139 bytes  received 44 bytes  366.00 bytes/sec
total size is 15  speedup is 0.08

$ rsync --version
rsync  version 2.6.9  protocol version 29
Copyright (C) 1996-2006 by Andrew Tridgell, Wayne Davison, and others.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
How can I build a kernel module on one system for a diff. kernel on a diff. system? slacker_ Linux - General 18 09-15-2014 06:55 PM
[SOLVED] How to compare only the file names between two directories? (diff?) SparceMatrix Linux - General 11 09-24-2013 02:39 PM
setting git-diff --color-words as git-diff potuz Linux - Software 5 09-09-2010 02:31 PM
How to compare timestamp of same file residing in two diff directories prasenjit.patra AIX 1 10-21-2009 05:45 PM
Dual Boot diff Hard Disk diff OS on Suse 9.1 wilhem Linux - Newbie 1 08-13-2004 07:06 PM > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 03:29 AM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration