LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 12-19-2013, 01:32 PM   #1
bennypr0fane
Member
 
Registered: Jan 2012
Location: Central Europe
Distribution: Manjaro, Kubuntu
Posts: 150

Rep: Reputation: Disabled
Rsync vs. Unison


I wonder why Rsync seems to be THE go-to standard for virtually all syncing everywhere in the *nix-world, whenever there is some kind of file syncing involved, it's always Rsync, even though it syncs one-way only.
Maybe it can be scheduled to sync first one way, then the other, in 2 separate procedures, or however that works. Anyway, I believe it is used a lot for syncing in both directions and I don't understand why a two-way-sync protocol like e.g. Unison is not more commonly used in the first place.
Or maybe my perception is just plain wrong, and Unison et al. is just as common.

I am trying to figure this out because I have a cloud storage provider that offers mounting the cloud storage as a network drive in any system via smb share, nfs, ftp or webdav.
That means though there are no local copies of the files stored automatically like e.g. in Dropbox, and any access to cloud files is actually an internet download.
That's why I want to have a local directory synced with the cloud drive - two-way, like with Dropbox.
The provider allows for syncing access, you guessed it, via Rsync.
If figure it'd be possible to just use Unison instead, but because of my initially described impression that Rsync is sorta what you're supposed to use, I'd like to know if anyone thinks Rsync would be somehow advantageous.
If it turns out so, then I need to figure how to achieve 2-way-syncing with a 1-way-tool, so how does that work?
 
Old 12-19-2013, 01:42 PM   #2
pan64
LQ Guru
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 16,258

Rep: Reputation: 5457Reputation: 5457Reputation: 5457Reputation: 5457Reputation: 5457Reputation: 5457Reputation: 5457Reputation: 5457Reputation: 5457Reputation: 5457Reputation: 5457
I do not really understand how that two way syncing should work. I think you need to specify one source and all the others can be synced (by rsync), but what to do if a file was modified here and there? What to do if a file was removed here and modified there? ..... How to handle renaming, moving files/dirs ....
 
Old 12-19-2013, 02:37 PM   #3
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
I can rsync in two directions.
 
Old 12-19-2013, 03:46 PM   #4
bennypr0fane
Member
 
Registered: Jan 2012
Location: Central Europe
Distribution: Manjaro, Kubuntu
Posts: 150

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Habitual View Post
I can rsync in two directions.
Well would you care for a short (or longer) explanation how you do that?
 
Old 12-19-2013, 03:59 PM   #5
bennypr0fane
Member
 
Registered: Jan 2012
Location: Central Europe
Distribution: Manjaro, Kubuntu
Posts: 150

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
I do not really understand how that two way syncing should work. I think you need to specify one source and all the others can be synced (by rsync), but what to do if a file was modified here and there? What to do if a file was removed here and modified there? ..... How to handle renaming, moving files/dirs ....
I guess with 2-way-syncing it's probably more common to run into conflicts that need to be resolved by the user, but generally I figure you just need well-made guidelines for which changes get priority over others.
If all ends are connected to the cloud whenever you change something, it should be easy to go chronologically.
If you make a change that is not synced immediately, and then make a different change to another location/instance before syncing the first change, then syncing that first one will lead to two conflicting statuses. I've seen that very rarely though with Dropbox, everything goes the way I want it to go most of the time, so they must have a pretty smart solution for this.

However I am looking for confirmation or rejection of my hypothesis that Rsync
1. Can sync both ways and
2. is preferable over a true two-way solution for technical reasons like e.g. greater speed, reliability, flexibility etc.

Last edited by bennypr0fane; 12-19-2013 at 04:00 PM.
 
Old 12-19-2013, 04:25 PM   #6
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Quote:
Originally Posted by bennypr0fane View Post
Well would you care for a short (or longer) explanation how you do that?
Code:
man rsync
or
http://www.thegeekstuff.com/2010/09/...mand-examples/
examples #4, and $5
 
Old 12-19-2013, 05:05 PM   #7
zhjim
Senior Member
 
Registered: Oct 2004
Distribution: Debian Squeeze x86_64
Posts: 1,748
Blog Entries: 11

Rep: Reputation: 233Reputation: 233Reputation: 233
Now thats a habit, its a classic.

OT: Depends on your use case. If oyu just want some kind of backup rsync is your friend. If you want to modify files regardless of the location. Say online and offline you would use unison. Never played with unison but with rsync you just mirror stuff. If you dont edit the same file you can 2 way sync with rsync but as soon as the same file gets changed on both location rsync screws.
Maybe think of rsync as a featured cp command. And unison more as of a revision control. That just make clear where to use which one.
 
2 members found this post helpful.
Old 12-19-2013, 06:48 PM   #8
bennypr0fane
Member
 
Registered: Jan 2012
Location: Central Europe
Distribution: Manjaro, Kubuntu
Posts: 150

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Habitual View Post
Code:
man rsync
or
http://www.thegeekstuff.com/2010/09/...mand-examples/
examples #4, and $5
If I understand these samples correctly, neither will resolve a situation where changes have been made to both ends. In such a case, if for instance there were changes made to the same file, only one of the two will make through, no?
 
Old 12-19-2013, 07:45 PM   #9
astrogeek
Moderator
 
Registered: Oct 2008
Distribution: Slackware [64]-X.{0|1|2|37|-current} ::12<=X<=14, FreeBSD_12{.0|.1}
Posts: 5,584
Blog Entries: 11

Rep: Reputation: 3606Reputation: 3606Reputation: 3606Reputation: 3606Reputation: 3606Reputation: 3606Reputation: 3606Reputation: 3606Reputation: 3606Reputation: 3606Reputation: 3606
Quote:
Originally Posted by bennypr0fane View Post
If I understand these samples correctly, neither will resolve a situation where changes have been made to both ends. In such a case, if for instance there were changes made to the same file, only one of the two will make through, no?
That is correct.

So what you mean by two-way sync is not a sync at all, but a merge.

Can unison do that - automatically and safely?

*** UPDATE ***

Being unfamiliar with Unison, and having a short break, I looked into it, here.

From that page:

Code:
Unison shares a number of features with tools such as configuration management packages (CVS,
PRCS, Subversion, BitKeeper, etc.), distributed filesystems (Coda, etc.), uni-directional
mirroring utilities (rsync, etc.), and other synchronizers (Intellisync, Reconcile, etc)...

Unlike simple mirroring or backup utilities, Unison can deal with updates to both replicas of
a distributed directory structure. Updates that do not conflict are propagated automatically.
Conflicting updates are detected and displayed.
So, although they call it "syncing", it it more akin to a revision control system and does in fact allow you to merge changes from both ends, but still requires human intervention to resolve conflicting updates, as would be expected.

For that use I think Git is probably a more useful comparison than rsync. And asking why Git is used more widely instead of Unison would be a more appropriate question. Rsync does not perform such merges.

Last edited by astrogeek; 12-19-2013 at 10:01 PM.
 
1 members found this post helpful.
Old 12-19-2013, 09:07 PM   #10
bennypr0fane
Member
 
Registered: Jan 2012
Location: Central Europe
Distribution: Manjaro, Kubuntu
Posts: 150

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by zhjim View Post
Now thats a habit, its a classic.
What is?
 
Old 12-20-2013, 01:35 AM   #11
zhjim
Senior Member
 
Registered: Oct 2004
Distribution: Debian Squeeze x86_64
Posts: 1,748
Blog Entries: 11

Rep: Reputation: 233Reputation: 233Reputation: 233
Quote:
Originally Posted by bennypr0fane View Post
What is?
Just a line from a song I had in mind.
 
Old 12-20-2013, 09:46 AM   #12
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
you'll need 2 unison profiles, as an example:

vi .unison/sendup.prf
Code:
# Unison preferences file
root = /home/<user>
root = /media/Keepers # or other qualified mount/point
perms=0
and run it with:
Code:
unison default -batch -ui text -auto -force /home/<user>
vi .unison/bringdown.prf
Code:
# Unison preferences file
root = /media/Keepers # or other qualified mount/point
root = /home/<user>
perms=0
and run it with:
Code:
I'll leave this as homework/study/self-improvement
Reliable backups and I can't emphasize this enough, aren't "enough". They need to be tested.
I hope you have some.

Have fun!
 
Old 12-22-2013, 08:15 PM   #13
bennypr0fane
Member
 
Registered: Jan 2012
Location: Central Europe
Distribution: Manjaro, Kubuntu
Posts: 150

Original Poster
Rep: Reputation: Disabled
It appears that Unison needs another instance of Unison to be installed on a remote server in order to sync with it, which sucks very bad. That server is only storage, I can't make it run any kind of program.
Remote sync can be done via SSH, RSH or TCP.
I wonder though, if the server is mounted under my local file system, e.g. with SMB, would Unison still consider that a remote system?
 
Old 12-23-2013, 01:56 AM   #14
zhjim
Senior Member
 
Registered: Oct 2004
Distribution: Debian Squeeze x86_64
Posts: 1,748
Blog Entries: 11

Rep: Reputation: 233Reputation: 233Reputation: 233
I'd say that it takes it as a local filesystem so should work.

Last edited by zhjim; 12-23-2013 at 01:58 AM.
 
Old 12-23-2013, 08:29 AM   #15
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
Quote:
Originally Posted by bennypr0fane View Post
I wonder though, if the server is mounted under my local file system, e.g. with SMB, would Unison still consider that a remote system?
I believe it would not.
It should behave as any locally mounted FS.
 
  


Reply

Tags
cloud, network, rsync, syncing


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
creating File Synchronization software like unison or rsync alienmayank Linux - Networking 1 11-10-2011 10:12 PM
Help with Unison Simply_S Linux - Software 0 11-02-2008 06:42 AM
Unison Setup FragInHell Linux - Software 0 02-21-2007 10:34 PM
rsync, unison from Win to Linux FTP/Web Akonbobot Linux - Server 0 10-07-2006 01:17 AM
question about unison nephish Linux - Software 1 11-18-2005 07:54 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 09:11 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration