LinuxQuestions.org - [SOLVED] Auto-synchronizing a user's document directory.

- Linux - General (http://www.linuxquestions.org/questions/linux-general-1/)

- - Auto-synchronizing a user's document directory. (http://www.linuxquestions.org/questions/linux-general-1/auto-synchronizing-a-users-document-directory-4175433400/)

Auto-synchronizing a user's document directory.

Hello,

I have a user who has used one Fedora 17 workstation. Now she will use one of two machines. I currently do not have a server where both machines could have a share to. They machines might not be up at the same time, so I don't want to symbolic link the directories together.

I would like to automatically synchronize the two directories by a cron job. If a file is created on one machine, I would like to see it show up on the other machine. If a file is updated on one machine, I would like to see the new version propagated to the other machine and vice versa. If a file is deleted on one machine, I don't necessarily want it deleted (right away) on the other machine in case it was accidentally deleted.

Do you think I can do this with RSYNC from one machine? Anyone with experience doing this?

- Raj Upadhyaya

This is trivially easy to do, but you should take the time to read the rsync documentation so that you understand what's going on. You can always resort to google for many examples of how others do this.

Looks like I might have it.. I mounted the Workstation 1 file system home share under the /home/raj/c directory. Here is what I have. First, I will sync from workstation 2 to workstation 1.

Code:

rsync -a -s -t -u --numeric-ids --no-owner --no-group --progress /home/virginia/Documents /home/raj/c/virginia

Then immediately after that, I will sync from workstation 1 to workstation 2 if there are any new files on the workstation 1.

Code:

rsync -a -s -t -u --numeric-ids --no-owner --no-group --progress /home/raj/c/virginia/Documents /home/virginia

Does that make sense? I think if I schedule it from cron I will remove the --progress tab and replace it with -q (which is quiet mode).

- Raj

Add the Documents directory onto the end of the command:

Code:

rsync -a -s -t -u --numeric-ids --no-owner --no-group --progress /home/virginia/Documents/ /home/raj/c/virginia/Documents/

And vice versa. The trailing slash here IS important.

Also, make sure your user knows that if they ever want to delete OR RENAME a file, they must do so on BOTH workstations in between cron jobs.

You'll also want a check to make sure the remote mount is up before running these commands.

From my root crontab.

Code:

5 0 * * * /bin/rsync -a -s -t -u --numeric-ids --no-owner --no-group  /home/virginia/Documents/ /home/raj/c/virginia/Documents/

#### Rest for a few minutes

10 0 * * * /bin/rsync -a -s -t -u --numeric-ids --no-owner --no-group  /home/raj/c/virginia/Documents/ /home/virginia/Documents/

Does that look good? It should kick off at 5 minutes after midnight tonight, and then 10 minutes after midnight. I always make mistakes with the crontab schedule.

- Raj

I would put both rsync commands in a single script and run that from the cron, that way you don't run into any race condition where the second rsync starts before the first has finished. Depending on usage, you may want to run it more often than that, especially if the user will be modifying documents in both places during the day.

I got up early this morning, and took your advice. Instead of just wrapping the two rsync lines in a simple script, I made it more robust.. You can now execute it from the command line and increase its verbosity. Uggh. I have to get more sleep or a better hobby. Here it is. If you have a suggestion for improvement, I would appreciate it.
- Raj

Code:

#!/bin/bash

# By Raj Upadhyaya

# BSD License

#

# Redistribution and use in source and binary forms, with or

#   without modification, are permitted provided that the following

#   conditions are met:

#

# *  Redistributions of source code must retain the above

#    copyright notice, this list of conditions and the following

#    disclaimer.

#

# *  Redistributions in binary form must reproduce the above

#    copyright notice, this list of conditions and the following

#    disclaimer in the documentation and/or other materials

#    provided with the distribution.

#

# THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY

# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,

# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A

# PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR

# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,

# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED

# TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,

# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND

# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT

# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING

# IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF

# THE POSSIBILITY OF SUCH DAMAGE.



##  globals

verbosity=""

# Display help

function usage(){

    cat << EOF



    syncDirectory [options] username directory

        rsync a directory from one server to the other.



        -v     : verbose (number) 1 or 2

        -h     : display help





EOF

}

#  ___  _ __ | |_(_) ___  _ __  ___

# / _ \| '_ \| __| |/ _ \| '_ \/ __|

#| (_) | |_) | |_| | (_) | | | \__ \

# \___/| .__/ \__|_|\___/|_| |_|___/

#      |_|



# Parsing options

while getopts "v:h" option

do

    case $option in

        v)

        if [[ $2 -ne 2 ]]

        then

          verbosity="--progress"

        else

          verbosity="--progress -vv"

        fi

        ;;

        h)

        usage

        exit 0

        ;;

        *)

        ;;

    esac

done





# Mainline...



shift $((OPTIND-1));



# Checks number of options correct

if [[ $# -ne 2 ]]

then

    usage

    exit 1

fi



myUser=$1

myDirectory=$2

server1Location=/home/$myUser/$myDirectory/

server2Location=/home/raj/c/$myUser/$myDirectory/

rsyncOptions="-a -l -t -u --numeric-ids -g -o -p"



if id -u $myUser > /dev/null 2>&1;

then

        echo "user exists" > /dev/null

else

        echo "Who is $myUser?  I can't seem to find that user on this system."

        exit 1

fi



if [[ ! -d "$server1Location" ]]

then

    echo "Couldn't find the directory: $server1Location.  Are you sure it exists

?"

    usage

    exit 1

fi



## Here we go!!!

/bin/rsync $rsyncOptions $verbosity $server1Location $server2Location

/bin/rsync $rsyncOptions $verbosity $server2Location $server1Location

You might want to read this discussion of rsync vs unison.
(its quite short; see the latter posts)

Chris,

I found this discussion on http://www.linuxquestions.org. http://www.linuxquestions.org/questi...8/#post4509973

I installed Unison to test it out, and it is very nice. The only difference that I saw with the RSYNC option is I could create a cron job to execute the above script a few times a day. The Unison product is a GUI implementation which nice and easy to set up. The difference that I saw that was cool was if a file was updated on both sides. If you use Unison, it prompts you to resolve the conflict and it won't automatically copy either side over the other. The RSYNC option will find the newer file (even if it is by a second) and choose that version to sync with both sides. I think given the use case of one user using one of two workstations at a time, this shouldn't happen.

Well, I learned alot about Rsync, and created a BASH script that is probably 104 lines too long, but what the hey. :cool:

- Raj

Well done; sorry I seem to have dropped the link there ... ;)
Anyway, it sounded like something that might be applicable and certainly worth knowing for future ref

Looks like you can run Unison from the command line as well without the GUI.

Quote:

unison -ui text -auto -batch batchpref

With this, you could potentially run unison from a cron job.

I will look into this and document my findings for others reference.

- Raj

Looks like it will work as a cron job. Here is my output of the first run.

Code:

 Videos$ unison -ui text -auto -batch batchpref

Contacting server...

Looking for changes

Warning: No archive files were found for these roots, whose canonical names are:

        /home/raj/Documents

        /home/raj/c/raj/Documents

This can happen either

because this is the first time you have synchronized these roots, 

or because you have upgraded Unison to a new version with a different

archive format.  



Update detection may take a while on this run if the replicas are 

large.



Unison will assume that the 'last synchronized state' of both replicas

was completely empty.  This means that any files that are different

will be reported as conflicts, and any files that exist only on one

replica will be judged as new and propagated to the other replica.

If the two replicas are identical, then no changes will be reported.



If you see this message repeatedly, it may be because one of your machines

is getting its address from DHCP, which is causing its host name to change

between synchronizations.  See the documentation for the UNISONLOCALHOSTNAME

environment variable for advice on how to correct this.



Donations to the Unison project are gratefully accepted: 

http://www.cis.upenn.edu/~bcpierce/unison



Reconciling changes                                               

file     <-?-> file       output 2.pdf  

home/raj/... : file               modified on 2012-10-23 at  8:04:25  size 17850     rw-rw-r--

c/raj/Doc... : file               modified on 2012-10-22 at  4:39:55  size 19198     rw-rw-r--

file     <-?-> file       output 3.pdf  

home/raj/... : file               modified on 2012-10-23 at  8:06:59  size 93902     rw-rw-r--

c/raj/Doc... : file               modified on 2012-10-22 at  4:36:11  size 93523     rw-rw-r--

file     <-?-> file       output 4.pdf  

home/raj/... : file               modified on 2012-10-23 at  8:08:07  size 34097     rw-rw-r--

c/raj/Doc... : file               modified on 2012-10-22 at  4:32:32  size 33754     rw-rw-r--

file     <-?-> file       output.pdf  

home/raj/... : file               modified on 2012-10-23 at  8:01:25  size 6055      rw-rw-r--

c/raj/Doc... : file               modified on 2012-10-22 at  4:38:56  size 6032      rw-rw-r--

file     ---->            HR 3962 - Medical Devices.doc  

Propagating updates

UNISON 2.40.63 started propagating changes at 11:30:58.42 on 23 Oct 2012

[CONFLICT] Skipping output 2.pdf

[CONFLICT] Skipping output 3.pdf

[CONFLICT] Skipping output 4.pdf

[CONFLICT] Skipping output.pdf

[BGN] Copying HR 3962 - Medical Devices.doc from /home/raj/Documents to /home/raj/c/raj/Documents

[END] Copying HR 3962 - Medical Devices.doc

UNISON 2.40.63 finished propagating changes at 11:30:58.53 on 23 Oct 2012

Saving synchronizer state

Synchronization complete at 11:30:58  (1 item transferred, 4 skipped, 0 failed)

  skipped: output 2.pdf

  skipped: output 3.pdf

  skipped: output 4.pdf

  skipped: output.pdf

Looks good; try amending a file and creating a new one and see what happens.
Also try deleting a test(!) file...