[SOLVED] Auto-synchronizing a user's document directory.
Linux - GeneralThis Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I have a user who has used one Fedora 17 workstation. Now she will use one of two machines. I currently do not have a server where both machines could have a share to. They machines might not be up at the same time, so I don't want to symbolic link the directories together.
I would like to automatically synchronize the two directories by a cron job. If a file is created on one machine, I would like to see it show up on the other machine. If a file is updated on one machine, I would like to see the new version propagated to the other machine and vice versa. If a file is deleted on one machine, I don't necessarily want it deleted (right away) on the other machine in case it was accidentally deleted.
Do you think I can do this with RSYNC from one machine? Anyone with experience doing this?
This is trivially easy to do, but you should take the time to read the rsync documentation so that you understand what's going on. You can always resort to google for many examples of how others do this.
Looks like I might have it.. I mounted the Workstation 1 file system home share under the /home/raj/c directory. Here is what I have. First, I will sync from workstation 2 to workstation 1.
5 0 * * * /bin/rsync -a -s -t -u --numeric-ids --no-owner --no-group /home/virginia/Documents/ /home/raj/c/virginia/Documents/
#### Rest for a few minutes
10 0 * * * /bin/rsync -a -s -t -u --numeric-ids --no-owner --no-group /home/raj/c/virginia/Documents/ /home/virginia/Documents/
Does that look good? It should kick off at 5 minutes after midnight tonight, and then 10 minutes after midnight. I always make mistakes with the crontab schedule.
I would put both rsync commands in a single script and run that from the cron, that way you don't run into any race condition where the second rsync starts before the first has finished. Depending on usage, you may want to run it more often than that, especially if the user will be modifying documents in both places during the day.
Last edited by suicidaleggroll; 10-21-2012 at 11:33 PM.
I got up early this morning, and took your advice. Instead of just wrapping the two rsync lines in a simple script, I made it more robust.. You can now execute it from the command line and increase its verbosity. Uggh. I have to get more sleep or a better hobby. Here it is. If you have a suggestion for improvement, I would appreciate it.
- Raj
Code:
#!/bin/bash
# By Raj Upadhyaya
# BSD License
#
# Redistribution and use in source and binary forms, with or
# without modification, are permitted provided that the following
# conditions are met:
#
# * Redistributions of source code must retain the above
# copyright notice, this list of conditions and the following
# disclaimer.
#
# * Redistributions in binary form must reproduce the above
# copyright notice, this list of conditions and the following
# disclaimer in the documentation and/or other materials
# provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
# PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR
# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
# TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
# IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
# THE POSSIBILITY OF SUCH DAMAGE.
## globals
verbosity=""
# Display help
function usage(){
cat << EOF
syncDirectory [options] username directory
rsync a directory from one server to the other.
-v : verbose (number) 1 or 2
-h : display help
EOF
}
# ___ _ __ | |_(_) ___ _ __ ___
# / _ \| '_ \| __| |/ _ \| '_ \/ __|
#| (_) | |_) | |_| | (_) | | | \__ \
# \___/| .__/ \__|_|\___/|_| |_|___/
# |_|
# Parsing options
while getopts "v:h" option
do
case $option in
v)
if [[ $2 -ne 2 ]]
then
verbosity="--progress"
else
verbosity="--progress -vv"
fi
;;
h)
usage
exit 0
;;
*)
;;
esac
done
# Mainline...
shift $((OPTIND-1));
# Checks number of options correct
if [[ $# -ne 2 ]]
then
usage
exit 1
fi
myUser=$1
myDirectory=$2
server1Location=/home/$myUser/$myDirectory/
server2Location=/home/raj/c/$myUser/$myDirectory/
rsyncOptions="-a -l -t -u --numeric-ids -g -o -p"
if id -u $myUser > /dev/null 2>&1;
then
echo "user exists" > /dev/null
else
echo "Who is $myUser? I can't seem to find that user on this system."
exit 1
fi
if [[ ! -d "$server1Location" ]]
then
echo "Couldn't find the directory: $server1Location. Are you sure it exists
?"
usage
exit 1
fi
## Here we go!!!
/bin/rsync $rsyncOptions $verbosity $server1Location $server2Location
/bin/rsync $rsyncOptions $verbosity $server2Location $server1Location
I installed Unison to test it out, and it is very nice. The only difference that I saw with the RSYNC option is I could create a cron job to execute the above script a few times a day. The Unison product is a GUI implementation which nice and easy to set up. The difference that I saw that was cool was if a file was updated on both sides. If you use Unison, it prompts you to resolve the conflict and it won't automatically copy either side over the other. The RSYNC option will find the newer file (even if it is by a second) and choose that version to sync with both sides. I think given the use case of one user using one of two workstations at a time, this shouldn't happen.
Well, I learned alot about Rsync, and created a BASH script that is probably 104 lines too long, but what the hey.
Well done; sorry I seem to have dropped the link there ...
Anyway, it sounded like something that might be applicable and certainly worth knowing for future ref
Looks like it will work as a cron job. Here is my output of the first run.
Code:
Videos$ unison -ui text -auto -batch batchpref
Contacting server...
Looking for changes
Warning: No archive files were found for these roots, whose canonical names are:
/home/raj/Documents
/home/raj/c/raj/Documents
This can happen either
because this is the first time you have synchronized these roots,
or because you have upgraded Unison to a new version with a different
archive format.
Update detection may take a while on this run if the replicas are
large.
Unison will assume that the 'last synchronized state' of both replicas
was completely empty. This means that any files that are different
will be reported as conflicts, and any files that exist only on one
replica will be judged as new and propagated to the other replica.
If the two replicas are identical, then no changes will be reported.
If you see this message repeatedly, it may be because one of your machines
is getting its address from DHCP, which is causing its host name to change
between synchronizations. See the documentation for the UNISONLOCALHOSTNAME
environment variable for advice on how to correct this.
Donations to the Unison project are gratefully accepted:
http://www.cis.upenn.edu/~bcpierce/unison
Reconciling changes
file <-?-> file output 2.pdf
home/raj/... : file modified on 2012-10-23 at 8:04:25 size 17850 rw-rw-r--
c/raj/Doc... : file modified on 2012-10-22 at 4:39:55 size 19198 rw-rw-r--
file <-?-> file output 3.pdf
home/raj/... : file modified on 2012-10-23 at 8:06:59 size 93902 rw-rw-r--
c/raj/Doc... : file modified on 2012-10-22 at 4:36:11 size 93523 rw-rw-r--
file <-?-> file output 4.pdf
home/raj/... : file modified on 2012-10-23 at 8:08:07 size 34097 rw-rw-r--
c/raj/Doc... : file modified on 2012-10-22 at 4:32:32 size 33754 rw-rw-r--
file <-?-> file output.pdf
home/raj/... : file modified on 2012-10-23 at 8:01:25 size 6055 rw-rw-r--
c/raj/Doc... : file modified on 2012-10-22 at 4:38:56 size 6032 rw-rw-r--
file ----> HR 3962 - Medical Devices.doc
Propagating updates
UNISON 2.40.63 started propagating changes at 11:30:58.42 on 23 Oct 2012
[CONFLICT] Skipping output 2.pdf
[CONFLICT] Skipping output 3.pdf
[CONFLICT] Skipping output 4.pdf
[CONFLICT] Skipping output.pdf
[BGN] Copying HR 3962 - Medical Devices.doc from /home/raj/Documents to /home/raj/c/raj/Documents
[END] Copying HR 3962 - Medical Devices.doc
UNISON 2.40.63 finished propagating changes at 11:30:58.53 on 23 Oct 2012
Saving synchronizer state
Synchronization complete at 11:30:58 (1 item transferred, 4 skipped, 0 failed)
skipped: output 2.pdf
skipped: output 3.pdf
skipped: output 4.pdf
skipped: output.pdf
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.