LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 10-21-2012, 06:17 PM   #1
Rupadhya
Member
 
Registered: Sep 2012
Location: Hoffman Estates, IL
Distribution: Fedora 20
Posts: 167

Rep: Reputation: Disabled
Auto-synchronizing a user's document directory.


Hello,

I have a user who has used one Fedora 17 workstation. Now she will use one of two machines. I currently do not have a server where both machines could have a share to. They machines might not be up at the same time, so I don't want to symbolic link the directories together.

I would like to automatically synchronize the two directories by a cron job. If a file is created on one machine, I would like to see it show up on the other machine. If a file is updated on one machine, I would like to see the new version propagated to the other machine and vice versa. If a file is deleted on one machine, I don't necessarily want it deleted (right away) on the other machine in case it was accidentally deleted.

Do you think I can do this with RSYNC from one machine? Anyone with experience doing this?

- Raj Upadhyaya
 
Old 10-21-2012, 07:53 PM   #2
aetratus
LQ Newbie
 
Registered: Jul 2006
Location: Internet
Distribution: Fedora, CentOS, Archlinux
Posts: 27

Rep: Reputation: 6
This is trivially easy to do, but you should take the time to read the rsync documentation so that you understand what's going on. You can always resort to google for many examples of how others do this.
 
Old 10-21-2012, 08:52 PM   #3
Rupadhya
Member
 
Registered: Sep 2012
Location: Hoffman Estates, IL
Distribution: Fedora 20
Posts: 167

Original Poster
Rep: Reputation: Disabled
Looks like I might have it.. I mounted the Workstation 1 file system home share under the /home/raj/c directory. Here is what I have. First, I will sync from workstation 2 to workstation 1.
Code:
rsync -a -s -t -u --numeric-ids --no-owner --no-group --progress /home/virginia/Documents /home/raj/c/virginia
Then immediately after that, I will sync from workstation 1 to workstation 2 if there are any new files on the workstation 1.
Code:
rsync -a -s -t -u --numeric-ids --no-owner --no-group --progress /home/raj/c/virginia/Documents /home/virginia
Does that make sense? I think if I schedule it from cron I will remove the --progress tab and replace it with -q (which is quiet mode).

- Raj
 
Old 10-21-2012, 10:50 PM   #4
suicidaleggroll
Senior Member
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 2,795

Rep: Reputation: 990Reputation: 990Reputation: 990Reputation: 990Reputation: 990Reputation: 990Reputation: 990Reputation: 990
Add the Documents directory onto the end of the command:
Code:
rsync -a -s -t -u --numeric-ids --no-owner --no-group --progress /home/virginia/Documents/ /home/raj/c/virginia/Documents/
And vice versa. The trailing slash here IS important.

Also, make sure your user knows that if they ever want to delete OR RENAME a file, they must do so on BOTH workstations in between cron jobs.

You'll also want a check to make sure the remote mount is up before running these commands.

Last edited by suicidaleggroll; 10-21-2012 at 10:51 PM.
 
Old 10-21-2012, 11:15 PM   #5
Rupadhya
Member
 
Registered: Sep 2012
Location: Hoffman Estates, IL
Distribution: Fedora 20
Posts: 167

Original Poster
Rep: Reputation: Disabled
From my root crontab.
Code:
5 0 * * * /bin/rsync -a -s -t -u --numeric-ids --no-owner --no-group  /home/virginia/Documents/ /home/raj/c/virginia/Documents/
#### Rest for a few minutes
10 0 * * * /bin/rsync -a -s -t -u --numeric-ids --no-owner --no-group  /home/raj/c/virginia/Documents/ /home/virginia/Documents/
Does that look good? It should kick off at 5 minutes after midnight tonight, and then 10 minutes after midnight. I always make mistakes with the crontab schedule.

- Raj
 
Old 10-21-2012, 11:31 PM   #6
suicidaleggroll
Senior Member
 
Registered: Nov 2010
Location: Colorado
Distribution: OpenSUSE, CentOS
Posts: 2,795

Rep: Reputation: 990Reputation: 990Reputation: 990Reputation: 990Reputation: 990Reputation: 990Reputation: 990Reputation: 990
I would put both rsync commands in a single script and run that from the cron, that way you don't run into any race condition where the second rsync starts before the first has finished. Depending on usage, you may want to run it more often than that, especially if the user will be modifying documents in both places during the day.

Last edited by suicidaleggroll; 10-21-2012 at 11:33 PM.
 
1 members found this post helpful.
Old 10-22-2012, 07:36 AM   #7
Rupadhya
Member
 
Registered: Sep 2012
Location: Hoffman Estates, IL
Distribution: Fedora 20
Posts: 167

Original Poster
Rep: Reputation: Disabled
I got up early this morning, and took your advice. Instead of just wrapping the two rsync lines in a simple script, I made it more robust.. You can now execute it from the command line and increase its verbosity. Uggh. I have to get more sleep or a better hobby. Here it is. If you have a suggestion for improvement, I would appreciate it.
- Raj
Code:
#!/bin/bash
# By Raj Upadhyaya
# BSD License
#
# Redistribution and use in source and binary forms, with or
#   without modification, are permitted provided that the following
#   conditions are met:
#
# *  Redistributions of source code must retain the above
#    copyright notice, this list of conditions and the following
#    disclaimer.
#
# *  Redistributions in binary form must reproduce the above
#    copyright notice, this list of conditions and the following
#    disclaimer in the documentation and/or other materials
#    provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
# THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
# PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR
# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
# TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
# IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
# THE POSSIBILITY OF SUCH DAMAGE.

##  globals
verbosity=""
# Display help
function usage(){
    cat << EOF

    syncDirectory [options] username directory
        rsync a directory from one server to the other.

        -v     : verbose (number) 1 or 2
        -h     : display help


EOF
}
#  ___  _ __ | |_(_) ___  _ __  ___
# / _ \| '_ \| __| |/ _ \| '_ \/ __|
#| (_) | |_) | |_| | (_) | | | \__ \
# \___/| .__/ \__|_|\___/|_| |_|___/
#      |_|

# Parsing options
while getopts "v:h" option
do
    case $option in
        v)
        if [[ $2 -ne 2 ]]
        then
          verbosity="--progress"
        else
          verbosity="--progress -vv"
        fi
        ;;
        h)
        usage
        exit 0
        ;;
        *)
        ;;
    esac
done


# Mainline...

shift $((OPTIND-1));

# Checks number of options correct
if [[ $# -ne 2 ]]
then
    usage
    exit 1
fi

myUser=$1
myDirectory=$2
server1Location=/home/$myUser/$myDirectory/
server2Location=/home/raj/c/$myUser/$myDirectory/
rsyncOptions="-a -l -t -u --numeric-ids -g -o -p"

if id -u $myUser > /dev/null 2>&1;
then
        echo "user exists" > /dev/null
else
        echo "Who is $myUser?  I can't seem to find that user on this system."
        exit 1
fi

if [[ ! -d "$server1Location" ]]
then
    echo "Couldn't find the directory: $server1Location.  Are you sure it exists
?"
    usage
    exit 1
fi

## Here we go!!!
/bin/rsync $rsyncOptions $verbosity $server1Location $server2Location
/bin/rsync $rsyncOptions $verbosity $server2Location $server1Location
 
Old 10-22-2012, 08:02 PM   #8
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,264

Rep: Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028
You might want to read this discussion of rsync vs unison.
(its quite short; see the latter posts)
 
Old 10-23-2012, 12:32 AM   #9
Rupadhya
Member
 
Registered: Sep 2012
Location: Hoffman Estates, IL
Distribution: Fedora 20
Posts: 167

Original Poster
Rep: Reputation: Disabled
Thumbs up

Chris,

I found this discussion on http://www.linuxquestions.org. http://www.linuxquestions.org/questi...8/#post4509973

I installed Unison to test it out, and it is very nice. The only difference that I saw with the RSYNC option is I could create a cron job to execute the above script a few times a day. The Unison product is a GUI implementation which nice and easy to set up. The difference that I saw that was cool was if a file was updated on both sides. If you use Unison, it prompts you to resolve the conflict and it won't automatically copy either side over the other. The RSYNC option will find the newer file (even if it is by a second) and choose that version to sync with both sides. I think given the use case of one user using one of two workstations at a time, this shouldn't happen.

Well, I learned alot about Rsync, and created a BASH script that is probably 104 lines too long, but what the hey.

- Raj
 
Old 10-23-2012, 05:51 AM   #10
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,264

Rep: Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028
Well done; sorry I seem to have dropped the link there ...
Anyway, it sounded like something that might be applicable and certainly worth knowing for future ref
 
Old 10-23-2012, 11:27 AM   #11
Rupadhya
Member
 
Registered: Sep 2012
Location: Hoffman Estates, IL
Distribution: Fedora 20
Posts: 167

Original Poster
Rep: Reputation: Disabled
Looks like you can run Unison from the command line as well without the GUI.
Quote:
unison -ui text -auto -batch batchpref
With this, you could potentially run unison from a cron job.

I will look into this and document my findings for others reference.

- Raj
 
Old 10-23-2012, 11:33 AM   #12
Rupadhya
Member
 
Registered: Sep 2012
Location: Hoffman Estates, IL
Distribution: Fedora 20
Posts: 167

Original Poster
Rep: Reputation: Disabled
Looks like it will work as a cron job. Here is my output of the first run.
Code:
 Videos$ unison -ui text -auto -batch batchpref
Contacting server...
Looking for changes
Warning: No archive files were found for these roots, whose canonical names are:
	/home/raj/Documents
	/home/raj/c/raj/Documents
This can happen either
because this is the first time you have synchronized these roots, 
or because you have upgraded Unison to a new version with a different
archive format.  

Update detection may take a while on this run if the replicas are 
large.

Unison will assume that the 'last synchronized state' of both replicas
was completely empty.  This means that any files that are different
will be reported as conflicts, and any files that exist only on one
replica will be judged as new and propagated to the other replica.
If the two replicas are identical, then no changes will be reported.

If you see this message repeatedly, it may be because one of your machines
is getting its address from DHCP, which is causing its host name to change
between synchronizations.  See the documentation for the UNISONLOCALHOSTNAME
environment variable for advice on how to correct this.

Donations to the Unison project are gratefully accepted: 
http://www.cis.upenn.edu/~bcpierce/unison

Reconciling changes                                               
file     <-?-> file       output 2.pdf  
home/raj/... : file               modified on 2012-10-23 at  8:04:25  size 17850     rw-rw-r--
c/raj/Doc... : file               modified on 2012-10-22 at  4:39:55  size 19198     rw-rw-r--
file     <-?-> file       output 3.pdf  
home/raj/... : file               modified on 2012-10-23 at  8:06:59  size 93902     rw-rw-r--
c/raj/Doc... : file               modified on 2012-10-22 at  4:36:11  size 93523     rw-rw-r--
file     <-?-> file       output 4.pdf  
home/raj/... : file               modified on 2012-10-23 at  8:08:07  size 34097     rw-rw-r--
c/raj/Doc... : file               modified on 2012-10-22 at  4:32:32  size 33754     rw-rw-r--
file     <-?-> file       output.pdf  
home/raj/... : file               modified on 2012-10-23 at  8:01:25  size 6055      rw-rw-r--
c/raj/Doc... : file               modified on 2012-10-22 at  4:38:56  size 6032      rw-rw-r--
file     ---->            HR 3962 - Medical Devices.doc  
Propagating updates
UNISON 2.40.63 started propagating changes at 11:30:58.42 on 23 Oct 2012
[CONFLICT] Skipping output 2.pdf
[CONFLICT] Skipping output 3.pdf
[CONFLICT] Skipping output 4.pdf
[CONFLICT] Skipping output.pdf
[BGN] Copying HR 3962 - Medical Devices.doc from /home/raj/Documents to /home/raj/c/raj/Documents
[END] Copying HR 3962 - Medical Devices.doc
UNISON 2.40.63 finished propagating changes at 11:30:58.53 on 23 Oct 2012
Saving synchronizer state
Synchronization complete at 11:30:58  (1 item transferred, 4 skipped, 0 failed)
  skipped: output 2.pdf
  skipped: output 3.pdf
  skipped: output 4.pdf
  skipped: output.pdf
 
Old 10-23-2012, 09:08 PM   #13
chrism01
Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.5, Centos 5.10
Posts: 16,264

Rep: Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028Reputation: 2028
Looks good; try amending a file and creating a new one and see what happens.
Also try deleting a test(!) file...
 
1 members found this post helpful.
  


Reply

Tags
cron, fedora, rsync


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Openldap Synchronizing with Windows Active Directory aravind1024004 Linux - Server 0 12-06-2008 07:08 AM
synchronizing user accounts? jedimastermopar Linux - Server 5 04-25-2008 04:59 PM
Konqueror freezes when I try to access user document directory. GlennsPref Mandriva 7 02-18-2008 12:17 PM
Auto Create user home directory on Samba m2azer Linux - Networking 2 01-15-2007 11:56 AM
how do i create a document, then a directory and save the document in it. jtdodge Linux - Newbie 9 06-15-2005 11:03 AM


All times are GMT -5. The time now is 06:37 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration