Some Rsync questions...

sigint-ninja · 10-15-2013, 02:38 PM

hi guys,

dont really know where this belongs

i have a windows server 2008 r2 that i want to backup using rsync
i have cygwin installed and am starting to do this...just have a few questions

i want to backup with rsync to 5 external hard drives...switching them every day...will Rsync write to the different drives as long as the drive letter is the same? does it only backup the changes by default...would it also only save the changes to say a mail PST file?

i would like to have it append to any one of the 2tb drives in at the time with no problem...also want to run a Rsync backup over the internet to another linux box in the office 20km away...i know there are numerous rysnc tutorials....just want to get these nagging questions out the way.

does this seem feasible???

lleb · 10-15-2013, 08:13 PM

well most of your questions can be directly answered via the man page for rsync, but here is some other info.

as for writing to the external HDDs, that will depend on your script. yes when the drive letters in MS change if you have the script hard coded to a specific letter and that letter is not found, just like anything in the MS world it will fail.

if you use the -a flag you will "archive" data. this is an append by default. without the --delete flag it will never remove files on the backup that have been removed from the source.

before you run 2TB via WWW highly consider physically performing the data transfer LOCAL... think about your upload speed and do the math for how long it will take to write 2TB via the web.

typically when I am setting that up the first rsync I run is to either the system physically in the same LAN, or to an external drive that i can move physically to the "remote" server to transfer the initial data. as long as you are not storing multiple copies of the same data (that is a good idea btw) you will only need to move the data once and rsync will then just append the data via the WWW without issue or massive bandwidth consumption. i typically keep 7 days worth of backups on my backup server. just put them in day of the week folders

...

good luck.

sigint-ninja · 10-16-2013, 04:44 PM

thanks for that...i will start reading the man pages!!!
so i could copy everything i need onto a hard drive than move that data to my other linux box in the other office
then rsync will only save changes...thats sweet!!!
and i could even have that save a copy everyday to other external drives on the linux box

thanks again....got a lot of reading to do,,,

lleb · 10-17-2013, 09:12 PM

yeah rsync is really powerful. also dont be afraid to hit up google and look for rsync examples. i have my web site rsync to both my backup server here at the house and to 2 other servers across the country all via cron.

i have all of the laptops in my household rsync to my backup server manually.

i use basically the same rsync script in both cases.

evo2 · 10-17-2013, 09:25 PM

Hi,

there are many existing solutions for making backups - many of which (can) use rsync as a backend. Instead of reinventing the wheel why not use something that is already known to work. Eg backuppc.

Evo2.

sigint-ninja · 10-21-2013, 03:34 PM

Hi Evo,

dont you think it would be better for me (as a Linux noob) to learn to write rsync, as well as other scripts?
or would my time be better suited elsewhere...thanks for your reply

sn

lleb · 10-21-2013, 04:57 PM

what ever floats your boat. for me many of the pre-packaged backup tools out there were not available for me when i first started learning to write basic scripts and how to use rsync. for me it is faster and easier just to write a simple backup rsync script.

a good idea might be to play with both options so you can choose what is better for you and your solution requirements.

evo2 · 10-21-2013, 08:11 PM

Hi,

Quote:

Originally Posted by sigint-ninja

dont you think it would be better for me (as a Linux noob) to learn to write rsync, as well as other scripts?

No, not better. Yes, it is good to learn to write scripts and to use rsync, but backups are very important, and I would not trust my backups to a script hacked together by a noob and nor would I suggest should you.

Quote:

or would my time be better suited elsewhere...thanks for your reply

You're likely to learn plenty just by configuring whatever backup system you choose.

Anyway as lleb says, choose what you think is best for you, but in doing so, be sure to make an informed choice.

Cheers,

Evo2.

sigint-ninja · 10-22-2013, 01:43 PM

was looking at a few tools...from other posts too.....not many support windows server 2008 r2
i have cygwin installed on my windows server and have a shell running...so i think i will run a script/s

are rsync scripts just bash scripts? i could use vim to write the script and save it with a script extension?
im practicing writing scripts on my centos box...not the production server.

thanks

lleb · 10-22-2013, 08:40 PM

Quote:

Originally Posted by sigint-ninja

was looking at a few tools...from other posts too.....not many support windows server 2008 r2
i have cygwin installed on my windows server and have a shell running...so i think i will run a script/s

are rsync scripts just bash scripts? i could use vim to write the script and save it with a script extension?
im practicing writing scripts on my centos box...not the production server.

thanks

yes a script is just a simple BASH script.

here is an example of a backup script that I use on all of my laptops with some personal data redacted.

Code:

$ cat /usr/bin/Backup 
#!/bin/bash

### Jan 12, 2013
### adding excludes.txt to ignore directories like
### .wine, .cache, etc...

### Jan 5, 2013
### Setting up for daugher to backup to sparky daily, or
### manually.  

### Jan 5, 2013 added ~/.ssh/config to the system for pre-
### defined ports and protocols as follows:

#	Host	foo.nu
#		Port		22222
#
#	Host	*
#		Protocol		2
#		ForwardAgent		yes
#		ForwardX11		yes
#		ServeraliveInterval	30
#		ServerAliveCountMax	5
#		TCPKeepAlive		yes

### Then set the permissions to 644 on that file.  This tells
### ssh to use the above port when contacting foo.nu.
### As a results modified the variables and rsycn command below.

### Dec. 7, 2012 modified to add 2nd location for backup
### The second computer is my Intel iMAC

### RSYNC backup between web server and sparky

### Dec. 4, 2012 edited script to run to old sparky
### located at my home.  This will change soon as we
### will be relocating to friend's home.

###########################################################
### Created by Ray Brunkow with help from Bryan Smith
#
# Copyright (C) 2012 Raymond L. Brunkow.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 2 or version 3 of the
# license, at your option.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
#
##########################################################

### Setting Variables
#####################################
dtstamp="`date +%F-%A-%H.%M.%S `"
dow=`date +%A`
HOMEDIR="$HOME"
PWD=`pwd`
log=${PWD}/logs/`date +%Y-%m-%d-%A`-rsync.log
RUSER=user
RHOST=server
RDIR=/path/to/save/data/

### Checking for logs directory, if not there then create it
#####################################

[ ! -d "${PWD}/logs" ] && mkdir -p ${PWD}/logs >> /dev/null 2>&1

rsync -aviS --exclude-from=${HOMEDIR}/excludes.txt ${PWD}/ ${RUSER}@${RHOST}:${RDIR}/${dow}/ >> ${log} 2>&1

### Clean up log directory to only keep 16 days worth of logs

find ${PWD}/logs/*.log -mtime +15 -exec rm '{}' \;

exit

feel free to modify that script as you require and ask as you need. its a nice simple little backup script for home directories.

sigint-ninja · 10-24-2013, 01:33 AM

thanks for that!!! will do