LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 06-28-2007, 09:55 AM   #1
pezdspencer
LQ Newbie
 
Registered: Jun 2007
Posts: 5

Rep: Reputation: 0
Using RSync to backup a secondary off-site backup server


Hi all,

I have a conceptual question regarding backup strategy for a secondary off-site backup implementation. I currently have a rsync-implemented on-site backup system running on FC5. It is actually using rsnapshot, with which I'm sure you're all familiar with.

I now have access to an offsite facility and would like to also perform an off-site backup of my backup server itself for redundancy. For security reasons, I would like to configure my firewalls to allow access to/from the backup servers only for this. I have the following in mind to accomplish this, but could use some advice:

I would like to use rsync over SSH to create a nightly mirror of the backup server itself. This seems to be the most straightforward, but since rsnapshot uses hardlinks and since it incrementally moves the directory structure nightly (day.0 to day.1; day.1 to day.2, etc. etc.) is this the best method? Even though the entire backup directory structure changes nightly, will rsync see that most of it is just hardlinks and simply/quickly keep a mirror and handle the directory changes without attempting to re-download each day.X since they daily change?

I know that this isn't the typical way in which a mirror works, since the directory structure doesn't usually change like that. But it is more confusing to me trying to keep a mirror of a backup server itself. I also know I could write a script that handles moving the off-site backup server's directory structure to match the on-site's prior to rsync, but I'd like to keep it as simple as possible. Also, most of my backed up files each night are incremental, only ~2G actually change daily.

Thanks,

PezD
 
Old 06-28-2007, 11:19 AM   #2
hob
Senior Member
 
Registered: Mar 2004
Location: Wales, UK
Distribution: Debian, Ubuntu
Posts: 1,075

Rep: Reputation: 45
There are two key points:

- Don't confuse mirroring with backups. Mirroring gives you an additional copy of the files as they are, whilst backup systems like rsnapshot retain multiple copies from different points in time. Mirroring is useful for hot stand-bys etc., but not really for file recovery, since it doesn't give you access to copies of past versions.

- Since Fedora is constructed from packages you can reconstruct the entire system using the installer and a single Kickstart file...apart from the logs, and the files that you created or changed yourself. So, you don't need extra copies of most of the files on the system for backup purposes, just stuff like /etc, /home, and portions of /var/. rsnapshot etc. will help you here.
 
Old 06-28-2007, 12:09 PM   #3
pezdspencer
LQ Newbie
 
Registered: Jun 2007
Posts: 5

Original Poster
Rep: Reputation: 0
hob,

Thanks for the helpful reply. My apologies, however, as I'm afraid I wasn't entirely clear.

I have been using rsnapshot as a backup server for over a year now to backup critical company data across 15 servers of different Linux distros and one netware box. This backup server only contains critical data I've assigned to backup from these servers.

However, for the secondary off-site backup, I'd like for it to be a duplicate of that backup server with the same exact data on it as well. This way I have redundancy of my backup data.

Thus, for my secondary off-site backup server, could I not "mirror" the primary backup server which has my incremental data? Could/should I use rsync for this? I ask because I am not sure how rysnc will react when it attempts to sync with the primary on-site backup server and sees that all of the daily directories are different (because the incremental backup moves the daily folders: day.0 -> day.1; day.1-> day.2) Will rsync simply update each daily directory since rsnpashot uses hardlinks for non-changed incremental files, or will it attempt to re-download each incremental daily directory since they are different, having been moved incrementally?

I know this isn't the standard use of mirroring, especially since mirrors don't usually have an entire directory change daily, just only additions/deletions/modifications of files in those directories. All I need is an off-site "mirror" of a backup server which contains data in directories that change daily because of incremental backups.

Last edited by pezdspencer; 06-28-2007 at 12:13 PM.
 
Old 06-28-2007, 03:31 PM   #4
hob
Senior Member
 
Registered: Mar 2004
Location: Wales, UK
Distribution: Debian, Ubuntu
Posts: 1,075

Rep: Reputation: 45
Quote:
Originally Posted by pezdspencer
hob,

Thanks for the helpful reply. My apologies, however, as I'm afraid I wasn't entirely clear.

I have been using rsnapshot as a backup server for over a year now to backup critical company data across 15 servers of different Linux distros and one netware box. This backup server only contains critical data I've assigned to backup from these servers.

However, for the secondary off-site backup, I'd like for it to be a duplicate of that backup server with the same exact data on it as well. This way I have redundancy of my backup data.
Ah, I think that I understand: you would like a backup arrangement that gives you two identical backup repositories, each on different servers.

My gut instinct is that rsync is smart enough, but that I wouldn't trust such an arrangement not to hit a problem and break the clone of your rsnapshot repository at some point. For a small-scale system I would personally go for one of two approaches:

- A script that makes a zipped copy of the rsnapshot repository and copies the zip file to the secondary backup server.

- Write your rsync script so that it syncs with a different clone of your repository on different days. Then, if the rsync job breaks once you still have some usable clones on your secondary backup server.

Since your aim is simply to recreate the rsnapshot repository in the event of a disaster you only need a small number of copies of it, but they must be accurate, which is why I wouldn't want rsync to mess with a sole secondary copy.

With 16 servers, you have probably hit the point where you need to invert your strategy, though - rather than trying to capture all the different files from all of these systems after the fact you should should probably now look to consolidate things. Facilities like Puppet and BCFG will let you manage your configurations systematically, and are designed to cope with heterogenous networks. You can configure the syslog services to route messages to a central log server, rather than having to backup so many log directories. Other measures would depend on the services involved.

It's been a long time since I did anything with Netware, but I beleive that Novell "Open Enterprise Server" gives you Netware services on a Linux box, so that might help you reduce the number of servers required as well.
 
Old 06-29-2007, 03:40 PM   #5
pezdspencer
LQ Newbie
 
Registered: Jun 2007
Posts: 5

Original Poster
Rep: Reputation: 0
hob,

Thanks again for the reply. I appreciate the information!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
backup with rsync muhammednavas Linux - Networking 4 12-01-2006 06:58 AM
Using rsync to backup data. Best way to backup folders? Micro420 Linux - General 2 11-23-2006 01:13 AM
Using rsync for local server backup. Some issues. enygma Linux - Networking 0 02-03-2005 01:46 PM
How do i backup a dns server? I am using rsync but the key dont match abefroman Linux - Networking 2 03-11-2004 08:16 AM
How do i backup a dns server? I am using rsync but the key dont match abefroman Linux - Newbie 1 03-09-2004 07:39 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 11:12 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration