Review your favorite Linux distribution.
Go Back > Forums > Linux Forums > Linux - Software
User Name
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.


  Search this Thread
Old 08-04-2008, 04:39 AM   #1
LQ Newbie
Registered: Dec 2005
Location: Tønsberg, Norway
Distribution: Ubuntu
Posts: 9

Rep: Reputation: 0
Question Backup system ala rdiff-backup, but without mirror and with dst encryption

I'm currently using rdiff-backup to backup my (remote) web server down to my local machine (behind a DSL line).

This is working great, but there is one catch. My upstream is very slow, so if I need the restore, it could take weeks or months to get the data back to a new server in case of a total crash.

So I thought that instead of running the backup to my local network I would run it to another server somewhere out there with a lot more bandwidth.

But to keep my costs down (this server doesn't give me any income, so I need to keep the cost down) I got a cheap web host which gives me SSH access and lots of disk space. I do not have root access. This means basically that I have to deal with what they give me. Because this server is controlled by someone else I cannot trust its integrity, so anything I store there would have to be encrypted (AES256, Twofish, Blowfish or whatever that gives decent protection). My access methods are ssh, scp, sFTP and traditional FTP.

It is of course important that you just transfer a diff (increments), to keep bandwidth costs down to a minimum.

If rdiff-backup was able to save all the files in one big (or several) encrypted file(s) and work without software installed on the destination, it would be perfect.

I have been thinking about a consept like this:


lvm snapshot -> dump -0 -f - / -> bzip2 -> mcrypt -key xxx
-> ssh user@host -C "cat >fs.dump"

ssh user@host -C "cat fs.dump"
-> mcrypt -d -key xxx -> bunzip2 -> restore -rf - /restore-dir/
The problem is that my origin server is not running ext2/3, but reiserfs (luckily it has lvm, but I don't have any space left for snapshots), so dump/restore is out of the question. Got a suggestion for a something to replace dump/restore that works with pipes?

Is tar useful for this purpose (remember big database files that change rapidly)? I would think not, as it doesn't support binary diffs.

The system above is good for one file, but to create increments smart you'd have to change the destination to a directory, and create a sort of map of the backup files present, so that you can purge old stuff (don't want to run out of space on destination).

To protect against changes in the stored backup data some kind of integrity check should be possible to do. I think we can assume that md5sum is available at the remote host for this purpose. Maybe it could be possible to calculate the md5sum as you transfer the data, to avoid being dependent on md5sum on the destination.

So, do any of you have a suggestion for a backup/restore software that handle this setup? Most of the stuff I can do myself, with some clever scripting, but the diff/incremental backup stuff is a bit above my level.

If you have suggestions for parts of the chain I would be most grateful for feedback.

Update: It seems like duplicity,, might be the solution to my trouble. Will report back when I have tested it out.

Last edited by dr_dex; 08-04-2008 at 07:22 AM. Reason: Possible solution found


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Remote backup with encryption miek Linux - Software 5 10-12-2007 08:15 AM
rdiff-backup and ssh - I'm a bit confused TheFourthDoctor Linux - Software 4 08-14-2007 03:24 PM
Sharing Rsync'd Backup Mirror? elliotfuller Linux - Server 1 03-11-2007 05:49 PM
Looking for simple backup/mirror solution agentchange Linux - Software 5 10-31-2006 01:52 AM
rdiff-backup problem dave`2005 Linux - Software 0 03-19-2006 04:16 PM > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 08:09 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration