LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)
-   -   Backup system ala rdiff-backup, but without mirror and with dst encryption (https://www.linuxquestions.org/questions/linux-software-2/backup-system-ala-rdiff-backup-but-without-mirror-and-with-dst-encryption-660283/)

dr_dex 08-04-2008 03:39 AM

Backup system ala rdiff-backup, but without mirror and with dst encryption
 
I'm currently using rdiff-backup to backup my (remote) web server down to my local machine (behind a DSL line).

This is working great, but there is one catch. My upstream is very slow, so if I need the restore, it could take weeks or months to get the data back to a new server in case of a total crash.

So I thought that instead of running the backup to my local network I would run it to another server somewhere out there with a lot more bandwidth.

But to keep my costs down (this server doesn't give me any income, so I need to keep the cost down) I got a cheap web host which gives me SSH access and lots of disk space. I do not have root access. This means basically that I have to deal with what they give me. Because this server is controlled by someone else I cannot trust its integrity, so anything I store there would have to be encrypted (AES256, Twofish, Blowfish or whatever that gives decent protection). My access methods are ssh, scp, sFTP and traditional FTP.

It is of course important that you just transfer a diff (increments), to keep bandwidth costs down to a minimum.

If rdiff-backup was able to save all the files in one big (or several) encrypted file(s) and work without software installed on the destination, it would be perfect.

I have been thinking about a consept like this:

Backup:

Code:

lvm snapshot -> dump -0 -f - / -> bzip2 -> mcrypt -key xxx
-> ssh user@host -C "cat >fs.dump"

Restore:

Code:

ssh user@host -C "cat fs.dump"
-> mcrypt -d -key xxx -> bunzip2 -> restore -rf - /restore-dir/

The problem is that my origin server is not running ext2/3, but reiserfs (luckily it has lvm, but I don't have any space left for snapshots), so dump/restore is out of the question. Got a suggestion for a something to replace dump/restore that works with pipes?

Is tar useful for this purpose (remember big database files that change rapidly)? I would think not, as it doesn't support binary diffs.

The system above is good for one file, but to create increments smart you'd have to change the destination to a directory, and create a sort of map of the backup files present, so that you can purge old stuff (don't want to run out of space on destination).

To protect against changes in the stored backup data some kind of integrity check should be possible to do. I think we can assume that md5sum is available at the remote host for this purpose. Maybe it could be possible to calculate the md5sum as you transfer the data, to avoid being dependent on md5sum on the destination.

So, do any of you have a suggestion for a backup/restore software that handle this setup? Most of the stuff I can do myself, with some clever scripting, but the diff/incremental backup stuff is a bit above my level.

If you have suggestions for parts of the chain I would be most grateful for feedback.

Update: It seems like duplicity, http://duplicity.nongnu.org/, might be the solution to my trouble. Will report back when I have tested it out.


All times are GMT -5. The time now is 10:50 AM.