Tweaking site backup

Ashex · 06-05-2008, 11:49 PM

Hey Guys,

Several weeks ago I went through the painful process of rebuilding a forum on a domain I help administrate (also known as, my dad has a site and I fix it when it breaks) after it was exploited due to being out of date.

I've since setup a bunch of backup procedures. daily mysqldumps of all databases, weekly backups of sites, and a full backup of everything every two weeks.

These are all placed in a visible password-protected directory so my dad can access them if he wanted to. I have a cron job to rsync the directory locally, I'm going to maintain a two week backup of everything to keep the size down.

Currently, this is the script I use for backups:

Code:

#!/usr/bin/perl
use strict;
use warnings;

my $fileOrFolder = $ARGV[0] or die "You forgot to specify a file or folder name!"; #The name of the file or folder provided during execution of the script
my $upper = "\U$fileOrFolder"; #Convert the file or folder name to uppercase for the label

my $date = `date +%Y_%m_%d`; #Store the date in a variable
chomp($date); #Remove the new line that gets added at the end

print `tar -cvvzhlf "/foo/backups/$fileOrFolder-$date.tar.gz" /foo/$fileOrFolder --atime-preserve --label $upper`;

Someone helped me set it up. I have modified copies of it for the full backup and mysql backups.

After it ran successfully last weekend (had to tweak it a little to work out the kinks), I realized that the full backup is going to take up a lot of space quickly. So I need to modify the script to work a little differently for me.

I'm hoping that someone here can help me set this up. Basically all I need it to do, is check for an existing backup, such as mysite-2008_06_01.tar.gz, then to just append any new files/changes to it, then either rename it to mysite-2008_06_05.tar.gz, or delete the old backup and create the new one.

I would use something like bacula for this, but it's on a shared host.

billymayday · 06-05-2008, 11:59 PM

At the start of my weekly backup script (this is more or less a full backup), I use

Code:

find /storage/backups/ -name '*' -type f -mtime +30 -exec rm -f {} \;

to keep versions less than 30 days old (so effectively 4 old versions)

Appending to compressed files doesn't work in tar if I recall correctly, so what you are suggesting won't work.

Otherwise, you could do incremental backups and less frequently do a new full. My nightly incrementals at home look like

Code:

find /home -daystart -mtime -2 -type f -print0 | xargs -0 tar rf "$DEST"

for example (plus other key directories)

Ashex · 06-06-2008, 12:14 AM

Nuts! I was hoping to do incremental/differential backups of some sort. I was originally going to use rsync and then tar it up locally, but this computer is on a 6Mb connection and that would take far too long :/
I didn't think about using find to limit the number of backups. I'll add that to the local rsync script that runs after all the backups have gone off. I staggered all the backups so the account doesn't get suspended for excessive i/o usage.

billymayday · 06-06-2008, 12:44 AM

Check that I was correct on the tar issue before you give up

Ashex · 06-06-2008, 12:57 AM

I double checked and the -A trigger will append to an existing tar. The man mentions flags for differential. I'll probably need to read up a bit more to learn the proper usage of them.

billymayday · 06-06-2008, 01:06 AM

Yes - but I don't think it works on compressed tars - try it