Automatic backup program/script for selected directories

JonathonReinhart · 02-19-2006, 02:05 AM

Hello. I've searched thru the archives and couldn't find exactly what I needed. I'm looking for a program that will backup my /srv and /etc directories to a 2nd harddrive, and also my mysql database. Anything simple that could to this, maybe as a cron job?

thanks!
Jonathon

gilead · 02-19-2006, 02:35 AM

I have the following set up in my crontab:

Code:

0 23 * * fri (/usr/local/bin/backup.sh 2>/dev/null | mail -s "Backup complete" root)

/usr/local/bin/backup.sh contains the following (I've modified the keepers line to have /srv and /etc and the BACKUPDIR would be modified to your output locaction):

Code:

#!/bin/bash
#
# $Id: backup.sh,v 1.3 2006/02/03 19:53:02 steve Exp $
#
# Backup config/data files to a bzip2'ed tar file
#

# Locations
BACKUPDIR="/usr/local/stuff/backup"
TMPDIR="/tmp"
KEEPERS="/srv /etc"
ARCHIVE="`/bin/hostname`-`date '+%F'`.tar"
ARCHIVEBZ2=${ARCHIVE}.bz2
OPTS="-r --atime-preserve -f"
FSROOT="/"

[ -f $BACKUPDIR/$ARCHIVE ] && /bin/rm -f $BACKUPDIR/$ARCHIVE
[ -f $BACKUPDIR/$ARCHIVEBZ2 ] && /bin/rm -f $BACKUPDIR/$ARCHIVEBZ2

echo "Creating tar file"
for KEEPER in $KEEPERS; do
  /bin/tar $OPTS ${BACKUPDIR}/${ARCHIVE} -C $FSROOT $KEEPER
done

echo "Compressing tar file"
[ -f ${BACKUPDIR}/${ARCHIVE} ] && /bin/bzip2 -zv ${BACKUPDIR}/${ARCHIVE}

/usr/bin/chmod 0600 ${BACKUPDIR}/${ARCHIVEBZ2}

ls -l ${BACKUPDIR}/${ARCHIVEBZ2}

echo "Done."

Feel free to take what you want - or use it as the start of a discussion about other ways to backup. There are plenty of people with good advice on the subject here.

JonathonReinhart · 02-19-2006, 07:42 PM

alright looks good. I did have to make 2 symlinks for it to work:
/bin/bzip2 -> /usr/bin/bzip2
/usr/bin/chmod -> /bin/chmod

now does this make a new tar file every time or will it overwrite? i have a small backup drive and dont wanna quickly fill it.

also, where is the mysql database stored, or how can i make a recover-able backup of it?

thanks!

gilead · 02-19-2006, 10:45 PM

Yeah, you can do it with either symlinks or just changing the script - the MySQL database files are under /var/lib/mysql by default.

Running the script multiple times on one day will overwrite the file - running it on separate days creates new files.The line containing ARCHIVE="`/bin/hostname`-`date '+%F'`.tar" does that part. Have a look at the man page for tar, you don't have to do a full backup each time. I just prefer to be able to unzip an archive and get at all the data at once.

JonathonReinhart · 02-20-2006, 12:53 AM

actually, maybe i prefer to have it maybe just mirror the /etc /srv, and /var/lib/mysql directories without even tar-ing them. Would that just be a for each KEEPER in $KEEPERS cp $KEEPER /backup -r ? Or is there somethin else i'm missing? thanks!

gilead · 02-20-2006, 02:34 AM

cp -dpR is a useful command for creating a copy of a directory (and recursively copying the sub-directories) with the original permissions. If you like to keep the command brief, cp -a is equivalent to cp -dpR. So yes, putting it in the loop like that will work fine.

JonathonReinhart · 02-20-2006, 07:45 PM

Ok, changing my mind again, i think i'll keep the tar files, but will i have to manually delete them after a couple months or so, to keep that drive from getting filled? or is there an easy way to have cron do this to the old ones? like check the date and if its 2 months old delete it

JonathonReinhart · 02-20-2006, 07:58 PM

That deletion can be incorporated into that same script. Anyways, I'm not worried about that yet, I've got space to last me a while.

How do I add that to crontab? Isn't there a command that does it?

Thanks yet again

gilead · 02-20-2006, 08:31 PM

The fastest way I know to return the list of files in a particular directory, older than a particular number of days would be the following:

Code:

find /home/backup -type f -iname '*bz2' -mtime +60 -maxdepth 1 -exec basename {} \; 2>/dev/null

This returns the names of the bz2 files in /home/backup (non-recursively) that are older than 60 days. Instead of just running basename, you could run rm and delete them from there. This command could be run in a script or added to your crontab:

Code:

0 3 * * * (/usr/bin/find /home/backup -type f -iname '*bz2' -mtime +60 -maxdepth 1 -exec basename {} \; | mail -s "Deleted old files" root)

JonathonReinhart · 02-20-2006, 08:52 PM

Ahh, that was really fast! So if I edit my crontab to look like this:

Code:

0 23 * * fri (/usr/local/bin/backup.sh 2>/dev/null | mail -s "Backup complete" root)
0 3 * * * (/usr/bin/find /home/backup -type f -iname '*bz2' -mtime +60 -maxdepth 1 -exec rm {} \; | mail -s "Deleted old files" root)

I should be set? I dont understand that whole second line, but close enough. And what do those first few things mean? Like 0 23 * * fri and 0 3 * * * ? I think I could have these run every monday morning (It's a mobile web proxy server, so more use is expected on weekends). Also, for the 60 days is that the +60 (I'm assuming)?

Thanks a lot, you've been a ton of help!

gilead · 02-20-2006, 09:46 PM

No problem

A couple of notes to hopefully explain better what it's doing. The crontab columns are:

Quote:

Minute Hour Day Month DayOfWeek Command

So the first crontab entry runs at the 0th minute of the 23rd hour every Friday (11pm every Friday).
The second entry runs at the 0th minute of the 3rd hour every day (3am every day).
The find command bits are:

Code:

/home/backup   Location to search
-type f        Look for files (not directories, etc.)
-iname '*bz2'  Look for filenames ending in bz2 (case insensitive)
-mtime +60     Modification time of more than 60 days
-maxdepth 1    Don't search sub-directores
-exec rm       Execute the rm command
{} \;          This is find's variable containing the found filename e.g. backup.tar.bz2

JonathonReinhart · 02-20-2006, 10:08 PM

Got it! makes perfect sense. I love how I can learn so much in such a short time. They should teach linux in high school, I swear. Thanks again!!

gilead · 02-20-2006, 10:21 PM

Glad to help - I wish the teachers at my son's high school felt the way you do

JonathonReinhart · 03-10-2006, 08:55 PM

Is there any way that I can check to see if my crontab is set up correctly? It doesn't appear that my backup job executed automatically.

I was thinking that there are commands to actually add it to the crontab for you?

I suppose I could look all this up in the man but humans explain things a lot better!

Thanks yet again!

gilead · 03-10-2006, 09:33 PM

You can set up a dummy entry to test that cron is running. Use crontab -e or add the following to your system crontab. It will append the date to the file /tmp/cron.test every minute:

Code:

* * * * * /usr/bin/date >> /tmp/cron.test

Once you're happy it's running, you only need to change the asterisks to values that control when the job runs.