Tape backup (VERY important)
 

Hello all!

Ok, Here's what I got and what I need to do. I have a RH 8.0 linux server running DNS, sendmail, samba, routing, etc. The rest of the lan is all Windoze machines. Everything runs great, and I have it set up exactly as I want it. On the server, I have a SUN DLT 4000 tape drive hooked up to an internal Adaptec 2940 SCSI card. The drive itself is external. I know NOTHING about tape backup in linux!

What I want is to do ONE backup a week, Sunday night at midnight, every single week. This can't be hard. I have heard of many tools to use, tar, dump, etc. I know my tape drive is /dev/st0, but that is about it. Can anyone please tell me the best way to set this up so it is done automatically every Sunday? I'm looking for step by step instructions (or close to it). I am still in the learning process of linux, so I am no expert. Here are my comments and questions:

1. The drive is 20/40GB.
2. The hard drive is 120GB, but only 30GB is used now, no more than 40GB ever will be.
3. I want ALL data backed up in full. Every last bit!
4. It is a dual processor system running (2) instances of SETI, should I KILL the SETI processes before running a backup, ie. will they interfere with the backup?
5. Software-wise, do I have to anything to compress data, since the drive and tapes are 40GB compressed, only 20GB native... Or will the drive handle this? The drive's "compression" light is on.
6. Do I put this in cron to run every week at the same time, and if so, how do I do this?
7. Assuming I can get this working, and the HDD fails, how do I restore the system back to normal?
8. Any other pearls of linux wisdom?

I am looking to make this as simple as possible for right now. The server is extremely important (without it, we have no internet or e-mail), but this is a home, not a business. Like I said, I'm still learning and have had about a million questions answered here at LQ. You guys are the greatest. I'm even starting to get comfortable where I am beginning to answer questions! HA! But for this, I am definitely a :newbie: !!! Any help you guys could be would be most appreciated. Thank you very much!

Chris

i have seen stuff on tape backup but i always over look it and dont read it becuase it never pertained to me. But yes, you would use cron to do this. I dont mess with cron either, but that much i do remember.

using red hat, you may have installed Amanda, i think that is the tape backpu software. check out the amanda docs.

Tape backup
 

Take a look at www.storix.com -- They have a nice little backup program for linux (as well as AIX) - You can download the program for free and it does very well - The free version doesn't come with a scheduler - but you can use cron to submit the job (I do it every night) -

Sounds like you have a setup similar to what I used to run. Server does everything. The idea is to look at the problem from two perspectives:

1) What happens if the HD dies.
2) Someone fat fingers the delte button and wants a file back.

My contention would be that, if you can solve #1, #2 isn't a problem other than how stale the backup is.

If the hard disk dies, you need to get back /var, /home, /etc and /root plus any non-standard partitions you're using. Everything else you're going to re-install on the new HD. Even /etc you will probably need to restore elsewhere and then carefully put individual files back into place.

So set up a cron job that runs once a week to do the backup. For simple backups like this, I suggest just using tar. The backups are much more likely to be portable. Your tar command will look something like:

tar -czf /dev/st0 /var /home /etc /root

c = create
z = use gzip compression
f = back up to the following file/device (/dev/st0)

This is followed by a list of directories to backup. Add more if you think you need them.

Try running this from the command line and then do a tar -xzvf /dev/st0 in a partition big enough to hold everything. Make sure it restores everything you think you need by poking around in the directories. Make sure you don't do this in / or you'll overwrite everything on your system with the backup so create a directory if you have to and then untar (tar -x) the backup there. You can zap this stuff once you're satisfied you've backed up everything you need. You also then have your first backup tape.

Finally, set up a cron job to run the backup. As root, do a "crontab -e". You'll get dumped into vi and you can edit the root crontab. You want to insert a line that looks something like:

0 1 * * 0 /bin/tar -czf /dev/st0 ...(the directories you want to backup)

This will run the backup a 0 minutes past 1:00 in the morning on Sunday. Check into the section 5 man pages for crontab if you want something different. Once you have this set up, all you have to do is remember to change tapes and clean the tape drive when it wants it.

Why don't you try http://www.arkeia.org it works wonderfull

I've used Arkiea and it does work great. The only problem I found with it is that when I tried using it on RH 9 it wouldn't work correctly. Since your using RH 8 though it shouldn't be a problem at all.

Another thing you can try is using the Webmin backup section to create your backup and scheduler without having to do it all manually. That was what I did when I couldn't get Arkiea to run on RH 9. The webmin backup utility is just a front end to dump I believe and it will edit cron to schedule. The email works through sendmail for successful jobs.

Which is something that Arcserve for Windows says it can do but I could never for the life of me get working. (Maybe its just me) So I was thrilled when Linux did everything right the first time.

Anyway just another idea.

Here is a backup script that I use. Just change /dev/rmt0 to your own backup device.

#!/bin/bash
#-------------------------------------------------------------------------
# Backup Script.
# Written : Stan Lovisa (slovisfaritec.co.za)
# Date : 18-01-2001
# Mod :
#-------------------------------------------------------------------------
PATH=/bin:/usr/bin:/etc:/usr/local/bin:/sbin:/usr/sbin
BLIST="/dbdump /home /usr /prod /train"
#BLIST="/home/stan"
JNL=/usr/local/logs/backup.jnl
LOG=/tmp/backup.log
REP=/tmp/backup.rep
ERR=/tmp/backup.err

MAIL_LIST="slovisa@mail.faritec.co.za "


echo "--------------------------------------------------------------" >> $JNL
echo "Start Backup on `date`" >> $JNL
echo "Start Backup on `date`" > $LOG
echo "Start Backup on `date`" > $ERR
echo "Start Backup on `date`" > $REP
echo "Files backup paths are :$BLIST" | tee -a $REP $JNL > /dev/null
#-----------------------------------------------------------------------
# Mail relevant users of backup status
notify()
{
pr -f $REP $ERR | mail -s "$REP & $ERR" $MAIL_LIST
}
#-----------------------------------------------------------------------
# Start backup according to BLIST
echo "`sdu -R $BLIST` files to be backed up" | tee -a $JNL $REP

find $BLIST -print | sort | backup -q -v -i -f /dev/rmt0 >> $LOG 2>> $ERR
EXIT_CODE=$?
if [ $EXIT_CODE != 0 ]
then
echo "Backup to tape NOT OK!: `date`" | tee -a $JNL $LOG $REP
echo "Exit code is : $EXIT_CODE" | tee -a $JNL $LOG $REP
notify
exit 1
fi

echo "Start read tape from Backup on `date`" | tee -a $JNL $LOG $REP

restore -Tvqf /dev/rmt0 >> $LOG 2>> $ERR
EXIT_CODE=$?
if [ $EXIT_CODE != 0 ]
then
echo "Read from tape NOT OK!: `date`" | tee -a $JNL $LOG $REP
echo "Exit code is : $EXIT_CODE" | tee -a $JNL $LOG $REP
else
echo "Tape read OK!: `date`" | tee -a $JNL $LOG $REP
fi

echo "End Backup on `date`" | tee -a $JNL $LOG $REP
notify
mt offline


Stan.

That's all awesome guys, I think I can take it from there. I'll let you all know if I need anything else in a new thread. Thanks a million!!!

Chris

Hardware Vs. Software compression
 

Hello All,

I have noticed a couple people recommending using Software compression.
In my experience with a similar Tape Drive (HP DLT1 40/80) I was able to store more information by relying on the Tape Drive's built in compression. The drive was trying to compress an already compressed file, and in some cases the files were getting inflated.
Also, by turning on Software Compression one of my CPU's was getting maxxed (running Dual P3 500mhz) and it was severly affecting the transfer rate to the Tape drive. It would take over 10 hours to perform a backup that should only take 4.
The other issue I had was unreliable data. Their were many times where I had to attempt a restore from tape 3-5 times before I would get a file that worked, and sometimes I was unable to recover a file.

Once I turned off software compression all my problems have gone away and I am getting about 82 GB each on an 40/80 GB Tape, and I have not had a problem restoring from tape since.

I am using BRU 17 (Backup Restore Utility) from Tolis Group. http://www.tolisgroup.com
This software is very easy to use. You have a choice of CUI or GUI.
I run it from a shell script launched by cron every day.

I have used BRU on Every Red Hat release since 5.2, including 9

Hope this helps.

Does anyone know if there is someway to do incremental backups directly to a CD-RW drive?

Or, how can I tar the entire /htdocs/images directory as well as /usr/local/mysql/data directory into one tar file. I then want to copy it over to a windows drive on the network. Thanks.

cen17394

OK I just started with bru 17, tape works no problems out of the box. Excellent.

What is not excellent is DVD backup or even getting xbru to see the same tape that bru happily backs up to!

Any experience of that? I would also like now and again to dump to a DVD.

Hi again, guys. Been a while... I had one other question regarding system backup. Since setting all this up, I have added an Apache web server (I'm learning it, HTML, general website building as a hobby). One thing I am doing is using php and MySQL with Apache. I am NEW at all 3, especially MySQL. By backing up my entire HDD to tape, is this successfully backing up the MySQL database?

Basiclly, I am backing up the entire system with: tar -czf /dev/st0 /

That is backing up EVERYTHING, right?

Another question is: What are the exact steps to get they system running again in case of HDD failure? For instance, do I buy a new HDD, boot it, and hook up the tape? Do I install RH, then just tar everything back onto the disk overtop of it? I don't know what to do. This server is absolutely critical, and I want to make sure that if I have a failure, I KNOW HOW to get it all back.

Let's say I have a failure. What do I do to restore it all, and will it be identical to what I had? I'm just looking to know what I need to do exactly in case of failure. Thank you!

Chris

I am backing up using BRU which I think is brilliant from the command line, haven't bothered with xbru. No problems at all, and yes it copies everything:-

examples:-

bru -cvZBOVR -L "Full-20031213" -f /dev/st0 /var/lib/mysql

This backs up entirely your mySQL database. BE AWARE in the switches O = to overwrite the tape!

You can use this command to put the backup anwhere:-

bru -cvZBOVR -L "Full-20031213" -f /dev/hde1 /var/lib/mysql
bru -cvZBOVR -L "Full-20031213" -f /dev/sda1 /var/lib/mysql

These are three commands I know work perfectly to Tape/Toshiba 2GB PCMCIA card.

To restore:-

bru -xvf /dev/st0 FULLY restores all mySQL backups.

bru -C -xvf /dev/st0 /var/lib/mysql/test/* PARTIAL restore of just the test database.

Hope this helps.

I am using SuSE distribution 8,2 kernel 2.4. Have a desktop and laptop.

I always only back up my /home, Mysqlmanager and Mysql directories. Bear in mind BRU copies the hidden files as well as default.

This week Desktop had a motherboard go south. So I had to use the laptop. I do a backup every day, so I lost maybe 1 hours work on the desktop.

All I did was do a BRU restore to the laptop and carry on working for the 3 days before I had my desktop back. Each day backup was done and then I did the BRU restore directly to Toshiba card on laptop - took it out into a pccard reader and BRU restore onto Desktop.

This works fine for MySQL with no loss at all.

So to your main question
What are the exact steps to get they system running again in case of HDD failure? For instance, do I buy a new HDD, boot it, and hook up the tape?

Don't rely on just TAR because it will fail. Modify what I have told you to back up everything to your tape.

As far as I am aware if your hdd fails, you just go get a new one. Install BRU from your CD, and then issue the RESTORE command. I have never done it that way, but I have faith that it would work.

I do not work for Tolis.

Thanks guys, that helps. What about a full system restore? What steps need to be taken to get the system back up and running? After all, if the HDD were to fail, and a new one is installed, HOW do I get all the stuff from tape back onto the system and working as if nothing had ever happened? The restore process is what I don't know... Never had to do it before.

Chris