1) What's your preferred way of backing up? Do you back the entire OS or just the configuration and data and then try do a reinstallation into a new copy of the OS?

2) Tapes are pretty slow right? How does that affect the backup process when the system is live and a few dozen users are logged in?

I need to reinitiate tape back ups ASAP on these servers that have fallen under my control, but I'm not exactly sure how to do it. I'm only a transitional system manager (though if I can figure it out, enough stuff before they hire a replacement, I'm definitely going to apply for the job).

The tape size is 160 GB and I have 180 GB of total data. 130GB of it is user files spread across 3 drives (u1, u2, u4).

Any help would be appreciated.

Personally I hate tape for backups.. so time consuming to keep track of and constantly swap tapes. Depending on how much data their is to backup a tape library may be required. then theres managing off-site tape storage, tape rotation schedules, replacing media, cleaning drives.. Ughh...

I do all my system backups to disk, with one system in house (for quick restores) and a second system off-site for Disaster Recovery situations. The system I use makes a full backup of the system, after that point it only backs up changes to the system (functions similar to rsync). I can do point of time restores daily for a week (7), weekly for a month (4), monthly for a year (12) and never have to leave my desk. Flyback uses a backup method, which is very similar to the commercial system I currently use.

Creating a complete image of your system, then doing regular backups, would probably make for the best solution to handle a Bare-Metal-Restore quickly.

Here are a few options to look at.
mondorescue
backuppc
systemimager
flyback


As for your tape size you may be all right with compression. I am assuming the size you listed for your data is the uncompressed size. once the data is compressed it should be much smaller.

The server I have with the most data is currently at 1.3 TB and takes only 10 minutes to backup every night. (SQL Server)
The system with the most data changing daily (file and print server) takes a Half hour for a full backup each night.
All together I am backup up 12 servers to disk in a 2 hour window each night.
I don't believe you can touch those times, or that number of restore points, easily with a tape backup system.

Not to mention file restores....
I can restore any file, from any of the restore points I mentioned (roughly 23 restore points) for any system and NEVER have to search for a tape..

That sounds very good although we're just a small op and have never bothered with off site storage. Even so, back up to disk rather than tape is definitely the kind of thing I will be aiming for as time goes by. I might still do a tape back up once a month just to have a second layer (and maybe store those off site).

My old boss was using Bacula which can back up to disk but something went wrong and the Bacula backup of the 180 GB system (that's uncompressed) quickly ate up all the storage space on a disk array with 12 250 GB disks (3 TB). Even with Raid 1 that's 1.5 TB. I guess Bacula isn't in your list for a reason! Maybe my boss just configured it badly and had it creating complete copies every night until it ran out of disk space in a week.

So I actually already have the hardware that can handle that sort of thing already. I'm just not ready (i.e. qualified) to set it up yet. I would think that doing tape back ups manually would be a lot simpler for now and at least give me something to fall back on while I get up to speed. If you can offer any advice for using tapes while for the moment, I'd appreciate it.

In other words, I could actually use a specific tutorial (know any good links?). For example, put a tape in the drive, details on how to mount the tape, example commands for tarballing a directory/drive with the output copied to the tape... and any tips on the finer points of tape usage. I'm really pretty lost here!

As long as there are no disasters for a few months while I figure all this out I'll be alright.

Tape still has it's place and purpose but I'd agree, go with disk based if you can. Don't count out Amanda and Bacula in that list, both can handle disk based storage and backups as well.

But I don't agree with swapping of tapes all the time, that's why you buy a tape drive with a robot in it. Last tape library I dealt with had 4 robots, 12 LTO3 drives (could handle up to 64) and held 1500 LTO3 tapes at any given time. We even had two cameras inside to view remotely to make sure the robots were working properly if we got any errors reported about tapes, etc.

The main thing I like about tape though is it is easy to make secure offsite copies to archive data instead of piping it through a VPN tunnel or the like, over the network, etc.

If you really want to get fancy with backups and disk storage, look into something like DataDomain's products. It's disk based storage that uses deduping technology. You essentially cram like 40TB of data into a 4-8TB disk appliance with their deduping technology.

Also as tapes being slow, the actual write time is actually faster than most hard disks available. The only thing that slows them down is the time to load, rewind/forward, eject, etc.

LTO4 can write 120MB/s. Most systems can't even keep up with it's max speed when you involve disk, network, etc. But it really just depends on what type of tape technology you go with.

Also depending on your backup data, with LTO and most others, you'll get a better compression ratio than disk based drives using tar and gzip, etc. At the previous employer where I handled tapes, we were fitting close to 2TB of Oracle database data on one LTO3 tape with Veritas Netbackup. I think these were actually rated to only hold an average of 400GB native and 800GB compressed data.

I back up everything except such obvious throw-aways as /tmp and browser caches. The problem with a complete backup on tape is that you end up with a huge blob. This is OK for complete restores but complete restores are rare. A typical restore is a few files or a directory with its contained filed. Tapes are horrible when you try to do partial restores. I strongly prefer using hard drive, DVD, CD, or even floppy where you can backup a file tree in a form that can be browsed and selectively restored.

Any complete backup consumes a lot of time and resources. No matter what backup media you use you can save considerable time and money by using incremental backups. The first backup you back up everything. On subsequent backups you only back up the files and directories that have changed.

----------------------
Steve Stites

P.S. When people face the backup problem for the first time their first reaction is to back up everything into one huge blog and then they are protected no matter what happens. As they become more experienced they start considering other things like generations of backup. Do you want to restore to 5 minutes ago, a day ago, before the last OS install, or when? How much backup space do you have for how many generations of backup? trickykid mentioned compression. Compressing a tape uses less tape, the tape runs faster, and the compression soaks up a lot of CPU time. Which is your scarcer resource, backup space or CPU time?

What about partial restores? Is your backup method amenable to finding and restoring the secretary's letter to the SEC which she lost yesterday?

Thanks for all the info.

So can someone outline the basic process of backing up 180 GB to a simple tape drive with 160 GB of tape?

I mean... do I just pop a tape in the drive and go:


tar cvzf /dev/TAPE/u1.tar.gz /u1

Well, that would be the most basic way. You could take the suggestions of using backup tools to handle the backups automatically.

Is this 180GB native or already compressed? What type of tape drive is this?

It's 180GB native spread across 7 partitions or so.

The drive type is a "Quantum DLT-V4" with "HP DLT Tape VS1 160 GB". I have some tapes for a newer drive that say 400 GB with a note (assuming 2:1 compression) so they're really 200 GB. I don't know how the hell they get away with that. Anyway, those won't work in this drive.

I just looked it up, and it is indeed only an honest 80 GB. So I have 180. With compression slightly greater than 2:1 it will fit on two tapes (if I can partition it neatly). Well, I'll use 3 tapes if I have to.

I just have to figure out how a tape drive works.

Believe me, I want to use good back up tools in time, and I will revisit this post in the future. I'm very appreciative of all that information.

However, I am desperately in need of a simple low-tech back up solution for the moment as I have a million other things to do and don't have time to configure new software and hardware at the moment. Thanks!

Yeah, it's always safe to assume their compression ratios. Like I said previously, we were almost getting 4:1 with LTO3 and Oracle database data, which has a lot of duplicated data. Databases get really great compression ratios.

As for just the quick simple solution, yeah, a quick tar and gzip command to dump the data to tape is the quickest way until you explore other options and maybe better solutions.

Ok thanks. And forget the questions I just edited out, I need to stop this laziness when you people are being so helpful, I need to figure out for myself what I can.

I found this page: http://nic.phys.ethz.ch/readme/80

It goes over tape commands. Can I expect this to be about the same as my tape? I think I can experiment with those commands using our oldest tapes and eventually make the back-ups I need.

I only have one more concern. Is it normal to back up a 40 GB directory + subdirectories while users are logged in and using the files? I would expect it is otherwise you couldn't back up without disconnecting everyone first.

If there's no other pitfalls... I think I'm going to initiate make my first back ups tomorrow. That will be a huge relief.

That link looks pretty accurate.

As for users, sometimes backing up a file while it's open isn't a good way to backup. Always try to run your backups when the systems not in use. You could always get fancy in writing your backup scripts to check to make sure the files aren't open, if they are, skip them and attempt to back them up the next round.

Unless I want to come in at midnight once a week and issue a temporary down time notice once a week and unplug the network cable, I don't see how I can accomplish that. I mean, the email server alone is probably getting new email a few times a minute at least (maybe slightly less often but still).

If I skip open files, then they would be skipped entirely for the whole week? (and how do you do make the tar program do that using a script?)

What's the worst that would happen if I just tried to store with some files open? Some of those open files might be corrupted on the tape?

I want to try to back up tomorrow after 4:30, but I'd prefer not to have to take the server off-line while I do it.

If you had to restore such a file it might not make sense to the application program. For example if you backed up a payroll file while payroll was running and then later restored it the file might show that some people were paid on the latest payday and other people weren't.

-------------------
Steve Stites