LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices

Reply
 
Search this Thread
Old 08-14-2008, 12:53 PM   #1
davidstvz
Member
 
Registered: Jun 2008
Posts: 405

Rep: Reputation: 30
Questions about Tape Backup for Sys Admins


1) What's your preferred way of backing up? Do you back the entire OS or just the configuration and data and then try do a reinstallation into a new copy of the OS?

2) Tapes are pretty slow right? How does that affect the backup process when the system is live and a few dozen users are logged in?

I need to reinitiate tape back ups ASAP on these servers that have fallen under my control, but I'm not exactly sure how to do it. I'm only a transitional system manager (though if I can figure it out, enough stuff before they hire a replacement, I'm definitely going to apply for the job).

The tape size is 160 GB and I have 180 GB of total data. 130GB of it is user files spread across 3 drives (u1, u2, u4).

Any help would be appreciated.
 
Old 08-14-2008, 01:26 PM   #2
farslayer
Guru
 
Registered: Oct 2005
Location: Willoughby, Ohio
Distribution: linuxdebian
Posts: 7,228
Blog Entries: 5

Rep: Reputation: 189Reputation: 189
Personally I hate tape for backups.. so time consuming to keep track of and constantly swap tapes. Depending on how much data their is to backup a tape library may be required. then theres managing off-site tape storage, tape rotation schedules, replacing media, cleaning drives.. Ughh...

I do all my system backups to disk, with one system in house (for quick restores) and a second system off-site for Disaster Recovery situations. The system I use makes a full backup of the system, after that point it only backs up changes to the system (functions similar to rsync). I can do point of time restores daily for a week (7), weekly for a month (4), monthly for a year (12) and never have to leave my desk. Flyback uses a backup method, which is very similar to the commercial system I currently use.

Creating a complete image of your system, then doing regular backups, would probably make for the best solution to handle a Bare-Metal-Restore quickly.

Here are a few options to look at.
mondorescue
backuppc
systemimager
flyback


As for your tape size you may be all right with compression. I am assuming the size you listed for your data is the uncompressed size. once the data is compressed it should be much smaller.

The server I have with the most data is currently at 1.3 TB and takes only 10 minutes to backup every night. (SQL Server)
The system with the most data changing daily (file and print server) takes a Half hour for a full backup each night.
All together I am backup up 12 servers to disk in a 2 hour window each night.
I don't believe you can touch those times, or that number of restore points, easily with a tape backup system.

Not to mention file restores....
I can restore any file, from any of the restore points I mentioned (roughly 23 restore points) for any system and NEVER have to search for a tape..
 
Old 08-14-2008, 02:00 PM   #3
davidstvz
Member
 
Registered: Jun 2008
Posts: 405

Original Poster
Rep: Reputation: 30
That sounds very good although we're just a small op and have never bothered with off site storage. Even so, back up to disk rather than tape is definitely the kind of thing I will be aiming for as time goes by. I might still do a tape back up once a month just to have a second layer (and maybe store those off site).

My old boss was using Bacula which can back up to disk but something went wrong and the Bacula backup of the 180 GB system (that's uncompressed) quickly ate up all the storage space on a disk array with 12 250 GB disks (3 TB). Even with Raid 1 that's 1.5 TB. I guess Bacula isn't in your list for a reason! Maybe my boss just configured it badly and had it creating complete copies every night until it ran out of disk space in a week.

So I actually already have the hardware that can handle that sort of thing already. I'm just not ready (i.e. qualified) to set it up yet. I would think that doing tape back ups manually would be a lot simpler for now and at least give me something to fall back on while I get up to speed. If you can offer any advice for using tapes while for the moment, I'd appreciate it.

In other words, I could actually use a specific tutorial (know any good links?). For example, put a tape in the drive, details on how to mount the tape, example commands for tarballing a directory/drive with the output copied to the tape... and any tips on the finer points of tape usage. I'm really pretty lost here!

As long as there are no disasters for a few months while I figure all this out I'll be alright.
 
Old 08-14-2008, 02:50 PM   #4
trickykid
Guru
 
Registered: Jan 2001
Posts: 24,133

Rep: Reputation: 197Reputation: 197
Tape still has it's place and purpose but I'd agree, go with disk based if you can. Don't count out Amanda and Bacula in that list, both can handle disk based storage and backups as well.

But I don't agree with swapping of tapes all the time, that's why you buy a tape drive with a robot in it. Last tape library I dealt with had 4 robots, 12 LTO3 drives (could handle up to 64) and held 1500 LTO3 tapes at any given time. We even had two cameras inside to view remotely to make sure the robots were working properly if we got any errors reported about tapes, etc.

The main thing I like about tape though is it is easy to make secure offsite copies to archive data instead of piping it through a VPN tunnel or the like, over the network, etc.

If you really want to get fancy with backups and disk storage, look into something like DataDomain's products. It's disk based storage that uses deduping technology. You essentially cram like 40TB of data into a 4-8TB disk appliance with their deduping technology.
 
Old 08-14-2008, 02:55 PM   #5
trickykid
Guru
 
Registered: Jan 2001
Posts: 24,133

Rep: Reputation: 197Reputation: 197
Also as tapes being slow, the actual write time is actually faster than most hard disks available. The only thing that slows them down is the time to load, rewind/forward, eject, etc.

LTO4 can write 120MB/s. Most systems can't even keep up with it's max speed when you involve disk, network, etc. But it really just depends on what type of tape technology you go with.

Also depending on your backup data, with LTO and most others, you'll get a better compression ratio than disk based drives using tar and gzip, etc. At the previous employer where I handled tapes, we were fitting close to 2TB of Oracle database data on one LTO3 tape with Veritas Netbackup. I think these were actually rated to only hold an average of 400GB native and 800GB compressed data.
 
Old 08-14-2008, 03:25 PM   #6
jailbait
Guru
 
Registered: Feb 2003
Location: Blue Ridge Mountain
Distribution: Debian Wheezy, Debian Jessie
Posts: 7,527

Rep: Reputation: 177Reputation: 177
Quote:
Originally Posted by davidstvz View Post
1) Do you back the entire OS or just the configuration and data and then try do a reinstallation into a new copy of the OS?
I back up everything except such obvious throw-aways as /tmp and browser caches. The problem with a complete backup on tape is that you end up with a huge blob. This is OK for complete restores but complete restores are rare. A typical restore is a few files or a directory with its contained filed. Tapes are horrible when you try to do partial restores. I strongly prefer using hard drive, DVD, CD, or even floppy where you can backup a file tree in a form that can be browsed and selectively restored.

Quote:
Originally Posted by davidstvz View Post
2) Tapes are pretty slow right? How does that affect the backup process when the system is live and a few dozen users are logged in?
Any complete backup consumes a lot of time and resources. No matter what backup media you use you can save considerable time and money by using incremental backups. The first backup you back up everything. On subsequent backups you only back up the files and directories that have changed.

----------------------
Steve Stites

P.S. When people face the backup problem for the first time their first reaction is to back up everything into one huge blog and then they are protected no matter what happens. As they become more experienced they start considering other things like generations of backup. Do you want to restore to 5 minutes ago, a day ago, before the last OS install, or when? How much backup space do you have for how many generations of backup? trickykid mentioned compression. Compressing a tape uses less tape, the tape runs faster, and the compression soaks up a lot of CPU time. Which is your scarcer resource, backup space or CPU time?

What about partial restores? Is your backup method amenable to finding and restoring the secretary's letter to the SEC which she lost yesterday?

Last edited by jailbait; 08-14-2008 at 03:37 PM.
 
Old 08-14-2008, 03:26 PM   #7
davidstvz
Member
 
Registered: Jun 2008
Posts: 405

Original Poster
Rep: Reputation: 30
Thanks for all the info.

So can someone outline the basic process of backing up 180 GB to a simple tape drive with 160 GB of tape?

I mean... do I just pop a tape in the drive and go:


tar cvzf /dev/TAPE/u1.tar.gz /u1
 
Old 08-14-2008, 03:29 PM   #8
trickykid
Guru
 
Registered: Jan 2001
Posts: 24,133

Rep: Reputation: 197Reputation: 197
Quote:
Originally Posted by davidstvz View Post
Thanks for all the info.

So can someone outline the basic process of backing up 180 GB to a simple tape drive with 160 GB of tape?

I mean... do I just pop a tape in the drive and go:


tar cvzf /dev/TAPE/u1.tar.gz /u1
Well, that would be the most basic way. You could take the suggestions of using backup tools to handle the backups automatically.

Is this 180GB native or already compressed? What type of tape drive is this?
 
Old 08-14-2008, 03:53 PM   #9
davidstvz
Member
 
Registered: Jun 2008
Posts: 405

Original Poster
Rep: Reputation: 30
Quote:
Originally Posted by trickykid View Post
Well, that would be the most basic way. You could take the suggestions of using backup tools to handle the backups automatically.

Is this 180GB native or already compressed? What type of tape drive is this?
It's 180GB native spread across 7 partitions or so.

The drive type is a "Quantum DLT-V4" with "HP DLT Tape VS1 160 GB". I have some tapes for a newer drive that say 400 GB with a note (assuming 2:1 compression) so they're really 200 GB. I don't know how the hell they get away with that. Anyway, those won't work in this drive.

I just looked it up, and it is indeed only an honest 80 GB. So I have 180. With compression slightly greater than 2:1 it will fit on two tapes (if I can partition it neatly). Well, I'll use 3 tapes if I have to.

I just have to figure out how a tape drive works.

Last edited by davidstvz; 08-14-2008 at 03:56 PM.
 
Old 08-14-2008, 03:57 PM   #10
davidstvz
Member
 
Registered: Jun 2008
Posts: 405

Original Poster
Rep: Reputation: 30
Quote:
Originally Posted by trickykid View Post
Well, that would be the most basic way. You could take the suggestions of using backup tools to handle the backups automatically.

Is this 180GB native or already compressed? What type of tape drive is this?
Believe me, I want to use good back up tools in time, and I will revisit this post in the future. I'm very appreciative of all that information.

However, I am desperately in need of a simple low-tech back up solution for the moment as I have a million other things to do and don't have time to configure new software and hardware at the moment. Thanks!
 
Old 08-14-2008, 04:06 PM   #11
trickykid
Guru
 
Registered: Jan 2001
Posts: 24,133

Rep: Reputation: 197Reputation: 197
Yeah, it's always safe to assume their compression ratios. Like I said previously, we were almost getting 4:1 with LTO3 and Oracle database data, which has a lot of duplicated data. Databases get really great compression ratios.

As for just the quick simple solution, yeah, a quick tar and gzip command to dump the data to tape is the quickest way until you explore other options and maybe better solutions.
 
Old 08-14-2008, 04:15 PM   #12
davidstvz
Member
 
Registered: Jun 2008
Posts: 405

Original Poster
Rep: Reputation: 30
Quote:
Originally Posted by trickykid View Post
Yeah, it's always safe to assume their compression ratios. Like I said previously, we were almost getting 4:1 with LTO3 and Oracle database data, which has a lot of duplicated data. Databases get really great compression ratios.

As for just the quick simple solution, yeah, a quick tar and gzip command to dump the data to tape is the quickest way until you explore other options and maybe better solutions.
Ok thanks. And forget the questions I just edited out, I need to stop this laziness when you people are being so helpful, I need to figure out for myself what I can.

I found this page: http://nic.phys.ethz.ch/readme/80

It goes over tape commands. Can I expect this to be about the same as my tape? I think I can experiment with those commands using our oldest tapes and eventually make the back-ups I need.

I only have one more concern. Is it normal to back up a 40 GB directory + subdirectories while users are logged in and using the files? I would expect it is otherwise you couldn't back up without disconnecting everyone first.

If there's no other pitfalls... I think I'm going to initiate make my first back ups tomorrow. That will be a huge relief.

Last edited by davidstvz; 08-14-2008 at 04:49 PM.
 
Old 08-14-2008, 04:54 PM   #13
trickykid
Guru
 
Registered: Jan 2001
Posts: 24,133

Rep: Reputation: 197Reputation: 197
Quote:
Originally Posted by davidstvz View Post
Ok thanks. And forget the questions I just edited out, I need to stop this laziness when you people are being so helpful, I need to figure out for myself what I can.

I found this page: http://nic.phys.ethz.ch/readme/80

It goes over tape commands. Can I expect this to be about the same as my tape? I think I can experiment with those commands using our oldest tapes and eventually make the back-ups I need.

I only have one more concern. Is it normal to back up a 40 GB directory + subdirectories while users are logged in and using the files? I would expect it is otherwise you couldn't back up without disconnecting everyone first.

If there's no other pitfalls... I think I'm going to initiate make my first back ups tomorrow. That will be a huge relief.
That link looks pretty accurate.

As for users, sometimes backing up a file while it's open isn't a good way to backup. Always try to run your backups when the systems not in use. You could always get fancy in writing your backup scripts to check to make sure the files aren't open, if they are, skip them and attempt to back them up the next round.
 
Old 08-14-2008, 05:09 PM   #14
davidstvz
Member
 
Registered: Jun 2008
Posts: 405

Original Poster
Rep: Reputation: 30
Quote:
Originally Posted by trickykid View Post
That link looks pretty accurate.

As for users, sometimes backing up a file while it's open isn't a good way to backup. Always try to run your backups when the systems not in use. You could always get fancy in writing your backup scripts to check to make sure the files aren't open, if they are, skip them and attempt to back them up the next round.
Unless I want to come in at midnight once a week and issue a temporary down time notice once a week and unplug the network cable, I don't see how I can accomplish that. I mean, the email server alone is probably getting new email a few times a minute at least (maybe slightly less often but still).

If I skip open files, then they would be skipped entirely for the whole week? (and how do you do make the tar program do that using a script?)

What's the worst that would happen if I just tried to store with some files open? Some of those open files might be corrupted on the tape?

I want to try to back up tomorrow after 4:30, but I'd prefer not to have to take the server off-line while I do it.
 
Old 08-14-2008, 05:47 PM   #15
jailbait
Guru
 
Registered: Feb 2003
Location: Blue Ridge Mountain
Distribution: Debian Wheezy, Debian Jessie
Posts: 7,527

Rep: Reputation: 177Reputation: 177
Quote:
Originally Posted by davidstvz View Post

What's the worst that would happen if I just tried to store with some files open? Some of those open files might be corrupted on the tape?
If you had to restore such a file it might not make sense to the application program. For example if you backed up a payroll file while payroll was running and then later restored it the file might show that some people were paid on the latest payday and other people weren't.

-------------------
Steve Stites
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Questions for sys admins bhert Linux - General 5 07-15-2007 03:13 PM
Using a DDS5 tape drive to restore from a DDS3 backup tape. AndrewCAtWayofthebit Linux - Hardware 1 05-14-2006 09:15 AM
newbie tar/tape backup questions mattl Fedora 3 04-24-2006 10:07 PM
Doin a weekly tape backup - multiple questions - VERY important - Please help WorldBuilder Linux - General 10 04-25-2003 03:12 PM
2 questions(Re: backup to scsi tape drive) ascii2k Linux - General 5 07-16-2001 05:47 PM


All times are GMT -5. The time now is 10:42 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration