I am building a home server that will host a multitude of files; from mp3s to ebooks to FEA software and files. I don't know if RAID is the right thing for me. This server will have all the files that I have accumulated over the years and if the drive fails than I will be S.O.L.

I have seen discussions where someone has RAID 1 setup but they don't have their drives internally (to the case), they bought 2 separate external hard drives with eSata to minimize an electrical failure to the drives. (I guess this is a good idea)

I have also read about having one drive then using a second to rsync data every week.

I planned on purchasing 2 enterprise hard drives of 500 MB to 1 GB but I don't have any experience with how I should handle my data.

Any help and advice would be much appreciated!

MT

well yes, you'd want raid. That's not really much of a dilemma is it? The issue is more likely what kind of raid, hardware of software. IF you buy an external caddy that supports raid then there's nothing at all to configure on the linux side. if you do want to do it with separate normal drives, then the installer will more than likely allow you to configure it in a few seconds.

I was leaning to RAID 0 but didn't know about this rsync deal. I was thinking about getting something like two: Rosewill RX-358-S Hard Drive enclosures to do the raid (it will be software based). I guess it would be wise to get another drive and do an off-site backup also.

Thanks for your advice

MT

What you are looking for is a backup solution, not redundancy, which is what all RAID levels offer, with the exception of RAID 0. Redundancy means your server will continue working if a disk dies, but it is not really a backup. RAID has more to do with the availability of data than the backup of data.

For my home server I use rsync in a cron job, to synchronise my main data disk to another disk. The command is simple:

And if you want to mirror your primary data disk, deleting stale files in your backup disk:

Both disks are internal - in the event of a fire or flood my backup would be useless to me but I have to balance other concerns. I have never found external, USB-connected disks reliable - or rather, I have never found the power supply to these disks reliable. A good-quality internal PSU is far better than any cheap power adapter you get with external USB disks. And with the amount of data I need to backup DVDs are just impractical. DAT tape is good but the drives are expensive.

I tend to buy disks of equal size from different manufacturers, so that if there is a manufacturing defect in one I am unlikely to suffer a catastrophic failure of both disks at the same time. Believe it or not it has happened. The Maxtor 40GB was a disaster and the old IBM Deskstar as well. Something like a 750GB 5400rpm EcoGreen Samsung and a 750 EcoGreen Western Digital would be fine.

How often do you perform rsync?

Thanks for your thoughts!

MT

I seem to have sounded pretty dismissive about other ways to store data... keeping archives of certain data is certainly important as well, but 500gb I wouldn't expect to want to back up, just keep available. I back up things like photos to a remote location (jungledisk in fact) but that's only a few gig. For covering against accidental deletion of files, I would just use an lvm snapshot to periodically take a "backup" without the need for additional storage space. With that on a raid 1 setup you cover both user error and device failure.

With raid caddies I assume it's possible to just remove one of the drives and that implictly becomes a backup, to store in the garage incase the house burns down etc...

+1 on jungledisk for off site backup. Even works with headless servers and hard to beat the price. A mirrored raid will increase the up time of your system as others have pointed out, but can also copy errors from one drive to the other which means a corruption can hit both drives.

I have a cron job set to run every hour, and I occasionally run it manually as well.

It really depends on how much your data changes hour-by-hour and how much you can afford to lose. If it's important, back it up, even if that means running this command in a script every twenty minutes.

RAID is not Backup!
RAID protects you against hardware failures. To a degree.
Because it adds additional complexity, it also introduces more potential failures.
Disks fail, it's not a question of "if" but of "when", so not having RAID would be foolish, but having RAID and not making regular backups would be just as foolish. One doesn't go without the other.

pj

Since this is a server and you are planning to put all of your data on it RAID wouldn't be a bad idea. Especially considering the low cost of hard drives these days, and the ability to do software RAID in Linux there is really very little reason to not use it. There is some added complexity with RAID but it is minimal, and realistically these complexities are just during initial setup and when a drive fails. Most modern distros include support for setting up RAID using the installer, so the only real concern would be a rebuild when a failure occurs.

Also for a general setup there is no real reason to use an external system for the drives if you have space in your case. In fact it brings in more room for problems (for example a cable being unplugged accidentally). If everything is in the case you really only have main power to the system to worry about.

As mentioned though RAID is not a replacement for backup. You will need to make some decisions on how to backup the RAID, and if you want/need to have everything or just selected directories.

The use of Raid 1 (mirrored) drives can be used as a backup and disaster recovery strategy.
You purchase 3 drives labelled 1 2 & 3 and use an external caddie for your second drive. At regular backup intervals you fail and remove the drive in the caddie (you can do this simply by shutting down the machine and removing the drive from the caddie if it's not a hotswap sata device). Replace the second drive with the third then add and resync. Place the second drive in a secure location. At the next backup remove the third drive and replace it with the second and so forth. This way if a catastrophic disaster happens you can boot the spare drive and be back online within minutes. I've been using this technique now for 6 years to provide redundancy to my customers. The added advantage is that on failure you do not rebuild your linux distribution which could take days. By using software raid the solution is flexible and the resync time does not impact significantly upon the computers performance.