Hey All,

After finally organizing the parts for a server build and tons of hassles, lol, I've decided to go the route of RAID 1 on a hardware setup rather than the software (mdadm). Here are the specs of the server rig:

- AMD Athlon 64 6000+ 3.1 Dual Core CPU
- 8GB (4x2GB) OCZ 800mhz RAM
- Dell 300W (maybe 350W) PSU
- MSI K9N SLI ready MOBO
- Radeon ATI HD 4650 Video Card (mobo has no on board video)
- 60GB IDE HDD (either Hitachi or Western Digital) for Debian OS
- 2 x 2TB Western Digital Black series 7,200 rpm SATA HDDs (for data storage and RAID 1 setup)
- VANTEC 2 Channel 4 Port SATA6 PCIE RAID Controller Card (http://microcenter.com/product/38922...PCIe_Host_Card)


Now, the questions...

1.) I've been told that going the hardware route would be more reliable for RAID rather than using the software RAID setup.. true?

2.) How difficult is it to setup a hardware RAID on Linux (In my case it will be on Debian Squeeze)?

3.) What types of issues will I run into during setup and after setup? ie: Will I have to mount the drives or anything everytime the system is restarted? Basically, how much of a hassle will it be to setup/run/maintain a hardware RAID 1 setup?


So far, my intentions for this server are:
1 - website hosting (my few personal websites, and maybe a few "pro bono" sites for friends/family members)

2 - website development / testing (php and mysql, etc.)

3 - file sharing on my home network (not looking to share online with strangers, just my own devices on the network)

4 - music and video streaming (not necessarily streaming from various online sources... I want to be able to stream my own music and video files that I've downloaded to devices on my own home network

5 - of course, security... I need all of this to be secured so no one can have access to my data outside of my home network. I don't want my server to download random files, or allow anyone outside of the home network to access the server and download random (potentially threatening) files to my server (I don't know if this one is possible anyway).

6 - once I have the OS installed and running smoothly, I plan to begin accessing the server remotely from main, everyday use, computer.

Basically it will be my media hub for all of my networked devices (laptops, ps3, phones, computers) so they can stream the files/data from the one machine, and I can have all of my data saved on one machine rather than a number of loose hard drives.

Thanks to all who can help!

The card you listed is not a RAID card at all, it is simple SATA controller card. So if you buy that you are still stuck with software RAID. True RAID controller cards with their own CPU and real hardware RAID (not fake RAID) are not available in that price class.
My recommendation: Go for software RAID, no need for that card, just use the SATA-ports on the mainboard.

Yeah, I've been seeing online that "true" RAID cards are at least around $100 for an OK brand. I was more focused on buying the 2TB Black hard drives, so when I asked about a RAID card and they said it's better I asked for a cheap one. I just let them give me that card and I assumed it was all good (I know... stupid me, lol. But, they've been quite helpful when it comes to questioning them about parts for my rigs). But, now that I'm looking at the package, it doesn't even say anything about RAID. I did tell them all I want to do is just have my data copied from 2TB Drive A to 2TB Drive B (just data, not the OS or it's settings and stuff) so they may have given me this card thinking it was sufficient enough for my needs.

I had a chance to actually do some in depth research on the issue over the last day or so while waiting for replies. I have decided to just give the software RAID a shot. It seems to have much more flexibility if the array dies or is corrupted. I don't like the idea that I'd have to rely on the one brand / exact same RAID card if my array fails. Also, mdadm seems to be quite stable in the linux community.

If I go the software route... would I able to pull one of the 2TB drives and hook it up externally (by USB to SATA adapter or dock) to a windows PC to access / pull the data? This would be the only thing stopping me from needing RAID. The OS will be on a separate drive and I just want to make sure my files, design work, music, movies, and such like that are constantly backed up, but I don't want to have to manually back them up to an external drive once a day. I wanted to use RAID 1 so it would handle it automatically. Then, I'd back it up to an external drive once a month or so...

mdadm is stable for everything, in my experience (tried RAID-5, RAID-1 and RAID0).
Regarding pulling out a disk and using it on a Windows computer, I never tried that, but theoretically it should work, as long as you use a Windows-readable file-system.

Keep in mind that a RAID is in no way a backup solution and in no way is a substitute for regular backups. Its sole purpose is to keep down-times due to hardware errors to a minimum. You can still loose your files, for example due to a bug in mdadm, the driver for the storage controller, the file-system-driver, hardware errors unrelated to the actual disks (like memory faults or a broken disk controller) or simply the most common: user error. If you delete a file on a RAID system it will be gone on both disks.

Yeah, this was my thoughts as well. I was just hoping to confirm it.

I understand that, basically, both disks are treated as one and whatever information is written to / removed from Drive A happens to Drive B. What confuses me, is people saying it is not a backup. If I'm copying all of my data on Disk A to Disk B, that seems like it is a "backup" scenario, lol. If one drive fails or seems to be becoming a problem, I still have all my data on the second drive... which is what I understand a "backup" to be. I would hope that a bug or system error (that doesn't affect the HDDs) wouldn't wipe my HDDs clean of the previous data if something occured?

What I'm trying to accomplish, is a way to back up my storage disk full of data in an automatic fashion so that when something new is added to the system (music, movie, documents...) it is also applied to the backup drive. I'd also like to be able to setup certain times (like once a day or week) for the system to automatically backup the storage disk to the backup disk. This way, if one disk fails, I have all of my data. But, what I DON'T want to do, is to have to manually setup / start a backup process everyday or every time I acquire new data. I thought RAID 1 was my answer to this scenario, but now I'm not so sure..

So basically:

60GB HDD = OS and packages (I don't care if this dies/goes down. Everything would be relatively easy to setup again.)
2TB WD BLACK HDD A = (Where all my data is stored. Music, Movies, websites, documents, pictures, etc.)
2TB WD BLACK HDD B = (Where all my stored data on HDD A is copied to and backed up. This way, if something happens to Drive A, I have everything on Drive B to put in it's place.)

Is this a scenario for RAID 1? Or is there another way I should go about doing this? I don't want to use an external drive because I'll get better write times if they are internal. I will get a 3TB external in the next few months to do periodic "off site" backups for me and clients (I fix some friends and family computers).

A backup means: If you accidentally remove an important file from your main disk you can still recover it from your backup disk. This is absolutely not the case with RAID-1, if you remove the file it will be lost on both disks.

I would recommend to use a separate disks solution: You store your files on disk A and from time to time a script will mount the second disk, sync the disks and unmount it again. This can easily be setup using rsync and cron and you can determine how often that should happen.

That is what I would prefer. I don't really see the need for redundancy if I don't care about the OS disk. Is there a way to set these up so that when I save a new version a file or just add a new file in general the server will see the write request, then mount disk B and copy the file to it, then unmount it?

Also now I'm wondering, if I will not be doing RAID, if I actually need the WD Black drives? I can get two Seagate 2TB drives for like $100 a piece and it will save me like $120. Although I haven't had any problems, a lot of people do not reccommend Seagate Drives.. Should I go with cheaper drives if I'm just setting up a backup solution rather than RAID? I know WD Blacks are quite nice, but the only reason I bought them was because they are basically the only drives (besides the NEW Red series) reccommended for consumer based RAID setups.

While this is possible I wouldn't recommend it, this can cause some serious slowdowns if you copy a large amount of files to the disk. A time-based solution is more advisable, for example syncing the disks every ten minutes.

If you don't use a RAID you possibly don't want to buy drives specificly aimed for RAID, but keep in mind that this type of server usually runs 24/7, so you may want to buy high quality disks. regarding the manufacturers, I never have seen that a specific manufacturer's disks are less reliable than others. usually people that don't recommend a specific manufacturer had only one disk from that manufacturer and it failed, so they conclude that the manufacturer in general must be bad, which is of course not the case.

Sorry, I may have confused you. Let's say I have the two disks setup identically. Both have the same files / data on them. Now I go to my Photoshop and design a new graphic. I would save this new graphic file (a .jpg) to Drive A. Then I want Drive B to mount, sync with A, then unmount. This would happen for all new files created and all old files that are edited and saved.

If this is possible, wouldn't it be better to do this rather than sync the entire disks every ten minutes? When the syncing starts, won't it copy all data on Drive A to Drive B? What I'm thinking is that I'd rather it sync a single (or few files) at 1GB when I save them rather than sync 1.2TB of data every ten minutes?

Or, am I wrong, and the sync every 10 minutes only adds the new files it finds? Let's say I have 10GB of data that is identical on both drives. Then, I download / create a new file that is 1GB and save it to Drive A making its total 11GB while Drive B is still 10GB. When the sync kicks in at the next 10 minute mark, does Drive B only copy the missing 1GB of data from Drive A or will it copy ALL (11GB) of data from Drive A?

Yeah, the Black series are for "high performance" desktop setups and are ABLE to handle the consumer based RAID setups (according to WD). It seems like if you're just running a RAID 0 or 1, you should be fine with the Black drives. If you want to run a RAID 5, 10, or another level they "reccommend" you buy their RE (RAID Edition) drives. From chatting with a couple friends, Micro Center, and online searching, this is the conclusion I've come to. Since the server is planned to run 24/7 (as I will be hosting my personal websites) I think I may keep the WD Blacks because they are a higher quality disk. Plus, they have the best Manufacturer warranty out there (it seems... 5 years).

(Also, I just talked with a tech at Micro Center about what I'm trying to accomplish, and he said as far he knew I want to setup a RAID 1 array. He told me to talk to their "knowledge bar" techs (who does the repair services and such) but my lunch was running too late and I couldn't wait for them. I plan to go back after work I think..)

If you use programs like rsync (which I would recommend) only the files that are new or changed will be synced. Even better, if you have only made slight changes that only affect a few KB in a file that is a few GB large only the changed portion will be copied.

This sounds good, and if it's a more secure way for backing up, it's what I want, lol. As long as it isn't trying to sync my whole 1 - 1.5TB of data every 10 minutes (which I'm sure would cause problems). If it is only syncing new and edited files every 10 minutes, that is what I'm aiming for.

My next question (if you have any ideas on the matter) is how much power will it take from my other programs running to do the syncs and cause my system to run sluggishly? For example, if I have my computer sync once an hour and one of these syncs will need to copy multiple GBs of data... how badly will it affect / slow down my other processes like streaming video to a couple TVs, listening to music on the laptop, then the running web and file servers? (Assume all of these processes are running at the same time and then the sync kicks in at it's hour mark).

Actually I don't have any data on that, but I doubt that you will notice that, since your other services should be limited by the network rather than the disk speed.

True. I was wondering more about CPU/Processor speed..


Also, how would I go about partitioning my drives if I do not go RAID? The plan was to install EVERYTHING for the OS on the 60GB drive. Then through CLI I would setup the two 2TB drives and then mount my /home partition onto the 2 drives' RAID array. Will the syncing/backup solution be similar?

It is quite similar, put your OS on the 60GB disk, mount one of the disks under /home, then let the sync-script mount the other disk under /mnt/backup or something similar and unmount it after the work is done.

EDIT: I thought about that and came to the conclusion that it would be better to not mount the disk as /home, since you will format it with a Windows file-system, which knows nothing about Linux file permissions. It would be better to use a generic mountpoint, like /mnt/data.

Makes sense... but, is there a reason I should format as a Windows file system? I was just going to format it as ext4... will this cause issues trying to plug it in externally to a windows machine?