LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware
User Name
Password
Linux - Hardware This forum is for Hardware issues.
Having trouble installing a piece of hardware? Want to know if that peripheral is compatible with Linux?

Notices



Reply
 
Search this Thread
Old 03-20-2005, 02:02 PM   #1
omega71122
Member
 
Registered: Mar 2005
Location: Central Ohio, USA
Distribution: Fedora 7 x86_64 (home server), IPCop (home gateway), CentOS (work laptop), lots more at work.
Posts: 32

Rep: Reputation: 15
SATA RAID Support


I'm looking into building a file server for my home network using two SATA RAID 0+1 arrays. I've been doing some research, looking for SATA controllers that would work well under Linux. If anyone has any suggestions, I'd appreciate them. Also, if anyone has an opinion as to the best distro for my particular server, I'd appreciate that as well.

Basically, I want it to do the following:
Gigabit DHCP Server (to the other computers in my LAN, plus to my Xbox[10/100])
VPN Server
SAMBA Server (the other computers on my network will most likely run XP Pro)
FTP Server

It needs to have the following:
Gigabit network connection (to my internal network)
10/100 network connection (to my cable modem)
Support for two SATA RAID 0+1 arrays (300GB and 200GB for file storage) and a ATA/133 RAID 0+1 array (probably 80GB for the OS, server-side software and swap partitions).

I'm probably going to be using a Pentium 4 platform (unless an AthlonXP/Athlon 64 based platform would be better for my server, opinions are appreciated). For the RAID, I'll probably use PCI controller cards (unless I can find a mobo with at least one SATA RAID array onboard).

I've been reading about what can happen when a drive fails (that it can take out the bus it's on rendering the other drive(s) on that bus inoperable). If it were to take out that bus, is it possible for the other drive be recovered by installing a new controller card? An idea I had was to have a pair of 2 port controller cards and put one drive from each array on each card (ie - a 300GB and 200GB disk on each card) and rebuild the array from the other bus using software RAID. Could that (in theory) keep my data safe in the event that a drive fails and takes that bus with it?

Thanks everyone.

PS - If my concepts are wrong, PLEASE let me know. I'm more or less a newb when it comes to in depth hardware stuff.

Last edited by omega71122; 03-20-2005 at 03:35 PM.
 
Old 03-20-2005, 02:43 PM   #2
magnate
LQ Newbie
 
Registered: Mar 2005
Location: London, UK
Distribution: Debian
Posts: 22

Rep: Reputation: 0
I've just spent 9 months getting Linux up and running on a simple RAID0 stripe set on my Asus A7N8X (SiI3112 SATA RAID controller). I would have been much easier to use linux software raid instead of the controller's version, but this machine needs to dual-boot into Windows, so I needed to do it the hard way.

So, my first point: if your server does not need to dual boot, use linux software RAID instead of buying a SATA RAID controller. Linux software raid will give you identical performance and reliability (because SATA RAID controllers are NOT true hardware RAID controllers, they use the same CPU and i/o overhead as software RAID).

(If you really want high performance, buy a true hardware raid controller - your system is easily high enough spec to make this worthwhile. You could buy a caching controller which allowed hot-swapping, so you wouldn't even have to take the server down when you replaced a drive!)

My second point: if you do choose to go the SATA RAID route, don't use Debian. I'm a committed Debian user, but SATA RAID support on Debian is still very patchy. I had to find and fix a bug in the latest Debian installer myself, and even then I could only install a 2.4 kernel, and had to overcome several more hurdles to upgrade to 2.6 (the way SATA RAID is handled is COMPLETELY different in 2.6). I'm not an expert on other distros but the dmraid toolset (which is the critical part of SATA RAID management for 2.6) is maintained by RedHat so I'd suggest Fedora.

If you go with a true hardware raid controller, or with linux software raid, I'd wholeheartedly recommend Debian.

Whichever hardware solution you go for, I'd definitely recommend using LVM2 to manage your volumes.

All the best,

CC
 
Old 03-20-2005, 03:25 PM   #3
omega71122
Member
 
Registered: Mar 2005
Location: Central Ohio, USA
Distribution: Fedora 7 x86_64 (home server), IPCop (home gateway), CentOS (work laptop), lots more at work.
Posts: 32

Original Poster
Rep: Reputation: 15
It doesn't need to dual boot with Windows. This will be a dedicated server.

Hot swap won't be an issue. I'm not expecting any drives to fail catestrophically (sp?). If a drive does fail, it wouldn't be a problem to take the server offline for a half-hour to shut it down, replace the drive, and restart it. From what I read, the volume can be written to the new drive during regular use. Or I can just do the work at night and let the volume be restored overnight. I'm going to custom-build my kernel with just what I need, so the startup/shutdown should be as fast as possible.

I already use FC2, and would like to stick with it (I'll probably upgrade to FC3 or 4 if it's an official release by the time I start setting it up).

I just want to clear this up. By using software RAID, the controller I use to connect the SATA drives does not need to have RAID listed in its specifications (ie - can I just buy any Linux-compatible SATA controller and use software RAID)?

As for RAID management, if I do go the software route (which I will if the answer to my question above is yes), I'll just use whatever RAID management console/GUI/tools/whatever that comes in the Fedora distro I use.

I would like it to be able to transfer data over my network as fast as possible. I'm not sure off the top of my head what the listed peak transfer rate is for SATA, but I'd like to assume it's over 1Gb/sec. As long as that speed can be maintained (or at least good enough speed to stream music and video over a Gbit connection or however fast the network traffic would permit) then I'll be happy. In this respect, I don't think that performance will be a big issue (unless I'm overestimating SATA's peak data transfer). In other words, it doesn't have to be the fastest array ever, just fast enough to stream music, video, and data over a 10/100/1000 network to my computers, WiFi, and Xbox.

Another question. Would there be any noticible benefits if I were to use a AMD Athlon64 platform instead of a 32-bit P4 based on the same specs that my original post called for? How good is SATA RAID support in the 64-bit kernels and distros? Also, the server will be connected to a UPS for power failures. How good is support for APC based UPS's in 64-bit distros?

Last edited by omega71122; 03-20-2005 at 03:27 PM.
 
Old 03-20-2005, 04:20 PM   #4
ironwalker
Member
 
Registered: Feb 2003
Location: 1st hop-NYC/NewJersey shore,north....2nd hop-upstate....3rd hop-texas...4th hop-southdakota(sturgis)...5th hop-san diego.....6th hop-atlantic ocean! Final hop-resting in dreamland dreamwalking and meeting new people from past lives...gd' night.
Distribution: Siduction, the only way to do Debian Unstable
Posts: 506

Rep: Reputation: Disabled
Not for nothing but Debian handles sata raid no better/worse than anyone else.
Once the modules are in place
the kernel is adjusted just right
and raidtools like mdadm and other good stuff is in place,all is fine.

Iron@bumbie:/etc$ cat /proc/mdstat
Personalities : [linear] [raid0] [raid1] [raid5] [multipath] [raid6] [raid10]
md1 : active raid5 sdd3[3] sdc3[2] sdb3[1] sda3[0]
48821376 blocks level 5, 32k chunk, algorithm 2 [3/3] [UUU]

md2 : active raid5 sdd4[3] sdc4[2] sdb4[1] sda4[0]
22956672 blocks level 5, 32k chunk, algorithm 2 [3/3] [UUU]

md0 : active raid1 sdb1[1] sda1[0]
128384 blocks [2/2] [UU]

unused devices: <none>

4 wd raptors
4 partitions on each drive
boot to raid 1
root is md2 raid 5
Kanotix based on knoppix maintaining a debian sid system.

I installed to kernel 2.6.8 initially and am useing 2.6.11-kanotix-4

All software raid and I have the Asus A7N8X (SiI3112 SATA RAID controller)...maybe I'll restructure with 2 more drives as raid 1 to sil controller and keep my 4 raptors as raid 5 but as one partition for storage...maybe a lvm3 partitions on that.

Thanx for the heads up on the sil controller
 
Old 03-21-2005, 08:27 AM   #5
magnate
LQ Newbie
 
Registered: Mar 2005
Location: London, UK
Distribution: Debian
Posts: 22

Rep: Reputation: 0
Quote:
Originally posted by ironwalker
Not for nothing but Debian handles sata raid no better/worse than anyone else.
Once the modules are in place
the kernel is adjusted just right
and raidtools like mdadm and other good stuff is in place,all is fine.
That's my point - the essential userspace tool for managing SATA RAID arrays under 2.6 is dmraid, which is not yet part of Debian.

Quote:
I installed to kernel 2.6.8 initially and am useing 2.6.11-kanotix-4
This worked because you are using proper software RAID, not SATA RAID.

Quote:
Thanx for the heads up on the sil controller
You're welcome. If you ever find yourself wanting to boot 2.6 from the SiI3112, you'll probably find http://tienstra4.flatnet.tudelft.nl/~gerte/gen2dmraid/ as useful as I did. Note that the dmraidmkinitrd script is out of date and has hard-coded urls to specific rpm versions which have been updated. Update the urls and it works fine.

CC
 
Old 03-21-2005, 09:00 AM   #6
magnate
LQ Newbie
 
Registered: Mar 2005
Location: London, UK
Distribution: Debian
Posts: 22

Rep: Reputation: 0
Quote:
Originally posted by omega71122
It doesn't need to dual boot with Windows. This will be a dedicated server.

Hot swap won't be an issue. I'm not expecting any drives to fail catestrophically (sp?). If a drive does fail, it wouldn't be a problem to take the server offline for a half-hour to shut it down, replace the drive, and restart it. From what I read, the volume can be written to the new drive during regular use. Or I can just do the work at night and let the volume be restored overnight. I'm going to custom-build my kernel with just what I need, so the startup/shutdown should be as fast as possible.
Then I am absolutely convinced that software RAID is your best solution. The latest kernels support RAID10 (0+1) so should do exactly what you want. As you say, if a drive fails, you power down, open it up, replace the drive, and the array is rebuilt from the mirrored copy. You cannot lose data unless two of your four drives fail at the same time, which is pretty unlikely. (Even then they have to be one from each mirror set.)

Since you need four drives for RAID10, you could check out RAID5 instead, which gives you 50% more useable space (you get to use 3 of the 4 drives with one for parity info), the same reliability (you don't lose data unless 2 drives fail together), and potentially better performance (3 striped drives instead of 2, and the parity info for RAID5 is less overhead than mirroring).

Quote:
I already use FC2, and would like to stick with it (I'll probably upgrade to FC3 or 4 if it's an official release by the time I start setting it up).

I just want to clear this up. By using software RAID, the controller I use to connect the SATA drives does not need to have RAID listed in its specifications (ie - can I just buy any Linux-compatible SATA controller and use software RAID)?
Yes, that's exactly right. Linux software RAID will work with any supported disk controller, even old ESDI or MFM/RLL controllers! It's completely device-independent. You therefore don't have to use SATA controllers if you don't want to - it will work with PATA, SCSI or a combination. For example, you may want a high-performance SCSI controller for the main stripe set, and you can use your motherboard's built-in IDE for the mirrored copy since performance isn't an issue there (the mirroring is done very intelligently in the background so unless the server is extremely busy you won't notice a performance hit from the slower mirroring drives). This only applies to RAID10 of course - for RAID5 you can have the parity drive slower but you want all the others fast.

Quote:
As for RAID management, if I do go the software route (which I will if the answer to my question above is yes), I'll just use whatever RAID management console/GUI/tools/whatever that comes in the Fedora distro I use.
I'm pretty confident that this is LVM2 (Logical Volume Manager). It's maintained by RedHat so I'd think so. There are always so many ways to do anything in Linux!

Quote:
I would like it to be able to transfer data over my network as fast as possible. I'm not sure off the top of my head what the listed peak transfer rate is for SATA, but I'd like to assume it's over 1Gb/sec. As long as that speed can be maintained (or at least good enough speed to stream music and video over a Gbit connection or however fast the network traffic would permit) then I'll be happy. In this respect, I don't think that performance will be a big issue (unless I'm overestimating SATA's peak data transfer). In other words, it doesn't have to be the fastest array ever, just fast enough to stream music, video, and data over a 10/100/1000 network to my computers, WiFi, and Xbox.
Hmm. The performance of the drives is much more the issue here than the capacity of the controllers. My understanding is that SATA is limited to 150Mb/sec (cf. UDMA-133), giving you a theoretical maximum of 300Mb/sec for a RAID0 stripe set on a 2-channel controller. But I've never heard of any drive getting over 100Mb/sec, let alone 150. Plus, you don't get 100% performance increase with a stripe set. The best I've ever had is 85% (two Quantum Atlas IVs on an Adaptec 2940U2W gave me 37Mb/sec from 20Mb/sec each). My SiI3112 with two Maxtor DiamondMax 160s gives me only 55% (77.5Mb/sec from 50Mb/sec each).

Ah, wait a minute. You mention Gbit ethernet. 1Gbit/sec=128Mb/sec. So a RAID0 stripe set of two top-of-the-line drives (WD Raptors, maybe?) would get you pretty close to that. You don't need anywhere near that to stream music or video (which works fine over my 100Mbit network, ie. 12Mb/sec). So I think you're right, performance isn't really an issue here. I think you could have 4 standard (PATA) drives on a motherboard's built-in controller and easily have enough performance, providing the board and the two main drives are fairly modern.

Quote:
Another question. Would there be any noticible benefits if I were to use a AMD Athlon64 platform instead of a 32-bit P4 based on the same specs that my original post called for? How good is SATA RAID support in the 64-bit kernels and distros? Also, the server will be connected to a UPS for power failures. How good is support for APC based UPS's in 64-bit distros?
Sorry, I've never used a 64-bit platform and don't know much about them. My instinct is that you would not see a noticeable improvement in the performance of a given drive&controller combo just by going to a 64-bit CPU. If you were talking about going to a 64-bit PCIe controller it would be a different story, but I don't think you are ...

All the best,

CC
 
Old 03-22-2005, 12:52 AM   #7
omega71122
Member
 
Registered: Mar 2005
Location: Central Ohio, USA
Distribution: Fedora 7 x86_64 (home server), IPCop (home gateway), CentOS (work laptop), lots more at work.
Posts: 32

Original Poster
Rep: Reputation: 15
I never considered RAID 5. So with say 4 200GB hard drives, I would have 600GB effective storage space? With 4 300GB hard drives, I would have 900GB effective storage? And the other 200GB/300GB would be distributed (evenly?) throughout the four drives with parity information? (If memory serves from my digital logic class and my computer architecture class, it's used for error correction; is that the case for RAID 5?) This may be a better route. It would be a little more expensive (or less depending on the size of the drives I use) to start out with, but more effective data storage is always better.

I'll do a bit more research on RAID 5 and make my decision on what road I'll go down. My concern is that if one drive fails, the other drive on that bus might fail as I've read could happen (As of now, I'm gonna use a pair of 2 port controller cards). In that case, I wouldn't want to be screwed over.

Wow... it looks like I grossly overestimated SATA's speed. I'm still going to upgrade my entire wired network to 10/100/1000. Assuming network speeds follow the same conventions as HDDs, etc., the theoretical peak for gigabit network traffic would be either 1000MB/s or 1024MB/s. I don't know though. I'm probably wrong. If I am, then I'll probably hold off on the gigabit upgrade for a while. I already know that I'm gonna get a motherboard with at least gigabit LAN onboard, definately have two LAN controllers (one to the outside and one to the inside), so the upgrade in the future is nothing more than unhooking all my cables and reconnecting them in a new order to use the server as a DHCP server/firewall (since I have a regular router).

Based on talking to my fraternity brothers and other friends, I'll probably go with a 32 bit platform. My one friend recommended Slackware as a server platform. Any thoughts on it vs. FC2/3/4? Based on the fact that Red Hat maintains LVM, plus the fact that I'm already used to Red Hat (though I know it's not too difficult to switch distros), as well as the fact I have the Complete Fedora Core Desk Reference (for Dummies) will probably lead me to stick with FC.

Update: I did do a bit of research in the last few minutes on RAID 5. My biggest concern is that a drive will fail and take the other drive on that bus with it, disabling 2 drives and therefore my entire array. I kinda like my idea to put one drive from each array on each controller and go with the 0+1 setup. I can always add drives and cards or (if I come accross another system to hack into a server) another server with more drives (I do have my box I experiment on).
 
Old 03-22-2005, 04:06 AM   #8
magnate
LQ Newbie
 
Registered: Mar 2005
Location: London, UK
Distribution: Debian
Posts: 22

Rep: Reputation: 0
Quote:
Originally posted by omega71122
I never considered RAID 5. So with say 4 200GB hard drives, I would have 600GB effective storage space? With 4 300GB hard drives, I would have 900GB effective storage? And the other 200GB/300GB would be distributed (evenly?) throughout the four drives with parity information? (If memory serves from my digital logic class and my computer architecture class, it's used for error correction; is that the case for RAID 5?) This may be a better route. It would be a little more expensive (or less depending on the size of the drives I use) to start out with, but more effective data storage is always better.
I'm no expert, but yes you do get (N-1)x drive capacity in a RAID5 array of N identical drives. I think the parity information is all stored on the Nth drive, but I'm sure there's a version where it's distributed between all of them.

Quote:
My concern is that if one drive fails, the other drive on that bus might fail as I've read could happen (As of now, I'm gonna use a pair of 2 port controller cards). In that case, I wouldn't want to be screwed over.
I've never heard of this happening. I think you'd have to have a very dodgy controller to cause this. SATA only allows one drive per channel, so it *should* be even less likely to cause this than PATA or SCSI, because each drive has its own cable.

Quote:
Assuming network speeds follow the same conventions as HDDs, etc., the theoretical peak for gigabit network traffic would be either 1000MB/s or 1024MB/s. I don't know though. I'm probably wrong. If I am, then I'll probably hold off on the gigabit upgrade for a while.
You are wrong, but in a direction which makes your network upgrade more likely, not less:

Network speeds are measured in bits per second (aeons ago the term "baud" was used).

Drive speeds are measured in bytes (or rather megabytes) per second.

So there is always a factor of 8 difference.

1000 bits = 1kilobit = 128bytes.

56000 bits/sec = 56kbit/sec = 56x128bytes = ~7kb/sec - this is standard dialup modem speed

1000000 bits = 1 megabit = 128000 bytes = 128kb/sec - this is broadband (at least for me!)

10 megabit = 1280kb = 1.28Mb - this is standard ethernet.

100 megabit = 12800kb = 12.8mb - this is the 100 in "10/100 ethernet".

1000 megabit = 1gigabit = 128Mb - this is the 1000 in "10/100/1000 ethernet".

So "gigabit ethernet" has a maximum possible data transfer speed (before overhead) of 128 megabytes/sec. 100 megabit ethernet has a max transfer speed of 12.8Mb/sec and 10 megabit ethernet has a max of 1.28Mb/sec.

Quote:
I already know that I'm gonna get a motherboard with at least gigabit LAN onboard, definately have two LAN controllers (one to the outside and one to the inside), so the upgrade in the future is nothing more than unhooking all my cables and reconnecting them in a new order to use the server as a DHCP server/firewall (since I have a regular router).
Nice. The gigabit LAN faces inwards so your home net runs at full speed, while the other LAN controller deals with your net connection, which I assume isn't going to exceed 100Mbit any time soon! Here in the UK the fastest home net connection is 4Mbit, though I know in north america you can get 8 or more. Here's me limping along with one ...

Quote:
Based on talking to my fraternity brothers and other friends, I'll probably go with a 32 bit platform. My one friend recommended Slackware as a server platform. Any thoughts on it vs. FC2/3/4? Based on the fact that Red Hat maintains LVM, plus the fact that I'm already used to Red Hat (though I know it's not too difficult to switch distros), as well as the fact I have the Complete Fedora Core Desk Reference (for Dummies) will probably lead me to stick with FC.
I used to use Slackware a loooong time ago (1995-7) - it was the first distro I ever used so I'm quite fond of it. It's known as the hacker's distro, because it doesn't try so hard to coat everything with glossy package managers and stuff like the others - you get right in there with the tarballs (or at least you used to). I'm sure it would do a fine job, but I think your reasons for sticking to FC are sound. The fact that you will always have the latest LVM and will be able to submit any bugs direct to the LVM maintainers is a clear winner.

Quote:
Update: I did do a bit of research in the last few minutes on RAID 5. My biggest concern is that a drive will fail and take the other drive on that bus with it, disabling 2 drives and therefore my entire array. I kinda like my idea to put one drive from each array on each controller and go with the 0+1 setup. I can always add drives and cards or (if I come accross another system to hack into a server) another server with more drives (I do have my box I experiment on).
Surely the 0+1 setup has the same vulnerability if you do it that way? If one drive failing takes down the other drive on the same controller, it doesn't matter whether you've got RAID5 or RAID0+1, they're both toast. If you lose half the main stripe set AND half the mirror, you can't recover. If you have both main drives on one controller and both mirror drives on the other, then you're safe if one controller takes both its drives down. The better solution is to make sure you buy a controller which doesn't do this! I'd recommend investigating modern SCSI controllers - a decent Ultra320 SCSI controller will allow much greater expandability (up to 15 drives) and better performance. Sure it'll be more expensive than your two SATA cards (as will the drives), but you're not cutting corners with this machine.

Anyway, have fun building it!

CC
 
Old 03-22-2005, 12:11 PM   #9
omega71122
Member
 
Registered: Mar 2005
Location: Central Ohio, USA
Distribution: Fedora 7 x86_64 (home server), IPCop (home gateway), CentOS (work laptop), lots more at work.
Posts: 32

Original Poster
Rep: Reputation: 15
I see what happened there. We were confusing b (bits) and B (bytes).

RAID 5's parity information is spread across the drives. I think that RAID 3 or 4 has the parity information on one drive only.

I did investigate going with SCSI based drives, but the cost would outweigh any benefits (on newegg one 300GB drive is over $800). I'm gonna go with the idea to put the 2 main drives on one controller and the 2 mirror drives on the other.

Now that all that's finalized, all that's left is to decide which distro to use, pick a motherboard, choose a CPU speed, and get some RAM...
 
Old 03-22-2005, 01:05 PM   #10
ironwalker
Member
 
Registered: Feb 2003
Location: 1st hop-NYC/NewJersey shore,north....2nd hop-upstate....3rd hop-texas...4th hop-southdakota(sturgis)...5th hop-san diego.....6th hop-atlantic ocean! Final hop-resting in dreamland dreamwalking and meeting new people from past lives...gd' night.
Distribution: Siduction, the only way to do Debian Unstable
Posts: 506

Rep: Reputation: Disabled
Quote:
My concern is that if one drive fails, the other drive on that bus might fail as I've read could happen (As of now, I'm gonna use a pair of 2 port controller cards). In that case, I wouldn't want to be screwed over.
With 4 disks, 3 used as primary and 1 as spare,you will be safe from one disk failure..Raid 5 allows for hotspare...or 2.You can add 8 disks in raid 5 if you wanted to.Simply removeing the failed disk and adding the new one is simple...your array will be fine as the drive you just replaced becomes the spare.You wont even have downtime when rma'n the borked drive
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
SATA RAID support in FC2 yet? Phaethar Fedora 5 09-02-2004 04:29 AM
Mandrake 10 SATA Raid 0 support problem hepkat63 Linux - Software 11 07-24-2004 10:12 AM
Linux Distros that support Sata Raid N4N01D Linux - Software 6 05-22-2004 04:37 PM
does linux support the sata raid and ide raid in k7n2 delta ilsr? spyghost Linux - Hardware 10 04-16-2004 06:27 AM
Mandrake 10 SATA Raid 0 support problem hepkat63 Linux - Hardware 6 04-05-2004 08:23 AM


All times are GMT -5. The time now is 05:59 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration