I have a system I would like to put a RAID 5 configuration into, and have a few noob type questions.

Configuration: Gigabyte G77-D3H motherboard, 32 GB ram, Slackware 15.0
Drives: WD Red 10TB, also a small SSD for the system drive

Would like to use existing SATA2 interfaces, or possibly two SATA3 and one SATA2 interface on MB. Alternatively, I could consider a RAID interface card, but need some guidance.

-it is unclear how much support, as in parity, the MB provides.
-it is unclear whether a dedicated lower cost RAID interface board would perform parity calculations, and otherwise enhance performance
-what is the best/current guidance on configuring the system and setting up the RAID array?

Thanks for any guidance or pointers.

Go for software RAID. With hardware RAID, a failure of the motherboard or RAID board would require you to get an identical part before reassembling your array. With software RAID, you can reassemble your array on completely different hardware. Linux mdadm is mature and performant.

Edit : see Ser Olmy's post below. A much more knowledgeable post than mine, although I have used software RAID on both Linux and NetBSD since around 2010 without any problems.

2 members found this post helpful.

Most likely, using the SATA interfaces on your motherboard will leave you to software RAID using linux mdadm tools.

Many lower cost RAID cards will also only present the connected disks as a bunch of disks and rely on software RAID.

Some mid range cost RAID cards will offer hardware RAID support, but only for a limited number of RAID levels. It is usually cheapest to implement RAID0 and RAID1.

Expensive hardware RAID controllers have some kind of built in bios RAID functionality where you can configure RAID5, RAID6 and other RAID configurations. They also often support configuring disks at hot spare disks. A hot spare disk means that immediately when a disk is considerad bad, a hot spare will be assigned to take the role of the bad disk and parity will be used to calculate the new contents of the hot spare.

Be aware that RAID is no replacement for backup, RAID will not protect you from user errors like overwriting the wrong directory.

I stopped using RAID5 after seeing a RAID5 system fail to rebuild. All those disks of the same brand, model and age... After one disk had failed and all the other disks started working more than ever before trying to rebuild it was no big surprise that yet another disk would fail.

RAID6 is capable to keep running with two failed disks, but I have also seen RAID6 fail...

regards Henrik

2 members found this post helpful.

Yes. Good point. For software RAID-1 I always buy two disks from different manufacturers. For quick and dirty backup I rsync to a third disk, again from a different manufacturer.

1 members found this post helpful.

So it appears that a possible strategy for ongoing file integrity would be to rsync the raid portions of interest to a spare drive, either in the hosting machine, or remotely. I assume that there are not really any economically viable bulk media, except for hard drive. I think capacities exceed 8mm tape and anything else I have used in the past.

So to set up the RAID, what document would one suggest I consider for primary guidance, using Slackware 15.0? And I assume that there are few distribution nuances??

In the root of the slackware tree there is README_RAID.TXT

For example
https://mirrors.ocf.berkeley.edu/slackware/slackware64-15.0/README_RAID.TXT

2 members found this post helpful.

Slackware tries to follow upstream as closely as possible. So these are suitable:

https://raid.wiki.kernel.org/index.php/A_guide_to_mdadm
https://linux.die.net/man/8/mdadm

I ran with RAID for a while, but the SMR drives were flaky when used that way. I switched to btrfs using the devices directly, which worked better. The drives started dying again and I switched to SSDs.

Edit to add: mralk3 (above) has a better answer. I forgot about that doc!

Thanks!! This is the kind of guidance I was looking for.

Whenever the topic of RAID comes up, for some reason we always see the same, peculiar advice being handed out where Linux software RAID is touted as somehow being preferable to, or even superior to, hardware RAID.

I have no idea from where this misconception might originate, but just to (once again) put the record straight:

• Hardware RAID is more reliable than software RAID, not less.
  
  Sure, the controller can break (highly unlikely, but not impossible), but so can the controller(s) used to drive a software RAID.
  
  
• In the unlikely event that a hardware RAID controller does fail, the array absolutely can be assembled using the software RAID driver, if a compatible controller isn't available.
  
  All you need to know in order to assemble an array, is the structure of the array and the location of the metadata (in order to skip it). In fact, this is precisely what data recovery firms do.
  
  
• Linux software RAID is notoriously brittle. If you don't believe me, do a Google or forum search for "md raid not working" or something to that effect.
  
  There are a number of very good reasons for why this is the case: For one thing, the controllers and drivers in a regular SATA or SAS setup isn't designed to handle failing drives gracefully, and they certainly don't deal well with drives failing catastrophically and blocking the bus. A blocked channel can make the entire controller unresponsive, often resulting in a corrupted array.
  
  This is made worse by the fact that the non-enterprise drives most people use have a ridiculously high timeout for read errors and auto reallocation, resulting in drives with even a single, marginal block being summarily ejected from the array. A hardware RAID controller will disable automatic S.M.A.R.T. block reallocation and set the timeout to a very low value, in order to handle a read error and an eventual reallocation itself.
  
  
• Hardware RAID arrays do not suddenly fail. At all.
  
  If a degraded array fails to rebuild, it's because it wasn't verified/scrubbed regularly, and bad blocks were allowed to silently accumulate ("bit rot"). Most RAID controllers support automatic and/or scheduled background scrubbing, but this usually has to be configured.
  
  By the way, this is every bit as much an issue with software RAID as with hardware RAID, except the md software RAID driver doesn't support any kind of automatic background scrubbing; you have to write "check" to /sys/block/md<number>, either manually or using a cron job.
  
  
• Hardware RAID setups are either faster or a lot faster than software RAID, depending on the setup.
  
  The fewer disks, the more data overhead with software RAID, as all the redundancy data has to be sent over the PCIe bus: 100% overhead for RAID1, 50% overhead for a 3 drive RAID 5 array. Also, software RAID means the parity calculations are done by the CPU, which can represent a significant processing overhead in an I/O heavy scenario with many write operations, particularly if RAID 6 is being used. Hardware RAID controllers have dedicated processors that do this.
  
  
• Hardware RAID controllers can be equipped with battery-backed cache. This means you can enable writeback caching and still not have to worry too much about power outages or kernel panics.
  
  Sure, you can enable writeback caching on the individual drives in a software RAID array as well, if you don't really care about the integrity of your data.
  
  
• Hardware RAID controllers handle hot-plugging and dynamic expansion and transformation/migration of arrays really well.
  
  Whether a given SATA controller supports hot-plugging of drives is anybody's guess, especially if the controller is of the onboard variety.

Hardware RAID exists for a reason. If it wasn't any good, why would all high-end servers have such controllers?

In my almost 30 years of working with servers, I've hardly ever seen a RAID controller fail. I've lost a total of two due to failed firmware updates, and there was a somewhat flaky Mylex model back in the late 1990s, but that's about it. On the other hand, I've had to recover a significant number of software arrays in both Windows and Linux, and not all of them could be successfully reassembled.

5 members found this post helpful.

Raid provides short-term redundancy against disk failure. The failed disk must be replaced and a rebuild cycle completed. Raid5 at 75% full (15TB data w 10TB disks) it will take about 30 uninterrupted hours to rebuild @150MB/s (or restore from a backup).

Raid does not provide protection from data corruption. It detects parity mismatch, but cannot fix the mismatch.

Raid generally gives faster reads and slower writes depending on usage, eg, active application data vs less active backup/bulk storage. Mirrors are the fastest form.

* Hardware controllers are easier to setup but have almost zero transparency if something goes off.
* Motherboard/BIOS raid is never a good choice.
* mdraid (linux software raid) is very mature and has a lot of visiblilty if something goes off.

Entry level: Start with raid1 (mirror), have a spare disk on hand (3 disks). If the data stored is not easily recreated, make two mirrors and keep them in sync (5 disks). Mirrors are unique in being physically separatable, eg, portable.

Experience level: raid-6 (allows for 2 disk failures), openzfs or btrfs (accurately repair data corruption).

1 members found this post helpful.

This discussion is very helpful. May I broaden the discussion a little?

Let's argue that I would like to add an additional drive, which would be SSD, and use it as a cache drive. For reads and/or writes.

What (feature?) do I need with a hardware RAID to support that?

I am not as worried about wear on the SSD because I have perhaps a half dozen smaller ones (<260GB) around, and I suppose I could just use one.

Yeah, I once had backups on such 8 mm tapes. Later replaced it with a DLT tape station. Nowadays I store offline backups on big rotating hard drives which I put in a USB3 docking station to read or write.

Another media possible to consider would be bluray discs which can store up to 100 GB of uncompressed data, but still, that is only 100 GB. The nice thing with such a disc is that it is read only, so you will not overwrite a backup by mistake. But can sucn a bluray disk really be trusted to keep its data for many years.

For some time Seagate sold some rotating hard drives as "archive disks". I can't say for sure if they were really capable of keeping data for longer times, or if it was only a way to sell big cheap slow disks.

regards Henrik

2 members found this post helpful.

I haven't used any "cache drive" functionality, but I have experience from hardware raid systems with only SSD drives. In my experience, you will get a raw disk bandwidth of about 2 GB/s on a 20 SATA-disk RAID6-system regardless of if the disks are SSD or spinning drives. The hardware raid card will become the bottleneck as each spinning disk on its own has a capacity of about 200 MB/s (yes those seagate EXOS drives are rather impressive).

With enough disks in the RAID system spinning drives will give enough bandwidth. When it comes to latency that a cache might help with the battery powered RAM on the raid controller will probably be to more help than an extra SSD drive for cache.

regards Henrik

I did a media life study, or series of studies, for a media marketer about 25 years ago. The study was updated again about 11 years ago. Bluray, depending upon the process, was good for 20 years, so I figure that is good enough for my stuff, but the stacks of 100gb discs, 50 gb discs, keeping them in order, keeping them vertically stacked in cake cases, and so on was tedious. The same media group also looked at laser tape, for long term archival storage, but there weren't allot of drive manufacturers interested, so I assume it fizzled out. I wonder what big data centers like insurance companies use these days?

I can only hope that the average pile of drives lasts longer than I do.

My use case is that the RAID will be infrequently accessed. The existing (mirrored via rsync) drives which I use now are spun down after a period of time. Documents might be searched, or other edited, and have been thinking how a workflow might be developed to use cached copies on an SSD, and perhaps only when written or some other rule, write the SSD based file to the RAID. If I implemented that, my RAID drives might only spin up 500 times a year. Hard to say, because I don't know what the activity looks like from the desired RAID system.

I like to think that spinning down modern drives is a reasonable way to reduce wear, and there are certainly arguments against it. But it is effective at thermal management, and at least reducing spin time.

There are two main WD Red 4TB drives currently used, and they are a bit over 10 years old. They still survive the SMART long test. I guess I might get a hint someday when they die, and I do a postmortem. But I really won't have a good experience base to compare them against.

Sorry about the side tangent.