Hi. I'm jon.404, a Unix/Linux/Database/Openstack/Kubernetes Administrator, AWS/GCP/Azure Engineer, mathematics enthusiast, and amateur philosopher. This is where I rant about that which upsets me, laugh about that which amuses me, and jabber about that which holds my interest most: *nix.
RAID recommendations: my 2 cents
I recently read a heated debate on the age-old argument: should I use RAID 5 or RAID 10 for my database? I noted a particularly relevant treatment of the topic (read it here), and figured like any good DBA I'd chime in with my thoughts.
The basic premise for the argument (which was, oddly, from a business guru...NOT a tech geek) was that given a required performance level and budget, what setup would provide better data safety?
"Any DBA would ask you to give them a RAID 10 of small, fast disks", the author says. This is true. Let's look at some math:
Let's say I have a pair of RAID arrays...one is RAID 5, the other is RAID 10. Furthermore, let's say these arrays are based (sadly) on a minimal number of big disks (say, 1 TB SATA drives). The RAID 5 has 3 disks and the RAID 10 has 4 (both are 2 TB arrays). Now, if you follow my blog at all, you'll have picked up on a common theme I like to stress, and that's the fact that lessons are learned best when you're running in panic mode. I call it "divine" learning, because nothing can take the place of being personally bitten by a bad decision in terms of how memorable the lesson is.
The idea is to enter into panic mode **prepared** (i.e. having been there, done that). Let's say both of my RAID arrays mentioned above suffer a disk failure. I'm now in panic mode. If I paid attention in basic math, I would know that the RAID 5 is a much more critical state right now (all else being equal, i.e. data stored on each array is equally valuable to my future employment with this company, etc...). The reason is simple: should another disk fail during the RAID 5 rebuild, I lose the entire array. No if's, and's, or but's...I have a 100% chance of failure should another disk go belly up.
The RAID 10 is a different story. I have three disks remaining in the RAID 10...but only one of the three (the other disk in the mirror that's degraded) is a single point of failure. I have a 33% chance that if another disk fails, I've lost the entire array. 100% vs. 33%...take your pick.
Now, let's move on to more realistic "database" scenarios. Let's say I have two equally valuable RAID arrays (1 is RAID 5 the other is RAID 10 again). This time, they are 2 TB based on smaller disks. Let's say the RAID arrays are based on 146 GB disks:
RAID 5 has 15 disks (ok, it's slightly under 2 TB...close enough). The RAID 10 now requires 28 disks. Both lose a disk. Panic mode sinks in again.
With the RAID 5, I **still** have a 100% percent chance of failure on ANY second disk failure. Odds aren't scalable in this instance!
With the RAID 10, however, I now have 27 other disks that could fail, only ONE of them gives me total failure. That basically means I have roughly 4% chance that the critical disk fails, and I can further the odds in my favor by building my mirrors with disks from different manufacturers and/or batches.
100% vs 4%...suddenly RAID 10 is making a ton more sense.
But rocket! You overlooked something! The RAID 10 has nearly twice the disks! You've skewed the numbers in your favor because the odds are stacked!
Ok, then let's say we have 16x 146GB 15k SAS disks, and we need to figure out if we should use RAID 5 or RAID 10. RAID 5 would give over 2 TB of storage, whereas RAID 10 would only give roughly 1.2 TB of storage. All else being equal, I still have a 100% chance of failure if a second disk fails in the RAID 5. I now have a 7% chance that the critical disk in the RAID 10 will fail.
One thing worth mention is that RAID 5 does definitely get a "bad rap" in many IT circles because it isn't as fail-proof as RAID 10. So here's an interesting thought (getting back to the businessman's analysis I read earlier): If the workload contains a majority of writes, RAID 10 will outperform RAID 5 (i.e. x < x + 1...no parity calculations for RAID 10 will make it faster as long as everything else is constant). However (here's the interesting note), if the workload contains a major majority of reads, a similarly priced RAID 5 setup will outperform RAID 10. So the important thing to know is "what percentage of my workload is read vs. write?". Once you know what percentage of the workload is read and write, you can make a more informed decision.
Case in point: All of my production databases, regardless if they are PostgreSQL, SQL Server, or MySQL, are backed with a RAID 10 for the main data array (Most range from 60/40 read/write to 75/25 read/write). No compromises there. Many of my reporting databases, however, are backed with a RAID 5 because once the nightly ETL operations are done, the report databases are *strict* read-only. In addition, if an array fails no critical data is lost since it can be ETL'd again.
Now, for the truly "zen-like" DBA power-trip, determine the read/write percentage *per table*, and split the database on to a RAID 10 and a RAID 5 based on the tables' i/o profiles =) Audit tables and log tables should go on a RAID 10 of the smallest/fastest disks available, whereas read-only or "mostly-read" data should go on a RAID 5. PostgreSQL makes this overwhelmingly easy, as individual "tablespaces" can be set up (or even added in to an existing database) to further tweak db performance.
The basic premise for the argument (which was, oddly, from a business guru...NOT a tech geek) was that given a required performance level and budget, what setup would provide better data safety?
"Any DBA would ask you to give them a RAID 10 of small, fast disks", the author says. This is true. Let's look at some math:
Let's say I have a pair of RAID arrays...one is RAID 5, the other is RAID 10. Furthermore, let's say these arrays are based (sadly) on a minimal number of big disks (say, 1 TB SATA drives). The RAID 5 has 3 disks and the RAID 10 has 4 (both are 2 TB arrays). Now, if you follow my blog at all, you'll have picked up on a common theme I like to stress, and that's the fact that lessons are learned best when you're running in panic mode. I call it "divine" learning, because nothing can take the place of being personally bitten by a bad decision in terms of how memorable the lesson is.
The idea is to enter into panic mode **prepared** (i.e. having been there, done that). Let's say both of my RAID arrays mentioned above suffer a disk failure. I'm now in panic mode. If I paid attention in basic math, I would know that the RAID 5 is a much more critical state right now (all else being equal, i.e. data stored on each array is equally valuable to my future employment with this company, etc...). The reason is simple: should another disk fail during the RAID 5 rebuild, I lose the entire array. No if's, and's, or but's...I have a 100% chance of failure should another disk go belly up.
The RAID 10 is a different story. I have three disks remaining in the RAID 10...but only one of the three (the other disk in the mirror that's degraded) is a single point of failure. I have a 33% chance that if another disk fails, I've lost the entire array. 100% vs. 33%...take your pick.
Now, let's move on to more realistic "database" scenarios. Let's say I have two equally valuable RAID arrays (1 is RAID 5 the other is RAID 10 again). This time, they are 2 TB based on smaller disks. Let's say the RAID arrays are based on 146 GB disks:
RAID 5 has 15 disks (ok, it's slightly under 2 TB...close enough). The RAID 10 now requires 28 disks. Both lose a disk. Panic mode sinks in again.
With the RAID 5, I **still** have a 100% percent chance of failure on ANY second disk failure. Odds aren't scalable in this instance!
With the RAID 10, however, I now have 27 other disks that could fail, only ONE of them gives me total failure. That basically means I have roughly 4% chance that the critical disk fails, and I can further the odds in my favor by building my mirrors with disks from different manufacturers and/or batches.
100% vs 4%...suddenly RAID 10 is making a ton more sense.
But rocket! You overlooked something! The RAID 10 has nearly twice the disks! You've skewed the numbers in your favor because the odds are stacked!
Ok, then let's say we have 16x 146GB 15k SAS disks, and we need to figure out if we should use RAID 5 or RAID 10. RAID 5 would give over 2 TB of storage, whereas RAID 10 would only give roughly 1.2 TB of storage. All else being equal, I still have a 100% chance of failure if a second disk fails in the RAID 5. I now have a 7% chance that the critical disk in the RAID 10 will fail.
One thing worth mention is that RAID 5 does definitely get a "bad rap" in many IT circles because it isn't as fail-proof as RAID 10. So here's an interesting thought (getting back to the businessman's analysis I read earlier): If the workload contains a majority of writes, RAID 10 will outperform RAID 5 (i.e. x < x + 1...no parity calculations for RAID 10 will make it faster as long as everything else is constant). However (here's the interesting note), if the workload contains a major majority of reads, a similarly priced RAID 5 setup will outperform RAID 10. So the important thing to know is "what percentage of my workload is read vs. write?". Once you know what percentage of the workload is read and write, you can make a more informed decision.
Case in point: All of my production databases, regardless if they are PostgreSQL, SQL Server, or MySQL, are backed with a RAID 10 for the main data array (Most range from 60/40 read/write to 75/25 read/write). No compromises there. Many of my reporting databases, however, are backed with a RAID 5 because once the nightly ETL operations are done, the report databases are *strict* read-only. In addition, if an array fails no critical data is lost since it can be ETL'd again.
Now, for the truly "zen-like" DBA power-trip, determine the read/write percentage *per table*, and split the database on to a RAID 10 and a RAID 5 based on the tables' i/o profiles =) Audit tables and log tables should go on a RAID 10 of the smallest/fastest disks available, whereas read-only or "mostly-read" data should go on a RAID 5. PostgreSQL makes this overwhelmingly easy, as individual "tablespaces" can be set up (or even added in to an existing database) to further tweak db performance.
Total Comments 0
