Hey all,

I've got a server with 3 software RAID volumes:

md1: sda1 + sdb1
md2: sdc1 + sdd1 + sde1
md3: sdf1 + sdg1 + sdh1

All the disks are connected to a LSI 3081E controller. There's no more available connectors on this controller.

Because I need some more disks, I've bought another 3081E controller, but instead of just creating md4 from disks connected to this new controller, I thought I'd ask here for opinions.

Would I benefit in any way from spreading all the disks across the two controllers?

It could look something like this ((a) or (b) signifies which controller the disk is connected to):

md1: sda1(a) + sdi1(b)
md2: sdb1(a) + sdc1(a) + sdj1(b)
md3: sdd1(a) + sde1(a) + sdk1(b)
md4: sdf1(a) + sdg1(a) + sdl1(b)

The above would provide a bit of redundancy. If one controller dies, the RAID volumes can keep running on disks connected to the other controller.

But what would it do to performance? Better? Worse? Same?

My first inclination was to simply connect the three new disks to the new controller and then add those disks to the new md4 volume. I suspect that will give me the best performance. But if the performance loss from spreading the disks is negligible, then perhaps I'd rather have the added bonus of some redundancy.

Any and all opinions/experiences are more than welcome.

Performance across controllers would be better, but you may or may not get redundancy depending on what raid levels you're using and where the disks are located, for example:

If this is RAID 5 and you lose controller "a", then the array will be broken (only supports the loss of 1 disk not 2)
If this is RAID 5 and you lose controller "b", then the array will keep running (only 1 disk lost)

cheers

Yes, I completely forgot to mention that everything is setup as RAID1. Simple mirroring.

Then how will you be adding the extra disks ? .. only 2 disks in RAID 1

You can mirror lots of disks in RAID1. I do it all the time, for data where uptime is important. The more disks, the less the chance of them all crapping out at the same time.

I think the other poster meant that you can have only 2 active disks in any given RAID 1 md device at once.

That might be (or not ?) a limitation of mdadm. But it's certainly not a limitation of mirroring. I have seen a production system here at work using a triple mirror. It's an old cobol system writing to 'flat-files'. At lunch time the application is stopped, the 3rd mirror is broken, the application restarts on the 2-way mirror and a full backup is then made of the offline disk. Once the backup is complete, the 3rd disk is re-silvered. The "downtime" of the cobol application is then only a few seconds. The backup itself takes an hours or so and the resilvering is pretty quick - depends on how busy the system was.

You can most certainly have more than two active disks in a RAID1 setup. I know, because I've done it many times.

Mirroring is not limited to 2 active disks.

Back to the original question, since all your RAID are RAID 1, clearly redundancy is the priority, so if you can (possibly) insulate yourself from controller failure by spreading the disks over two controllers, that would be consistent with that goal.

I doubt the performance difference would be that great anyway, but have no benchmarks to back me up.