There seems to be a fair amount of discussion about the performance of LUKS, especially the kcryptd process as it appears to be single-threaded. My current setup is 3x2TB drives in a soft RAID5 (md), which has a LUKS volume ontop. When writing, I get around 40MB/s which could be better given a 3GHz quad CPU.

The low performance could be related to the fact only 1 kcryptd runs per device (this device being /dev/md1). My question is; would it be better to have a seperate LUKS volume per HDD, and them forming a RAID5 from these? This should spawn a kcrypt process per physical device (3) instead of just 1.

Has anyone tried this setup and does it make any difference on read/write performance?

40MB/s writing to RAID 5 is pretty good, especially with only three spindles.
I don't think your CPU has anything to do with it. Remember a "write" on
RAID 5 requires a two reads, a calculation, and two writes. Assuming a
16K block size, each 16K requires 4 disk operations, or one operation
per 4K. At 40 MB/s, you're getting 1000 IOPs.

Let us know when you benchmark it the other way, but I think
you'll see precious little difference - CPUs are WAY faster
than drives, so no matter what you do with the CPU the
drives are still only going to write the data at a certain
speed.

As to which is BETTER, I'd decide that by not based on
performance, but on logical design. To me, the RAID is at
the physical level - that data just happens to be on those
drives, but it could just as easily be anywhere else.
Which data should be encrypted is a _logical_ choice rather
than a _physical_ one. Logical sits on top of physical, so
encryption on top of RAID.

1 members found this post helpful.

Agree.

& if you add LVM into the mix, the same logic says it goes between the 2.

Thanks for the replies. Personally I don't see why the encryption layer cannot go on the lowest level, pending some testing. I set this up using 3 loopback devices and it worked OK, however it was on the same phyiscal hard disks so performance couldn't be tested. Another plus point I can see is resizing RAID arrays should be easier if the encryption was the lowest level as you only have to resize the RAID array. Otherwise you have to resize the RAID array first, then the LUKS "container".

When I initially configured this array it was unecrypted and the performance was quite a bit higher that when it is encrypted (I can't remember the actual numbers). Maybe extra kcryptd processes will help? I have a bunch of old drives laying about and will test the performance difference and post back. I'm not expecting much difference but it's worth a test

raymor, going by your information, would you expect read/write rates to increase with the number of physical disks? I could test this too..