Encryption of very large volumes 50TB+
 

Hello,
My company has about 50TB that is currently being stored on a Linux system spread across multiple ext3 formatted SAN volumes. The system functions as a NAS serving CIFS shares. But as the data grows we have been juggling dirs around on the ext3 vols. We need to replace this with a contiguous filesystem. We would use a NAS appliance but we requirement data-at-rest encryption and we would also like to use our existing SAN.
What I am purposing is a system with RedHat6 using XFS on top of LVM. That part should be straight forward, but I have never worked with disk encryption. I have been reading about LUKS and LUKS on LVM, LVM on LUKS, etc. But I have not found anyone working with volumes this large.
I am looking for advise and opinions on this. What are the best options? How would LUKS handle a volume like this? Ability to grow the volume? Key management? Other encryption methods? etc. Any help is appreciated. Let me know if there is more info I can provide.
Thanks in advance!

Disk encryption has three flavors.

Block/Disk Partition Encryption

Individual Files

File System or Directory Encryption.


The LUKS 'dmcrypt' method you mentioned is in the first group, and involved Formatting a partition so that everything is encrypted at the block level. You need the encryption key to mount the partition, and without it the disk does not have a generally reconisable file system.

If you NAS is limited to CIFS filesystems, then you can not use this encryption method.

Unless you make one really huge file in which to store the whole filesystem that you can then mount separately. I doubt however that your NAS provider would like a 50 Tb filesystem with a single 50Tb file of what is essentially purely random data on it. Of course that many not even work, as everything has to be able to handle a 50Tb file, from CIFS (instant fail), to network, to file seeking, or mounting, and so on!

I wouldn't even like to try!



Individual file encryption encripts the a individual file with a key. This generally requires the encryption built into the application that is using the file, or a decryption pipeline. In a pipeline files are typicaly also compressed before they are encrypted (not much good doing compression afterward).

It is good for small file, such as protecting some private data of some kind, but it is not generally good for large files or databases, unless the application itself handles the encryption. It is also typically what is used and advertised in 'Cloud' services, where they can encrypt/decrypt each file individually. It it is the cloud servive provider that does the encryption.



The newest form and the one you can use, is known as File System, or Directory Level Encryption (and I don't mean encrypting a directory archive).

This is where you mount a directory tree, and you see all the files as normal (unencrypted), but the other end of the mount is just another directory (such as a CIFS mount), also with a completely normal directory structure, but all the filenames and file data is encrypted.

The key with this is that the files and filenames are encrypted but the filesystem used is just a normal filesystem. You can use any normal filesystem CIFS, Dropbox (Cloud), USB stick, and you get the same file system limitations in both encrypted and decrypted directory trees.

If that file system is from a NAS, the NAS stores encrypted files but still sees it as a normal file system, that it can handle without any special requirements. In the cloud they also see a normal file system filles with normal but encrypted files. In both cases however the remote server does not have the encryption key, so teh system programmers that look after that system can not decrypt the data at all. They see and store the encrypted files, but only you 9with the password) can mount it in a decrypted form.

The current software that provides this type of filesystem is called EncFS (short for Encrypt FS). And it is not only for Linux and MacOSX, but with a small amount of restrictions, it is now available under windows too, so cloud solutions are also sharable across most desktop platforms.

The point its your data encrypted, and it works without anything special for the remote NAS or Cloud provider.


Wikipedia Page: http://en.wikipedia.org/wiki/EncFS

For windows see BoxCrypt: http://www.boxcryptor.com/index.html

Just an idea
 

How do you encrypt 50 TB of data?

Interesting question. Have you tried contacting admins at "friendly" organizations which deal with large amounts of data, such as WikiMedia, and asking for advice?

@A.Thyssen
Sorry, I realize I was not clear about the "CIFS" part. With this NAS we were planning to have 50TB of block storage attached, either from SAN or iSCSI. It would function as a Samba server thereby making the data available to other systems through SMB/CIFS shares. So we should be able to do block level encryption in that case. But thank you for the info on the filesystem encryption. It sounds interesting, I will check it out.

@Peufelon
I hadn't thought of going directly to other admins. My first inclination is Google, then Forums. I've never worked in IT on this level before. Guess I have to expand my resource pool. Thank you for your advice.

Just to follow up, I jumped onto the LVM/LUKS IRC (freenode.org #lvm). I got this response from one of the Devs:

Hope this helps anyone else looking for this information.