filesystems & encryptions
We are going to have a large flat file containing details of members, this file is constantly updated by members (changes of address, new members etc) and we want the data to be encrypted. Conventional encryption doesn't seem suitable since the whole file would need decryption-make changes-encryption for every change.
Questions: 1) Is it a reasonable solution to create a lot of small files holding just the details of one or a few members? 2) How can I calculate or know the space that is going to be occupied by an encrypted file for a given method/algorithm? 3) Is one encryption software better suited than others for what we want to do (we're contemplating using dm-crypt)? 4) It looks like the filesystem of choice would be XFS. Any comment? Thank you for your help. |
The Linux kernel can encrypt an entire file system. It can be an actual device, or just a regular file that you've mkfs'ed a file system on. Take a look at `man losetup`. You also need to make sure that you have loopback and cryptoloop support either built into your kernel or built as modules.
ta0kira |
Thanks to ta0kira.
The question we have is that it is impractical to decrypt a large file, then make changes to it then re-encrypt it, therefore is it practical to make a lot of small files holding the details of one or a few members so that the size of the file to be decrypted, changed and re-encrypted is as small as possible. |
Sounds like you have control over the file format, which helps out a lot. There are several ways around processing the whole file for every change. You could "containerize" the file by putting separately encoded sections within a single file for each user. You could also have a separate file for each user. I don't think it makes that big of a difference, except separate files might be easier to work with in your program, but then you have a whole bunch of files instead of just one.
What type of cypher do you use (block or stream)? The kernel uses block ciphers, and when you have an encoded file system it just applies the cipher to the part of the file system being changed. That brings me to another idea; you could have an encrypted disk image file (which allows you to transport it as one file) with separate files for each user. All you would have to do for this is mount the disk image with encryption on, then read/write like it was a normal disk. The kernel would take care of all of the encryption/decryption and would only process the part of the image being changed. This would require you use a built in cipher, or if you have your own cipher you would need to make a kernel module out of it (only if it's a block cipher). I can help you out if you want to do that. Oh, to answer your questions: 1) Yes, that sounds like a good idea. 2) If it's a block cipher, you need to find out how big the block size in bytes. You then round the size of your file to the nearest block size. If the method adds a header to the file (such as a checksum), you need to add that size also. If it compresses the file, you really need to wait until it's done to see what size it is. 3) I don't like dm-crypt all that much. It's too complex for everyday use. losetup gets the job done for me because it's very simple (I hear it has several bugs though). A lot of people use GPG, but I don't know how you interface software with it. 4) I don't know about XFS. I use ext3 and ReiserFS because they journal, and therefore recover extremely fast after a hard shutdown. ta0kira |
This is volunteer work for a Not-For-Profit. I have a free hand and have to set it all up as best I can.
Since you OK the idea, it looks like very small files are the way to go as the existing routines would interface very well with this system (they're written in assembly, I only have a very basic knowledge of C). We don't need portability. We can choose any cypher we want, the more secure the better, I only had dm-crypt in mind because it's the only one for which I found some sort of user manual on line. The encryption is intended to protect the data if the machine holding it is stolen or mis-appropriated, what we envisage is a system by which the key would be stored in a file in memory (ramdisk or similar) and is lost when power is turned off. From your explanation it seems we can use a kernel based block cypher. We wouldn't compress the data unless it reinforces the protection. If I understood what I read correctly, losetup is loop-AES, from which comes cryptoloop from which comes dm-crypt. Loop-AES is better than either and I remember reading its implementation has a lot of bugs but those wouldn't affect us because of the way we would use it although I can't remember the details. XFS also journal and is reported to be particularly good with lots of small files. I have my distro set up with ext3 which is not supposed to be safe for encryption and I'll probably set up Debian as the OS on the production machine. I don't know what an encrypted disk image file is but your explanation seems to indicate it might be what we need provided it doesn't imply encrypting a full hard disk. I'd be very grateful if you could make a few recommendations. Thank you very much. |
Quote:
If you want to make an encrypted disk image, do this: 1) Make sure you `modprobe loop` and `modprobe cryptoloop` unless you have these built in to your kernel. 2) Create a random disk image: Code:
> dd if=/dev/urandom of=my.img bs=1K count=1024 3) modprobe your ciper (I'd suggest aes or blowfish) 4) Loop the random image: Code:
> losetup -e aes /dev/loop0 my.img 5) Create your file system on the disk image: Code:
> mkfs -t ext2 /dev/loop0 Code:
> fsck /dev/loop0 Code:
> mount /dev/loop0 /mnt/hd Code:
> umount /dev/loop0 Code:
> losetup -e aes /dev/loop0 my.img Code:
> mount -o encryption=aes my.img /mnt/hd |
I got as far as the password. After entering this, I get a message:
ioctl: LOOP_SET_STATUS: Invalid argument I have compiled the kernel with AES_586, blowfish and others in the kernel (option= Y). I suppose that would not make any difference. I can't figure out what the invalid argument could be. Can you help again? Thank you very much for the script. |
Type `cat /proc/crypto` and see if aes shows up. I actually had to upgrade to kernel 2.6 to get this to work. Did you get any error messages when modprobing loop or cryptoloop? Look at lsmod and, unless you have them built in to your kernel (not default), then you should see loop, cryptoloop, and aes. If they all show up and you still get the error, AFAIK there is no easy fix except to upgrade kernels. Some people use dm-crypt instead, but that is quite complex and I'm not entirely sure how/if you can mount a disk image with it. The stuff I showed you is standard Linux stuff, which is why I like it; it should supposedly work on all Linux systems.
ta0kira |
cat /proc/crypto shows all the ciphers including aes and blowfish etc, all in the kernel.
I got no error modprobing loop and cryptoloop, they both show with lsmod I specially installed a second kernel (2.6.12.3) for this so if you've got another suggestion, I'll try it. |
Quote:
Quote:
Quote:
GH |
Thanks to freegianghu, this is a lot of enlightenment and a beautiful quote. In my case, this will definitely put encryption back on an unused testing machine.
And thanks to taOkira for the personal guidance. |
All times are GMT -5. The time now is 03:00 AM. |