Filesystem for flash memories
Hello everyone, I've been looking on the internet on what filesystem is best suited for flash memories like USB keys, mmc, SD cards and the likes, but I'm not feeling quite satisfied with the answers I got.
There are two use cases I'm dealing with: 1) I have an SD card that I want to use for storing some data I'm working on. I need to preserve file permissions, and I need to maximize seek and read speed, but the filesystem is accessed for writes quite often too. 2) I want to install a linux distribution on a USB key Now, I've been reading around that said memories are block devices, and that I should stick to regular filesystems. I went for ext2 for the first case, but I experienced data loss on a brand new PNY SD card, which is unacceptable. Also, I suspect ext2 is striving for writing files consecutively or in some other harddisk-optimized way, which looks like a complete waste to me. In addition, other filesystems would flood the memory with journal crap. I looked into JFFS2, UBIFS, LogFS and here's what I came up with (some statements are contraddictory): - journaling might shorten the memory's life, and it will definitely slow things down - wear leveling is always hardware-implemented, and a software wear leveling could cancel out with the hardware's effort - hardware wear leveling has often a cheap implementation, and a better software wear leveling is desirable - JFFS2 doesn't store any index on the device; this comes at the cost of a slow mount but the flash memory is accessed less often - LogFS should work well on memories up to 64mb - MTD-oriented filesystems can be used on block devices - ext3 = ext2 + journaling - ext3 is not just ext2 with journaling So, is there anyone who can add some clarification to all this? Should I go ext2 everywhere, or ext4, or should I try one of those MTD filesystems? If so which one? What's the risk/drawback of each solution? Thanks to anyone who can help! |
flash memories do have a specific lifetime, which is very long by the way..
i kinda don't get it why journaling slows things down because it's only a few kilobytes of read write i guess.. logFS is definitely something you don't want... 64MB is a very small quantity.... ext3 is ok for both linux and the files. you better use ext4 because it is said to have faster boot times and also faster filesystem checking... |
Quote:
|
This guy (http://joggler.exotica.org.uk/ubuntu/) seems to think BRFS with a relevant mount option is a good way to go for running Linux from a USB flash drive.
I'm using his BRFS image and this is what the relevant line from /etc/fstab looks like: Code:
LABEL=linux-root / btrfs noatime,compress,ssd 0 1 The contents of this post represents the sum of my knowledge regarding which filesystems are good for flash drives. :) |
I didn't know about btrfs! It seems promising although not very mature yet, I'll keep an eye on that!
|
I've been looking at this for some time (note; this is from the stuff that I've read, not from stuff that I've tried, given that I am getting ready to try) and here is the list of clarifications that seem 'knowable'; there is a lot on the 'net that is either rubbish (insert your own, more extreme word for 'rubbish', if desired), or only applicable to some of the situations, without saying which the author is talking about.
Quote:
SSDs (things that look to the system like a hard drive) have a controller with wear levelling, USB thumb drives, Compact Flash cards don't. So, if you don't know whether the author is talking about an SSD or not, it is probably best to disregard what they say. Some controllers are better than others (broadly, the more recent the better), but, if you've got a controller, you've probably got to trust the controller, even if you know that its not the finest controller in the world. Quote:
Quote:
Quote:
Quote:
In testing, BTRFS's SSD mode didn't seem to do much. I'd guess that at this point it just tells the system that unconventional block sizes are about to be encountered, but I don't know. Certainly the performance change is anything other than dramatic, but maybe it helps device life more than performance... |
Hi salasi, thanks for replying. By now I got convinced that my cheap USB stick is already doing wear leveling, so I just went for ext4. This old thread from 2003, specifically, talks about DataTraveler having wear level technology (my memory stick is a DataTraveler 8GiB), and as for SD cards, it seems that wear leveling is part of the standard.
Besides, I did try BTRFS, and I really noticed some speed gain with my usb stick, but unfortunately free space detection is unpredictable: for example, even if I still had 1.5GiB of free space reported, the copy of 700-1000MiB files couldn't be completed. I assume that either the reported free space is wrong, either the allocation is defective as memory starts to run low. I will certainly give it another try in the future. If anyone has anythong more to say, I will keep on reading this thread from time to time (and hopefully gathering informations here is helping someone else as well). |
Quote:
Quote:
There is some benchmarking over at phoronix, eg: http://www.phoronix.com/scan.php?pag...s_nilfs2&num=1 http://www.phoronix.com/scan.php?pag...tbook_fs&num=1 http://www.phoronix.com/scan.php?pag..._zfs_ssd&num=1 but broadly, the message is a bit complex. You can quite easily turn around the 'which is fastest, which is slowest' list by tweaking the test conditions. nilfs2 is the other filesystem that seems to worth consideration, but how it would work in practice (does it make the wear issue worse (it shouldn't, but does it deliver, in practice, the potential), does it play nicely both with and without controllers? what about the compressing (SF) controllers?) is another matter. |
All times are GMT -5. The time now is 10:35 PM. |