I'm not sure whether to put this in hardware, software, or security, but as the common denom is the UFD, hardware it goes; apologies if wrong...

I just bought a 32GB UFD and, while waiting for it to arrive, am doing some research. Never having owned anything with flash other than some embedded stuff like the cell (mobile) phone, I opened a can of worms research-wise.

I intend to use the stick both as a replacement for CD/DVD's of Linux and BSD install disks for use at install-fests or for recovery, as well as backup of important stuff: subbing for a [CD/DVD]-R(/W) rather than a ZIP or hard drive. With that in mind, here are my questions:

1. Write protect switches were common on UFD's as recent as a couple of years ago, but now are very rare. Encrypting the contents against reading is one thing, but how do you keep a possibly hostile machine (due to virus, luser error, etc) from modifying or, worse yet, erasing the contents?

2. Related to #1, are there any opinions here about using the dedicated flash file systems (JFFS[2], YAFFS, etc?) vs FAT32 or EXT2 vs the read-only cramFS or SquashFS? Could using one of those FFS's help with the write-protection discussed above? (Since a winblows machine can't read it and (most) Linux users are not lusers. Do any of the above FS's do bit-wise EC/D (error correction/detection)?

3. Related to flash itself, there seems to be some conflicting info on the web about data retention time in a modern NAND flash chip as well as the number of erase cycles. Does anyone have some more definitive info, especially about the former?

Thanks, Mike

Well, I've had a UFD (a 2GB Transcend JF V30) for about two years, and I write to it twice daily.

52 weeks at 5 days per workweek:

260 days x 2 = 520 writes in a year

so 1040 writes in two years. And it is still going fine...

As far as I know there is not much difference between writing and erasing (from the flashdisk's hardware point of view).

As for retention, from personal experience, we have some flashdisks the graphic designers use which have been unused for up to 4 months at a time, with no loss...

Thanks, Stefan! This was interesting info, but I'm looking at time retention in terms of years rather than months. Some devices advertise greater than a decade retention time while most devices don't advertise anything in that regard. Since they all get their flash from the same manufacturers (Hynix, Intel, AMD?, Toshiba, Micron, and maybe a couple more) I guess it would be down to breaking it open (if possible w/o destroying it, probably hard to do on a tough drive without destroying it) and reading the part numbers then checking on the manufacturer's web site.

In terms of number of writes, depending on the device, you may need one write, but usually 2: an erase cycle followed by a write. The question I have is that I've seen Wikipedia say 100k-1M writes for NOR and 100k or less for NAND, while another web site said just the opposite! Since ALL large flash is NAND, this really leaves question marks in my head. Another weirdness: Corsair says "typical" device lifetime is 10 years which is why they guarantee it for that long, Kingston gives a 5 year guarantee, while Patriot (the one I bought because of an incredible Black Friday special on Newegg--there, I said it!) gives a "limited lifetime guarantee"--but I have yet to read the fine print to see exactly how that's defined! There are also a couple of other makers saying the same "limited lifetime guarantee", again without defining just what that means. Meanwhile, older devices (8GB and under) have far less warranty, in some cases your typical one year warrantee you see with electronics. Is it because of using NOR rather than NAND, or is the manufacturing quality (cheap case and earlier troubles with lead-free solder) to blame for the lack of faith?

These are the kinds of things I need to answer. As I said, my 2 uses would be as a replacement for CD's and DVD's for hauling Linux distros to install-fests as well as an off-computer backup for important files. For this reason, my biggest concerns are data retention at the bit level and ability to resist modifying or erasing data when put into someone else's computer that may be infected with a virus. The FFS's do wear-levelling, which can be important, but I don't know if they do ECC. Likewise, if I use a read-only FS, what's to stop some virus from getting access to the drivers directly and sending a low-level write or erase command to the flash drive, whether mounted (under Linux) or not?

Mike

I suppose the 'definite' answer about durability can only be given by the technicians at the factory producing the flash cell chips.
Flash media (otherwise known as 'Solid State' storage or drive) is much more complicated than 'normal' magnetic media such as a hard drive, and many factors cloud the final result of whatever characteristic you wish to measure.
And given the relatively young age Flash technology is in for public consumption, I suppose that is your answer why nobody really knows (yet)


Non-scientific answer;
I currently have 3 Kingston DataTraveler USB flash drives (512MB, 2GB and 4GB) that are still working perfectly despite having seen multiple full disk wipes of around 15GB per wipe in the past year with 'Eraser', a secure disk wiping program.
Data retention has been perfect for years, for me.

Your mileage may vary however!

By the way, if a cell is bad, no amount of software error correction will help you, and you will receive no warning signs in advance when your flash drive is about to fail like a 'normal' hard drive may warn you (such as S.M.A.R.T. and clicks and slower performance and.. etc.).

That said, Flash drives themselves have spare flash cells available, and are already equipped with hardware-level error correction and write wear leveling mechanisms which are permanently active for the lifetime of the product.
Wear leveling is an attempt by the hardware to spread out disk writes over all cells equally, so no one cell sees more write activity than another.
Should a cell become unusable, error correction is used to mark that cell as bad and attempt to remap and migrate the data onto a spare cell and continue normal operation.

And concerning erase versus write;
Erasing and writing mean the same thing to a flash based drive; in order to write a data page the entire data block that page is on has to be erased and rewritten.

And once you run out of spare cells, well.. you do make backups do you?
Safety is not in numbers with flash drives. Perhaps price? SSDs may be 'safer' because of their manufacturing processes but are also much more expensive, and cheap USB/SD flash could be easy on your wallet but not on your conscience because of much! cheaper manufacturing processes than SSD.

Perhaps for your purposes, an SSD would be your best choice; you'll pay a bit more but you will have (with some models, choose wisely and ask lots of questions) the added benefit of a write protect jumper and a normal SATA interface, so you can choose to mount it in an USB/Firewire enabled carrier or insert it into the target PC itself.
Only write-protect under Linux I know of is to mount read-only.


Maybe this place gives you a little more insight into flash performance, it seems fairly level-headed:
http://storagemojo.com/category/ssdflash-disk/

True, that. I only partly understand why flash can achieve such higher cell densities than DRAM, although reading about NOR flash, I guess it has to do with all the extra address lines.

It sounds like you've given them quite a work out! And yes, I'm all too familiar with the varying stages of electronic mortality rates: in fact, I seem to be cursed with getting the lemon in many devices, or at least needing to do a lot more work to get them working.

My understanding was that only SOME devices have wear-levelling and/or ECC to detect cells going back in the controller. Is that not true?
I thought it was up to the driver to determine when a cell was going bad, but I realize I may be confusing UFD's with other, simpler flash media like SD, CF, etc.

Hmm...I thought it was considered wear every time you tampered with the charge on an individual cell. If that's the case, the worst-case scenario is you perform and erase and write operation that changes a bit from 0 to 1 and back to 0 again.

But then this just occurred to me: in NAND flash, since everything is done by the page, if a single cell goes bad, does that mark the entire page as bad and the device has to shift to a new, spare page rather than individual cells?

Which is why I'm getting a "tough" UFD; eventually, for my backups I want the Corsair Survivor (the one in the aluminum housing that can survive being boiled, froze, and run over with a bus!). But the thing is that I'm not writing to it constantly: one write a month, maybe. What I'm concerned with is how long those cells are going to retain their charge before they become unreadable. In other words, how many years after writing a file will that file have so many errors that whatever error correction is used can no longer recover it? This doesn't mean the flash cells are bad, simply that they lost their charge over time: a re-write would prove the cells are healthy. This is opposed to magnetic media, where the question is how long a field is retained in a spot and/or aging of the mechanical components of the drive (for hard disks), or optical media where chemical processes may affect readability (aging of plastics and dye) or physical (scratches and such).

Of course, for the Linux distro thing, none of these things apply: rather, I'm worried about sticking this thing in some machine and IT doing damage to the image inside! This is where lack of an external, physical write-protect switch is worrisome.

OOOHH, now this is interesting: you can write protect an SSD? I didn't know that! I will DEFINITELY reconsider it! As for software write-protect, like I said, I'm wondering if using CramFS or SquashFS, both of which are read-only like ISO9660, may work. It'll certainly keep Linux from mounting R/W, but to windows it'll appear to be an unformatted drive. Since it's MY drive, no one will format it, but I'm more worried about some bozo that has a boot-sector or BIOS virus in his machine that will try to copy itself to the UFD, possibly trashing some data in the process. Of course, winblows may try something on its own, too...that's a big ???

I'll check it out, thanks!

Mike