hi guys

i'm just wondering whats the state of linux support for hotswapping usb thumbdrives? i mean, if i'm copying file(s) from the hdd to the thumbdrive, and then rudely unplug the thumbdrives in the middle of the copy, to what extent does this foul things up?

- will entire thumbdrive be corrupted, unreadable, lost chains, etc?
- will any file(s) already copied be readable?
- will any existing file(s) residing on the thumbdrive be adversely affected? (ie. lost data)
- how will the file that was partially copied (when the thumbdrive was remove) affect the thumbdrive?

if this is not the right forum, i would appreciate it if somebody might point me to the right direction.

ps. yes, i understand this is not the "right" thing to do, but i just wanted to know ...

<removed advertising> flash disk branded NETAC which has patented Ultra Stable Technology (UST) which prevents disk and data corruption.

Many flash disks have this issue and may subject to disk or data corruption should they plug out while accessing. Files may not be synchronised and FAT table may be corrupted. In the worst case, the disk has to be thrown away as warranty has expired. This is painful for high capacity disk like 4GB and 8GB which still cost a fortune.

For UST OnlyDisk, tests show that files are verified after copied and corruption can be prevented. The only compromise is speed which is understandable as we cannot have the best of both worlds.

If you have an expandable flash disk, experiment with it. Be ready to throw away if corruption occurs. FYI, UST is not 110% guranteed but will reduce such possibilities.

<removed advertising>

right, so your usb thumbdrive does this ... but thats not the solution i'm looking for. i'm wondering what the current state of linux is with regards to using off-the-shelf nobrand thumbdrives.

Moved: This thread is more suitable in Linux-General and has been moved accordingly to help your thread/question get the exposure it deserves.

A: Maybe, maybe not
A: Maybe, maybe not
A: Maybe, maybe not
A: Maybe, maybe not

Poor answers you might think, but the truth is that there's no answer, because it all depends on the filesystem, and exactly what was or was not written to the drive when it was unplugged mid-transfer. That's why this is "not the right thing to do".

oh, the website with information was blocked and replaced with <no advertising>.

well, you know the brand and just add .com to complete the address.

For early days, file system for flash disk is always FAT16. Formatting to FAT32 or NTFS or linux FS may make the disk works but unstable. For new flash disks, you can format to newer FS but much depend on what the chip supports. The reason is clear.... 16GB and higher capacity are out.

If it is no brand flash disks, you are paying for what it does. Just don't tax it more 'cos you pay for what you get.

UST basically ensures the file is copied over before the next file is sent. For other brands, the idea is just focus on speed and dump everything to cache. That's why you can see the flash disk is STILL flashing few seconds even though the copy window is closed and you assume the file is transferred. User who plug the disk out during these few seconds has the highest risk of disk or data corruption.

If one hotswapping a USB thumbdrive in Windows one asks the operating system permission to do so. Same applies in Linux.

If you want to remove thumbdrive sda just use the terminal command to remove it
You may see the short burst of activities in the thumbdrive and the mounted folders disappear before OS can respond back with a prompt signalling it is ready for the removal.

Some distros keep the files you require for copying into the thumbdrive temporarily in the ram and execute them not always at time you wish them to for various reasons. Some may do it immediately. The only sure fire method to get the files copied is to unmount the device. This tells the OS it has no more chance to delay and must complete the file transfer immediately.

If one remove the thumbdrive without "umount" it then there exits the opportunity of some files may not get transferred or the transfer got interrupted resulting corruption to the filing system or the partition table.

Common sense really. If you fire an employee on the spot you can tell him

(a) Drop everything, here is your pay and you can leave now.

or

(b) Finish what you are doing, here is your pay and you can leave afterward.

That's bollocks. If I pull the device during write access
the file will be truncated, and the state of the FAT or
i-node table will be dubious; no matter what brand of disk
you use. The device itself has no impact on the file-system.

Nice marketing blurb, though. Happy hunting, even though I
honestly hope that not too many people will believe what you're
telling them.



Cheers,
Tink

I wanted to make my original question as generic as possible as I didn't want to taint responses. But now that I have gotten a couple, here's the whole story.

My company's making an embedded device which uses linux as OS and the hw has a micro-sd slot. The developers are citing several technical challenges (presumeably linux) blocking full hotswap support.

I think most folks expect that if you do this in the middle of a file transfer, you'd be screwed ... That said, one would expect that it is only the file that is in the middle of the file transfer that would be screwed. The other data on the card itself should not lose integrity. But thats just my opinion.

So I just wanted the low-down from people who actually dive into the code .. I'm more of a user than a programmer so I'm quite clueless on this

For a Linux system writing to a (V)FAT file-system just make
the line in fstab have sync,noatime ... that *should* do the
trick. If that's still not what they want ... :}



Cheers,
Tink

Not yet a widespread option, but using ZFS would prevent all existing data and file-system corruption.

well unfortunately, most removable flash memory in the market now will be FAT16 or FAT32 formatted, so you really can't fight the tide with a more superior filesystem ... gotta take the pragmatic approach

A normal USB flash drives is very slow and formatting it into a complicated filing system doesn't pay.

The new 4Gb I bought the other day states reading speed is 12Mb/s and writing is only 3.5Mb/s. A USB hard disk on the other hand can be written at 20 to 25Mb/s.

Seem to me jlliagre's recommendation of mounting the USB flash drive with "sync" option is the best way to prevent the file corruption.

So far my experiences with ZFS on Solaris 10/SPARC have
been negative :} .... the most notable were poor performance,
high memory consumption and causing of kernel panics.



Cheers,
Tink

Pulling a usb disk out is similar to sudden power failure during file transfer to usb disk. I am simply stating that data or disk corruption to usb disk (not the harddisk!) may occur, just like pulling your harddisk out of your PC during power on.

Why must OS perform a checkdisk when improper shut down? Why must we unmount everytime before plug out? If you are confident your usb disk will not fail, try pull it out without unmounting.

I am not trying to market but share information. After all, it is sad to tell customers that their usb disks are dead and they lost their valuable data inside which cannot replace by a new one.