corrupted ntfs partition table on usb drive
 

Hi all, first post here. I'm setting up a backup system based on Raspberry Pi and an external 2TB USB drive. A cron will drive rsync backup from my NAS. As I have to transfer in excess of 200GB of data I had formatted my USB drive with NTFS and have ntfs-3g on the Raspian box (Debian Wheezy optimised for arm processor) with the intention of using my Windows 7 PC to populate the USB drive via USB3 (complex set up - don't ask :) ).

It was working fine until I shut down the Raspian box and removed the USB drive (didn't umount first). The consequence of this was a corrupted ntfs partition table that neither Windows 7 PC nor any of my linux boxes could fix. Nothing is lost but I'm reluctant to repeat this and might take the pain of slow network transfer and just go ext4 for the USB drive. It just means I can't use it as a portable drive for a Windows laptop when I'm away.

The real question though, is ntfs-3g not reliable enough to use, or should I have unmounted the drive before shut-down, or is this just a one-off bad luck occurrence? Given how slow the Raspberry Pi is it is possible it hadn't quite finished shut down before I unplugged the USB drive a minute or two later.

I never had problems with vfat (well I did, but I deserved them).
It doesn't have the security hangups that ntfs has. Tools fairly good in both systems.

I don't get the windows 7 populate and ntfs-3g deal. Where is the flash drive installed at?

No, I spared you all the details. My NAS is 2TB. At the moment a Windows PC runs a winscp script as a daily batch job that copies NAS data to a 750GB external USB. That is the one I was going to use to initialise the new 2TB USB drive which will afterwards be connected to the Raspberry Pi and which will take over the NAS backup function. Then the 750GB USB drive will be re-deployed elsewhere. So really it's a one-off data transfer to get the 2TB backup disk sync'ed with the NAS 2TB drive. Then *if* I can use ntfs on the backup drive, I have the added bonus of being able to transport my media files with me when I travel, having access only to a Windows laptop. Hope this clears it up :)

Some of us lead (over)complicated lives :)

I'm not familiar with the Pi but I would assume that all file systems will be automatically unmounted during the shutdown process. If you are not waiting until the Pi is completely shutdown then it is possible to corrupt the drive.

If you can't wait to unplug drive until the Pi is shutdown then I would manually unmount it. Be sure to close any applications that are currently access the device as well as terminal windows where its working directory are pointing to the drive.

OK, so I am not more likely to suffer a disk corruption using ntfs-3g than ext4. Both are equally likely to become corrupted by premature removal of disk? In other words, my partition table damage was more likely due to premature disk disconnection than ntfs-3g issues?

User error. Writes are bufferred, and depending on the filesystem, possibly journalled as well. If you disconnect a drive during update, data has a very high chance of corruption. This is why power drops can have the same effect.

Always umount. For something like this I'd manually umount and not rely on shutdown scripts - they have been known to be impatient ...

Do you have evidence of partition table damage, or are you simply mis-interpreting filesystem corruption in NTFS ?.

I agree with syg00 and would not expect the partition table to be corrupted. Since we do not know how the drive is partitioned and with out knowing the exact error messages we can not say.

Typically NTFS would complain of a dirty of unclean shutdown. Any open files could become corrupted as well as losing data that was pending to be written to disk. As stated no different then ext filesystems.

I can't remember the exact message, something about the possibility of fake raid, running chkdsk /f and mounting windows twice (seemed an odd remark). My browser search history shows I was looking for way to mend a corrupt master file table. Anyway, several attempts at chkdsk /f and fsck on both Win7 & linux boxes failed to fix it and returned essentially the same corrupt file table message. No real harm done as the files were backups and I was just testing the drive. I'm just anxious to avoid a repeat when it goes into deployment.

michaelk: drive was a single ntfs partition on a 160GB USB drive (was testing the arrangement on an old drive) formatted to ntfs using gparted.

Ok, that's filesystem corruption.
Make sure it's umount'd properly and you should be fine - NTFS, ext4, FAT32, whatever.

Just for the record, I've opted for ext4 as nfts had really poor write speeds (assessed by time sh -c "dd if=/dev/zero of=/mnt/backup/ddfile bs=8k count=30000 && sync") and exfat caused errors with some of the rsync opts (file permissions - chgrp "/some/file" failed: Function not implemented ). Probably best to stay with 'native' filesystem anyway. Thanks everyone for the assistance.