I was reading a column on cnet about deleted files in the iPod hard drives. I knew that as far as FAT filesystems went, deleted files could be undeleted because files are flagged when they're 'deleted' and not actually obliterated.

Now, I don't know what filesystem the iPod uses on their hdds (I don't own one), but what about the ext3 and reiserfs filesystems? Is data security a matter of filesystem design, or is it a hardware limitation?

Is this just a matter of using magnetic drives that have this problem? When I delete something, I'd like to be sure it is truly gone, and not just flagged.

Does flash memory work the same way?

Almost all writable media works this way. Otherwise, erasing a 10GB file would mean writing 10GB - really 50 to 60GB with magnetic media, due to residual magnetic polarization.

Over time, files are overwritten. While there are some programs that claim to securely erase files (by multiply overwriting), their effectiveness is a function of the underlying filesystems allocation mechanism.

For example, from the man page for the Linux 'shred' command:

CAUTION: Note that shred relies on a very important assumption: that
the filesystem overwrites data in place. This is the traditional way
to do things, but many modern filesystem designs do not satisfy this
assumption. The following are examples of filesystems on which shred
is not effective:

* log-structured or journaled filesystems, such as those supplied with

AIX and Solaris (and JFS, ReiserFS, XFS, Ext3, etc.)

* filesystems that write redundant data and carry on even if some
writes

fail, such as RAID-based filesystems

* filesystems that make snapshots, such as Network Appliance’s NFS
server

* filesystems that cache in temporary locations, such as NFS

version 3 clients

* compressed filesystems

In addition, file system backups and remote mirrors may contain copies
of the file that cannot be removed, and that will allow a shredded file
to be recovered later.

None of the 'mainstream' filesystems actually overwrite the data sectors of files when deleting: fat, ntfs, ext2/3, xfs, or reiserfs. They just remove varying amounts of filesystem metadata. There is a program called 'shred' that can help, on some filesystems.

Interesting. Thanks for the insight.

But what about flash RAM media? Would those be susceptible to 'residual' data if the data is disposed of properly through say, the shred command, and it's on a FAT filesystem?

I would expect flash media to not have all the problems of magnetic media "if the data is disposed of properly". But the ultimate way to properly dispose of such data is unbeknownst to me.

You could try a little test. Write your phone number and address to your flashdrive memory. Then erase it as securely as you know how. Tag it with a sticker that says "Al-Quieda meeting, this Friday, 7:00pm, my house" and mail it off to the NSA. See if anybody shows up!

;-)

LOL @ haertig.

In all seriousness, flash will probably not have residual data issues, once the individual bytes/sectors have been overwritten.

The info page for 'shred' says that if it is used on a journaled file system it won't "reliably operate". Does this mean that it provides no shredding whatsoever or does it mean that files will be shredded to some extent but not as securely as on a non-journaled filesystem?

Reading this seems to suggest that its only insecure because the journal keeps track of the files that have been shredded. Other than this though, is the security level the same as on a non-journaled file system?

One really handy utility is "Darik's Boot and Nuke"--a mini-linux and secure erase program that boots from a floppy. It includes some of the routines that are used to erase classified data.

AFAIK, it should be. Provided, of course, that the filesystem does in-place overwrites. I am fairly certain ext3 does.

Qwerty: The problem on a journaled file system is that there's no way to guarantee that the writes are occuring to the sectors of the file (as opposed to newly alllocated sectors).

In that case, the only way to shred is at the device or partition level. That is shreding the partition /dev/hda1 (for example) will get rid of the data on that partition - because there's no filesystem in the way.

As an alternative to shred, you can look at scrub which "can also create a file and expand it until a file system is full in order to attempt to reclaim and scrub file systems blocks belonging to deleted files if scrubbing the raw disk device is not possible"

Either of the above methods will work under any filesystem, but it can take a long time (many hours) depending on the size of the partition or the amount of free space respectively.