How to Securely Delete Your Files???

I was wondering if there is a better shred utility. The shred utility is supposed to delete files securely from your computer. However, as the man page mentions:

       "CAUTION:  Note  that  shred relies on a very important assumption: that
       the filesystem overwrites data in place.  This is the  traditional  way
       to  do  things,	but many modern filesystem designs do not satisfy this
       assumption.  The following are examples of filesystems on  which  shred
       is not effective:

       * log-structured or journaled filesystems, such as those supplied with

	      AIX and Solaris (and JFS, ReiserFS, XFS, Ext3, etc.)"

I think almost everyone here now uses a journaled filesystem. So how would you securely delete your date (+ scramble the file name)? Is there any utility equivalent to shred that works on journaled filesystems? Thanks for your time..

In my search, I came across srm (secure rm - h**p://srm.sourceforge.net/). Unfortunately, it too has the problem. The following was taken from the README file that came with it:

	"All users, but especially Linux users, should be aware that srm will
	only work on file systems that overwrite blocks in place. In
	particular, it will _NOT_ work on resiserfs or the vast majority of
	journaled file systems. It should work on ext2, FAT-based file
	systems, and the BSD native file system. Ext3 users should be
	especially careful as it can be set to journal data as well, which is
	an obvious route to reconstructing information.

	As of this writing, the state of the Linux kernel prohibits any
	planning about solving the above problems prior to the 2.6 kernel
	series. Indeed, by then, other developments may make the entire issue
	a moot point. Either the LSM framework will make writing the kernel
	portion trivial, or encrypted file systems will be common enough to
	make srm nearly useless."

I don't quite understand the part about the 2.6 kernel solving this problem. Can anyone shed some light? Thanks...

Originally posted by BinaryDigit01 In my search, I came across srm (secure rm - h**p://srm.sourceforge.net/). Unfortunately, it too has the problem. The following was taken from the README file that came with it: "All users, but especially Linux users, should be aware that srm will only work on file systems that overwrite blocks in place. In particular, it will _NOT_ work on resiserfs or the vast majority of journaled file systems. It should work on ext2, FAT-based file systems, and the BSD native file system. Ext3 users should be especially careful as it can be set to journal data as well, which is an obvious route to reconstructing information. As of this writing, the state of the Linux kernel prohibits any planning about solving the above problems prior to the 2.6 kernel series. Indeed, by then, other developments may make the entire issue a moot point. Either the LSM framework will make writing the kernel portion trivial, or encrypted file systems will be common enough to make srm nearly useless." I don't quite understand the part about the 2.6 kernel solving this problem. Can anyone shed some light? Thanks...

I think it means that kernel 2.6 includes some options so that the developers of srm can fix the problem.

Sorry for bumping this thread up, but I'm sure there's got to be some way. Without the ability to securely delete files forever, companies would not store sensitive information on journaling filesystems. Even FAT32 and NTFS have ways of securely deleting files.

I don't know much about this, but I guess what I'd suggest is to either use a non-journaled filesystem for your sensitive data (one that will work with shredding utilities), or use an encrypted filesystem, which should make shredding unnecessary. There'd probably be some kind of performance hit for using an encrypted FS, but if security is that important, it won't matter.

And hey, if FAT32 and NTFS allow secure deletions, then use one of those filesystems! Unlike Windows, at least Linux gives you many filesystems to choose from

Originally posted by wapcaplet And hey, if FAT32 and NTFS allow secure deletions, then use one of those filesystems!

Hehe. Not quite. I have a strong dislike for MS and rarely use it unless I have to do some C# programming or play some games that don't quite work 100% under wine and winex. It's just that I'm a little upset and disappointed that there are a zillion programs under Windows to securely delete files yet I haven't come across a single one under Linux that can securely delete files under ANY journalled filesystem.

Well, I'd guess that journaled filesystems were not designed with that kind of security in mind. Journaling just doesn't seem very compatible with covering your tracks, and probably wasn't intended to be. Think about it - if you're a secret agent, and you want to conceal your knowledge and actions, you won't keep a diary and leave it lying around.

If you want the benefits that journaling provides, use it. But if you want the kind of security you're talking about, you have a number of other filesystems to choose from. Even ext2 (the default filesystem on many Linux distributions) allows secure deletion, since it does not use a journal. If you want the advantages of journaling and the capabitility to secure your sensitive data, you can use some kind of strong encryption on individual files.

P.S. - This discussion brings up a valid point - if you have important data, you're likely to create backups of it at some point, which means that you'd have to shred those backups too (which may be impossible, if they're on a write-once medium like CD-R. You'd have to destroy the whole disc, which may contain other stuff you want to keep.); overall, it makes more sense to just encrypt your sensitive files. Then you can back them up all you like, and shredding becomes irrelevant.

That link from wapcaplet also says NTFS is journal, but it is possible to shred files on that!

With an ext3 filesystem, you have chattr utility which allow you to set file attributes like this one:

When a file with the `s' attribute set is deleted, its blocks are zeroed and written back to the disk.

Oliv'

Originally posted by wapcaplet If you want the advantages of journaling and the capabitility to secure your sensitive data, you can use some kind of strong encryption on individual files.

Even if I do use strong encryption on files, I still have to remove/delete the source files. For example, I put all my sensitive docs, ppts, and excel sheets in a folder and tar it and then encrypt it using gnupg. That's fine. But when I come to delete the source files (the unencrypted tar and the doc, ppt, and excel files), I still have a problem, since they can't be securely deleted. So you can't really have the advantage of journaling and the capability to secure your sensitive data. I guess I have to stick to containers or a non-journaled file system.

Originally posted by Oliv' With an ext3 filesystem, you have chattr utility which allow you to set file attributes like this one:

According to the man page (man chattr), it says in the "BUGS AND LIMITATIONS" section:

	As of Linux 2.2, the 'c', 's', and 'u' attributes are not honored by the
	kernel filesystem code.  These attributes will be implemented in the future
	ext2fs version.

The above is for e2fsprogs version 1.32. Is there a new version that solves this issue?

Originally posted by BinaryDigit01 Even if I do use strong encryption on files, I still have to remove/delete the source files. For example, I put all my sensitive docs, ppts, and excel sheets in a folder and tar it and then encrypt it using gnupg. That's fine. But when I come to delete the source files (the unencrypted tar and the doc, ppt, and excel files), I still have a problem, since they can't be securely deleted. So you can't really have the advantage of journaling and the capability to secure your sensitive data. I guess I have to stick to containers or a non-journaled file system.

OK, one solution I can think of for that situation is to set aside a small portion of your hard drive (say 100MB) for just this purpose; put a non-journaling filesystem on it. Save all your sensitive source files there. After they're tarred and encrypted, shred them (or for that matter, shred the entire partition). Rinse, repeat.

Solution number two: Use an encrypted filesystem for all your sensitive work. EncFS, for instance.

Solution number three: Use a Loopback encrypted filesystem. A loopback filesystem is just a regular file on your existing (journaled) filesystem that you can mount. Everything on it is encrypted, so there is little risk of anyone recovering info from it. Use that filesystem to do all your sensitive work.

I've also heard about encrypted journaled filesystems, though I'm not sure which ones can do this.

I managed to get an encrypted ReiserFS with SuSE.

Thanks a lot wapcaplet, EncFS is just what I was looking for. Sorry for being such a nuisance

Hi all

I just dont understand how the journaling cause an issue in shred. Is it because:

1. journaling creates a copy of my "sensitive" data file so even if I shred the file, a copy if it is still stored in the journal. Sure enough over time the journaled space will be reuse but it wont be "securely" delete and we dont know when it gets reuse?

2. journaling causes my overwrite to write to a different data blocks in the harddisk rather than the orginal data block. In other words, journaling reallocate my data blocks?


my thoughts:
I think it is number one that is the issue right. Some body told me number two is the issue but I kind of doubt that journaling reallocate data blocks.

As a matter of fact, the ext2 filesystem does dynamic block allocation. Though it is not documented, I have tested a few times and pin point the case where shred would fail due to dynamic block allocation.

I create a file with hole: echo -n "X" | dd of=/tmp/hole bs=1024 seek=10
determine the blocks using : debugfs
copy the blocks to a tmp location using : fsgrab
shred the /tmp/hole file
determine the new blocks that the file occupy : debugfs
copy the blocks to another temp location using: fsgrab

well, guess what, some of the old blocks still contain old data, READ MY LIPS, they dont get overwritten.

scary isn't it. Many ask what's the chance of a file having enough holes to cause the problem. I am not sure. Any one know the answer to this ???

Regards,

BunZ