Need to delete a file from disk permenantly with out using third party tools
 

Hi,

I need to delete a file permenantly from disk, not just clearding discriptors but make it unrecoverable by any data recovery tools.

I have seen shred, wipe et al. I want such functionality but without using those tools as I am working on a properity hardware which has limited linux utilities.
Basically I want to implement what shred and other utilities provide.

Can anyone suggest how to proceed on this?
I have digged many websited in vain as all suggest to use readily available utility.
I would like to know if there is a way to know where exactly on disk is the file stored(may be in chunks) so that I can overwrite those areas.I know lsof can list this but is there any alternative to this as lsof isnt aviable on my box.
Any info is appreatiated.

Thanks,
Ravi

Interesting.
Could you explain what scenarios you want to protect against or why thwarting recovery is important?
If your scenario doesn't need to factor in Cold Boot attack couldn't you use tmpfs?
What file system does your "properity hardware which has limited linux utilities" use and what tools does it provide?

You may face a couple of obstacles depending upon the type of file system you're dealing with -- from the manual page for the shred utility:
Might be worth a check (if you didn't already) to see if shred is present on your target machine.

If you are lucky and the file system on that system overwrites in place, you can "roll your own" something like the following.

One method is to overwrite the file two or three times (or more!) with random bits (or bytes); you can use the system random number generator (seed it from the clock) to obtain as many random numbers as you need (be careful to limit the numeric values to 0 - 127!) then unlink the file. It's not going to wipe the disk but it is going to scramble the content beyond recovery... of most folks -- governments are another entity entirely and nobody really knows for sure what capabilities are out there and in use.

Write a little C program that starts at the beginning of the file and walks to the end writing the random numbers (0 - 127) in a loop to two or three passes. After the passes complete, unlink the file.

Now, what if you're not so lucky?

The source code for srm is available (at http://downloads.sourceforge.net/srm/srm-1.2.10.tar.bz2). I won't pretend to understand the algorithm used in srm but it appears to be effective and you ought to be able to port it to that platform. That's what I'd do if it's important enough to worry about; as far as I know, srm is the effective way to do what you need to.

Other than that, there really isn't much of anything that will do an effective job that I'm aware of that isn't going to involve a whole lot more fiddling around than you've go platform capability for doing.

Give srm a try.

Hope this helps some.

Thanks unSpawn and Tronayne.

Q) Could you explain what scenarios you want to protect against or why thwarting recovery is important?

[Ravi] I have an application running on a PPC board with customised ubuntu distro. The distro is very light in the sense it has very limited tools like grep find and other basic tools. It dosent even have lsof and stat.

The application running on the hardware creates several log file which holds confidential data. The customer wants no trace of any confidential data on the system.
Meaning he wants all the logs pertaining to an event permenaltly erased.

So, when a particular event occurs, all the identified logs should be erased forever.

As of now I am able to achieve this in a crude way.
What I do is
1) As soon as the event occurs, I initiate system reboot
2) During boot I have a code/functions in rcS to mount /data.
3) Delete the specified files, take a backup of all other files.
4 Umount /data, assign zeros to it using dd and then
5) Remount /data and copy back the backed up files

I know this is completely inefficient.
Now what I need to do is instead of rebooting the system each time the even occurs,
I would want to delete the specific files then and there and make it unrecoverable.

I have used shred, it does what I want ofcourse with limitiations pointed by tronayne.
But shred is not available on my target machine :(
Writing a c program is a way of doing it. Thanks tronayne. I will try this.

Also, I was thinking about runcating the file. does this work?? I mean does it anyway make data unrecoverable?? I am yet to try this.

Thanks,
Ravi

Since your scenario doesn't factor in Cold Boot attack why not use tmpfs? Or even better: ensure the application doesn't write logs in the first place?

Cannot use tmpfs as the same h/w is used by different customers and for this specific customer the security feature is activate d using a special command. So the product is same and the feature can be activated/deactivated at the customer's wish.
Disabling logs is not an option.

You can probably save yourself the trouble of writing a utility for removing the files you need to if you can try srm -- that's basically what it does:
The roll-your-own approach is, essentially, write your own srm.

Some years ago I wrote a "secure remove" (this is before the Internet existed in its present form) that did what I described above (it was the "approved" method) for classified data files. This was when you erased tapes with a big electromagnetic gadget, reduced disk platters to little metal chucks and all sorts of things. But that was Unix SVR4 boxes and you could get away with overwrite in place and there were no journaling file systems. Looking through the source code for srm, that's pretty much what it does -- with some extra bells and whistles -- and if I had to do it again, that's what I'd use. What the heck, the executable is 17K and all you have do is compile it (hopefully that system has the libraries, but, if it doesn't, you can build it static against the static libraries, which ought to be available on the box).

Of course, srm is significantly more sophisticated that the little utility I wrote; it's got to deal with more modern stuff than I did and, as far as I can tell, it does that rather well. Worth a shot, eh?

Hope this helps some.

Adding to that if you decide to write your own application consider having your code checked by somebody knowledgeable. Also test it well because misunderstanding file system specifics may lead to a false sense of security and create a liability.

One other thing to keep in mind is that the type of media matters. Most of the secure delete utilities were designed to work with magnetic media. If you are using FLASH based media, these algorithms may not work for you as the hardware will thwart your efforts via the wear leveling mechanisms. What happens is that the data is still present, but the pointers will change to a different segment of flash. Flash provides other challenges in that it works by erasing a whole segment and then writing once to a particular address. Consequently, when you delete a file, it may still be present and readable in raw format, but a bit is set that tells you it has been deleted.

I forget how to find the exact memory locations a file is located at. It should be simple stuff if you go back to the basics. Find that and then you could dd over the areas.

I think I might prefer ram drive with encryption.

Actually, this brings up a good point about the benefit of encryption. If the data is not stored on physical media in unencrypted format, a lot of the need for secure erasure go away. This can be particularly important in the case of FLASH based devices, where you do not have access to the actual physical media through it's logical interface. The case I mentioned above, with FLASH and solid state drives, a portion of the memory is going to be hidden from the user and not accessible. By keeping the data encrypted, it doesn't matter.

I personally feel that you should never assume that you can guarantee that the data that has been recorded on a disk-drive has ever been "irretrievably overwritten." There are just too many variables involved here; too many players in the game. The file system, maybe a storage-network, even the drive hardware itself.

If you can take the filesystem offline and time and writes are not an issue, the usual tar and move remaining files... dd over the device (maybe multiple times / bit patterns)... recreate a filesystem and untar. By no means efficient, but security is just an illusion anyway. Chances are the information traveled over a network and the middleware could have logged raw packets. Even encryption only does so much. WEP anyone?

If it's truly confidential, it should never reside on a computer in my opinion. And it should never physically leave "the vault".

I probably shouldn't ask as there probably is such a thing. But a hard driver shredder? Since it's the new paper for all intents. And which model, the molten lava? the wood chipper? the old take a ride on a floor buffer model? As I look at all my old hard drives that I'm still lugging around. Even the failed ones.

RAM drive with encryption sounds good. As long as it's local and not networked through other machines. But even that has limits. Having put /tmpfs into ram and tried watching hulu or youtube from linux running on a USB stick. With only 1GB of physical ram, and no swap space at about 40 minutes of an hour long show is when your web browser closes as the kernel protects itself from a lack of RAM.

Well, old drives (that can't leave the building, let alone the computer room safe) can be taken apart (the disk stack), take a ride on a bulk tape eraser (the magnetic jobs, leave your mechanical watch somewhere else), then sent through a chipper (makes little tiny chunks). That'll pretty much make it unreadable. I suspect a few minutes in an MRI machine would do some damage too, eh? Just bolt it down good before you hit the switch. You do the same thing with LaserDisc (remember them?), CD-ROM, DVD, Blu-ray (except for the bunk eraser part); i.e., chop the buggers into little bitty pieces. And they will melt into an unrecognizable mess with the help of a little lighter fluid or charcoal starter or a ride in a 500 degree oven (they do stink when you burn, though).

I'd never let a flash drive out of the safe without chipping it, either. Hell, I wouldn't allow one of damned things into the building in the first place if I had a choice (and if it came in, it stays in for eternity -- in the safe). Wouldn't plug it in to a network connected box under any circumstances.

'Way I understand it just polishing the magnetic material off a disk isn't good enough. Allegedly, the platter retains a change at the atomic level(!) that can be read. Lordy, lordy, lordy, who'd have the equipment to do that, me wonders (well, I kind suspect I know who does -- the guys that came up with the destruct rules -- but I don't think they'd care too much about anything on my stuff nowadays).

Can you get carried away with this stuff? Uh, yeah. But, if you've got a copier that you're going to scrap you might want to pull the disk driver out of it before you dump it if you ever copied any proprietary information you don't want out in the world.

Bottom line is if you care about it, physically destroy it when you're done with it.