LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (http://www.linuxquestions.org/questions/linux-software-2/)
-   -   Need to delete a file from disk permenantly with out using third party tools (http://www.linuxquestions.org/questions/linux-software-2/need-to-delete-a-file-from-disk-permenantly-with-out-using-third-party-tools-4175441116/)

ravirao1981 12-12-2012 07:22 AM

Need to delete a file from disk permenantly with out using third party tools
 
Hi,

I need to delete a file permenantly from disk, not just clearding discriptors but make it unrecoverable by any data recovery tools.

I have seen shred, wipe et al. I want such functionality but without using those tools as I am working on a properity hardware which has limited linux utilities.
Basically I want to implement what shred and other utilities provide.

Can anyone suggest how to proceed on this?
I have digged many websited in vain as all suggest to use readily available utility.
I would like to know if there is a way to know where exactly on disk is the file stored(may be in chunks) so that I can overwrite those areas.I know lsof can list this but is there any alternative to this as lsof isnt aviable on my box.
Any info is appreatiated.

Thanks,
Ravi

unSpawn 12-12-2012 08:18 AM

Quote:

Originally Posted by ravirao1981 (Post 4847665)
I need to delete a file permenantly from disk, not just clearding discriptors but make it unrecoverable by any data recovery tools.

Interesting.
Could you explain what scenarios you want to protect against or why thwarting recovery is important?
If your scenario doesn't need to factor in Cold Boot attack couldn't you use tmpfs?
What file system does your "properity hardware which has limited linux utilities" use and what tools does it provide?

tronayne 12-12-2012 08:51 AM

You may face a couple of obstacles depending upon the type of file system you're dealing with -- from the manual page for the shred utility:
Quote:

CAUTION: Note that shred relies on a very important assumption: that the file sys‐
tem overwrites data in place. This is the traditional way to do things, but many
modern file system designs do not satisfy this assumption. The following are exam‐
ples of file systems on which shred is not effective, or is not guaranteed to be
effective in all file system modes:

* log-structured or journaled file systems, such as those supplied with AIX and
Solaris (and JFS, ReiserFS, XFS, Ext3, etc.)

* file systems that write redundant data and carry on even if some writes fail,
such as RAID-based file systems

* file systems that make snapshots, such as Network Appliance's NFS server

* file systems that cache in temporary locations, such as NFS version 3 clients

* compressed file systems

In the case of ext3 file systems, the above disclaimer applies (and shred is thus
of limited effectiveness) only in data=journal mode, which journals file data in
addition to just metadata. In both the data=ordered (default) and data=writeback
modes, shred works as usual. Ext3 journaling modes can be changed by adding the
data=something option to the mount options for a particular file system in the
/etc/fstab file, as documented in the mount man page (man mount).

In addition, file system backups and remote mirrors may contain copies of the file
that cannot be removed, and that will allow a shredded file to be recovered later.
Might be worth a check (if you didn't already) to see if shred is present on your target machine.

If you are lucky and the file system on that system overwrites in place, you can "roll your own" something like the following.

One method is to overwrite the file two or three times (or more!) with random bits (or bytes); you can use the system random number generator (seed it from the clock) to obtain as many random numbers as you need (be careful to limit the numeric values to 0 - 127!) then unlink the file. It's not going to wipe the disk but it is going to scramble the content beyond recovery... of most folks -- governments are another entity entirely and nobody really knows for sure what capabilities are out there and in use.

Write a little C program that starts at the beginning of the file and walks to the end writing the random numbers (0 - 127) in a loop to two or three passes. After the passes complete, unlink the file.

Now, what if you're not so lucky?

The source code for srm is available (at http://downloads.sourceforge.net/srm/srm-1.2.10.tar.bz2). I won't pretend to understand the algorithm used in srm but it appears to be effective and you ought to be able to port it to that platform. That's what I'd do if it's important enough to worry about; as far as I know, srm is the effective way to do what you need to.

Other than that, there really isn't much of anything that will do an effective job that I'm aware of that isn't going to involve a whole lot more fiddling around than you've go platform capability for doing.

Give srm a try.

Hope this helps some.

ravirao1981 12-13-2012 12:22 AM

Thanks unSpawn and Tronayne.

Q) Could you explain what scenarios you want to protect against or why thwarting recovery is important?

[Ravi] I have an application running on a PPC board with customised ubuntu distro. The distro is very light in the sense it has very limited tools like grep find and other basic tools. It dosent even have lsof and stat.

The application running on the hardware creates several log file which holds confidential data. The customer wants no trace of any confidential data on the system.
Meaning he wants all the logs pertaining to an event permenaltly erased.

So, when a particular event occurs, all the identified logs should be erased forever.

As of now I am able to achieve this in a crude way.
What I do is
1) As soon as the event occurs, I initiate system reboot
2) During boot I have a code/functions in rcS to mount /data.
3) Delete the specified files, take a backup of all other files.
4 Umount /data, assign zeros to it using dd and then
5) Remount /data and copy back the backed up files

I know this is completely inefficient.
Now what I need to do is instead of rebooting the system each time the even occurs,
I would want to delete the specific files then and there and make it unrecoverable.

I have used shred, it does what I want ofcourse with limitiations pointed by tronayne.
But shred is not available on my target machine :(
Writing a c program is a way of doing it. Thanks tronayne. I will try this.

Also, I was thinking about runcating the file. does this work?? I mean does it anyway make data unrecoverable?? I am yet to try this.

Thanks,
Ravi

unSpawn 12-13-2012 05:30 AM

Quote:

Originally Posted by ravirao1981 (Post 4848195)
The application running on the hardware creates several log file which holds confidential data. The customer wants no trace of any confidential data on the system. Meaning he wants all the logs pertaining to an event permenaltly erased. So, when a particular event occurs, all the identified logs should be erased forever.

Since your scenario doesn't factor in Cold Boot attack why not use tmpfs? Or even better: ensure the application doesn't write logs in the first place?

ravirao1981 12-13-2012 05:51 AM

Cannot use tmpfs as the same h/w is used by different customers and for this specific customer the security feature is activate d using a special command. So the product is same and the feature can be activated/deactivated at the customer's wish.
Disabling logs is not an option.

tronayne 12-13-2012 08:25 AM

You can probably save yourself the trouble of writing a utility for removing the files you need to if you can try srm -- that's basically what it does:
Quote:

srm removes each specified file by overwriting, renaming, and truncating it before
unlinking. This prevents other people from undeleting or recovering any informa‐
tion about the file from the command line. By default srm uses 35 passes to over‐
write the file's contents. If this seems overkill you can use use the --dod,
--doe, --openbsd, --simple option which use less passes. If you specify more than
one option (of those listed above) they are executed in the order shown above.
The roll-your-own approach is, essentially, write your own srm.

Some years ago I wrote a "secure remove" (this is before the Internet existed in its present form) that did what I described above (it was the "approved" method) for classified data files. This was when you erased tapes with a big electromagnetic gadget, reduced disk platters to little metal chucks and all sorts of things. But that was Unix SVR4 boxes and you could get away with overwrite in place and there were no journaling file systems. Looking through the source code for srm, that's pretty much what it does -- with some extra bells and whistles -- and if I had to do it again, that's what I'd use. What the heck, the executable is 17K and all you have do is compile it (hopefully that system has the libraries, but, if it doesn't, you can build it static against the static libraries, which ought to be available on the box).

Of course, srm is significantly more sophisticated that the little utility I wrote; it's got to deal with more modern stuff than I did and, as far as I can tell, it does that rather well. Worth a shot, eh?

Hope this helps some.

unSpawn 12-13-2012 08:46 AM

Adding to that if you decide to write your own application consider having your code checked by somebody knowledgeable. Also test it well because misunderstanding file system specifics may lead to a false sense of security and create a liability.

Noway2 12-13-2012 10:17 AM

One other thing to keep in mind is that the type of media matters. Most of the secure delete utilities were designed to work with magnetic media. If you are using FLASH based media, these algorithms may not work for you as the hardware will thwart your efforts via the wear leveling mechanisms. What happens is that the data is still present, but the pointers will change to a different segment of flash. Flash provides other challenges in that it works by erasing a whole segment and then writing once to a particular address. Consequently, when you delete a file, it may still be present and readable in raw format, but a bit is set that tells you it has been deleted.

jefro 12-13-2012 12:26 PM

I forget how to find the exact memory locations a file is located at. It should be simple stuff if you go back to the basics. Find that and then you could dd over the areas.

I think I might prefer ram drive with encryption.

Noway2 12-13-2012 01:28 PM

Quote:

I think I might prefer ram drive with encryption
Actually, this brings up a good point about the benefit of encryption. If the data is not stored on physical media in unencrypted format, a lot of the need for secure erasure go away. This can be particularly important in the case of FLASH based devices, where you do not have access to the actual physical media through it's logical interface. The case I mentioned above, with FLASH and solid state drives, a portion of the memory is going to be hidden from the user and not accessible. By keeping the data encrypted, it doesn't matter.

sundialsvcs 12-13-2012 11:43 PM

I personally feel that you should never assume that you can guarantee that the data that has been recorded on a disk-drive has ever been "irretrievably overwritten." There are just too many variables involved here; too many players in the game. The file system, maybe a storage-network, even the drive hardware itself.

Shadow_7 12-14-2012 11:36 AM

If you can take the filesystem offline and time and writes are not an issue, the usual tar and move remaining files... dd over the device (maybe multiple times / bit patterns)... recreate a filesystem and untar. By no means efficient, but security is just an illusion anyway. Chances are the information traveled over a network and the middleware could have logged raw packets. Even encryption only does so much. WEP anyone?

If it's truly confidential, it should never reside on a computer in my opinion. And it should never physically leave "the vault".

I probably shouldn't ask as there probably is such a thing. But a hard driver shredder? Since it's the new paper for all intents. And which model, the molten lava? the wood chipper? the old take a ride on a floor buffer model? As I look at all my old hard drives that I'm still lugging around. Even the failed ones.

RAM drive with encryption sounds good. As long as it's local and not networked through other machines. But even that has limits. Having put /tmpfs into ram and tried watching hulu or youtube from linux running on a USB stick. With only 1GB of physical ram, and no swap space at about 40 minutes of an hour long show is when your web browser closes as the kernel protects itself from a lack of RAM.

tronayne 12-14-2012 01:28 PM

Well, old drives (that can't leave the building, let alone the computer room safe) can be taken apart (the disk stack), take a ride on a bulk tape eraser (the magnetic jobs, leave your mechanical watch somewhere else), then sent through a chipper (makes little tiny chunks). That'll pretty much make it unreadable. I suspect a few minutes in an MRI machine would do some damage too, eh? Just bolt it down good before you hit the switch. You do the same thing with LaserDisc (remember them?), CD-ROM, DVD, Blu-ray (except for the bunk eraser part); i.e., chop the buggers into little bitty pieces. And they will melt into an unrecognizable mess with the help of a little lighter fluid or charcoal starter or a ride in a 500 degree oven (they do stink when you burn, though).

I'd never let a flash drive out of the safe without chipping it, either. Hell, I wouldn't allow one of damned things into the building in the first place if I had a choice (and if it came in, it stays in for eternity -- in the safe). Wouldn't plug it in to a network connected box under any circumstances.

'Way I understand it just polishing the magnetic material off a disk isn't good enough. Allegedly, the platter retains a change at the atomic level(!) that can be read. Lordy, lordy, lordy, who'd have the equipment to do that, me wonders (well, I kind suspect I know who does -- the guys that came up with the destruct rules -- but I don't think they'd care too much about anything on my stuff nowadays).

Can you get carried away with this stuff? Uh, yeah. But, if you've got a copier that you're going to scrap you might want to pull the disk driver out of it before you dump it if you ever copied any proprietary information you don't want out in the world.

Bottom line is if you care about it, physically destroy it when you're done with it.


All times are GMT -5. The time now is 07:31 PM.