LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 12-12-2012, 06:22 AM   #1
ravirao1981
LQ Newbie
 
Registered: Mar 2012
Posts: 8

Rep: Reputation: Disabled
Need to delete a file from disk permenantly with out using third party tools


Hi,

I need to delete a file permenantly from disk, not just clearding discriptors but make it unrecoverable by any data recovery tools.

I have seen shred, wipe et al. I want such functionality but without using those tools as I am working on a properity hardware which has limited linux utilities.
Basically I want to implement what shred and other utilities provide.

Can anyone suggest how to proceed on this?
I have digged many websited in vain as all suggest to use readily available utility.
I would like to know if there is a way to know where exactly on disk is the file stored(may be in chunks) so that I can overwrite those areas.I know lsof can list this but is there any alternative to this as lsof isnt aviable on my box.
Any info is appreatiated.

Thanks,
Ravi
 
Old 12-12-2012, 07:18 AM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Quote:
Originally Posted by ravirao1981 View Post
I need to delete a file permenantly from disk, not just clearding discriptors but make it unrecoverable by any data recovery tools.
Interesting.
Could you explain what scenarios you want to protect against or why thwarting recovery is important?
If your scenario doesn't need to factor in Cold Boot attack couldn't you use tmpfs?
What file system does your "properity hardware which has limited linux utilities" use and what tools does it provide?
 
Old 12-12-2012, 07:51 AM   #3
tronayne
Senior Member
 
Registered: Oct 2003
Location: Northeastern Michigan, where Carhartt is a Designer Label
Distribution: Slackware 32- & 64-bit Stable
Posts: 3,541

Rep: Reputation: 1065Reputation: 1065Reputation: 1065Reputation: 1065Reputation: 1065Reputation: 1065Reputation: 1065Reputation: 1065
You may face a couple of obstacles depending upon the type of file system you're dealing with -- from the manual page for the shred utility:
Quote:
CAUTION: Note that shred relies on a very important assumption: that the file sys‐
tem overwrites data in place. This is the traditional way to do things, but many
modern file system designs do not satisfy this assumption. The following are exam‐
ples of file systems on which shred is not effective, or is not guaranteed to be
effective in all file system modes:

* log-structured or journaled file systems, such as those supplied with AIX and
Solaris (and JFS, ReiserFS, XFS, Ext3, etc.)

* file systems that write redundant data and carry on even if some writes fail,
such as RAID-based file systems

* file systems that make snapshots, such as Network Appliance's NFS server

* file systems that cache in temporary locations, such as NFS version 3 clients

* compressed file systems

In the case of ext3 file systems, the above disclaimer applies (and shred is thus
of limited effectiveness) only in data=journal mode, which journals file data in
addition to just metadata. In both the data=ordered (default) and data=writeback
modes, shred works as usual. Ext3 journaling modes can be changed by adding the
data=something option to the mount options for a particular file system in the
/etc/fstab file, as documented in the mount man page (man mount).

In addition, file system backups and remote mirrors may contain copies of the file
that cannot be removed, and that will allow a shredded file to be recovered later.
Might be worth a check (if you didn't already) to see if shred is present on your target machine.

If you are lucky and the file system on that system overwrites in place, you can "roll your own" something like the following.

One method is to overwrite the file two or three times (or more!) with random bits (or bytes); you can use the system random number generator (seed it from the clock) to obtain as many random numbers as you need (be careful to limit the numeric values to 0 - 127!) then unlink the file. It's not going to wipe the disk but it is going to scramble the content beyond recovery... of most folks -- governments are another entity entirely and nobody really knows for sure what capabilities are out there and in use.

Write a little C program that starts at the beginning of the file and walks to the end writing the random numbers (0 - 127) in a loop to two or three passes. After the passes complete, unlink the file.

Now, what if you're not so lucky?

The source code for srm is available (at http://downloads.sourceforge.net/srm/srm-1.2.10.tar.bz2). I won't pretend to understand the algorithm used in srm but it appears to be effective and you ought to be able to port it to that platform. That's what I'd do if it's important enough to worry about; as far as I know, srm is the effective way to do what you need to.

Other than that, there really isn't much of anything that will do an effective job that I'm aware of that isn't going to involve a whole lot more fiddling around than you've go platform capability for doing.

Give srm a try.

Hope this helps some.
 
1 members found this post helpful.
Old 12-12-2012, 11:22 PM   #4
ravirao1981
LQ Newbie
 
Registered: Mar 2012
Posts: 8

Original Poster
Rep: Reputation: Disabled
Thanks unSpawn and Tronayne.

Q) Could you explain what scenarios you want to protect against or why thwarting recovery is important?

[Ravi] I have an application running on a PPC board with customised ubuntu distro. The distro is very light in the sense it has very limited tools like grep find and other basic tools. It dosent even have lsof and stat.

The application running on the hardware creates several log file which holds confidential data. The customer wants no trace of any confidential data on the system.
Meaning he wants all the logs pertaining to an event permenaltly erased.

So, when a particular event occurs, all the identified logs should be erased forever.

As of now I am able to achieve this in a crude way.
What I do is
1) As soon as the event occurs, I initiate system reboot
2) During boot I have a code/functions in rcS to mount /data.
3) Delete the specified files, take a backup of all other files.
4 Umount /data, assign zeros to it using dd and then
5) Remount /data and copy back the backed up files

I know this is completely inefficient.
Now what I need to do is instead of rebooting the system each time the even occurs,
I would want to delete the specific files then and there and make it unrecoverable.

I have used shred, it does what I want ofcourse with limitiations pointed by tronayne.
But shred is not available on my target machine
Writing a c program is a way of doing it. Thanks tronayne. I will try this.

Also, I was thinking about runcating the file. does this work?? I mean does it anyway make data unrecoverable?? I am yet to try this.

Thanks,
Ravi
 
Old 12-13-2012, 04:30 AM   #5
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Quote:
Originally Posted by ravirao1981 View Post
The application running on the hardware creates several log file which holds confidential data. The customer wants no trace of any confidential data on the system. Meaning he wants all the logs pertaining to an event permenaltly erased. So, when a particular event occurs, all the identified logs should be erased forever.
Since your scenario doesn't factor in Cold Boot attack why not use tmpfs? Or even better: ensure the application doesn't write logs in the first place?
 
Old 12-13-2012, 04:51 AM   #6
ravirao1981
LQ Newbie
 
Registered: Mar 2012
Posts: 8

Original Poster
Rep: Reputation: Disabled
Cannot use tmpfs as the same h/w is used by different customers and for this specific customer the security feature is activate d using a special command. So the product is same and the feature can be activated/deactivated at the customer's wish.
Disabling logs is not an option.
 
Old 12-13-2012, 07:25 AM   #7
tronayne
Senior Member
 
Registered: Oct 2003
Location: Northeastern Michigan, where Carhartt is a Designer Label
Distribution: Slackware 32- & 64-bit Stable
Posts: 3,541

Rep: Reputation: 1065Reputation: 1065Reputation: 1065Reputation: 1065Reputation: 1065Reputation: 1065Reputation: 1065Reputation: 1065
You can probably save yourself the trouble of writing a utility for removing the files you need to if you can try srm -- that's basically what it does:
Quote:
srm removes each specified file by overwriting, renaming, and truncating it before
unlinking. This prevents other people from undeleting or recovering any informa‐
tion about the file from the command line. By default srm uses 35 passes to over‐
write the file's contents. If this seems overkill you can use use the --dod,
--doe, --openbsd, --simple option which use less passes. If you specify more than
one option (of those listed above) they are executed in the order shown above.
The roll-your-own approach is, essentially, write your own srm.

Some years ago I wrote a "secure remove" (this is before the Internet existed in its present form) that did what I described above (it was the "approved" method) for classified data files. This was when you erased tapes with a big electromagnetic gadget, reduced disk platters to little metal chucks and all sorts of things. But that was Unix SVR4 boxes and you could get away with overwrite in place and there were no journaling file systems. Looking through the source code for srm, that's pretty much what it does -- with some extra bells and whistles -- and if I had to do it again, that's what I'd use. What the heck, the executable is 17K and all you have do is compile it (hopefully that system has the libraries, but, if it doesn't, you can build it static against the static libraries, which ought to be available on the box).

Of course, srm is significantly more sophisticated that the little utility I wrote; it's got to deal with more modern stuff than I did and, as far as I can tell, it does that rather well. Worth a shot, eh?

Hope this helps some.
 
Old 12-13-2012, 07:46 AM   #8
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Adding to that if you decide to write your own application consider having your code checked by somebody knowledgeable. Also test it well because misunderstanding file system specifics may lead to a false sense of security and create a liability.
 
Old 12-13-2012, 09:17 AM   #9
Noway2
Senior Member
 
Registered: Jul 2007
Distribution: Gentoo
Posts: 2,125

Rep: Reputation: 781Reputation: 781Reputation: 781Reputation: 781Reputation: 781Reputation: 781Reputation: 781
One other thing to keep in mind is that the type of media matters. Most of the secure delete utilities were designed to work with magnetic media. If you are using FLASH based media, these algorithms may not work for you as the hardware will thwart your efforts via the wear leveling mechanisms. What happens is that the data is still present, but the pointers will change to a different segment of flash. Flash provides other challenges in that it works by erasing a whole segment and then writing once to a particular address. Consequently, when you delete a file, it may still be present and readable in raw format, but a bit is set that tells you it has been deleted.
 
Old 12-13-2012, 11:26 AM   #10
jefro
Moderator
 
Registered: Mar 2008
Posts: 21,937

Rep: Reputation: 3619Reputation: 3619Reputation: 3619Reputation: 3619Reputation: 3619Reputation: 3619Reputation: 3619Reputation: 3619Reputation: 3619Reputation: 3619Reputation: 3619
I forget how to find the exact memory locations a file is located at. It should be simple stuff if you go back to the basics. Find that and then you could dd over the areas.

I think I might prefer ram drive with encryption.
 
Old 12-13-2012, 12:28 PM   #11
Noway2
Senior Member
 
Registered: Jul 2007
Distribution: Gentoo
Posts: 2,125

Rep: Reputation: 781Reputation: 781Reputation: 781Reputation: 781Reputation: 781Reputation: 781Reputation: 781
Quote:
I think I might prefer ram drive with encryption
Actually, this brings up a good point about the benefit of encryption. If the data is not stored on physical media in unencrypted format, a lot of the need for secure erasure go away. This can be particularly important in the case of FLASH based devices, where you do not have access to the actual physical media through it's logical interface. The case I mentioned above, with FLASH and solid state drives, a portion of the memory is going to be hidden from the user and not accessible. By keeping the data encrypted, it doesn't matter.
 
Old 12-13-2012, 10:43 PM   #12
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,610
Blog Entries: 4

Rep: Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905Reputation: 3905
I personally feel that you should never assume that you can guarantee that the data that has been recorded on a disk-drive has ever been "irretrievably overwritten." There are just too many variables involved here; too many players in the game. The file system, maybe a storage-network, even the drive hardware itself.
 
Old 12-14-2012, 10:36 AM   #13
Shadow_7
Senior Member
 
Registered: Feb 2003
Distribution: debian
Posts: 4,137
Blog Entries: 1

Rep: Reputation: 874Reputation: 874Reputation: 874Reputation: 874Reputation: 874Reputation: 874Reputation: 874
If you can take the filesystem offline and time and writes are not an issue, the usual tar and move remaining files... dd over the device (maybe multiple times / bit patterns)... recreate a filesystem and untar. By no means efficient, but security is just an illusion anyway. Chances are the information traveled over a network and the middleware could have logged raw packets. Even encryption only does so much. WEP anyone?

If it's truly confidential, it should never reside on a computer in my opinion. And it should never physically leave "the vault".

I probably shouldn't ask as there probably is such a thing. But a hard driver shredder? Since it's the new paper for all intents. And which model, the molten lava? the wood chipper? the old take a ride on a floor buffer model? As I look at all my old hard drives that I'm still lugging around. Even the failed ones.

RAM drive with encryption sounds good. As long as it's local and not networked through other machines. But even that has limits. Having put /tmpfs into ram and tried watching hulu or youtube from linux running on a USB stick. With only 1GB of physical ram, and no swap space at about 40 minutes of an hour long show is when your web browser closes as the kernel protects itself from a lack of RAM.
 
Old 12-14-2012, 12:28 PM   #14
tronayne
Senior Member
 
Registered: Oct 2003
Location: Northeastern Michigan, where Carhartt is a Designer Label
Distribution: Slackware 32- & 64-bit Stable
Posts: 3,541

Rep: Reputation: 1065Reputation: 1065Reputation: 1065Reputation: 1065Reputation: 1065Reputation: 1065Reputation: 1065Reputation: 1065
Well, old drives (that can't leave the building, let alone the computer room safe) can be taken apart (the disk stack), take a ride on a bulk tape eraser (the magnetic jobs, leave your mechanical watch somewhere else), then sent through a chipper (makes little tiny chunks). That'll pretty much make it unreadable. I suspect a few minutes in an MRI machine would do some damage too, eh? Just bolt it down good before you hit the switch. You do the same thing with LaserDisc (remember them?), CD-ROM, DVD, Blu-ray (except for the bunk eraser part); i.e., chop the buggers into little bitty pieces. And they will melt into an unrecognizable mess with the help of a little lighter fluid or charcoal starter or a ride in a 500 degree oven (they do stink when you burn, though).

I'd never let a flash drive out of the safe without chipping it, either. Hell, I wouldn't allow one of damned things into the building in the first place if I had a choice (and if it came in, it stays in for eternity -- in the safe). Wouldn't plug it in to a network connected box under any circumstances.

'Way I understand it just polishing the magnetic material off a disk isn't good enough. Allegedly, the platter retains a change at the atomic level(!) that can be read. Lordy, lordy, lordy, who'd have the equipment to do that, me wonders (well, I kind suspect I know who does -- the guys that came up with the destruct rules -- but I don't think they'd care too much about anything on my stuff nowadays).

Can you get carried away with this stuff? Uh, yeah. But, if you've got a copier that you're going to scrap you might want to pull the disk driver out of it before you dump it if you ever copied any proprietary information you don't want out in the world.

Bottom line is if you care about it, physically destroy it when you're done with it.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Running third party tools on Ubuntu which are known to work on Fedora cricballa Linux - Software 2 09-17-2009 09:47 PM
How to Install some 3rd party tools in linux? ashokg1981 Linux - Software 3 07-22-2007 02:53 PM
LXer: Two tools for building third-party installers LXer Syndicated Linux News 0 01-20-2007 12:54 AM
need info. on 3rd party tools Uday123 Programming 9 12-27-2005 11:12 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 02:34 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration