LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices

Reply
 
Search this Thread
Old 11-12-2009, 02:27 PM   #1
catkin
LQ 5k Club
 
Registered: Dec 2008
Location: Tamil Nadu, India
Distribution: Servers: Debian Squeeze and Wheezy. Desktop: Slackware64 14.0. Netbook: Slackware 13.37
Posts: 8,551
Blog Entries: 28

Rep: Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176Reputation: 1176
Multiple HDD writes required to render data irretrievable?


Hello

It is sometimes asserted that writing to HDDs does not render the previous data irrecoverable. Has this phenomenon ever been reliable demonstrated to allow recovery of significant data? Or is it a theoretical urban myth?

The question arose in this LQ thread, specifically in this post and this post; in this post unSpawn wrote "I remember I did test different methods of deletion and recovery a year or so ago. I don't have the results at hand right now but I could run tests again and post them (in a way anyone could reproduce them with the right tools). Anyone interested is invited to create a separate thread and we'll take it from there.".

This is such a thread!

Best

Charles
 
Old 11-12-2009, 03:05 PM   #2
Jim Bengtson
Member
 
Registered: Feb 2009
Location: Iowa
Distribution: Ubuntu 9.10
Posts: 164

Rep: Reputation: 38
Quote:
It is sometimes asserted that writing to HDDs does not render the previous data irrecoverable. Has this phenomenon ever been reliable demonstrated to allow recovery of significant data? Or is it a theoretical urban myth?
Apparently it is possible to recover data from a wiped hard drive, but it's also a lot of work. The US Government polices state that if the data is classified (I don't know if it's "top secret" or higher) then the hard drive should be destroyed, not just cleared. Government documents lists four categories of hard drive disposal:
  • Disposal - Disposal is the act of discarding media with no other sanitization considerations. This is most often done by paper recycling containing non-confidential information but may also include other media.
  • Clearing - Clearing information is a level of media sanitization that would protect the confidentiality of information against a robust keyboard attack. Simple deletion of items would not suffice for clearing. Clearing must not allow information to be retrieved by data, disk, or file recovery utilities. It must be resistant to keystroke recovery attempts executed from standard input devices and from data scavenging tools. For example, overwriting is an acceptable method for clearing media.
  • Purging - Purging information is a media sanitization process that protects the confidentiality of information against a laboratory attack. For some media, clearing media would not suffice for purging. However, for ATA disk drives manufactured after 2001 (over 15 GB) the terms clearing and purging have converged. A laboratory attack would involve a threat with the resources and knowledge to use nonstandard systems to conduct data recovery attempts on media outside their normal operating environment. This type of attack involves using signal processing equipment and specially trained personnel.
  • Destroying - Destruction of media is the ultimate form of sanitization. After media are destroyed, they cannot be reused as originally intended. Physical destruction can be accomplished using a variety of methods, including disintegration, incineration, pulverizing, shredding, and melting.

Read section 2.4 and Appendix D of this NIST document:

Guidelines for Media Sanitization
NIST Special Publication 800-88
http://csrc.nist.gov/publications/ni...00-88_rev1.pdf

Last edited by Jim Bengtson; 11-12-2009 at 03:24 PM. Reason: Found additional information.
 
Old 11-12-2009, 03:09 PM   #3
Jim Bengtson
Member
 
Registered: Feb 2009
Location: Iowa
Distribution: Ubuntu 9.10
Posts: 164

Rep: Reputation: 38
Also this document and it's attachments:

Spectrum-OSS
Practices & Policies
SUBJECT: Disposition of computer hard drives and the data contained within.
http://oss-spectrum.org/HD1.htm
 
Old 11-12-2009, 03:41 PM   #4
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,367
Blog Entries: 54

Rep: Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867
Sure, but wouldn't it be way more interesting to actually test out stuff and share actual results? I'm not talking hardcore 35 pass Gutmanns but 'rm' vs 'shred' vs 'BcWipe' vs 'srm' et cetera like wrt the original thread?
 
Old 11-12-2009, 03:56 PM   #5
smeezekitty
Senior Member
 
Registered: Sep 2009
Location: Washington U.S.
Distribution: M$ Windows / Debian / Ubuntu / DSL / many others
Posts: 2,229

Rep: Reputation: 173Reputation: 173
how about a bash script that runs dd over and over on a for loop from /dev/random and /dev/zero to the drive? good luck recovering that.
 
Old 11-12-2009, 04:19 PM   #6
unSpawn
Moderator
 
Registered: May 2001
Posts: 27,367
Blog Entries: 54

Rep: Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867Reputation: 2867
Oh come on!

Quote:
Originally Posted by smeezekitty View Post
how about a bash script that runs dd over and over on a for loop from /dev/random and /dev/zero to the drive? good luck recovering that.
With all due respect but half of the reason you made "Senior Member" is because at least half of your posts are either completely OT drive-by oneliners or otherwise meant jocularly. So while I'm sure your remark here was meant jocularly as well, if you can't contain yourself and contribute in a more fitting way, by all means please confine yourself to posting in /General, OK? Thanks for understanding.
 
Old 11-12-2009, 05:47 PM   #7
Quakeboy02
Senior Member
 
Registered: Nov 2006
Distribution: Debian Squeeze 2.6.32.9 SMP AMD64
Posts: 3,245

Rep: Reputation: 121Reputation: 121
The reason this even poses the potential for a problem is due to two things: 1) the guard space between data tracks, and 2) inaccuracies of head placement due to heat. When you combine these two together, you see that the data doesn't get written at exactly the middle of a track each time. If this week you are writing data to the area closer to the spindle, then a sort of thin strip of the data from last week's write might be left over in the guard space further out toward the next track. It's this thin strip of leftover data that gives the vulnerability.

In the old days, when there was only 5MB on a disk, it wasn't that daunting of a task to go through and skew the heads off one way or the other to see if there was any leftover data. Today, with multi-terabyte drives and closer track spacing, this has become that much harder to do. Note that it's just harder, not impossible. So, at some point, it comes down to ROI (return on investment): is there enough potential reward to go through the man-weeks and -months it takes to manually search though that much disk for usable old data? For a low-level crime, probably not. For corporate or national espionage, then it's a matter of what had been on the disk.
 
Old 11-13-2009, 07:38 AM   #8
H_TeXMeX_H
Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269
Actually it's not possible to recover meaningful data from a zeroed HDD, even just one zeroing.

I posted about this too:
http://www.linuxquestions.org/questi...Fzero+wipe+HDD

Nobody has yet proven that it is possible to recover meaningful data after one zeroing. Of course, some people want to be extra sure, in which case you can use /dev/urandom, but it's a waste of time, IMO.

If you still don't believe it, submit it to the Mythbusters ... but they probably won't figure it out either.
 
Old 11-13-2009, 07:51 AM   #9
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728
For a variety of sound technical reasons, it is theoretically plausible that data can be recovered after just one pass of all zeros. When the government is dealing with top-secret data, "theoretically plausible" is enough to justify some simple precautions. Since disk drives have become so cheap, I suppose we'll see a trend to simply destroying the whole drive.

As a practical matter, it would be pretty hard for a mere mortal to recover anything after one pass of random + one pass of zeros---that's all I ever do.
 
Old 11-13-2009, 08:55 AM   #10
H_TeXMeX_H
Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269
In theory it's possible to recover a few bits here and there, but not within any level of certainty or accuracy. You can't be certain that what you recover is actually meaningful data. Either way, even if this is true, I don't see why a single pass with /dev/urandom would not be enough even for the most sensitive of data. I don't even know of any sound theoretical explanation for the multi-pass.
 
Old 11-13-2009, 09:02 AM   #11
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728
Quote:
Originally Posted by H_TeXMeX_H View Post
In theory it's possible to recover a few bits here and there, but not within any level of certainty or accuracy. You can't be certain that what you recover is actually meaningful data. Either way, even if this is true, I don't see why a single pass with /dev/urandom would not be enough even for the most sensitive of data. I don't even know of any sound theoretical explanation for the multi-pass.
In the view of the government security people, the idea is to be certain that you can NOT recover anything.

The theory of all this is pretty basic: Writing (anything) to magnetic media leaves some random amount of residual from the last write. This thread describes various ways that this can happen, but I don't recall discussion of one basic mechanism, to wit: the hysteresis in the basic process of changing the state of magnetization.

Regardless, by using multiple writes with random data, the residual from any **real** data is reduced to the point that it is not detectable.
 
Old 11-13-2009, 09:27 AM   #12
Jim Bengtson
Member
 
Registered: Feb 2009
Location: Iowa
Distribution: Ubuntu 9.10
Posts: 164

Rep: Reputation: 38
Lightbulb

I don't know if this settles it or not...

"Data overwritten once or twice may be recovered by subtracting what is expected to be read from a storage location from what is actually read. Data which is overwritten an arbitrarily large number of times can still be recovered provided that the new data isn't written to the same location as the original data (for magnetic media), or that the recovery attempt is carried out fairly soon after the new data was written (for RAM). For this reason it is effectively impossible to sanitise storage locations by simple overwriting them, no matter how many overwrite passes are made or what data patterns are written. However by using the relatively simple methods presented in this paper the task of an attacker can be made significantly more difficult, if not prohibitively expensive."
-- Secure Deletion of Data from Magnetic and Solid-State Memory, Peter Gutmann, Dept. of Computer Science, University of Auckland, 1996


"Gutmann explains that when a 1 bit is written over a zero bit, the 'actual effect is closer to obtaining a .95 when a zero is overwritten with a one, and a 1.05 when a one is overwritten with a one'. Given that, and a read head 20 times as sensitive as the one in a production disk drive, and also given the pattern of overwrite bits, one could recover the under-data.

The references Gutmann provides suggest that his piece is much overwrought. None of the references lead to examples of sensitive information being disclosed. Rather, they refer to experiments where STM microscopy was used to examine individual bits, and some evidence of previously written bits was found.
"
-- Can Intelligence Agencies Read Overwritten Data? (2004)

Abstract. Often we hear controversial opinions in digital forensics on the required or desired number of passes to utilize for properly overwriting, sometimes referred to as wiping or erasing, a modern hard drive. The controversy has caused much misconception, with persons commonly quoting that data can be recovered if it has only been overwritten once or twice. Moreover, referencing that it actually takes up to ten, and even as many as 35 (referred to as the Gutmann scheme because of the 1996 Secure Deletion of Data from Magnetic and Solid-State Memory published paper by Peter Gutmann) passes to securely overwrite the previous data. One of the chief controversies is that if a head positioning system is not exact enough, new data written to a drive may not be written back to the precise location of the original data. We demonstrate that the controversy surrounding this topic is unfounded.
-- Overwriting Hard Drive Data: The Great Wiping Controversy, by Craig Wright, Dave Kleiman, and Shyaam Sundhar R.S.


"A paper published in December last year; “Overwriting Hard Drive Data: The Great Wiping Controversy” by Craig Wright, Dave Kleiman and Shyaam Sundhar R.S. as presented at ICISS2008 and published in the Springer Verlag Lecture Notes in Computer Science (LNCS) series, proves beyond doubt that data can’t be recovered from a wiped drive even if one uses an electron microscope. As Craig Wright puts it in a post on the SANS Computer Forensics blog:

'Although there is a good chance of recovery for any individual bit from a drive, the chances of recovery of any amount of data from a drive using an electron microscope are negligible. Even speculating on the possible recovery of an old drive, there is no likelihood that any data would be recoverable from the drive. The forensic recovery of data using electron microscopy is infeasible… The fallacy that data can be forensically recovered using an electron microscope or related means needs to be put to rest.'
"
http://itknowledgeexchange.techtarge...ttled-at-last/
 
Old 11-13-2009, 09:43 AM   #13
H_TeXMeX_H
Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269Reputation: 1269
Ok, well in that case, if you want to be extra sure use a large magnet or sledgehammer depending on your style.

I'm not that paranoid and am convinced one /dev/zero is good enough.
 
Old 11-13-2009, 09:53 AM   #14
Jim Bengtson
Member
 
Registered: Feb 2009
Location: Iowa
Distribution: Ubuntu 9.10
Posts: 164

Rep: Reputation: 38
Quote:
I'm not that paranoid and am convinced one /dev/zero is good enough.
It probably depends on what's on the old hard drive. If it's top-secret government data, destroy the hard drive. If it's a bunch of old photos from your summer vacation 5 years ago, wipe it and go on with your life.
 
Old 11-13-2009, 10:08 AM   #15
Jim Bengtson
Member
 
Registered: Feb 2009
Location: Iowa
Distribution: Ubuntu 9.10
Posts: 164

Rep: Reputation: 38
Quote:
Originally Posted by lewc View Post
that is something else to consider what are the possible reprecautions of the data being recovered / accessed by others schools have me destroy images that childeren draw in mspaint for child protection, do they think some dumpster diver will follow lil timmy 10 years later... "look timmy you drew this" lol
True, but I highly doubt anyone would devote the resources needed (electron microscope?) to recover overwritten data just so that they can torment Timmy 10 years later.
 
  


Reply

Tags
data, hdd, multiple, write


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
How Linux writes/deletes from a hdd tkbonito Linux - General 2 07-29-2009 11:21 AM
Reduce HDD writes? Cyberman Linux - Hardware 1 09-29-2008 02:49 PM
FC6 Server Randomly Writes Data To HDD jmoschetti45 Linux - Server 6 05-22-2007 03:16 PM
Using OpenOffice source to render data for charts into images ? WhiskeyTangoFoxtrot Programming 4 01-26-2007 05:25 AM
Gnome 2.6, odd hdd writes. slackMeUp Slackware 11 10-07-2004 02:42 AM


All times are GMT -5. The time now is 11:12 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration