military grade disk wipe
 

Anyone know of a good program that will run within my existing linux OS (aka not a boot disk) that will completly low level wipe a given device?

Right now I'm using a program called copywipe in windows but I'd like to do my drive wiping downstairs in the server room (too hot in my room to have such operation going on) and hook it up to the Linux server instead. I just use a USB connection with an enclosure then point a fan to the drive (gets VERY hot otherwise). The key is I need the wipe to be extremely good so that even if the FBI or other such snoopy organizations got ahold of it, they would not be able to retrieve anything even by removing the platters and analysing the magnetics etc...

This process can take a week per drive, I'm fine with that. I just want to basically fire it up and forget about it then come back and check it later.

Before you say "just physically destroy the drive" these are practically brand new and I rather recycle then throw out.

Just so you know, if your drive was made around 2001 or so and later, it's not vulnerable to laboratory microscope attacks. The technology has changed, and you only need to overwrite data once now. The reason why the government still requires many overwrites is in case someone comes out with a similar attack in the future. At least that's what they said in the SANS GCFA training.

Good to know. I'd still like to know of such software though so I can still do a decent wipe, but keeping that in mind I'll only do it like 5 times.

http://dban.sourceforge.net is a bootable CD which gives you many options for wiping hard drives. One of which, from memory, gives you 7 wipes and rewrites and is, apparently, recommended by the NSA.

That looks good but I'm hoping for something that I can install/run rather the boot off of. I'll be attaching the disks to be wiped via usb external enclosure and plugging them into my live server, don't want to have to boot out of it for that long and my vmware idea did not work. I tried booting off it using my backup server which I can afford to have down for a few days, but it would not recognize usb, guessing it runs at a rather low level dos'ish type format, and not full blown linux with all drivers etc.

Well, there are a lot of tools that could help you.
For example, I always shred my data instead of rm-ing it. The shred utility is available on most *NIX-es nowadays. But if you want to do a full disk wipe, you can use the wipe utility Debian provides or just dd-ing it or shred-ing from a LiveCD with some Linux distribution on it.

Couldn't you do something like dd if=/dev/random of=/dev/usbdevice? If not, there are some programs on SourceForge

I think that's all those shredding/wiping programs do. They just organize how many cycles are done, and what is in each cycle. You could do 8 full wipes to a disk with some being all 0s, some being all 1s, some being random, and some being specific patterns.

Can someone explain that command in details? I have an idea of what its doing but the --help file gives me absolutely no info that helps me understand it. I could just write a bash script and have some paterns setup in advanced. Or would copying /dev/urandom a couple times be very efficient?

Right now I found a program called wipe so I did "wipe /dev/sdc" and its at 2% since like, this morning lol.

I figure the odds of goverment agencies getting ahold of the drive (or people with same ability as them) is VERY slim, but rather not take chances. And I rather stay green and have someone else put the drive to good use, then to just smash it. Though smashing it and doing other physical destruction *IS* much more fun. But I'd feel bad doing it to 3 rather new 500GB drives. :P

'dd' would be the simplest and most effective. Software like 'wipe' makes too many assumptions and doesn't necessarily work, nor would it affect any blocks which it believes is unused.

So:

dd if=/dev/random of=/dev/your_device_to_be_wiped

You can use 'urandom' instead of 'random' if it is faster.
Just run that dd line a few times and everything's gone; better if you can script it because it can take a few hours per pass. Some 'random' devices are pretty slow, so you'd better check on the progress of the 'dd' - otherwise it can take you months to do a single pass. You can check on dd's progress via:

kill -SIGUSR1 $(pidof dd)

I agree about dd. It is the easiest and best way. Just set up a bash script to run it as many times as you think necessary. If you really want to, you could set up pattern files and write those to the drive, but I would just use /dev/urandom a few times.

Does DD actually end though? like does it know when it reaches the end of the disk?

Yes, it will stop at the end of the device you specify (in your case either a disk or a partition on a disk.)

You'd probably be better off doing something overnight like:

The NSA specification was something like 8 passes. Two or three should be good though.

If you're worried about time definitely use urandom because if you walk away from your computer overnight there might not be that much activity on /dev/random. To counteract this, you could try shuffling a playlist of all your music - then /dev/random will most likely not be your weakest link.

Yes. You can also get it to report its progress - see the manpage. Also use a (decent) blksize to speed things up.

Why not just copy a partition or 2 or 3 & maybe a bunch of jpgs or mp3s from a system you don't care about. Maybe install Vista. It will look like the disk is just a backup or was removed from a working system. An empty, wiped disk would raise some suspicions all by itself that a disk with innocuous data would not.

I know there is a forensic version of dd that would tell you the progress, but I didn't know that it was built in now. I'll have to check that out.

You're right about increasing the block size. bs=4k should help a lot. conv=noerror would continue if it ran into an error as well.

Some general remarks only.

Not to belittle or chide anyone but one should note the standard mentioned that doesn't argue for different methods (or which incineration is just one) and n passes to be used because it's "good", it's about uniformity, quality and certainty. And wrt to standards and threads about wiping data one thing that's often overlooked is the verification stage: you can use all sorts of n passes schemes but if you fail to verify the result, the task isn't finished. One could argue it's a rule only people who handle classified or sensitive data work with, but IMHO it is just common sense to make certain the data is gone.


Also note there's a fast alternative to /dev/urandom called frandom (not for crypto purposes):

If you seize a disk anywhere out of its factory-sealed packaging, knowing the seizure location and the fact disks leave the factory patterned, then from an investigative point of view that's a very good observation. Next to emptied disks the ones that contain b0rken filesystems or containing patterned locations are the easiest way to spot "interesting" usage.

It is negative information. You could just be looking at a privacy advocate who takes steps beyond the ordinary...just because.

And I would love to see the prosecutor: "Your Honor, we are sure this suspect was up to no good because his hard drive was wiped and patterned. We know he was doing bad things because there was no evidence..."

Of course, in the current Orwellish environment, that might just happen...

Speaking just for myself, every now and then I fill up my /home and my / directories with a file in /tmp, and one in ~/tmp, that I fill from /dev/null, then delete, then fill from /dev/urandom - and I set it to run dd until the drive is full.

Ain't perfect, but should get rid of anything that I deleted.

FYI the military standard for cleaning a disk can be found here: http://www.dtic.mil/whs/directives/c...ml/522022m.htm

When I needed to send my hard disk back to Dell when it failed, I cleaned it with this: http://abaababa.ouvaton.org/wipe/ which is conveniently located in the Ubuntu repositories. The disk was 230 GB and it took all night and into the morning to complete all 8 passes.

You'll obviously want to read through the manual page to see what the options are all about, but I'm pretty sure if you apply the default settings to a full hard disk, and have the time to wait, that your data will be irrecoverable.

I mean, what do you really have on there anyway? If someone is capable of getting your stuff after 8 passes it means they have a lot of money and really really want it. No one (no government/military) is going through all that trouble for your cookies or even for some financial information. If you have data that is that seriously private then you need to disassemble the hard disk, burn it, and bury the parts over a number of hard to find locations. Then station a loyal troll to protect each of said locations.

What you've given there is a possible explanation. When you investigate things you have to be careful to deal with facts only and not let personal perception, opinion, influence what you (think you) are seeing. There are settings where regulations and policies mandate wiping. Most of the time those will be formally documented and accompanied by an auditing trail. Cutting things short, if there's no regulations or policies in effect then wiping data requires the user to have knowledge of wiping data as a solution and be technically knowledgable enough to chose a tool and perform the wiping. True, that is not a conclusion or rocket science but simply an observation that could trigger a more in-depth investigation.



You don't have to be a lawyer to see that's cutting things a wee bit too short. Humourous nonetheless.

Agreed. But presuming that the person being investigated is competent (and wiping data is evidence of competence) then the more in-depth investigation should lead noplace.

Also, the more in-depth investigation might be constrained by either time, funds, or the interest of the relevant authorities, based upon the severity of the suspected problem.

Bottom line is that a private individual (or organization) who has any reason at all to fear that any of his personal data on his computer could be compromising in any fashion should wipe hard drives. If this causes the authorities to become suspicious, so be it.

I once was hauled before Immigration and Naturalization Service because I married a foreigner. "We are going to investigate your marriage to make sure it is not a sham to get her papers". The interviewer started asking me questions. My answers came slower and slower as the questions got more personal. Finally I told the interviewer that I would answer no further questions.

The interviewer became irate: "Why won't you answer my questions? What do you have to hide?"

My response: "This interview is OVER, you f---ing NAZI!!! I WILL NOT PERMIT A GOVERNMENT FUNCTIONARY TO SPEAK TO ME IN THIS FASHION. I WILL SPEAK TO YOUR SUPERVISOR RIGHT NOW!!!". And I did. And I reamed him, and his flunky: "How DARE THIS BITCH SPEAK TO ME THIS WAY??? YOU WILL NOT INVADE MY PRIVACY. YOU WILL NOT SUGGEST THAT MY INSISTENCE THAT YOU KEEP YOUR BUREAUCRATIC NOSE OUT OF MY BUSINESS SUGGESTS THAT I AM DOING ANYTHING AT ALL WRONG. I will remind you of the law of the land: I am innocent until proven guilty. If you can come up with ANY EVIDENCE AT ALL of wrongdoing by me, then you may approach me. Until that time stay the F*** OUT OF MY FACE AND OUT OF MY WAY."

He took it; he had no choice. My wife got her permanent visa, and later her citizenship.

The point is that they can be as suspicious as they want. But in the US anyway (at least, pre-Patriot Acts) suspicion gets them nothing. They need evidence.

"Should" does not equal "state with onehundred percent certainty" and so, without knowing the strategy and inventory of the theoretically seized items and even if just an exercise, that is an assumption and one I can not support.


...or be sped up by certain persons forgetting to wipe their sticks, long-forgotten slash off-site backups, et cetera ;-p

The only disk that is most likely impossible to recover data is the one that has turned to dust. I use to work for a data recovery company, it was amazing the data they pulled from drives that had been wiped, damaged, burned and so on.. So wipe all you want, there's someone out there that can recover something from it.

...Unless it's the RIAA or other copyright type company. :/ It makes me sick to the stomac the stuff they get away with.


also this is something wacked I thought of, but given /dev/random and /dev/urandom generate random data based on stuff going on in the system, is there a possibility of it writing say, memory? I could see using this method being dangeraus, and using a set of patterns being more secure.

Sure I'm being paranoid, but I just like considering these type of things. I rarely sell/give HDDs away, only time I'll really get rid of one is if it fails, and I usually just physically destroy it because its fun.


Also using dd how do I manage to output a pattern, but make it repeat?

if I do something like

dd if=pattern1.txt of=/dev/hda

it will just copy pattern1 once. I don't want to have to generate a 500GB file, I want to make like a 100 byte one, then have it repeat.

Of all the dd-related tools 'dcfldd' is the only one which has a builtin "pattern" and "textpattern" option (see the manual), for others (at least with 'dd') you can echo the pattern and pipe it through it.

...how? If the magnetic coating is gone, the data goes with it, right? Or does the coating not get burned...

Out of curiosity, have you ever heard of a recovery from a piece of a platter?

Along a similar line, what if I took a powerful magnet and slid it all over the platters? Would that effectively destroy the magnetic patterns in it?

As I understand it, the only reason that anyone wipes anything twice+ is because of floppies. It was a medium that could leave magnetic traces behind, and made data recoverable in theory.

Hard disk drives that are wiped one time with zero's/random data have never been successfully recovered -- there is no proof anywhere that it has happened. I think its just a relic of the floppy disk/magnetic media days and a continued urban legend propagated by paranoids.

There is no proof that you can recover anything even wiping with /dev/zero. However, for the extra paranoid the best option is to encrypt the entire HDD. Multiple runs of a PRNG don't add significant extra security, because PRNGs are not cryptographically secure. Mersenne twister is the most common PRNG used and it can be entirely predicted using 624 consecutive outputs.
https://blog.spideroak.com/201212051...n-ruby-and-php

So, if you were able to look at the HDD and identify the bits that were there previously and recover them all (supposedly it can be done, but is extremely tedious), then all you would have to do is decypher the data, which is easy for Mersenne twister, but difficult if it is encrypted.

No - it used to work in the really old days. The way disks worked (using a linear recording), left large gaps between tracks. Because the mechanics of head movement, two separate seeks almost never got exactly the same location... so different seeks would be offset slightly in different directions.

Disk drives even has sub-track positioning capability (on the CDC 9600 specifically I know had this) with up to 20 offset positions on either side of the center of the track. You could actually have a complete track recorded by using offset 20 and offset -20... but there could be some bleed over. Using the offsets was a way to attempt to recover from a read error. A sector was never identified as bad, unless all 41 reads (using all possible offsets + the center). If one of the passes worked, then the sector was good.

Making a single pass overwrite would not remove the possibility of recovering prior data by looking at the extreme offsets. Prior data could more easily be retrieved using specialty equipment that had higher resolution than that available to the general head positioning mechanics. (besides just the surface reading, the magnetic domain also had a depth dimension where the surface could be removed, then the domains underneath retrieved). So using multiple overwrites would count on the seek imprecision to write to slightly different locations...

Most of this died when disk manufacturing switched from horizontal recording to vertical recording (the depth field vanished), and the higher precision of of head positioning (and incredibly smaller size of the magnetic domain grains) has made multiple overwrites mostly unnecessary.

What remains though are bad sectors - these are never overwritten, and can still carry sensitive information. Thus the process of removing the platters, degaussing, and grinding to powder is the final method.

Out of curiosity, why can't bad sectors be written to? Is it something that the OS is currently incapable of, or does the HDD itself prevent you from doing it?

Because the disk drive formatter board prevents it. What happens is that the address for the sector gets mapped to a replacement sector. Thus any attempted access to the bad sector address has the address replaced with the address of its replacement, and the bad sector isn't processed.

The formatter board has taken over the function of remapping bad sectors from the driver, making some errors impossible to recover from. A disk that has "too many" bad sectors cannot be recovered from anymore.

What used to be done was that the "manufacturers bad block" was copied into the driver on first access. This list was originally created when the drive was first formatted with sector headers/checksums (the original definition of the process of "formatting the drive"). The driver then used the replacement list itself, and could even expand the list as the driver detected additional errors. When the reserve list was used up the DRIVER would report an error. If the admin desired, the disk could be backed up, and reformatted using the driver function to reformat (the disk controller/disk formatter board had nothing do to with it). This allowed disks to be recovered, and during a verify pass errors could again be added to the bad block list - and the bad block sectors on the disk updated. This frequently returned disks to usability. It was even possible to expand the bad block list to extend the usable life of the disk.

When I first started working with computers (back around 1975 as an operator of a DEC System 10) disk drives would not be taken out of service unless there was a head crash (the disk read/write heads damaged, and thus the disk pack in use physically destroyed) or a spindle on a disk pack broke (destroyed both disk pack and drive). Otherwise, a disk pack would just be reformatted and the bad block list expanded. It was normal for there to be 10-20 errors due to media defects, but there could be over 100 soft errors added (mostly due to header checksum errors exhausting the retry/offset recovery). Such soft errors COULD be recovered by reformatting. Didn't mean they wouldn't happen again, but it could take a month or two first. If the system admin chose, they could extend the list for the drive... The only requirement for the list is that the list had to be stored at the beginning of the drive, and the reserve list only created during a format phase to designate the size of the replacement list. Once the disk format/verify was completed could a filesystem be put on it. It did mean that the available storage got slightly smaller as the replacement list expanded.

All of that went away when the formatter board got its own processor and embedded with the disk drive and disk pack. The resulting cartridge was faster (and a LOT smaller), and head crashes a rather rare occurrence since the enclosed disk prevents general contamination. But a good bit of flexibility was also lost.

Sometimes a low level (device command) reformat will recover the sector (this is actually recovering from a soft failure rather than a hard failure, but the drive can't tell the difference) and overwrite it. But if a surface defect has been finally detected, that won't happen.

BLANCO is what I've used in the past.

Those products have the same limitation - they cannot wipe the bad sectors.

Which is sufficient for most things.

shred -x -v -n 0 -z /dev/sd?

zero means 1 pass , 1 = 2 passes.

That is the command DBAN uses. I think shred is part of core-utilities.
Some distros do not include shred
http://www.howtoforge.com/how-to-sec...ves-with-shred

Don't some modern hard drives have a "secure wipe" or is that only available on previously hardware encrypted (usually laptop) drives? I know the utility (Linux and Windows versions available) from my SSD manufacturer has a "secure wipe" option too though I've not researched what that does.

Oh, a legal word of caution (mostly paranoia though) for those in the UK using random data to wipe -- if the authorities can convince a judge that it is encrypted data on there you can go to prison for something like 5 years for not giving up the key and repeatedly be sent there until you can prove your innocence.

Well, then I guess it is better to use a PRNG like mersenne twister, because then you can prove that it is a PRNG by extracting the seed. It is not as secure, but should be good enough. I usually use /dev/zero, which should also be good enough.

Personally, I would make do with using /dev/zero. However, if there is data which may be misused I suppose it's the trade between the two.
Personally, I have no data which I feel the need to delete apart from bank details -- I fear javascript more than an electron microscope attack.

I've heard a lot about using multiple passes of random data, but I've also heard that no information has ever been successfully recovered from a drive that's been zeroed out. Which is true?

Both.

The only issue is HOW the disk is being recovered, and who is doing the recovery.

For low expense, just making a dump of the disk sufficient - and zeroing the disk is sufficient to neutralize any recovery. Low expense would be around $1,500 - $5,000 as it usually needs to replace the formatter board with a special purpose board, lots of calibration, and a read.

For HIGH expense the recovery can use a magnetic microscope. If only one pass at reading is done, then some data could be recovered from a single zero pass. (The reason is that writing the same value takes less space than writing alternating values - thus some space at the end of a sector might retain old data; and HIGH expense can be around $15k).

For REALLY REALLY high expense, multiple passes can be made. And the price goes higher if they choose to remove the top layer of the recording media, and make multiple passes. Sometimes more data can be recovered. To cover this, multiple overwrites can/will reduce the data. But REALLY REALLY high means prices for a disk scan will be well over $15,000 PER SCAN. (It requires a cleanroom environment...)

With the newest disks, it is usually uneconomical to do this as almost no data is that valuable.

some good and not so good info here.....

magnetic polarization leaves remnants of original state even after you change the magnetic polarization, thus recovery of data is still possible even after wiping (not with any gear you would have though, etc). multi-pass wipes with flipping the bits helps to stop this type of forensics work.

secure wipe is a HD controller function in most modern day HD's, but many times the mobo bios will block such function command to protect the HD from getting accidentally wiped. some mobo bios's have access to HD commands from the bios util itself, but its rare from what i have seen.

i believe the HD controller can be told to attempt to write all sectors, even the bad ones. a "bad sector" is usually marked when a read failure occurs.

the place to go for magnetic stuff is CMRR @ http://cmrr.ucsd.edu/ ,you can email the folks here and they usually have a good response time.

What if you cracked it open and scraped the powerful magnet inside all over the platters?

A powerful magnet would work.

It has never been proven to work, but if you are worried about it, I would encrypt the data instead of doing multiple passes.

Just to be clear.. The military(army at least) physically destroys higher security classed disks. They do not "wipe" them. "Military grade" or no.

https://www.youtube.com/watch?v=yd_O7-rqcHc

There are also brute force levers that destroy the disk. But the shredder is what I've seen used.

When I was contracted to the Navy, that was ALSO their solution. It required special wavers for contracts that had "return to vendor" requirements. The only time this wasn't done was when the disk was being reused within the same group, which allowed some projects to have a sizable storage rack composed of "recycled" disks...

Nice video... It looked like one of the drives tried to escape

It looked like the hard drives came out the bottom in fairly large size chunks... If you could get the chunks from one hard drive (instead of that whole box), couldn't you use the magnetic microscope on the platter remnants, and get some info from that? Or are the shredder "jaws" magnetized as well?

Take a cruise to the Bahamas. Or somewhere. Throw the damn things overboard. :D (Then, go get other Margarita.)

Quite seriously, your problem is now solved.

If you seriously regret whatever-it-is that you stored upon a disk drive, the best thing to do is to (wipe it somehow and then ...) consign it to a place where it will never be found. (Then, clever soul that you are, "have a drink.")

Note: If what you have is really so dammed important that the CIA launches another Glomar Challenger mission to recover it, be sure to get yourself a piece of the movie.

The 'jaws' are not electrified. But, when magnetized metal bends(ie: platters), the data becomes VERY problematic to recover. If its possible at all.