Ahhh, man is every time it is mentioned a marvel -- I'd never have guessed it has an entry on null . Thanks for sharing, onebuck.

First of all, if you write zeroes over an entire drive, only sophisticated recovery methods can possibly recover anything useful. /dev/zero is a kernel feature. The linux kernel produces the zeroes.

/dev/urandom is held in a kernel entropy pool. If entropy runs out, it uses the same entropy over again. /dev/random stops until the kernel has gathered unique entropy.

To use a random file you would have to write a shell script that loops an runs the dd command repeatedly. You would use a variable for the seek='' parameter, and increment it by the size of the random file, so you didn't keep writing the same spot on the disk.

I've never tried this before, but it's theoretically possible. I don't overwrite whole disks with random data. If you really want to make it tough to recover an overwritten drive, first tighten up the disk tracks:

dd if=/dev/sda of=/dev/sda

which rewrites the data to the drive, which gets rid of magnetic flux on the outer edges of the tracks. Then:

dd if=/dev/zero of=/dev/sda

There will be nothing recoverable unless someone takes the drive apart and uses electron micrography.

1 members found this post helpful.

Understood. Thanks for clarification and elaboration, AwesomeMachine.

Great information. Thanks! Going in my delicious NOW.

Thanks AwesomeMachine.

A lot of effort and skill went into this post... I'm seriously impressed.

A short question. I just swithced to ext4 and Ubuntu 10.10.

Are there any known problems with using dd on ext4 file systems?

No problems with ext4 that I've heard of. The only problem file systems I'm aware of are encrypted file systems in which sometimes dd will not copy the encryption header, because someone put the encryption header on a CD and deleted it off the hard drive.

Thanks AwesomeMachine

I will do some testing in the next month and if I have problems with dd and my encrypted folders I will report the results.

Hi,

This would be my first post. I just needed to make an image of my newly installed windows system. The HDD in the system is a TB large and I wouldnt be able to copy the whole thing bit for bit. Is there a way I could use the DD command to copy only what is being used on the Hard drive. Currently I have a little over 900GB free.

Scenario: I'm using Ubuntu to boot from CD and hoping to image only what's being used on the 1TB HDD. I'm looking to output the file to a USB which is 32GB.

Any help is greatly appreciated.

David

First of all, thanks AwesomeMachine for this great post!

I'm fairly new to Linux/Ubuntu. I've Ubuntu 10.10 installed on a 32GB Corsair USB (fdisk -l shows 32.5 GB) and I want to do a 1:1 copy of thsi installation to another USB drive. The new one is a 32GB SanDisk (fdisk -l shows 32.0 GB for this one). What I tried: Boot with SystemRescueCD, plug in source and destination USB (fdisk - l shows me source as /dev/sdc and dest as /dev/sdd. I simply did "dd if=/dev/sdc of=/dev/sdd", but the new USB doesn't boot (from dd I got a success message). Is it because source is 32.5 GB and dest 32.0 GB? Another weird thing that I notice: fdisk -l shows the 3 partitions /dev/sdc and /dev/sdd exactly the same; gparted instead shows /dev/sdd "unallocated". Any thoughs? What would happen, if I shrink a little the ext4 partition of my Ubuntu 10.10 installation, try to create the partions manually and try to copy partitions instead of the whole device?

Thanks AwesomeMachine for such a wonderful thread.

I have a simple question. Suppose I am using dd without "notrunc" option and I have two partitions sda1 and sda2 of same size. Assume that sda1(as well as sda2) has 10000(512 bytes) blocks. For sda1, first 2000 and last 2000 blocks have data on it and rest are all zero blocks. When i run
blocks which are zero in between will be abbreviated by *****.
My question. Will the operating system understand those 5 asteriks?
That is, when I mount my partition sda2, will the data on last 2000 blocks of sda1 be accessible?

Hi David,

Try this:

dd if=/dev/sda bs=32k | gzip | dd of=/dev/sdb1

OR

mount the flash drive to /mnt/sdb1 and try:

dd if=/dev/sda bs=32k | bunzip2 > /mnt/sdb1/partition.img

But bzip2 compression of a disk really stresses the memory subsystem. Gzip is easier on the hardware.

Certain memory weaknesses cause problems only when running bzip2 full blast for a half hour or more!

Some unrelated trivia: the most common cause of unrecoverable memory errors are gamma radiation and other cosmic rays!

I think the culprit is something called: U3. You can google it if you wish. Write 400 MB of /dev/zero to the target drive, twice.

dd if=/dev/zero of=/dev/sdd bs=1M count=400

After it's finished, hit the up arrow and do it again. It'll take 2 minutes or so. Pull the target drive out, wait 10 seconds, and put it back.

Now it should work, unless you have U3 on the source drive! If that's the case, you can mount the drive, use rsync -rv <drive_mount_point/*> <some_empty_directory/>,

Wait for rsync to finish! Do the dd trick to the source drive the same way. Partition the drive and format as vfat. Mount it and rsync the directory tree you just copied off it, back to it.

Then install isolinux to the drive. I'm not promising this will for sure work in every possible case. You might come out ahead using the Ubuntu method to reinstall Linux to the source drive.

But I want readers to learn something about what's really going on underneath those USB installation scripts!

I'm going to bow and say, "Thank you," to everyone for the recent applause! Thank you.

-Awesome Bows.

The discussion thread sort of evolved, although I'm impressed you read that far into it. Hexdump is the program that represents successive zero blocks as *****.

You are correct. It is my mistake. If you read a little further on, I corrected it.

Thankyou again AwesomeMachine for the clarification. I read most of the pages of the thread, some how I missed your earlier clarification. Sorry for bothering you.

I started this tread to help others. Feel free to ask questions. I'm a no nonsense writer, so sometimes I come across indifferent or stern. I encourage readers to experiment. That was my original intention for the OP.