AwesomeMachine, I found the info pages on dd quite short on error messages with dd.
When I backup partitions (using notrunc,noerror and from a live cd)
I always get a +1 next to the number of records transferred.
Saying that the OS on the backup partition seems to be working fine.
(I know it is short on details; I am not asking for a solution to this;
also I may have a "wobbly" HD)

I wondered if you knew a bit more about dd error messages (or lack of),
that could be added to your tutorial.
Thanks

As cryptic as its syntax are dd error messages. Dd is a bit lacking in verboseness. There is no -v switch. If I could find a place that lists all the dd messages I would add them to the post. The 'xxxxxxxxx records +1' means there was one piece of data on the source device that was less than one block. Block size is determined by 'bs=' in the dd command. Say you specify 'bs=8192'. That's 16 sectors per block. Sometimes the last block isn't a full block. It might only be 13 sectors. Dd skips the data at the end of the partition which doesn't make a full block. You can get around this by leaving out 'bs=', and block size will default to 'bs=512'. 512 bytes is one sector on an IDE or SATA (which are also IDE (Integrated Drive Electronics) drives. With 512 byte block size there can't be an incomplete block because drives are are divided in 512 byte chunks called sectors. You can also use cylinders for blocks, as each partition is made of only complete cylinders:

bs=16065b

Will accomplish this, but that is a very large block size for dd and may degrade performance. Cylinders are 255 (heads) * 63 (sectors per track) = sectors per cylinder * 512, or in dd language 'b' = bytes per cylinder. Partitions must have complete cylinders. There are no fractions of cylinders in a partition.

Thanks, that makes sense and is enlightening (much more than the doc itself).
I always use bs=4096.

Well, I will be on the lookout, one never knows

Does it matter if I use dd while the system is running other processes, or do I need to switch to single user mode? What happens to the contents of /proc when you restore a image?

Hey Guys

I'm new to linux, but wanted to know if it is possible for dd to clone a 80gb hdd to a 20gb hdd. Their is only about 4gb of data on the 80gb hdd.

Thanx in advance

It really doesn't matter if the system is running, or even if a drive is mounted or not. /proc is a virtual file system. It doesn't exist on the HDD.

In theory it is possible to do this, but in practice you are better off just copying the files. If you need the drive bootable, use a linux install CD, boot into rescue mode, and install grub.

alot of thanks

Thank you for this article. It helped me very much.

Since there is only 4gb it's possible as long as you don't wipe the fs

What would the command look like.

If you copy a drive, or partition, dd copies the entire file system. Dd does not discriminate based on what is usable data, and what is empty, but formatted space on the partition. For copying user data you use "cp -r". Dd is a pretty useful tool, and in my post I made a little more out of it than is necessary. Some things are just cool to do with dd. But, copying user data is not something dd was meant to do. That's why there are other linux tools besides dd. If you could use one command for everything, linux wouldn't be so hard to learn, but you can't. Use the best tool for the job. If you take the time to read this entire thread you will understand why you can't use dd for what you want to do.

Here is an example of using tar or dd

dd can naturally clone a hard disk if the target is identical in size or larger as the excess space will become unallocated space by default.

The opposite way doesn't work because if a large disk is cloned into a small disk the partition table will be ruinned and cannot be understood by the operating system or ourselves since the big disk addresses are outisde the bound of the small disk. If a 80Gb disk is fully used up then its partition table will have a record of the address of the end of the 80Gb disk. How can an operating system find the same address in a small disk with only 20Gb?

To migrate a 80Gb disk say with only one partition of hda1 into a smaller 20Gb disk formatted with only one partition say hdc1 the following steps are one of the many alternatives

(1) make temporary directories to mount hda1 and hdc1 if they are not parts of the booted up filing system.
(2) change root to the directory you wish to clone the information out. In this case it is the source hda1
(3) Just one tar command is needed to clone the filing system
The entire original filing system with all the settings and protections are cloned from hda1 to hdc1. As Awesomemachine pointed out previously the filing-system cloning does not copy the boot sector across because the boot loader is not part of any filing system. This makes dd different from all the other archiving commands because it duplicates or mirrors every byte in the partition or disk to be cloned and so the boot sector is automatically included. If the partition happens to be the first one then the partition table will be copied too.

If a user has a need to make the new disk hdc bootable then he/she will need to change root into the new partition to restore the boot loader.
(4) If the disk hda1 was originally booted by Grub the command to put Grub into the MBR of the new home hdc is simply
For Lilo booting hda1 originally the equivalent command is
Please be aware after change-root a user is actually inside hdc1 and must issue
to return back to the Linux where everything starts.

Had the target disk been the same size or bigger dd would eat it for breakfast with just one line command (omitting parameters to show its simplest form) of
Hope the above clarifies the main difference between dd and other archiving commands.

A user will appreciate the usefulness of dd if he/she is cloning a large disk with inside many different partitions for Dos, Windows, Linux, BSD... and can rely on the target disk to boot exactly same as the orginal.

Observed difference between dd and other commercial software like Ghost/Acronis True Image

dd is simpler, usually faster and more reliable.

To be fair the commercial software can clone a big source disk into a small target disk, assuming the files can be accommodated in the small target disk, by the same command as cloning a small source into a big target. Due to the same technique being used the type of operating systems inside the disk/partitions is immaterial too. In order word commercial cloning software are equally capable.

I used to rely on Ghost for cloning but since into Linux I never have had a need to go back to the commercial software, except for comparison studies.

THANK YOU! Very helpful post with examples. Why can't MAN pages, in general, contain examples such as this?

Question: I have my Ubuntu system running on RAID 1 with 2x160GB hard drives. I am wanting to expand them to 2x750GB hard drives also running in RAID 1. Will `dd` work with a RAID 1 system and will the new drives know that they will be working in RAID 1??

I believe that is down to the kernel. If the kernel recognises the RAID disks as you want it to then you are home dry.

The kernel does have to cope with different chipsets and if it fails the two disks in the RAID may be treated as two separate disks.

You should have a pretty good idea of the hard disk recognition if you issue the "fdisk -l" command requesting all disks to be displayed with their partitions tabulated.