Dd does not require the partition to be mounted. In fact, it is better the partition be unmounted. As long as linux recognizes the partition info you can copy the partition. The file system is independent of the partition info. So long as the partition is a standard PC format, and I have never heard of one that wasn't on a PC, dd will copy the data. Otherwise, linux will certainly recognize the drive itself (i.e. sda) and the whole drive will dd to another drive.

This is a great thread, I've read all the way through it and it's top notch, answers just about every dd question a guy could have.

I did have two myself, though--I apologize if these were already answered, and if so, I just didn't understand.

Anyway, here goes: I'm using the SysRescCD distro (sysresccd.org) and I want to make a clone of a hard disk that's 120GB in capacity with 1 huge NTFS (Windows XP in this case) partition on it... But I don't have any comparably sized HDDs lying around, and, even if I did, I wouldn't want to clone the whole blasted thing because only 13-14GB of disk space is actually being used. I would use partimage, but when I ran it I was notified that partimage's NTFS support is experimental, and I REALLY don't want to tempt fate with this drive. Since dd clones at the lowest level I know it can safely handle NTFS because it doesn't even know or care what FS it's cloning... But is there a way for me to clone all the essential boot info / etc. from the drive and ONLY the used space of the partition? Or if partimage is safe to use, that'd be even easier... Basically, I have some 10GB hard drives and a CD burner, so my options are somewhat limited. I'm sure I'm going to need to get a DVD burner eventually... But I digress. Can I clone just the used space of the partition with dd?

My second question is related to the one above: I know that the first 512 byte sector of all PC HDDs is used as the boot sector, but are there any other low level items like the boot sector that I need to make sure and back up when I'm cloning my drives? I'd like to have each item cloned to a seperate file, just in case I need to restore ONLY the boot sector but not data, etc. I've heard about 'volume boot sectors', and I was wondering if there are items at the ends of partitions that delimit them that I should clone too, or what? What all is there that I don't know about?

Any help at all will be much appreciated.

With dd you can copy an entire drive to another entire drive, or to a partition. You can also copy a partition to another partition. The tricky thing is sector 1. Sector 1 is The Master Boot Record, or MBR. This contains the bootstrap code and the partition table. If you copy an entire drive:

dd if=/dev/sda of=/dev/sdb bs=4k conv=notrunc,noerror

sector 1 is copied. If you only copy a partition, sector 1 is not copied. For instance, if you:

dd if=/dev/sda3 of=/dev/sdb bs=4k conv=notrunc,noerror

and sdb has no partitions, you data will be inaccesible because the partition table was not copied from sector 1 on the source drive. Else:

dd if=/dev/sda of=/dev/sdb bs=4k conv=notrunc,noerror

and /dev/sdb is smaller than /dev/sda, you will copy a partition table that tells /dev/sdb it is larger than its capacity.

OK, so here's the trick. Partition the smaller destination drive before you do:

dd

All you have to do is make a partition. If the program you are using to partition won't make an NTFS partition, type 07, then make a linux partition and do:

dd if=/dev/sdb count=1 | hexdump -C > file.bin

Open file.bin in a hex editor and in the line that begins:

000001c0

change the third byte from 83 to 07. Save the file, then do:

dd if=file.bin of=/dev/sdb count=1

and the partition is now NTFS, or type 07.

You have to make sure your hex editor is set to 16 column mode for 000001c2 to end up in the third column. In 16 column mode, the last 4 bytes in the third column are partition type bytes. The actual offsets are:

000001c2
000001d2
000001e2
000001f2

Sometimes written:

0x1c2h
0x1d2h
0x1e2h
0x1f2h

Now, when you do dd, leave out the notrunc option because the destination drive is smaller. This doesn't copy adjacent sectors of zeroes. So, if the last 100 GB is unused space, dd will abbreviate it with *****. Also, skip sector 1 on the source drive, and seek past sector 1 on the destination drive.

Here is your command:

dd if=/dev/sda of=/dev/sdb skip=1 seek=1 bs=1b conv=noerror

If you want to check for errors on the source drive, leave out noerror.

hi awesomemachine,

can you look into this my other post and suggest something useful i could do.
thanx.

http://www.linuxquestions.org/questi...=453368&page=4

Hey Awesome, thanks a million, that really answered my question. I just have two more regarding the answer you gave me:


Okay, first off, you said "adjacent sectors of zeroes." What are these sectors adjacent to? Are you just saying IOW "empty/unused sectors"?

You said regarding unused space that "dd will abbreviate it with *****." How exactly does it do this? Do you mean that it adds 5 asterisks ( * ) at the end of whatever you're copying? If so, do I need to make sure I have room at the end of my destination for the asterisks?

Finally, you said to use this command to accomplish what I want:

"dd if=/dev/sda of=/dev/sdb skip=1 seek=1 bs=1b conv=noerror"

I notice that for the block size you specified "1b". Does that mean 1 byte or 1 bit or something else? And is there a specific reason why you use "1b" instead of, for example, "bs=4096"?

Thanks again, you've really helped me alot.
Cheers!

Very useful. Thanks for posting it.

If there are 30,000 sectors of zeroes all adjacent to each other, dd will write five asterisks to the first sector and omit the 29,999 other sectors.

See above

the b suffix means 1 sector. If you want to omit just sector 1, with skip and seek, the block size needs to be 1 sector, or 512 bytes. bs=4096 could be written bs=8b. bs=512 is abbreviated bs=1b.

A nicely written article. Its clear, full of content and very useful for the type of systems I will be supporting. I'm an OpenSource supporter and use Linux for most of my IT work. However, the MS OS is always lurking in the "wings", so there are "things" that I have to know to address some issues related to this OS.

I have a HP laptop computer with a 40 GB hard drive that dual boots into MS Windows XP or Fedora 3. I'm running out of hard disk space so I purchased a Seagate 100 GB HD drive. I want to image the current 40 GB HD to the new 100 GB drive.

After reading your article, I decided to use the Linux "dd" command to transfer the OS(s) and the data to the 100 GB drive. The transfer is running as I write this note.

******************** (the transfer is now Finished)
Bad news, Lots of errors listed on the console.
The MS XP OS partly works (it boots), but it lost drive E: and F: and the Fedora core-3 OS would not boot. I read in one of the article about the "dd" command that transferring to a much larger drive from a smaller drive can present a problem? I'm seraching the literature for some answers.

The 100 GB drive was blank at the start of the transfer.

Any comments will be helpful.

Bennett

Did you copy the partition table as well wit dd? (The 416 bytes or so from the first sector)

What kind of errors did you get on the console?

jlinkels

Thanks again Awesome, that answers all my questions! I really appreciate it!

Cheers!

In the disk clone of dd
dd if=/dev/hda of=/dev/hdb conv=noerror,notrunc

What if the sizes of hda and hdb are different

I suppose dd stops at the end of hda if hda is smaller than hdb

What if hda is larger than hdb? Does dd stop at the end of hdb?

Yes, dd would stop at the end of hdb. You cannot put a 10G disk image onto a 5G disk (for example).

OK, the best way to do what you want is to partition the new drive before you copy everything. You can make the partitions any way you want as long as there are enough of them. Sector 1 of drive 0, sda, hda contains the MBR. The MBR is 446 bytes, but there are signature bytes at the end of sector 1, "55 AA" that have to be there for anything to read the partitions. So, the exact mix of partitions is not the important thing, that is primary and logical. If you install MS Windows first it will install to an extended partition if you tell it to. I always put Windows in a logical drive on an extended partition, along with swap. Then I make two primary partitions for /home and /. I actually make / first, then /home, then the extended parttition, and then MS Windows partition, and finally swap. A DOS boot disk with a Partition Magic second emergency floppy disk works wonders. You just boot with the DOS floppy, or make a floppy image with dd and use cdrecord to make a cd out of it. Boot, take out that disk, and put in the second floppy of Partition Magic and type:

PQMAGIC

and there's you fully featured partition program, MS Windows and linux. PM 8.0 works for everything.

Since you want to keep the partition tables separate:

dd if=/dev/hda1 of=/dev/hdb1

These should be the / directories for linux.

Then do every partition. When you get done, boot from the linux install CD into rescue mode and tell it to fix everything, esp GRUB. Unless you use NTLDR to boot, but please don't do that. The rescue mode should fix the the first 429 bytes of sector 1, or boot strap code, without touching the partition table.

If you get hung up on doing an NTFS partition, which needs to be type "07" for MS Windows to even work, first write zeroes to all of the new disk:

dd if=/dev/zero of=/dev/hdb

Then partition. You can make the MS Windows partition linux type "83" and fix it later. Heres how you fix it so it's NTFS:

If it's in an extended partition:

dd if=/dev/hdb | hexdump -C | grep '00 83'

Look where it finds that on the terminal screen. Go there with dd:

dd if=/dev/hdb of=/directory/file.bin skip=10000 count=1

But figure out, by the offset in hexdump, where dd has to go. In the above example, dd is going to start at decimal offset 5,120,000. That's the default bs=512 times skip=10000. You're skipping 10000 blocks of 512 bytes, then recording a single sector, count=1. Hex offset is calculated with a calculator in linux, 5,120,000 d = 0x4e2000 h. Your screen is going to give you hex values, but dd works in decimal. You have to convert the hex offset where "00 83" are found to decimal so you can make dd go to the right spot. When you capture file.bin, open it with a hex editor and change "00 83" to "0007", then:

dd if=/directory/file.bin of=/dev/hdb seek=10000

That writes the new value to the extended partition.

If MS Windows is in a primary partition, is it partition 1, 2, 3, or 4?

0x1c2
0x1d2
0x1e2
0x1f2

Contain the partition type indicators for the drive. Do:

dd if=/dev/hdb of=/directory/file.bin count=1

That will record sector 1, the MBR, which contains the primary partition. Type "05" is extended, type "83" is linux, type "82" is linux swap, "01" is DOS, "07" is NTFS/HPFS. So, if your partitioning program doesn't make NTFS partitions, and you made a linux partition for MS Windows, change the corresponding type indicator:

location hex offset in sector 1:
0x1c2
or
0x1d2
or
0x1e2
or
0x1f2

with a hex editor. Screw it up afew times. Use fdisk -l in linux to see the aprtition is NTFS, and to see all your partitions. Now don't freak, just take it slow, easy, like you would with a really attractive, classy woman on a date. Give her what she wants, and she'll give you what you want. That's what linux is all about.

just one of them, depending on which primary partition, 1-4 you want to be NTFS, "07". Then, write the file back to the HDD. The partition is now NTFS type

You're welcome!!

If there is less than 5 GB of data on the 10 GB drive, and you omit conv=notrunc when you make the image of the 10 GB drive, it will fit on the 5 GB drive. Conv=notrunc writes out all the empty sectors. If you omit this, empty sectors are left off the image.