Required help in HDD Cloning

saikee · 12-13-2007, 06:16 PM

David1357,

I beg your pardon!

When you clone a hard disk you clone its geometry too!

dd operates on the sector basis. It stops whenever the number of sectors exhausted in either the source or the target disk.

If the target is bigger than the source the target clone has the same partition table as the original source and the excess hard disk space is unallocated. Nothing is unusual here because there is no law requiring a hard disk must be fully partitioned before an operating system can use it.

If the target is smaller it still has the same partition table of the larger hard disk and that is trouble. No OS in the world knows what to do with a hard disk where the partition table indicates more space than it physically got. Just ask yourself what would you do if you are an operating system. Remember an operating system has to guarantee the integrity of the partition if it can read/write it. Thus no operating system would want to have anything to do with such a hard disk.

So disks never need to be identical. The safe way is always have a target larger than the source. You can do exceptions if you know where you are going.

For example last month I cloned a Vista from a newly bought laptop from a 2.5" 160Gb to a 3.5" 500Gb. Then from the 3.5" 500Gb I clone it back into another 2.5" 250Gb. I am using the Vista in the last 2.5" 250Gb disk in the laptop to write this thread. I managed to clone from a bigger 3.5" 500Gb into a 2.5" 250Gb because the partition table is for a 160Gb disk. The dd stopped the cloning operating after the last sector in the 250Gb has been reach. dd actually cloned the space from 160Gb to 250Gb.

You may ask why didn't I clone the 2.5" 160Gb directly into the 2.5" 250Gb disk. The answer is I had already done that. I bought 3 No. of 2.5" 250Gb disk and one 3.5" 500Gb disk and was experiementing them with eSata connection instead of the USB connection. eSata is 3 to 4 times faster than USB2.

I have cloned between Pata and Sata, Sata and eSata, internal with external disk, 3.5" with 2.5" disk etc etc. Requiring two disk identical size from identical manufacturer is a misinformation.

fareedreg · 12-14-2007, 03:31 AM

Thanks friends but if I apply the command

# dd if=/dev/hda of=/mnt/hdb1/tmp/hda.clone

Then after copying all the old HDD 40GB Maxtor to 80GB Segate will i remove my old one and can install new one.
Will It keep the boot record and system files in my new hdd.

One more question plz.
How can i partition after copying all old data to new hdd?

fareedreg · 12-14-2007, 03:56 AM

Actually what i want is This

I have fedora 8 installed on my pc along with working data on 30 GB HDD MAXTOR. I would like to install a new HDD of 80 GB in such a manner so it can clone my old HDD periodically basis. In case If my old HDD got corrupted the new one will automatically run without interpt my client.

Is there any software required or will I do it on command line or by script?

Thanks

masterclassic · 12-14-2007, 09:07 AM

I don't understand very well: what you mean by "automatically"?
Safety/data integrity in today's systems is usually assured by raid plus backup. Are you sure that the disk got corrupt after last cloning and not before?
Further, it is perhaps better to secure your system against internet intrusions.

saikee · 12-14-2007, 10:46 AM

Two things needed to considered.

(1) If you want the target disk bootable as the original then it must have the same partition table so that the boot loader on loading its second by going into the specified hard disk address will find the right files. This means keeping the image on a file is out.

(2) If you are cloning the whole disk, which is the easiest, you only need to boot up a Linux live CD or just use the existing Fedora Linux in the hard disk and issue just one command in the terminal

I assume your 80Gb hard disk is attached as an external disk or internally and called device sdb. Your source disk is device sda

Code:

dd if=/dev/sda of=/dev/sdb bs=32256

The bs=32256 is to do each record as one complete track of 63 sectors with 512 bytes per sector. Without it the transfer default to 512 bytes (1 sector) per record and it will take 63 operations to clone one track. The 32256 is about the optimum I have tried.

The target disk will be bootable same as the original if the original disk is removed and substituted.

To clone a disk you don't need to partition it. In fact I always tell people to cut off the seal from a new purchase and stick the raw disk in. The first sector cloned of the hard disk has the partition table and why bother to recreate something that will be overwritten immediately?

David1357 · 12-15-2007, 12:45 PM

Quote:

Originally Posted by saikee

When you clone a hard disk you clone its geometry too!

While it is impossible to clone the drive geometry, I understand why you would make such an absurd assertion.

After thinking about it for a few hours, I finally figured out how you are able to use dd to clone drives without screwing up the partition table. As long as the BIOS uses LBA to boot the machine, it will not look at the C/H/S values in the partition table. However, dd will not work for BIOSes that only grok C/H/S addressing. Since BIOSes that old are not likely to be in use very much any more, dd will usually work.

However, if you are saving the image using Partimage or the equivalent, and then using fdisk to create a larger partition on a new drive and trying to restore it to that, the process of managing the MBR is very tricky. I just did it for a friend who wanted to move his 6 GB Windows 2000 install to a 120 GB drive, and I had to think about it overnight before I realized what the magic step was.

uncle-c · 12-15-2007, 01:45 PM

Gentlemen, just to clarify things which would simplify matters considerably ( and help me and the OP ). Imagine we have this possible scenario:

SDA = 10Gb OS = Linux

SDB = 120Gb 100% unallocated space ( no OS or filesystem)

Code:

# dd if=/dev/sda of=/dev/sdb bs=32256

I assume that dd would copy boot record and partition table to sdb ? And thus the remaining 110Gb of sdb will not be shown when does does fdisk -l , but will be shown as "unallocated space" when using a graphical partitioning tool such as gparted. Is this right ?
Now taking out sda and replacing sdb as master drive, would one have to mess about with re-installing grub or should the drive boot normally ?

Thanks !

uncle

saikee · 12-15-2007, 04:09 PM

uncle-c,

I have said it in the previous post and explained in my link. In your case the excess hard disk space will become unallocated space, just as if one uses up the first 10Gb of the hard disk and leaves the remaining 110Gb unallocated. You still have access to the full 120Gb and can use the unallocated space in the existing partitions by resizing them, using programs like Gparted or Parted Magic.

The cloning will execute on the first 10Gb only and will terminate once the number sectors in the device sda has been exhausted.

I have repeated myself already. The first sector of a hard disk is the MBR and the partition of the hard disk is between the 447th to 510th bytes. Thus as soon as the first sector is cloned, which has to be the very first cloning process, the target disk will immediately has a partition identical to the source.

Also as along as you make sure which one is the source which will only be read and not written the cloning operation is very safe. In my link I suggested various way to double check the process before committing the action, which I labeled as the second line of defense against mistake.

David1357,

I have cloned disks with different geometries before. After cloning the target took on the same geometry as the source. You will have a problem of cloning a very old disk that doesn't use LBA whereas all new disks are now big enough requiring LBA mode.

uncle-c · 12-15-2007, 04:30 PM

Thanks a lot saikee, and sorry for making you repeat some material. Differing opinions often lead to confusion for newbies.
You're a legend !

David1357 · 12-16-2007, 10:47 AM

Quote:

Originally Posted by saikee

I have cloned disks with different geometries before. After cloning the target took on the same geometry as the source. You will have a problem of cloning a very old disk that doesn't use LBA whereas all new disks are now big enough requiring LBA mode.

A disk drive is a piece of hardware. Its geometry is determined by the manufacturer. And there are actually two geometries: The real physical geometry (usually 2 or 3 platters and 4 or 6 heads) and the virtual geometry reported by the hard drive interface. Neither one can be changed by modifying values in the master boot record. If you can show me a line from the ATA 6 spec. (or the later version of your choosing) that says something different, I will concede defeat.

When you run fdisk, it does a little magic behind the scenes where it interrogates the hard drive about its geometry. That geometry will vary from drive to drive. However it is fixed and cannot be changed.

As I said before, you are relying on LBA addressing to hide the differences in geometry between two drives. However, you will see problems if you use an old motherboard with a BIOS that uses C/H/S addressing to load the boot code.

Many Linux distros recognize the fact that someone might try to install the O/S on a system that has an old BIOS, and that is why the auto-partitioning part of the install creates a small "/boot" partition that is smaller than the 1024 cylinder limit of C/H/S addressing.

fdisk also knows about this type of problem, and will warn you when it starts if the first partition crosses the 1024 cylinder boundary.

saikee · 12-16-2007, 12:07 PM

David1357,

There is no such thing of conceding defeat in the forum. We all try to contribute with our own experience to the forum on the understanding of the subject matters. Everybody's information is useful. The readers take in whatever they believe.

The manufacturer's geometry of a hard disk is one thing and the freedom of an operating system to read it in another geometry is another. As I understand Linux's fdisk, cfdisk and sfdisk do allow a user to select a geometry different to the manufacturer's setting and LBA itself is a prime example. Nearly every hard disk I purchased in the last three years are always read by Linux having 63 sectors and 255 heads, as seen from my current PC

Code:

debian:/home/saikee# fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1       12158    97659103+  17  Hidden HPFS/NTFS
/dev/sda2   *       12159       24316    97659135    7  HPFS/NTFS
/dev/sda3           24317       36598    98655165    5  Extended
/dev/sda4           36599       60801   194410597+  83  Linux
/dev/sda5           24317       24438      979933+  82  Linux swap / Solaris
/dev/sda6           24439       25654     9767488+  83  Linux
/dev/sda7           25655       26870     9767488+  83  Linux
/dev/sda8           26871       28086     9767488+  83  Linux
/dev/sda9           28087       29302     9767488+  83  Linux
/dev/sda10          29303       30518     9767488+  83  Linux
/dev/sda11          30519       31734     9767488+  83  Linux
/dev/sda12          31735       32950     9767488+  83  Linux
/dev/sda13          32951       34166     9767488+  83  Linux
/dev/sda14          34167       35382     9767488+  83  Linux
/dev/sda15          35383       36598     9767488+  83  Linux

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1       15808   126977728+  15  Unknown
/dev/sdb2           15809       29184   107442720   15  Unknown
/dev/sdb3           29185       42560   107442720    5  Extended
/dev/sdb4           42561       60801   146520832+  15  Unknown
/dev/sdb5           29185       30400     9767488+  83  Linux
/dev/sdb6           30401       31616     9767488+  a9  NetBSD
/dev/sdb7           31617       32832     9767488+  83  Linux
/dev/sdb8           32833       34048     9767488+  83  Linux
/dev/sdb9           34049       35264     9767488+  83  Linux
/dev/sdb10  *       35265       36480     9767488+  83  Linux
/dev/sdb11          36481       37696     9767488+  83  Linux
/dev/sdb12          37697       38912     9767488+  83  Linux
/dev/sdb13          38913       40128     9767488+  83  Linux
/dev/sdb14          40129       41344     9767488+  83  Linux
/dev/sdb15          41345       42560     9767488+  83  Linux

Disk /dev/hdb: 320.0 GB, 320072933376 bytes
255 heads, 63 sectors/track, 38913 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/hdb1   *           1        3039    24410736   af  Unknown
/dev/hdb2            3040        6078    24410704+  b5  Unknown
Partition 2 does not end on cylinder boundary.
/dev/hdb3            6078        8578    20081313   bf  Solaris
/dev/hdb4            8579       11765    25599577+  b5  Unknown
debian:/home/saikee#

Many restrictions in the older kernels are no longer applicable.

This 1024 cylinder limit has been written all oever the Internet. The fact of the matter is when I entered Linux 3.5 years ago both Lilo and Grub have cured this problem and I have never met it in practice. The oldest Grub I managed was from DSL version 0.91 (current being 0.97) and I could use it to boot a distro at the end of a 500Gb disk (with 60801 cylinders).

As a rule I always install a Linux or any OS including BSD and Solaris in a single partition and do not believe a seaprate /boot partition is necessary for a non-server PC myself. Red Hat uses a separate /boot partition can be for the apllicatiuon of LVM as no boot loader can read it. In the sdb disk above I have 44 partitions inside each with a bootable operating system. Each operating system has a /boot but that is a subdirectory of its root filing system "/". Each boot loader inside each /boot boots the operating system regardless what position it is inside the hard disk.

This 1024 cylinder limit for the boot loader is history as far as I am concerned because I never met it myself. I am sure it was an issue before but users nowadays will not be able to find it in any current distro.