Another look at DD for cloning disks
 

I'm still very much a Linux newb, but have spent many hours studying my new OS since my last post here, and want to follow up on disk cloning questions first posed in:

https://www.linuxquestions.org/quest...ve-4175552363/

Is DD really such a terrible option? If so, please tell me why, and spare me from any grief I may get into. To me, it now looks like the best means to produce my next disk clone.

Last time around, as explained in the above-linked thread, I wasted time in Clonezilla, then successfully used the Windows program Macrium to produce a clone, albeit at great temporary cost to my Win 7 desktop.

If I had it to do again, I'd:

(*) Install a target SSD in the computer alongside an identical source drive.
(*) Boot Linux from a CD (so neither internal drive will be mounted).
(*) Open a terminal window, and issue a command like mount or sudo parted -l or gnome-disks to be absolutely sure that my source is, say, dev/sda, and that my target is dev/sdb.
(*) Type dd if=/dev/sda of=/dev/sdb

And I'm done!

Last time around, syg00 dubbed DD the worst backup option besides no backup at all, and suicidaleggroll wrote that he'd never trust a clone made of a mounted disk. Linux experts, please share any negative thoughts about DD, if you think I'm putting my faith in the wrong program. I want to benefit from your experience.

Also, if there's a reason to use DD switches when making a clone to an identical drive, please let me know.

I use dd all the time, but not as a backup solution. I use it for things like preserving an installed system image (equivalent to a factory restore option), replacing a systems hard drive, moving a disk to a virtual machine, copying downloaded iso to media (such as creating a live usb).

Do double/triple check that your if= and of= specs are correct. The first time you get them backwards you won't be happy.

For cloning/copying a hard drive, I usually boot the source system with a live media, and connect an external hard drive dock disk, and use dd to save the disk image as a file on the external disk. This takes longer than disk-disk, but gives you additional advantages. You can mount partitions from the image file in linux to access files without needing to put it on a physical drive first. You can restore the image over a network, which can be helpful in circumstances when an external disk isn't feasible.

<OPINION> I worked with clonezilla for a few days, but dropped it from my toolbox. Poor documentation and a lot of extra work to accomplish the same result as a dd image were my gripes.</OPINION>

The disadvantage of dd for cloning is that if you have, for example, a 500GB disk with a filesystem that is 30% full, dd is still going to copy the entire 500GB. An intelligent cloning program would skip over the free space and copy only the parts that are used. Of course if you're making a copy for forensic analysis or data recovery from a damaged filesystem, then you want the full copy, and dd (or ddrescue for drives with some bad sectors) is the tool you want.

As rknichols says, you end up copying the entire drive, bit for bit.

Whenever I do it, which is not that often, I zero out the free space, pipe the output to bz2 or xz which will compress the resulting image. Then upon restoring pipe the output from bz2/xz to dd.

This is time consuming, and there are likely better ways.

ETA: I agree with never backing up a mounted disk. Files can change. inodes can change. If you're going to use dd, unmount the source disk first.

A newbie and guru can bork a system pretty quickly with dd of course.

You have other options that may be faster too. Things like clonezilla/redobackup/rsync/cpio/tar and even things like gparted can copy off data.

dd has two major failings. You are copying ALL THE DATA, including the UNUSED disk space. And you will have difficulty if the destination drive or partition is not identical to the source drive or partition. With various other annoyances if you want to change things between the source and destination like partition sizes or file systems. But it has various still relevant modern uses as well.

Thanks for the feedback, everyone! I think I'll now use DD the next time I need to clone, with appropriate care while typing, and only when sure that the drives are unmounted.

yup. this is what i do:
http://www.linuxquestions.org/questi...9/#post5408615

Limitations:

1. does not copy filesystems mounted read/write (the usual case)
2. does not allow for changing the volume names or labels
3. does not allow for changing size of the filesystem (either expanding or shrinking)
4. copies everything bit for bit.

#1 - can be worked around by remounting filesystems read only.
#2 - you have to manually change the names or labels. If you don't a reboot will be unable to identify the disks to use other than by device name - which can change (it depends on spin up time, controller order, and physical controller...)
#3 - can't easily be worked around: you have to copy by partition, not by disk, but then you loose the boot blocks.
#4 - good only for making images... sucks if you have a 4TB disk that is only 10 filled; and it is SLOW for large disks

One good thing it does is copy the disk for an image backup (copying boot blocks, partition tables, and any filesystems) - but don't do it for large disks as it can take hours.

Another thing is that it doesn't care what is on the disk. All other methods have to understand the filesystem(s) on the disk or they cannot resize the partitions and filesystems. Resizing a partition requires adjusting the filesystems definition of its size, it can also require moving data blocks around if the data is scattered.

Personally, I use tar to create backups. Since it operates at the file level, it doesn't care what kind of filesystem is used as that is handled by the kernel. The resulting backup can be restored to any filesystem on any available partition as long as it is large enough for the data (conversions from things like NTFS to btrfs are trivial). To finish a rebuild, I reinstall the boot, and adjust the fstab file to properly mount the various filesystems on boot. Then boot single user to double check everything.

The only place I have used dd to make a backup was on raspbian (Raspberry PI) and that was an 8GB dump. But moving to a 16GB sdcard was a bit of a booger as the extended partition was between two primary partitions and a third. Only the third primary partition could be extended... So, delete two partition, create new ones - then restore from a tar backup I also made. And using dd isn't even really needed there, unless you are trying to preserve a RISCOS installation (which doesn't know squat about partitions or filesystems).

I swore by dd as a commercial solution for years thinking time was less of an issue than data integrity.

In 2009 I switched to rsync and I think it's the best solution for PCs.

If you're commercial or using novel hardware (raspberry pi was mentioned earlier), you may want or even need dd. For home use you'll be way happier learning to write filer files (takes about 2 hours if you apply yourself)

after that backups are $FAST. If you want periodic backup, you can script something likemake it a cron job and forget it until you need the backup.

your rules file might look likecheck out any of the rsync tutorials on the web. It's very powerful software

Clonzilla.

From other post. "and deposited a big new file on the USB drive. "

That means you copied the drive to a file and not a clone.