LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-16-2015, 07:06 PM   #1
Adams Seven
LQ Newbie
 
Registered: Sep 2015
Distribution: Linux Mint 17.2
Posts: 23

Rep: Reputation: Disabled
Another look at DD for cloning disks


I'm still very much a Linux newb, but have spent many hours studying my new OS since my last post here, and want to follow up on disk cloning questions first posed in:

https://www.linuxquestions.org/quest...ve-4175552363/

Is DD really such a terrible option? If so, please tell me why, and spare me from any grief I may get into. To me, it now looks like the best means to produce my next disk clone.

Last time around, as explained in the above-linked thread, I wasted time in Clonezilla, then successfully used the Windows program Macrium to produce a clone, albeit at great temporary cost to my Win 7 desktop.

If I had it to do again, I'd:

(*) Install a target SSD in the computer alongside an identical source drive.
(*) Boot Linux from a CD (so neither internal drive will be mounted).
(*) Open a terminal window, and issue a command like mount or sudo parted -l or gnome-disks to be absolutely sure that my source is, say, dev/sda, and that my target is dev/sdb.
(*) Type dd if=/dev/sda of=/dev/sdb

And I'm done!

Last time around, syg00 dubbed DD the worst backup option besides no backup at all, and suicidaleggroll wrote that he'd never trust a clone made of a mounted disk. Linux experts, please share any negative thoughts about DD, if you think I'm putting my faith in the wrong program. I want to benefit from your experience.

Also, if there's a reason to use DD switches when making a clone to an identical drive, please let me know.
 
Old 09-16-2015, 07:25 PM   #2
Doug G
Member
 
Registered: Jul 2013
Posts: 749

Rep: Reputation: Disabled
I use dd all the time, but not as a backup solution. I use it for things like preserving an installed system image (equivalent to a factory restore option), replacing a systems hard drive, moving a disk to a virtual machine, copying downloaded iso to media (such as creating a live usb).

Do double/triple check that your if= and of= specs are correct. The first time you get them backwards you won't be happy.

For cloning/copying a hard drive, I usually boot the source system with a live media, and connect an external hard drive dock disk, and use dd to save the disk image as a file on the external disk. This takes longer than disk-disk, but gives you additional advantages. You can mount partitions from the image file in linux to access files without needing to put it on a physical drive first. You can restore the image over a network, which can be helpful in circumstances when an external disk isn't feasible.

<OPINION> I worked with clonezilla for a few days, but dropped it from my toolbox. Poor documentation and a lot of extra work to accomplish the same result as a dd image were my gripes.</OPINION>
 
1 members found this post helpful.
Old 09-16-2015, 07:50 PM   #3
rknichols
Senior Member
 
Registered: Aug 2009
Distribution: CentOS
Posts: 4,360

Rep: Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001Reputation: 2001
The disadvantage of dd for cloning is that if you have, for example, a 500GB disk with a filesystem that is 30% full, dd is still going to copy the entire 500GB. An intelligent cloning program would skip over the free space and copy only the parts that are used. Of course if you're making a copy for forensic analysis or data recovery from a damaged filesystem, then you want the full copy, and dd (or ddrescue for drives with some bad sectors) is the tool you want.
 
1 members found this post helpful.
Old 09-16-2015, 08:04 PM   #4
goumba
Senior Member
 
Registered: Dec 2009
Location: New Jersey, USA
Distribution: Current: Debian and OpenSUSE. Past: Arch, RedHat (pre-RHEL). FreeBSD & OpenBSD novice, Hackintosh
Posts: 1,192
Blog Entries: 7

Rep: Reputation: 335Reputation: 335Reputation: 335Reputation: 335
As rknichols says, you end up copying the entire drive, bit for bit.

Whenever I do it, which is not that often, I zero out the free space, pipe the output to bz2 or xz which will compress the resulting image. Then upon restoring pipe the output from bz2/xz to dd.

This is time consuming, and there are likely better ways.

ETA: I agree with never backing up a mounted disk. Files can change. inodes can change. If you're going to use dd, unmount the source disk first.

Last edited by goumba; 09-16-2015 at 08:55 PM.
 
1 members found this post helpful.
Old 09-16-2015, 08:47 PM   #5
jefro
Moderator
 
Registered: Mar 2008
Posts: 19,502

Rep: Reputation: 3014Reputation: 3014Reputation: 3014Reputation: 3014Reputation: 3014Reputation: 3014Reputation: 3014Reputation: 3014Reputation: 3014Reputation: 3014Reputation: 3014
A newbie and guru can bork a system pretty quickly with dd of course.

You have other options that may be faster too. Things like clonezilla/redobackup/rsync/cpio/tar and even things like gparted can copy off data.
 
Old 09-17-2015, 11:16 AM   #6
Shadow_7
Senior Member
 
Registered: Feb 2003
Distribution: debian
Posts: 4,000
Blog Entries: 1

Rep: Reputation: 846Reputation: 846Reputation: 846Reputation: 846Reputation: 846Reputation: 846Reputation: 846
dd has two major failings. You are copying ALL THE DATA, including the UNUSED disk space. And you will have difficulty if the destination drive or partition is not identical to the source drive or partition. With various other annoyances if you want to change things between the source and destination like partition sizes or file systems. But it has various still relevant modern uses as well.
 
Old 09-17-2015, 11:39 AM   #7
Adams Seven
LQ Newbie
 
Registered: Sep 2015
Distribution: Linux Mint 17.2
Posts: 23

Original Poster
Rep: Reputation: Disabled
Thanks for the feedback, everyone! I think I'll now use DD the next time I need to clone, with appropriate care while typing, and only when sure that the drives are unmounted.
 
Old 09-17-2015, 11:48 AM   #8
schneidz
LQ Guru
 
Registered: May 2005
Location: boston, usa
Distribution: fc-15/ fc-20-live-usb/ aix
Posts: 5,272

Rep: Reputation: 914Reputation: 914Reputation: 914Reputation: 914Reputation: 914Reputation: 914Reputation: 914Reputation: 914
Quote:
Originally Posted by goumba View Post
As rknichols says, you end up copying the entire drive, bit for bit.

Whenever I do it, which is not that often, I zero out the free space, pipe the output to bz2 or xz which will compress the resulting image. Then upon restoring pipe the output from bz2/xz to dd.

This is time consuming, and there are likely better ways.

ETA: I agree with never backing up a mounted disk. Files can change. inodes can change. If you're going to use dd, unmount the source disk first.
yup. this is what i do:
http://www.linuxquestions.org/questi...9/#post5408615
 
Old 09-17-2015, 11:49 AM   #9
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,844

Rep: Reputation: 1478Reputation: 1478Reputation: 1478Reputation: 1478Reputation: 1478Reputation: 1478Reputation: 1478Reputation: 1478Reputation: 1478Reputation: 1478
Limitations:

1. does not copy filesystems mounted read/write (the usual case)
2. does not allow for changing the volume names or labels
3. does not allow for changing size of the filesystem (either expanding or shrinking)
4. copies everything bit for bit.

#1 - can be worked around by remounting filesystems read only.
#2 - you have to manually change the names or labels. If you don't a reboot will be unable to identify the disks to use other than by device name - which can change (it depends on spin up time, controller order, and physical controller...)
#3 - can't easily be worked around: you have to copy by partition, not by disk, but then you loose the boot blocks.
#4 - good only for making images... sucks if you have a 4TB disk that is only 10 filled; and it is SLOW for large disks

One good thing it does is copy the disk for an image backup (copying boot blocks, partition tables, and any filesystems) - but don't do it for large disks as it can take hours.

Another thing is that it doesn't care what is on the disk. All other methods have to understand the filesystem(s) on the disk or they cannot resize the partitions and filesystems. Resizing a partition requires adjusting the filesystems definition of its size, it can also require moving data blocks around if the data is scattered.

Personally, I use tar to create backups. Since it operates at the file level, it doesn't care what kind of filesystem is used as that is handled by the kernel. The resulting backup can be restored to any filesystem on any available partition as long as it is large enough for the data (conversions from things like NTFS to btrfs are trivial). To finish a rebuild, I reinstall the boot, and adjust the fstab file to properly mount the various filesystems on boot. Then boot single user to double check everything.

The only place I have used dd to make a backup was on raspbian (Raspberry PI) and that was an 8GB dump. But moving to a 16GB sdcard was a bit of a booger as the extended partition was between two primary partitions and a third. Only the third primary partition could be extended... So, delete two partition, create new ones - then restore from a tar backup I also made. And using dd isn't even really needed there, unless you are trying to preserve a RISCOS installation (which doesn't know squat about partitions or filesystems).
 
Old 09-17-2015, 09:20 PM   #10
polaris96
Member
 
Registered: Jan 2015
Distribution: Slackware, LFS, OpenIndiana, debian wheezy
Posts: 55

Rep: Reputation: Disabled
I swore by dd as a commercial solution for years thinking time was less of an issue than data integrity.

In 2009 I switched to rsync and I think it's the best solution for PCs.

If you're commercial or using novel hardware (raspberry pi was mentioned earlier), you may want or even need dd. For home use you'll be way happier learning to write filer files (takes about 2 hours if you apply yourself)

after that backups are $FAST. If you want periodic backup, you can script something like
Code:
#!/bin/bash
#This assumes BACKUP_PARTITION is locally connected

mount /dev/$BACKUP_PARTITION /mnt
rsync -a --filter="merge $FILTERFILE" / /mnt
umount /mnt
make it a cron job and forget it until you need the backup.

your rules file might look like
Code:
- /dev/
- /media/
- /mnt/
- /proc/
- /sys/
- /run/
- /tmp/
check out any of the rsync tutorials on the web. It's very powerful software
 
Old 09-18-2015, 08:55 PM   #11
jefro
Moderator
 
Registered: Mar 2008
Posts: 19,502

Rep: Reputation: 3014Reputation: 3014Reputation: 3014Reputation: 3014Reputation: 3014Reputation: 3014Reputation: 3014Reputation: 3014Reputation: 3014Reputation: 3014Reputation: 3014
Clonzilla.

From other post. "and deposited a big new file on the USB drive. "

That means you copied the drive to a file and not a clone.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
cloning several disks to one? unclejed613 Slackware 10 05-19-2014 09:02 PM
[SOLVED] added a couple of new disks and now /dev/sd* disks are all messed up nass Linux - Hardware 4 10-30-2012 11:16 AM
[SOLVED] RAID 5 with 4 hard disks... array started with only 3 out of 4 disks kikinovak Slackware 9 08-11-2012 07:33 AM
Installation Disks not recognized as official disks when booting with floppy coal-fire-ice Mandriva 13 12-09-2004 05:29 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 03:15 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration