dd copy running for 4 days, can i check status somehow?

rwreed · 02-25-2009, 10:32 AM

Hi All,

My hard drive started acting up so I thought I'd back it up to a usb drive I have. Its a 250gb sata and I dd it to a 350gb usb drive with the following command dd if=/dev/sda of=/dev/sdc Its now been running about 4 days continuously. I know its working because I see the hard drive light flashing (intermittently and slowly). Is there any way to figure out if its almost done, how much it has done or how much longer it will take?

Thanks in advance.
Randy

******* · 02-25-2009, 11:38 AM

You should be able to do something like 'pkill -USR1 dd' to get stats. See 'man dd' for the actual signal.

PTrenholme · 02-25-2009, 12:18 PM

Note that the command you stated you used will have replaced the partition table of the USB drive with a copy of the partition table of the SATA drive. That partition table will, at a minimum, need to be repaired before the image will be usable, and you may find that the table is irreparable.

There is a useful "how-to" for dd in the tutorials section.

That being said, there is another version of dd called dcfldd that has a "progress bar," and another (ddrescue) that will try to "skip" errors in the source drive that the noerror option to dd might find problematic.

The tutorial suggests that using the bs=4096 conv=noerror,notrunc options to dd will improve the performance of dd.

In what way was your sda "acting up?" There may be other things you can try besides imaging the drive.

rwreed · 02-25-2009, 12:32 PM

Thank you. Your response got me searching with a new set of parameters. I had read the man page but I didn't understand it.

For those who come after these instructions really helped from http://linuxcommando.blogspot.com/2008/06/show-progress-during-dd-copy.html

Code:

First, find out the process id of the dd process by running the following in the new virtual terminal.

$ pgrep -l '^dd$'
8789 dd
$



To send the USR1 signal to the dd prcoess:

$ kill -USR1  8789
$



Note that as soon as the USR1 signal is detected, dd will print out the current statistics to its STDERR.

$ dd if=/dev/random of=/dev/null bs=1K count=100
0+14 records in
0+14 records out
204 bytes (204 B) copied, 24.92 seconds, 0.0 kB/s

CJS · 02-25-2009, 12:36 PM

Quote:

Originally Posted by PTrenholme

No, but the command you stated you used will have replaced the partition table of the USB drive with a copy of the partition table of the SATA drive. That partition table will, at a minimum, need to be repaired before the image will be usable, and you may find that the table is irreparable.

Just curious, but why do you say the partition table will need to be repaired? Assuming his original partition table on the sda drive was not corrupt, I don't see anything wrong with copying that partition table to the new drive along with all the partitions. Of course none of the partitions will extend past 250 GB since that is the capacity of the sda drive, but that's not a problem.

pixellany · 02-25-2009, 02:00 PM

Independent of all other comments.....

Before using dd to copy large amounts of data, I will always do a partial dry run (eg copy just 1GB) to see how long it's going to take. In the dry run, I can try different block sizes to see what is fastest.

Quick experiment:
Imaging from my disk to an image file on the same disk (I used a block size of 500M): ~32MBytes/second (Because it's writing to the same disk, I'd expect this to be slower than disk to disk.
At this rate, 320 GBytes would take 10,000 seconds, which is just under 3 hours.

Based on this, I'd say that with something running for days, there is a problem....

Also, I think it's good to specify a count, and not just let it run to the end.

Tinkster · 02-25-2009, 02:22 PM

Quote:

Originally Posted by CJS

Of course none of the partitions will extend past 250 GB since that is the capacity of the sda drive, but that's not a problem.

If one doesn't mind losing 100 GB ...

Cheers,
Tink

syg00 · 02-25-2009, 05:55 PM

Quote:

Originally Posted by pixellany

Based on this, I'd say that with something running for days, there is a problem....

And the problem is probably USB. I tried to recover a mates NTFS partition once, but dd'd it off first. Problem was (I later figured out), he only had USB 1 - ugh ...

Tinkster · 02-25-2009, 05:58 PM

heh

150KB/s isn't much fun with a large file system ;*)

PTrenholme · 02-25-2009, 08:46 PM

Quote:

Originally Posted by CJS

Just curious, but why do you say the partition table will need to be repaired? Assuming his original partition table on the sda drive was not corrupt, I don't see anything wrong with copying that partition table to the new drive along with all the partitions. Of course none of the partitions will extend past 250 GB since that is the capacity of the sda drive, but that's not a problem.

That sounds plausible, but I would consider loosing 100 Gb on a drive a problem worth fixing. And, when I did the dd tango whilst replacing a 80Gb drive in a laptop with a 160 Gb one, my recollation is that GRUB would not boot the new drive 'till I repaired the partition table. That may have been a particularity of the specific configuration I had on that laptop, and my generalizing of the problem to this thread may have overstated the case (for which I appoligize), but I think a caution about the partition table was warrented.

frieza · 02-25-2009, 09:27 PM

Quote:

Originally Posted by rwreed

Hi All,

My hard drive started acting up so I thought I'd back it up to a usb drive I have. Its a 250gb sata and I dd it to a 350gb usb drive with the following command dd if=/dev/sda of=/dev/sdc Its now been running about 4 days continuously. I know its working because I see the hard drive light flashing (intermittently and slowly). Is there any way to figure out if its almost done, how much it has done or how much longer it will take?

Thanks in advance.
Randy

that actually is the wrong way to do it, you have the right idea if you want a bit copy and your usb hard drive has enough room (the size of your hard drive you wanna back up + a bit extra), which as you stated, it has more then enough room.
you would mount the usb drive on say /mnt/usbdrive

then dd if=/dev/sda of=/mnt/usbdrive/backup.img
this creates a raw dump of your drive to a file stored on the usb drive's filesystem that can be recovered by dd if=/mnt/usbdrive/backup.img of=/dev/sda from a live cd or a restore cd
note, not sure if its absolutely necessary but.. just to make sure, after restoring using dd, fdisk the drive and without making any changes, write the partition table just to make sure it's written properly
however either way i'm not surprised that it was taking so long
dd copies every single bit of that 250 gigs, literally EVERY bit, bit for bit, even the free space and usb transfer rates, even usb2.0 are slow by comparison to sata