LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 06-13-2011, 04:21 AM   #1
Dtag
LQ Newbie
 
Registered: May 2009
Posts: 7

Rep: Reputation: 0
ddrescue speed issue


Hi,

I am trying to rescue a large harddisk (1TB) by making an image with ddrescue.
As suggested on this page:
http://www.cgsecurity.org/wiki/Damaged_Hard_Disk
I started off with:
./ddrescue -n /dev/old_disk /dev/new_disk rescued.log

Now the program was running at like 30 Mb/s, but the speed keeps decreasing. Now after over 30 hours, its only at 330GB, and the transferrate is at 1.7Mb/s - still getting lower and lower all the time.

Why does it keep getting slower? Is there anything I can do ti speed it up? I also tried interrupting and then restarting it with the --block-size=4KiB parameter that didn't change anything. The transferrate remains.

Any ideas?

Thanks
 
Old 06-13-2011, 04:47 AM   #2
smoker
Senior Member
 
Registered: Oct 2004
Distribution: Fedora Core 4, 12, 13, 14, 15, 17
Posts: 2,279

Rep: Reputation: 250Reputation: 250Reputation: 250
Why don't you run it with the verbose option flag ?
If the disk has problems then the read rate will go down as errors are found.
 
Old 06-13-2011, 05:18 AM   #3
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,413

Rep: Reputation: 4197Reputation: 4197Reputation: 4197Reputation: 4197Reputation: 4197Reputation: 4197Reputation: 4197Reputation: 4197Reputation: 4197Reputation: 4197Reputation: 4197
What [s]he said ...
If it's a dodgy disk, what does time matter ?
 
Old 06-13-2011, 06:22 AM   #4
H_TeXMeX_H
LQ Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301Reputation: 1301
It's probably dieing slowly. It will eventually reach 0, and then it will be dead. (except maybe for the freezer hack, which is a last resort)
 
Old 06-13-2011, 06:30 AM   #5
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,413

Rep: Reputation: 4197Reputation: 4197Reputation: 4197Reputation: 4197Reputation: 4197Reputation: 4197Reputation: 4197Reputation: 4197Reputation: 4197Reputation: 4197Reputation: 4197
Never had the freezer trick work. But, I'm anal about backups ...

Get kicked in the balls often enough, you take notice ...
 
Old 06-13-2011, 07:16 AM   #6
Dtag
LQ Newbie
 
Registered: May 2009
Posts: 7

Original Poster
Rep: Reputation: 0
As for the verbose option: I have enabled it now but so far there hasnt been any more output than without verbose (except for some lines right at the start). Also I haven't had any errors for quite a while, and so far only like 60 MB have been tagged as errorneous. So I'm kind of doubting that the slowdown is due to errors -- also note that the slowdown is slow, but steady -- if it was due to errors, i wouldnt expect a steady slowdown.

As for why time matters: Well I don't care if it takes a day, or a few days... but if it gets well below 1MB/s which is going to be very soon if the slowdown continues, its going to take weeks of even longer -- if its gonna finish at all.

I'm also kind of doubting the disk is really dieing - it looks like there was something in ddrescue that makes the speed degrade for huge files.

Oh by the way I found someone who appeared to have the same problem some time: http://www.mail-archive.com/bug-ddre.../msg00365.html Unfortunately noone replied :/
 
Old 06-13-2011, 08:06 AM   #7
smoker
Senior Member
 
Registered: Oct 2004
Distribution: Fedora Core 4, 12, 13, 14, 15, 17
Posts: 2,279

Rep: Reputation: 250Reputation: 250Reputation: 250
The speed will not degrade for huge files, in fact the opposite is true. Lots of small files take longer because the process keeps getting interrupted, and so it cannot build up to full speed.
 
Old 06-13-2011, 01:18 PM   #8
Dtag
LQ Newbie
 
Registered: May 2009
Posts: 7

Original Poster
Rep: Reputation: 0
Well I was talking about the size of the destination image file of course.

Status update: 411Gb now and it is down to 100 kb/s If i just restart the process and make it write to another file, it starts off with the full 30MB/s again and then starts dropping again
 
Old 07-28-2011, 02:01 PM   #9
skapunker
LQ Newbie
 
Registered: Jul 2011
Posts: 4

Rep: Reputation: Disabled
I've always had the same issue with ddrescue, I'm a computer tech and use it all the time for backing up data for customer's when the drive is either failing or they have accidentally deleted files and I need to recover their files or partition. I am currently backing up a 320 GB HDD that I know is not failing, I'm just recovering data on it. It started off at a decent speed but now it's done 220 GB after for running for 2 days and is copying at 390 kB/s. I know ddrescue will slow down when it's trying to copy bad blocks, but this is not the first time I've had it progressively get slower on a perfectly good drive, so I know it's not the drive.

Just wanted to let Dtag know you're not crazy.

Still searching for an answer to this issue.

Running Ubuntu 10.10, ddrescue 1.14

Last edited by skapunker; 07-28-2011 at 02:09 PM.
 
Old 07-28-2011, 04:17 PM   #10
qrange
Senior Member
 
Registered: Jul 2006
Location: Belgrade, Yugoslavia
Distribution: Debian stable/testing, amd64
Posts: 1,071

Rep: Reputation: 47
i would use larger blocksize, for eg 256K.
perhaps its heating up. try to keep it as cool as possible.

if all else fails, backup it piece by piece, by skipping some blocks on input and output (btw, be careful with that option, always remember block size)

Last edited by qrange; 07-28-2011 at 04:18 PM.
 
Old 07-28-2011, 05:28 PM   #11
Tinkster
Moderator
 
Registered: Apr 2002
Location: earth
Distribution: slackware by choice, others too :} ... android.
Posts: 23,067
Blog Entries: 11

Rep: Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928Reputation: 928
A random note: if both devices are dangling of the same usb channel
that may cause issues; have you tried writing the output to a file
on the built-in HDD and compare the speeds?

And then see how it goes writing from that intermediate file to
the clone disk.

Last edited by Tinkster; 07-28-2011 at 05:29 PM.
 
Old 07-28-2011, 08:08 PM   #12
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,417
Blog Entries: 55

Rep: Reputation: 3627Reputation: 3627Reputation: 3627Reputation: 3627Reputation: 3627Reputation: 3627Reputation: 3627Reputation: 3627Reputation: 3627Reputation: 3627Reputation: 3627
Quote:
Originally Posted by syg00 View Post
Never had the freezer trick work.
That's because it's a busted myth. It doesn't apply to modern drives.


Quote:
Originally Posted by Dtag View Post
Status update: 411Gb now and it is down to 100 kb/s If i just restart the process and make it write to another file, it starts off with the full 30MB/s again and then starts dropping again
Could try dd_rescue? From the README: You can tell dd_rescue to start from the end of a file and move backwards and it uses two block sizes, a large (soft) block size and a small (hard) block size. In case of errors, the size falls back to the small one and is promoted again after a while without errors.


Quote:
Originally Posted by Tinkster View Post
if both devices are dangling of the same usb channel that may cause issues
Good point. I usually took source disks out, favoring a HW write blocker / reader or motherboard connection over whatever an enclosure provided.
 
Old 07-29-2011, 02:59 AM   #13
Dtag
LQ Newbie
 
Registered: May 2009
Posts: 7

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by skapunker View Post
Just wanted to let Dtag know you're not crazy.
Thanks


It's been a while since I had that problem -- I didn't really solve it. I only found out that ddrescue had an option to attempt to read the filesystem and it actually worked! I was really quite surprised as mounting previously failed with horrible error messages and everything seemed very broken and all my attempts to fix it failed. I then proceeded to copy the files over (which only took like a day). Some files are broken but like 95% are ok - that was fine with me.
 
Old 08-04-2011, 08:11 PM   #14
skapunker
LQ Newbie
 
Registered: Jul 2011
Posts: 4

Rep: Reputation: Disabled
Good to hear it worked out.

As far as the speed issue I believe (at least for me) it was a hardware configuration issue (device or driver). With the same machine I had a very similar slowing transfer issue with rsync but after reloading Ubuntu it has still had issues and now with ddrescue as well.

The configuration I use is a storage drive hooked up to a motherboard sata port and then the drive to a sata card. I hooked the drive up the same way to a different machine and resumed ddrescue with the log where the other computer had left off while running slow and the rest of the transfer went quickly.

So my recommendation for those having this issue is try a different machine and/or hardware. Wish I knew exactly what caused it though.
 
Old 01-28-2014, 09:18 AM   #15
njank
LQ Newbie
 
Registered: Jan 2014
Posts: 6

Rep: Reputation: Disabled
ddrescue in file copy mode?

Quote:
Originally Posted by Dtag View Post
I only found out that ddrescue had an option to attempt to read the filesystem and it actually worked! I was really quite surprised as mounting previously failed with horrible error messages and everything seemed very broken and all my attempts to fix it failed. I then proceeded to copy the files over (which only took like a day). Some files are broken but like 95% are ok - that was fine with me.
Sorry for resurrecting an old thread, but as this is the only time I've heard someone mention that ddresuce could be set to what sounds like a filecopy mode, I wanted to ask if anyone could clarify. to my knowledge and a scan of the ddrescue manual, I thought it only did raw disk or partition copies. I'm recovering a failing drive with only 40GiB of non-consolidated data spread over a ~240GiB disk. so I'm waiting a long time for ddrescue to copy good but empty sectors. The filesystem mounts fine, but the drive goes nonresponsive after repeatedly hitting a bad sector as file copy tools like to do (even HBCD's unstoppable copy proved quite a hands-on chore). ddrescue plays very nicely with the drive, but being able to restrict to used sectors would be a godsend... (drive frequently drops to <100k/sec)
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
network speed issue varun_saa Red Hat 1 11-22-2006 01:05 PM
Network Speed issue cereal83 Linux - Networking 4 05-31-2006 09:56 AM
Linux, an issue of speed. elliotfuller Linux - General 46 08-09-2005 07:53 PM
RH9 proFTP speed issue benbroad Linux - Software 8 12-01-2004 02:33 AM
k3b speed issue Pathian Linux - Hardware 2 10-22-2003 05:51 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 04:50 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration