LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)
-   -   Splitting and merging large file (~1.5TB) with dd involving multiple hard drives (https://www.linuxquestions.org/questions/linux-software-2/splitting-and-merging-large-file-%7E1-5tb-with-dd-involving-multiple-hard-drives-4175461337/)

zahoo 05-09-2013 03:50 PM

Splitting and merging large file (~1.5TB) with dd involving multiple hard drives
 
Hi All,

First, I hope this is the correct forum/category to post this question, it's kind of in between hardware and software.

I describe my hardware/OS environment at the end.

The issue I'm about to solve is that I've experienced recently a hard drive failure on a 2TB hard drive. As I write, I'm cloning the failed/corrupted drive with the help of ddrescue to another 2TB hard drive into an image. As of good practice, I'll copy the cloned image so that I can safely use the cloned image for data recovery. Here are the critical issues in actually copying this cloned image:

1. I only have free space of 2TB distributed across multiple drives (please note that I'm already using 2 x 2TB drives, i.e. the original and the one hosting the cloned image)

2. The drive hosting the cloned image gave me this warning when running fdisk:

"The device presents a logical sector size that is smaller than
the physical sector size. Aligning to a physical sector (or optimal
I/O) size boundary is recommended, or performance may be impacted.


Command (m for help): p

Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes
81 heads, 63 sectors/track, 765633 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
"

3. When I'll copy the cloned image, I'll do it by splitting the image into multiple parts, so that each part will fit to a drive providing that much of space.


Questions:

1. When I define the block size for splitting the cloned image, what block size should I be using (512, 4096 or else)? Please note the warning message and the difference in logical / physical sector size above.

2. When it comes to merging the parts, is it a problem that different drives might have different sector sizes? If so, do I need to use the block size of the actual drive hosting the part at hand, or the one I used to split the cloned image file? Or should I use the block size of the target drive?


Thank you so much for your help!

Best,

zahoo


Environment: Debian testing (installed ~8 months before the current release of Wheezy) for 64bit arch (amd64), kernel: 3.2, Intel Core i5-750 CPU, Asus P7P55D motherboard.

syg00 05-09-2013 06:33 PM

Personally I would trust the software. Pick a (large) multiple of 4k as a "bs" - "dd" just acts on sequential sectors regardless. There might be a performance penalty, but in a recovery scenario that should be the last of your concerns.

For "ddrescue" on the split file, that's a bit more interesting as it only reads the allocated sectors - and tends to be all over the place (as in deliberately non-sequential). I might be inclined to also let it default to 512 - you might be able to just feed it the segments separately and set the appropriate (hardware) sector size, but I don't know if I'd bother.

zahoo 05-10-2013 05:25 PM

Thank you so much, syq00!

So I started the ddrescue with default settings. In the first step I defined the source partition, the target image and a logfile and used the -n flag to avoid bad sector splitting. No blocksize was defined. In the next step, I used the -d -r3 flag, and source partition, target image and logfile to zero in on the bad sectors. Once the procedure was over, took about 2 days (step 1 and 2), I tried to mount the image which I could. Since everything seemed to be the way it was before the disk error, I started copying the files instead of my original goal, i.e. first backing up the cloned image by splitting it. Anyway, your answer was very helpful, it has reassured me that getting the blocksize right has to do with performance as opposed to the outcome of dd, i.e. success/failure.

Thank you again!

Best,

zahoo


All times are GMT -5. The time now is 01:06 PM.