LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 05-09-2013, 03:50 PM   #1
zahoo
Member
 
Registered: Nov 2004
Location: EU, HUNGARY, Budapest
Distribution: Debian SID
Posts: 40

Rep: Reputation: 15
Splitting and merging large file (~1.5TB) with dd involving multiple hard drives


Hi All,

First, I hope this is the correct forum/category to post this question, it's kind of in between hardware and software.

I describe my hardware/OS environment at the end.

The issue I'm about to solve is that I've experienced recently a hard drive failure on a 2TB hard drive. As I write, I'm cloning the failed/corrupted drive with the help of ddrescue to another 2TB hard drive into an image. As of good practice, I'll copy the cloned image so that I can safely use the cloned image for data recovery. Here are the critical issues in actually copying this cloned image:

1. I only have free space of 2TB distributed across multiple drives (please note that I'm already using 2 x 2TB drives, i.e. the original and the one hosting the cloned image)

2. The drive hosting the cloned image gave me this warning when running fdisk:

"The device presents a logical sector size that is smaller than
the physical sector size. Aligning to a physical sector (or optimal
I/O) size boundary is recommended, or performance may be impacted.


Command (m for help): p

Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes
81 heads, 63 sectors/track, 765633 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
"

3. When I'll copy the cloned image, I'll do it by splitting the image into multiple parts, so that each part will fit to a drive providing that much of space.


Questions:

1. When I define the block size for splitting the cloned image, what block size should I be using (512, 4096 or else)? Please note the warning message and the difference in logical / physical sector size above.

2. When it comes to merging the parts, is it a problem that different drives might have different sector sizes? If so, do I need to use the block size of the actual drive hosting the part at hand, or the one I used to split the cloned image file? Or should I use the block size of the target drive?


Thank you so much for your help!

Best,

zahoo


Environment: Debian testing (installed ~8 months before the current release of Wheezy) for 64bit arch (amd64), kernel: 3.2, Intel Core i5-750 CPU, Asus P7P55D motherboard.
 
Old 05-09-2013, 06:33 PM   #2
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,103

Rep: Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117
Personally I would trust the software. Pick a (large) multiple of 4k as a "bs" - "dd" just acts on sequential sectors regardless. There might be a performance penalty, but in a recovery scenario that should be the last of your concerns.

For "ddrescue" on the split file, that's a bit more interesting as it only reads the allocated sectors - and tends to be all over the place (as in deliberately non-sequential). I might be inclined to also let it default to 512 - you might be able to just feed it the segments separately and set the appropriate (hardware) sector size, but I don't know if I'd bother.
 
1 members found this post helpful.
Old 05-10-2013, 05:25 PM   #3
zahoo
Member
 
Registered: Nov 2004
Location: EU, HUNGARY, Budapest
Distribution: Debian SID
Posts: 40

Original Poster
Rep: Reputation: 15
Thank you so much, syq00!

So I started the ddrescue with default settings. In the first step I defined the source partition, the target image and a logfile and used the -n flag to avoid bad sector splitting. No blocksize was defined. In the next step, I used the -d -r3 flag, and source partition, target image and logfile to zero in on the bad sectors. Once the procedure was over, took about 2 days (step 1 and 2), I tried to mount the image which I could. Since everything seemed to be the way it was before the disk error, I started copying the files instead of my original goal, i.e. first backing up the cloned image by splitting it. Anyway, your answer was very helpful, it has reassured me that getting the blocksize right has to do with performance as opposed to the outcome of dd, i.e. success/failure.

Thank you again!

Best,

zahoo
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Splitting/Merging Large Files thund3rstruck Linux - Software 3 05-26-2012 02:40 PM
Splitting Large directory over multiple blank DVDs Euler2 Linux - Newbie 5 05-20-2010 04:30 PM
Best File System for Large Drives and Multiple Operating Systems. Gonzee Linux - General 3 06-09-2009 05:53 AM
Splitting A Large Xml File anirudh Programming 7 09-03-2004 04:28 AM
splitting large tar file derfberg Linux - Newbie 1 01-08-2003 07:44 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 08:31 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration