LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 05-09-2006, 07:11 PM   #16
haertig
Senior Member
 
Registered: Nov 2004
Distribution: Debian, Ubuntu, LinuxMint, Slackware, SysrescueCD, Raspbian, Arch
Posts: 2,327

Rep: Reputation: 357Reputation: 357Reputation: 357Reputation: 357

I don't have any experience resizing NTFS partitions from Linux. I messed around years ago using Partition Magic, but that was only for data I didn't really care about.

I would start a new thread with a very specific title asking for help on your resizing problem. You should get better responses than in this thread where the title mentions nothing about resizing partitions. Then come back here after you get it resized (or start a new very specific thread) with any problems you run into with dd. The more specific the title to a thread, the better you are able to attract the people who know the answer. I wish I could help you on this one, I just don't have experience resizing NTFS.

A word of caution: I would personally consider resizing a partition to be dangerous. I would backup the data first, assuming it's important stuff you don't want to risk losing. And if you've got good backups (test them!), then there's really no need to resize. Just wipe the partitions out, repartition from scratch, and restore data as needed from your backups.

Make SURE!!! you've got your if and of parameters correct, and are targeting the correct destination partition before you hit <enter> on that dd command. Double, triple, quadruple check! After all the work you've done on this, it would be horrible to have it all go down the drain because of a single character typo. An "i" for an "o", an "a" for a "b", a "1" for a "2" ... and you're screwed!
 
Old 05-09-2006, 08:31 PM   #17
vbsaltydog
Member
 
Registered: Nov 2005
Distribution: CentOS
Posts: 154

Original Poster
Rep: Reputation: 15
Your comments and assistance are greatly appreciated. I figured out the ntfs error. It was because the last time the windows OS was up, it was shutdown without using the start|shutdown menu so the ntfs boot was set to go to the boot menu screen (the one with start windows normally, safe mode, etc), once I brought windows up and then shutdown cleanly the knoppix/ntfsresize command worked.

I am finishing the ntfsresize now and will do the dd procedure next. I will report the results to the board.
 
Old 05-09-2006, 08:58 PM   #18
Electro
LQ Guru
 
Registered: Jan 2002
Posts: 6,042

Rep: Reputation: Disabled
It is better to use
dd if=/dev/hda of=image.bin conv=noerror

This means you need to format another drive and make a partition that is big enough to handle the image. The drive that will contain the image file have to be greater than 250 GB. A 300 GB will do. After you are done making an image, completely disconnect the failed drive.

DO NOT touch the source drive. Just make an image of the drive using the dd command. You do not need to resize anything.

Steps:
1) insert new hard drive that is 300 GB
2) make one partition on the 300 GB drive
3) format the 300 GB hard drive
4) use dd to make image of the failed drive and save the image file to
5) disconnect the failed drive.
6) extract data from the image file

The possibility of retriving all the files is a very, very slim chance. The possibility of the client being happy is also a very, very slim chance.
 
Old 05-09-2006, 10:28 PM   #19
vbsaltydog
Member
 
Registered: Nov 2005
Distribution: CentOS
Posts: 154

Original Poster
Rep: Reputation: 15
Well here's the current status....ehhhh.

Failed 80Gb HD as a slave.
Good 120Gb HD as the Master (divided into a 40GB NTFS partition and a FAT16 80GB partition).

The Plan is to dd the failed drive to the available partition and recover data from recovery partition.

dd if=/dev/hdb of=/dev/hda2 conv=noerrors

Well, when I run this command I get a ton of IO errors and 0 bytes in and out messages with 0 files transferred.

I was under the impression that dd does a raw, byte level copy and would not error on disk scratches or whatever is causing the read errors. What am I missing here guys?

Thanks to everyone for your assistance,

-vbsaltydog
 
Old 05-10-2006, 12:22 AM   #20
haertig
Senior Member
 
Registered: Nov 2004
Distribution: Debian, Ubuntu, LinuxMint, Slackware, SysrescueCD, Raspbian, Arch
Posts: 2,327

Rep: Reputation: 357Reputation: 357Reputation: 357Reputation: 357
Quote:
Originally Posted by vbsaltydog
I was under the impression that dd does a raw, byte level copy and would not error on disk scratches or whatever is causing the read errors. What am I missing here guys?
Reread post #5 in this thread. dd will error out. dd_rescue tries a bit harder, but can be very slow when errors are encountered. dd_rhelp is probably your best choice for a damaged disk. No guarrantees on any of this. If your disk is so screwed, ... well, you just might be ... screwed. I hope not, but remember you're working with a pretty damaged thing here (per your reports). Your persistance may pay off, but it might not be easy or quick.

In the example dd command you posted, you said "dd ... conv=noerrors" I assume that was a minor typo. It should be "conv=noerror", not "conv=noerrors". However, even this parameter will not force dd to continue past every possible error. It can still stop.
 
Old 05-10-2006, 07:52 PM   #21
Electro
LQ Guru
 
Registered: Jan 2002
Posts: 6,042

Rep: Reputation: Disabled
Quote:
Originally Posted by vbsaltydog
Well here's the current status....ehhhh.

Failed 80Gb HD as a slave.
Good 120Gb HD as the Master (divided into a 40GB NTFS partition and a FAT16 80GB partition).

The Plan is to dd the failed drive to the available partition and recover data from recovery partition.

dd if=/dev/hdb of=/dev/hda2 conv=noerrors

Well, when I run this command I get a ton of IO errors and 0 bytes in and out messages with 0 files transferred.

I was under the impression that dd does a raw, byte level copy and would not error on disk scratches or whatever is causing the read errors. What am I missing here guys?

Thanks to everyone for your assistance,

-vbsaltydog
DO NOT use dd or a similar program to copy sector by sector of a partition. It will not work and it will only catch part of the data. Only use dd or similar utilities on raw device like /dev/hda and then output it to one BIG file. Spliting also destroyes the data. Use one big hard drive (300+ GB) to store the image of the WHOLE ENTIRE DRIVE not as a partition. Then use grep, cat, or any utility to extract the valuable data. Extracting all the data is too time consuming. Valuable files like documents, spreadsheets, presentations, database, music, video, photos, project files, email, bookmarks/favorites should be extracted from the image file. Applications and the OS should never be extracted, because both the OS and the applications can be re-installed.

PyFlag looks good to extract the data from an image file. You could also use tct, The Sleuth Kit/autopsy, foremost, and many others to help you extract more files.
 
Old 05-10-2006, 08:42 PM   #22
haertig
Senior Member
 
Registered: Nov 2004
Distribution: Debian, Ubuntu, LinuxMint, Slackware, SysrescueCD, Raspbian, Arch
Posts: 2,327

Rep: Reputation: 357Reputation: 357Reputation: 357Reputation: 357
Quote:
Originally Posted by Electro
DO NOT use dd or a similar program to copy sector by sector of a partition. It will not work and it will only catch part of the data. Only use dd or similar utilities on raw device like /dev/hda and then output it to one BIG file. Spliting also destroyes the data.
With all due respect, I think every statement made in the these sentences is incorrect. Can you site some examples or documentation that supports what you're saying? dd works on entire disks, partitions, and files. You can split the data and reconstruct it later. No problems.
Quote:
Then use grep, cat, or any utility to extract the valuable data.
From a binary image of the drive?
Quote:
Valuable files like documents, spreadsheets, presentations, database, music, video, photos, project files, email, bookmarks/favorites should be extracted from the image file.
Grepping mp3's and jpegs from a binary disk image? That would be quite the accomplishment in my book.

Again, with all due respect, you appear to be pooh-poohing standard practice, and then recommending the impossible in its place. I don't quite understand...
 
Old 05-10-2006, 10:05 PM   #23
vbsaltydog
Member
 
Registered: Nov 2005
Distribution: CentOS
Posts: 154

Original Poster
Rep: Reputation: 15
The plot thickens. As I mentioned before I can run dd on the partition to copy it to another partition on a good disk but I get a ton of IO read errors and after letting dd run for 12 hours I had to abort dd since it was tying up my workstation.
I figured I might give dd_rhelp a try so tonight I booted to Knoppix, downloaded dd_rhelp and installed it, then ran
dd_rhelp /dev/hdb /dev/hda2
but instead of getting IO read errors as with dd, I get a cant find source error and dd_rhelp stops.

I did not try to dd the bad disk to an .iso yet since there is some controversy.

Any more ideas anyone?
 
Old 05-10-2006, 11:10 PM   #24
haertig
Senior Member
 
Registered: Nov 2004
Distribution: Debian, Ubuntu, LinuxMint, Slackware, SysrescueCD, Raspbian, Arch
Posts: 2,327

Rep: Reputation: 357Reputation: 357Reputation: 357Reputation: 357
Quote:
Originally Posted by vbsaltydog
dd_rhelp /dev/hdb /dev/hda2
Like I said earlier, I have never personally used dd_rhelp ... only researched it. So I don't know the syntax of it's command line. But what you illustrated above looks fishy. You are telling it to copy an entire hardisk into a partition. I would be surprized if this is correct usage of dd_rhelp. In post #10 of this thread you said you were trying to recover a 120Gb partition. "/dev/hdb" would be the entire harddisk, not a partition. You can dd a partition and then mount that image. But if you dd an entire disk, I don't think you can mount THAT image.

Maybe explain again EXACTLY what you are trying to do. Something like "My client has a 250Gb disk with three partitions on it and I want to recover data off the second partition, which is 120Gb in size."

Quote:
I did not try to dd the bad disk to an .iso yet since there is some controversy.
This would be a pointless thing to attempt at this point. If you're getting READ errors, it doesn't matter where you're eventually planning on WRITING to (a partition or a file), since you can't read the source in the first place.
 
Old 05-11-2006, 05:06 PM   #25
Electro
LQ Guru
 
Registered: Jan 2002
Posts: 6,042

Rep: Reputation: Disabled
Quote:
Originally Posted by haertig
With all due respect, I think every statement made in the these sentences is incorrect. Can you site some examples or documentation that supports what you're saying? dd works on entire disks, partitions, and files. You can split the data and reconstruct it later. No problems.
From a binary image of the drive?
Grepping mp3's and jpegs from a binary disk image? That would be quite the accomplishment in my book.

Again, with all due respect, you appear to be pooh-poohing standard practice, and then recommending the impossible in its place. I don't quite understand...
dd is ment to be used as a sector by sector copy. Doing it from a partition will give weird results depending on the filesystem. The dd utility have to be used to do a sector by sector copy of the whole entire disk. If you do not have enough space for the destination file, tough luck on your part.

grep and cat could be used, but it is best to use a hex editor to extract the data. Using a hex editor is very, very time consuming on a very, very, very big drive, so programs need to be used to automate the process.

Data recovery is an impossible task if the destinatiion drive is not larger than the source, the user does not have enough time, and do not have another computer besides the primary computer. Recoverying data is not some small task. It can take a week to a month to recover the clients valuable data. Data recovering will use close to 100% of the computer's resource during the process, so that is why another computer should be used to do the data recovery process. The secondary computer should have parity and ECC memory. Also should be using a 100+ US dollar power supply with a true UPS device.

Quote:
Originally Posted by vbsaltydog
I did not try to dd the bad disk to an .iso yet since there is some controversy.
The reason why there is controversy is because haertig does not know the process of data recovery. haertig did post about the two tools to extract the data from a failed drive. It is safer to use the tools and direct their output to a file instead of a raw device. This means you need another hard drive that has a larger capacity than the failed drive, so in your case a 300+ GB hard drive is needed. Then run the tools that I mentioned to help you retrieve the data.

Data recovery services uses a clean room, similar requirements to make intergrated circuits, to open up the hard drive to inspect the plattors. After they inspect the plattors, they call the customer to tell him or her what options they have that might get the data back. The options could be to replace the heads, actuator, and maybe the electronics. Another option could be using a laser to read each side of the plattors. The laser is the safest and costly way but only if the plattors are not damage very, very bad. Replacing the parts is the cheapest and a little dangerous, so they use a similar program to do a sector by sector copy to make an image of drive. All options do require another drive that is equal to or greater than the capacity after data recovery is done. Data recovery services goes to the next step that is extracting the data byte by byte. They use more powerful programs that I mentioned earlier to retrieve all data. After the data is recovered, they are placed on the drive that the customer has sent them. The cost of data recovery depends how much you value your data.

The reason why I describe what a data recovery service does is because to make you understand that outputting the data to a file from a sector by sector utility is the safest way to recover data and data recovery services alway output to an image file. Outputing the data to another raw disk is not the safest because there is no way of knowing that the blocks are corrupted.

I have done data recovering but I was lucky that is was the partition table that got corrupted and the XFS fix utility recovered all the data. vbsaltydog, your problem is even worst than mine. You will have to buy two hard drives one that is a similar model, brand, and capacity and another that is larger than the failed drive. The second or the largest hard drive will be used to store the image file of the failed drive. The image file could then be copied to the replacement drive, so Windows can try to fix the filesystems. If Windows is able to fix the filesystems, you can give the replacement drive to the client. If he or she can not find certain file(s), you can use the image file to extract additional files that the Windows utilities that had fix the drive could not fix.
 
Old 05-11-2006, 05:47 PM   #26
haertig
Senior Member
 
Registered: Nov 2004
Distribution: Debian, Ubuntu, LinuxMint, Slackware, SysrescueCD, Raspbian, Arch
Posts: 2,327

Rep: Reputation: 357Reputation: 357Reputation: 357Reputation: 357
Quote:
Originally Posted by Electro
dd is ment to be used as a sector by sector copy. Doing it from a partition will give weird results depending on the filesystem. The dd utility have to be used to do a sector by sector copy of the whole entire disk.
I totally disagree, I and many others have personally dd'ed partitions and MBR's with no problems, but no need to debate the point. The OP can do his own research regarding how dd can be used to copy files, disks, partitions, MBR's, etc. and draw his own conclusions.

Quote:
The reason why there is controversy is because haertig does not know the process of data recovery.
I freely admit that I am not a data recovery expert. But at this point, we're not really talking hard-core data recovery. That MAY come in the future depending on what the OP runs into, but for now I was advising the simple things to try BEFORE going hard-core and hiring a recovery professional. I am certainly not advising on how to be your own recovery firm.

The plan I was trying to conveigh to the OP was to pull as much data off the failed drive as possible. I think dd_rhelp may be one of the better tools to accomplish this. Take that data and put it into a file (or raw partition) or whatever. I really don't care. If you say files are better than raw partitions I won't argue. I think where you're misunderstanding my intent is the next step. I am NOT recommending combing through that image looking to recover data directly. Now THAT would fall into the category of "data recovery", and it would be a nightmare for the OP.

So the OP now has a image of the corrupted drive (as best as possible). The image would not be mountable because it too would be corrupted. What I was recommending is to take that image (be it file OR partition), and running it through fsck. This MAY repair the corruption enough to make the image mountable. If the OP can then mount the image, it's easy to pull files off of it. Of course some files may be missing, because fsck sent them to La-La land when it detected them as corrupted, but hopefully there wouldn't be too many of them. If a file is found missing that is absolutely critical, then the OP will need to send the original drive off to a data recovery firm. It would no longer be a do-it-yourself project.
 
Old 05-11-2006, 08:07 PM   #27
Electro
LQ Guru
 
Registered: Jan 2002
Posts: 6,042

Rep: Reputation: Disabled
The programs that I listed will go through the image file and extract the data. Also losetup can setup a loop device to mount the image.

I disagree that dd is able to extract the partition. dd will miss a lot of data in the process. Filesystems like JFS and XFS can not be done this way anyway. The filesystem utilities have to be used instead because they place some data in the boot record not the master boot record. Since the drive fail, it is better to image the whole entire drive. I do not recommend using fsck to fix FAT12/16/32 and NTFS partitions. Windows utilities should be used first. If the Windows utilities said that the partitions have to be formated, ignore or cancel the message and then check through fsck. I suggest running fsck through a test option that shows what may be done.
 
Old 05-11-2006, 10:45 PM   #28
haertig
Senior Member
 
Registered: Nov 2004
Distribution: Debian, Ubuntu, LinuxMint, Slackware, SysrescueCD, Raspbian, Arch
Posts: 2,327

Rep: Reputation: 357Reputation: 357Reputation: 357Reputation: 357
Quote:
I do not recommend using fsck to fix FAT12/16/32 and NTFS partitions. Windows utilities should be used first.
Yea! We finally agree!

If a Windows utility will work, definitely use that. I think the OP tried this, but it failed. OP also said he didn't know if he was dealing with FAT or NTFS. I was hoping for FAT, because there is an fsck for that. I didn't know there was an fsck variant for NTFS. If we're dealing with NTFS, this really makes things tough. The Windows utilities won't work - OP already tried that. The only hope I can come up with is that Linux could mount this partially-recovered corrupted NTFS volume even though Windows can't. Or Linux could pull most of the data off the failing disk, then this recovered stuff could be written back to a new disk, and even though corrupted, the new disk might be recognizeable by Windows for a run at chkdsk. This is a long shot, but it could happen if Windows bails quickly on any disk error and dd_rhelp keeps chugging away until it somewhat succeeds. Windows might view the recovered image as being in better shape than the original disk, thanks to dd_rhelp's persistance.

Quote:
I suggest running fsck through a test option that shows what may be done.
Good idea. I agree. Twice now!
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
data recovery from crashed disk. on boot grub promt appears vinshe Linux - Newbie 7 09-11-2006 06:18 AM
How to dual boot Suse linux and XP with only recovery disks and one hard drive Jonnyk429 Linux - Newbie 1 02-09-2006 02:51 PM
Data & Photo Recovery for Linux headly21 Linux - Software 2 01-24-2006 05:24 AM
Data Recovery from Linux. duffmckagan Linux - Software 14 07-04-2005 02:36 PM
Data Recovery Software for Linux WarlockofVirgo Linux - Software 1 10-06-2004 05:23 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 07:35 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration