LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 07-12-2016, 10:36 AM   #16
JaredDM
LQ Newbie
 
Registered: Jul 2016
Location: Providence, RI
Distribution: Mostly Windows to be honest.
Posts: 9

Rep: Reputation: Disabled

Just because you've gotten lucky since 1980, doesn't mean it always works out that way. You've heard from two data recovery professionals now who both see this on a regular basis when it doesn't work out so well. But, you're free to believe what you want. I'm sure there's people who've walked through a minefield a dozen times and were never harmed too, but not everyone is so lucky.
 
Old 07-12-2016, 10:57 AM   #17
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,912

Rep: Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513
No - most data recovery people only see the worst case. Not the normal case. That biases their view such that they assume that ANY file system errors are catastrophic.

I have seen catastrophic failures... broken spindles, bearings, head crashes... A system crash isn't one.

Now deliberate damage is something else (and data recovery people do see these as well). But that is also not what is being discussed.

As long as the history of the disk is reasonable - and free from physical handling damage (dropped, submerged, fire, ...) a UNIX/Linux filesystem is VERY recoverable.

Last edited by jpollard; 07-12-2016 at 11:00 AM.
 
Old 07-12-2016, 11:01 AM   #18
JaredDM
LQ Newbie
 
Registered: Jul 2016
Location: Providence, RI
Distribution: Mostly Windows to be honest.
Posts: 9

Rep: Reputation: Disabled
I'm not saying that every file system error is catastraphic, most may not be. I'm just saying that it's stupid to run any file system repair utility when you don't have a backup. In fact, not having a backup is stupid in and of itself. All I'm suggesting is that before you run that, it's a really good idea to at least make a ddrescue image so you have a backup. I've got hundreds of past customers who probably all wish someone had given them this advice before they listened to someone like you.
 
Old 07-12-2016, 11:15 AM   #19
hydrurga
LQ Guru
 
Registered: Nov 2008
Location: Pictland
Distribution: Linux Mint 21 MATE
Posts: 8,048
Blog Entries: 5

Rep: Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925Reputation: 2925
Interesting debate, guys. Personally, after a system crash I would, if able, first back up the drive (on a "most important data first" basis in case the crash is indicative of imminent woe), then look at the SMART data. If the SMART data looked reasonable then I would run the filesystems through a fsck, taking a note of the extent of the repairs. If those repairs were reasonably minimal, I would carry on with the repaired system but be watchful for any shenanigans. If the repairs were any more than minimal, I would revert to a previous backup and update any files from my post-crash backup if necessary.

In general, if I feel the system is on shaky ground in any way, it gets reverted. I'd rather nip any problems in the bud than face further problems down the line.

However, I am more than welcome to have this approach torn apart by the good LQ burghers.
 
Old 07-12-2016, 11:30 AM   #20
LukeRFI
Member
 
Registered: Jun 2016
Location: Canada
Distribution: Various versions of Fedora
Posts: 30

Rep: Reputation: Disabled
Quote:
Originally Posted by hydrurga View Post
Interesting debate, guys. Personally, after a system crash I would, if able, first back up the drive (on a "most important data first" basis in case the crash is indicative of imminent woe), then look at the SMART data. If the SMART data looked reasonable then I would run the filesystems through a fsck, taking a note of the extent of the repairs. If those repairs were reasonably minimal, I would carry on with the repaired system but be watchful for any shenanigans. If the repairs were any more than minimal, I would revert to a previous backup and update any files from my post-crash backup if necessary.

In general, if I feel the system is on shaky ground in any way, it gets reverted. I'd rather nip any problems in the bud than face further problems down the line.

However, I am more than welcome to have this approach torn apart by the good LQ burghers.
You are on the right track. The safest practices followed by any data recovery professional are as follows:

1. Resolve physical issues (PCB, heads, firmware, etc) to get drive functional
2. Image the hard drive using data recovery hardware imager. If possible, select used sectors by bitmap and image those sectors first going from lowest to highest (forward) or highest to lowest (backward) LBA, skipping any damaged areas on the first pass and gradually digging down to read around the bad sectors, playing with the drive read timeouts, putting it into PIO mode, adjust software, hardware and physical resets of the drive.
3. Recover the files from the image on the healthy hard drive
4. Verify that the recovery results are good
 
1 members found this post helpful.
Old 07-12-2016, 11:57 AM   #21
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,912

Rep: Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513
Quote:
Originally Posted by JaredDM View Post
I'm not saying that every file system error is catastraphic, most may not be. I'm just saying that it's stupid to run any file system repair utility when you don't have a backup. In fact, not having a backup is stupid in and of itself. All I'm suggesting is that before you run that, it's a really good idea to at least make a ddrescue image so you have a backup. I've got hundreds of past customers who probably all wish someone had given them this advice before they listened to someone like you.
Most people cannot afford the cost of a data recovery facility either, as the last data recovery effort I had contact with would cost over $10,000 for a single disk.

Backups are a separate issue. Yes, backups are good and should be made.

But it is also not necessarily good to use ddrescue... It can on its own scrape the disk (rendering it useless even for data recovery experts) and STILL not get any data.

If you have physical damage and the data is more valuable than the cost of a recovery... send it out. Do nothing to the disk.

If you have a system crash? reboot. On linux the normal activity is to recover the disk automatically (most systems used a "fsck -y" to do that). Current journaling filesystems will do the recovery itself just by replaying the journal.

fsck on its own is not damaging - it lets you CHOOSE what to do. And for most filesystems, that is sufficient (it can be a pain if you have a 60TB filesystem with 50-70 million files).

I do recommend using fsck in a read-only mode first to find out WHAT appears wrong, and find out what the error messages mean.

For the cheapo recovery ... (the under $1000 range) there was likely nothing wrong with the disk in the first place. Disks Windows won't read... Linux had no problem with. Data recovery was easy: mount read only, copy the data (use ddrescue if you want). Read errors identified the files that were actually damaged. (This happens most often on laptop disks.)

Last edited by jpollard; 07-12-2016 at 12:06 PM.
 
1 members found this post helpful.
Old 07-12-2016, 12:08 PM   #22
JaredDM
LQ Newbie
 
Registered: Jul 2016
Location: Providence, RI
Distribution: Mostly Windows to be honest.
Posts: 9

Rep: Reputation: Disabled
You are quoting prices from a different era my friend. The average cost of recovery at most labs these days is only around $500-700. $10K was back in the days when only a few companies existed and they could charge whatever they wanted.

Pro recovery using hardware imaging tools is always best, but if you're not willing to pay for it ddrescue is the next best thing as it'll only access each sector once and has algorithms to skip around areas of bad sectors rescuing as much data as possible before going back and retrying the bad areas at a smaller block read size. If ddrescue can read the disk beginning to end just fine, you've accomplished two things.

#1. You've confirmed that a simply logical issue
#2. You've made a backup in case something goes wrong with the fsck repair

If ddrescue is hitting a lot of errors, now you know there is media damage and can make an educated decision whether to opt for pro recovery, or let ddrescue continue and hope for the best. Running a file system repair against a drive which may have bad sectors in the file tables is a blind, foolish thing to do and often leads people to permanent data loss, or at the very least makes it impossible to get back files with any sort of names & folder hierarchy.
 
1 members found this post helpful.
Old 07-12-2016, 12:24 PM   #23
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,912

Rep: Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513
Quote:
Originally Posted by JaredDM View Post
You are quoting prices from a different era my friend. The average cost of recovery at most labs these days is only around $500-700. $10K was back in the days when only a few companies existed and they could charge whatever they wanted.
It still costs a lot when a disk has to be dissassembled to get the platters, then rebuild a replacement disk to read the platters in a safe way. Anything less than $1000 implies the only failure was the formatter... or there was nothing wrong in the first place. Any physical damage and you can't trust the internal read heads anyway.
Quote:

Pro recovery using hardware imaging tools is always best, but if you're not willing to pay for it ddrescue is the next best thing as it'll only access each sector once and has algorithms to skip around areas of bad sectors rescuing as much data as possible before going back and retrying the bad areas at a smaller block read size.
Depends on the "hardware imaging tools". Many just use ddrescue or its equivalent... assuming the internal heads are still valid. If they aren't - you are destroying the data.
Quote:
If ddrescue can read the disk beginning to end just fine, you've accomplished two things.

#1. You've confirmed that a simply logical issue
#2. You've made a backup in case something goes wrong with the fsck repair
Assuming you have duplicate storage space handy. The problem with ddrescue is that it does a block by block copy - and you have to have enough storage (and possibly metadata storage too) to hold the entire filesystem. And no - using a normal backup disk doesn't work unless you have more than 2 (then you can sacrifice one of them and still have at least two backups). Businesses might have a big enough budget for that... but most individuals don't.
Quote:

If ddrescue is hitting a lot of errors, now you know there is media damage and can make an educated decision whether to opt for pro recovery, or let ddrescue continue and hope for the best. Running a file system repair against a drive which may have bad sectors in the file tables is a blind, foolish thing to do and often leads people to permanent data loss, or at the very least makes it impossible to get back files with any sort of names & folder hierarchy.
And the same thing can be said of an fsck run. You don't seem to realize that fsck doesn't write to the disk until after it has all the information. If fsck -y detects other errors, it doesn't do anything.
 
Old 07-12-2016, 12:34 PM   #24
JaredDM
LQ Newbie
 
Registered: Jul 2016
Location: Providence, RI
Distribution: Mostly Windows to be honest.
Posts: 9

Rep: Reputation: Disabled
I said "PRO" recovery using hardware imaging tools. No actual professional data recovery company is just using ddrescue. Computer repair shops who tinker in DR might, but not pro recovery places. We all use tools like PC-3000 and DeepSpar Disk imager.

Also I have no idea what this "formatter" you're talking about is. Must be an obsolete term from before my time. However the prices I told you are accurate. I do about half dozen read/write head replacements and about a dozen other projects relating to corrupted firmware, bad sectors, etc. every week. Granted there are still some companies like Kroll and DriveSavers who are charging those extremely high rates, but it's not the norm these days. In all the recoveries we've done here, I've rarely ever seen a single drive recovery go over much more than $1200-$1300. And those were the really tough cases we didn't even want to deal with. Our typical price for most drives needing new read/write heads is $650 + the cost of parts. Only certain models & circumstances which we know are problematic will run up the cost higher than that.
 
Old 07-12-2016, 12:51 PM   #25
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,912

Rep: Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513
Quote:
Originally Posted by JaredDM View Post
I said "PRO" recovery using hardware imaging tools. No actual professional data recovery company is just using ddrescue. Computer repair shops who tinker in DR might, but not pro recovery places. We all use tools like PC-3000 and DeepSpar Disk imager.
As I also said, or equivalent.

Those tools assume the disk heads and media are not physically damaged.
Quote:

Also I have no idea what this "formatter" you're talking about is. Must be an obsolete term from before my time. However the prices I told you are accurate. I do about half dozen read/write head replacements and about a dozen other projects relating to corrupted firmware, bad sectors, etc. every week. Granted there are still some companies like Kroll and DriveSavers who are charging those extremely high rates, but it's not the norm these days. In all the recoveries we've done here, I've rarely ever seen a single drive recovery go over much more than $1200-$1300. And those were the really tough cases we didn't even want to deal with. Our typical price for most drives needing new read/write heads is $650 + the cost of parts. Only certain models & circumstances which we know are problematic will run up the cost higher than that.
I'm surprised you don't know what a formatter card is then. It happens to be that little circuit card on the back of every disk made. The one handling head positioning, CRC checks, handles low level formatting...

Replacing heads is worthless if the media is actually damaged. You just end up damaging more heads. The only way to check for that is disassembly of the unit to inspect each under a microscope. And both require a clean-room environment to do. I'm not even sure that can be done with some of the newer disks - they are requiring a helium environment for operation.
 
Old 07-12-2016, 01:58 PM   #26
JaredDM
LQ Newbie
 
Registered: Jul 2016
Location: Providence, RI
Distribution: Mostly Windows to be honest.
Posts: 9

Rep: Reputation: Disabled
What you're talking about is called a PCB (printed circuit board) by pretty much everyone else on the planet. I've never heard it referred to as a "formatter" so it's either an obsolete term or just something you made up, I don't know which.

And yes, all read write head replacements should be done in a clean room, that's what we do here anyway. To answer the other question you raise, yes all newer disks can have the read/write heads replaced including the HGST helium filled ones. However given that these are only enterprise drives, it's a rare occasion that one would even need recovery. Most people buying enterprise grade drives understand the importance of a good backup. Experiments have shown that these drives will still operate (though possibly not at full potential) even after the helium has been released, and can operate long enough that way to still image all of the data without issue. However most of us have methods which we've thought up which can re-fill the drive with helium just to be safe.
 
Old 07-12-2016, 02:43 PM   #27
LukeRFI
Member
 
Registered: Jun 2016
Location: Canada
Distribution: Various versions of Fedora
Posts: 30

Rep: Reputation: Disabled
Quote:
Originally Posted by jpollard View Post
It still costs a lot when a disk has to be dissassembled to get the platters, then rebuild a replacement disk to read the platters in a safe way. Anything less than $1000 implies the only failure was the formatter... or there was nothing wrong in the first place. Any physical damage and you can't trust the internal read heads anyway.
My lab charges $700+parts (in Canadian dollars) for head changes making it very rare that our cost ever exceeds $1000USD, which is what I think you are suggesting. My experience since I got into the game in the mid 90's is that people have a single experience with a single data recovery lab and then use that as the expected experience every time with every data recovery lab. Yes, there are still a few labs out there who are more than happy to pry every penny they can from your tightly closed fists, but there are plenty of very reputable labs who charge a lot less and, in many cases, have a better success rate than the larger more expensive labs.
 
1 members found this post helpful.
Old 07-12-2016, 05:44 PM   #28
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,912

Rep: Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513
Quote:
Originally Posted by JaredDM View Post
What you're talking about is called a PCB (printed circuit board) by pretty much everyone else on the planet. I've never heard it referred to as a "formatter" so it's either an obsolete term or just something you made up, I don't know which.
Nope. A PCB is just a printed circuit board. A formatter has a specific function for a disk.
Quote:

And yes, all read write head replacements should be done in a clean room, that's what we do here anyway.
Not "should" but "must". The spacing between the head and the surface is so small (measured in nanometers for the high capacity ones) that any speck of dust will cause a head crash, causing physical media damage and head damage.
Quote:
To answer the other question you raise, yes all newer disks can have the read/write heads replaced including the HGST helium filled ones. However given that these are only enterprise drives, it's a rare occasion that one would even need recovery.
There is no manufacturing difference between an "enterprise" drive and a "consumer" drive. Only the warranty. And any disk that fails MAY require recovery. That is up to the business or individual to decide.

What is a waste is sending a disk out for recovery that doesn't need it.
Quote:
Most people buying enterprise grade drives understand the importance of a good backup.
Some of the worst backup policies are used by those same people. It is only through experience with failures that will teach some people the value of recovery services. But finding a good one is rather difficult.
Quote:
Experiments have shown that these drives will still operate (though possibly not at full potential) even after the helium has been released, and can operate long enough that way to still image all of the data without issue.
MAYBE - but it is a very failure prone approach. Flying heads designed for helium will NOT fly properly without it. Using heads NOT designed for it risks losing data.
Quote:
However most of us have methods which we've thought up which can re-fill the drive with helium just to be safe.
And might work if the media hasn't been damaged, which you really won't know without disassembly of the platters and inspecting them.

Good competent data recovery services are expensive. And most charge more the larger the drive.

All I've been trying to get across is that fsck is not something to be feared. It is just a tool that can repair logical errors with a minimum of expense.

The problem has been that Microsoft has been unable to implement decent filesystems in its entire history. This, combined with their desire to keep people ignorant of their devices, is what has made several businesses profitable from that ignorance.

Last edited by jpollard; 07-12-2016 at 05:52 PM.
 
Old 07-12-2016, 05:51 PM   #29
JaredDM
LQ Newbie
 
Registered: Jul 2016
Location: Providence, RI
Distribution: Mostly Windows to be honest.
Posts: 9

Rep: Reputation: Disabled
I'm done with this conversation. Now you are just speaking total nonsense and almost everything you said above is just made up by you. What would I know anyway, it's not like I'm a data recovery professional.....oh wait! That's exactly what I am, but I guess some old timer who paid for data recover once back in the 80's must know more than I do.
 
Old 07-12-2016, 06:02 PM   #30
jpollard
Senior Member
 
Registered: Dec 2012
Location: Washington DC area
Distribution: Fedora, CentOS, Slackware
Posts: 4,912

Rep: Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513Reputation: 1513
Quote:
Originally Posted by JaredDM View Post
I'm done with this conversation. Now you are just speaking total nonsense and almost everything you said above is just made up by you. What would I know anyway, it's not like I'm a data recovery professional.....oh wait! That's exactly what I am, but I guess some old timer who paid for data recover once back in the 80's must know more than I do.
Evidently that is true.

And I have been recovering data since about the mid 80s anyway. Even from disks that don't spin up. Still do - though now the disks are mine instead.

Last edited by jpollard; 07-12-2016 at 06:06 PM.
 
  


Reply

Tags
errors, filesystems, fsck



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to fix an accidentally-fsck'ed RAID5 member kevinfishburne Linux - Server 1 01-12-2016 11:08 AM
[SOLVED] Boot fails at fsck but I get no error when I run fsck myself. new install on ssd. rlx Slackware 1 12-02-2012 02:29 PM
Will fsck fix this problem and will it break anything? DJOtaku Linux - Hardware 11 12-22-2011 05:54 PM
How to fix fsck? anon004 Linux - Hardware 4 01-17-2009 11:34 AM
fsck - doesn't fix problem? Ishkabibble Linux - Hardware 2 04-07-2008 09:33 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 07:52 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration