LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > General
User Name
Password
General This forum is for non-technical general discussion which can include both Linux and non-Linux topics. Have fun!

Notices


Reply
  Search this Thread
Old 01-21-2014, 08:20 PM   #1
metaschima
Senior Member
 
Registered: Dec 2013
Distribution: Slackware
Posts: 1,982

Rep: Reputation: 491Reputation: 491Reputation: 491Reputation: 491Reputation: 491
HDD longevity report, is it believable ?


There was a recent report on HDD longevity using a rather large number of HDDs:
http://blog.backblaze.com/2014/01/21...-should-i-buy/

My concern is that the report may not be believable.

My concerns are:
1) Seagate (12,765 drives) and Hitachi (12,956 drives) are believable and comparable, but the rest are not: Western Digital (2,838 drives), Toshiba (58 drives), Samsung (18 drives). If WD had more drives tested, close to 12,000 like the others then it would be comparable.

2) A limited number of each HDD was tested, and they varied widely in number and failure rate. Picking the most comparable:
Hitachi GST Deskstar 5K3000, 3.0TB, 4592 drives, 0.9% annual failure rate, average age = 1.7 years
Seagate Barracuda, 3.0TB, 4252 drives, 9.8% annual failure rate, average age = 1.4 years

The least comparable and possible outliers:
Seagate Barracuda 7200, 1.5TB, 539 drives, 25.4% annual failure rate, average age = 3.8 years
Seagate Barracuda Green, 1.5TB, 51 drives, 120.0% annual failure rate, average age = 0.8 years

How did they even get 120% annual failure rate ? Is their calculation of annual failure rate accurate ? They say it is " the average number of failures you can expect running one drive for a year", but how did they do the actual calculation ? Is it the number of drives that they have had fail per year ? If so, then it depends on how old the drive is and you need to observe a large number of drives for a long time for it to approach reality. I'd really like to see raw numbers on this, before I make any investments.

EDIT: Here's another study by the same site, with some hints on how they calculate annual failure rate:
http://blog.backblaze.com/2013/11/12...k-drives-last/
Quote:
Imagine you have a disk drive supplier who provides drives that are 100% reliable for six months, but then all fail at that point. Whatís the annual failure rate? If you have to keep 100 drives running at all times, youíll have to replace the drive in every slot twice a year. That means that youíll have to replace 200 drives each year, which makes your annual failure rate 200%. So, in theory at least, there is no worst possible failure rate. If every drive failed after one hour of use, the annual failure rate would be 876,000%.
Another description is here:
http://en.wikipedia.org/wiki/Annualized_failure_rate
"Note that annualized failure rate will increase towards and beyond the end of the service life of a device or component." This means that annual failure rate is totally dependent on the age of the drive, so you can only compare drives with the same age. Right ?

3) They mention that some of the drives were believed to be refurbished. This can really skew results.

Last edited by metaschima; 01-22-2014 at 02:34 PM. Reason: I mistook Toshiba for Hitachi :(
 
Old 01-22-2014, 07:53 AM   #2
salasi
Senior Member
 
Registered: Jul 2007
Location: Directly above centre of the earth, UK
Distribution: SuSE, plus some hopping
Posts: 4,070

Rep: Reputation: 896Reputation: 896Reputation: 896Reputation: 896Reputation: 896Reputation: 896Reputation: 896
Quote:
Originally Posted by metaschima View Post
There was a recent report on HDD longevity using a rather large number of HDDs:
http://blog.backblaze.com/2014/01/21...-should-i-buy/

My concern is that the report is not believable.
Could you clarify the basis on which you find this report not to be believable, please?

There is a lot of information around; unfortunately most of it is based on low numbers of disks, too low to be used for the purposes of extrapolation, and the internet then goes to work multiplying 'anecdote' into 'data'. This article (and several others) do not seem to be examples of this unfortunate waywardness with data, and so should be given at least a little time, more than 'I bought one and it was good' (probably reported two weeks after purchase).

Or have you spotted some methodological inadequacy with it?

Quote:
Originally Posted by metaschima View Post

Now, I bought the one with the highest number of highest reviews at 1TB, which was the last link I posted. Did I make a mistake ? Is Toshiba really better ? Is there a better benchmark somewhere ? Are all these reviews fake or what ?
I certainly do not believe that all of the reviews are fake (some may be, and I do worry on any reports of this kind how easy it would be for an unscrupulous manufacturer to edge their reviews a bit higher or the competition's a bit lower by planting one or two doctored reports), but if you take into account that you would expect 80% of disk drives to be surviving after five years, you expect 80% of reviewers to have no useful information on reliability at the five year mark (assuming one drive purchase...for RAID buyers they get a little earlier information, but they probably aren't the majority of purchasers)...and, at the one year mark that would scale appropriately, but not linearly.

Note also that the majority of users of large numbers of disk drives appear to feel constrained not to publish brand names, and that is one area in which backblaze's blogs stand out - they buy their disks under the same sort of arrangement that you or I would (and, presumably, Google and Amazon don't) and don't feel constrained to protect the reputations of the guilty parties.

Also note that it seems that every disk drive manufacturer has the occasional bad patch when they produce disk drives that are less reliable than usual, just for the more reliable ones this is less frequent than for the less reliable ones. So, even if you see someone who bought ten drives for his RAID array and three failed, it might just be a (very) bad batch, or it might be something more long term. It is difficult to tell.
 
Old 01-22-2014, 10:33 AM   #3
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 9,078
Blog Entries: 4

Rep: Reputation: 3173Reputation: 3173Reputation: 3173Reputation: 3173Reputation: 3173Reputation: 3173Reputation: 3173Reputation: 3173Reputation: 3173Reputation: 3173Reputation: 3173
Dunno ... there are lots of reports about which light-bulbs last longer, but in the end, what really matters most is that you are in the dark.
 
Old 01-22-2014, 11:35 AM   #4
cascade9
Senior Member
 
Registered: Mar 2011
Location: Brisneyland
Distribution: Debian, aptosid
Posts: 3,753

Rep: Reputation: 934Reputation: 934Reputation: 934Reputation: 934Reputation: 934Reputation: 934Reputation: 934Reputation: 934
Quote:
Originally Posted by metaschima View Post
There was a recent report on HDD longevity using a rather large number of HDDs:
http://blog.backblaze.com/2014/01/21...-should-i-buy/

My concern is that the report is not believable. I've checked reviews around the net before I bought my recent HDD and Seagate and WD have good reviews, while Toshiba has bad reviews (with many DOA). Any site will do, but these are some examples from newegg, I've picked ones of similar price range and also included the WD and Seagate drives with the lowest reviews just for comparison with Toshiba:
Quote:
We donít have enough Toshiba or Samsung drives for good statistical results.
http://blog.backblaze.com/2014/01/21...-should-i-buy/

Newegg 'reviews' are kinda pointless IMO (salasi is prefectly corrent with the "multiplying 'anecdote' into 'data'" comment).

As far as the backblaze article goes, note that all the HDDs used are standard 3 year or less warranty models.

If you're really woried about long term relability, you'd be better off spending a few more $$$ and getting 'enterprise' drives, or at least the semi-enterprise drives (e.g. WD 'black'HDD get a 5 year warranty).

Just FYI, seagate is (or at least was) reducing the warranty length on HDDs. Most 'desktop' drives only have 2 years now, and even the 7200 RPM 'enterprise' drives only get 3 years now.

While a warranty doesnt mean the drive wont fail, it does show the confidence the manufacturer has in the drive.

It fits with what I've been hearing, that seagate quality has dropped over the years, and is probably lower than WD/hitachi/samsung now. But that is still just 'anecdote' *nods at salasi*.

While its totally non-scientific, I'm in the 'if its worked for you in the past, stick with it' camp. If seagate has done it for you in the past, and/or WD/hitachi/samsung have failed, maybe you just have good luck with seagate. I've seen it happen before, many times.
 
Old 01-22-2014, 12:21 PM   #5
metaschima
Senior Member
 
Registered: Dec 2013
Distribution: Slackware
Posts: 1,982

Original Poster
Rep: Reputation: 491Reputation: 491Reputation: 491Reputation: 491Reputation: 491
Quote:
Originally Posted by salasi View Post
Or have you spotted some methodological inadequacy with it?
One thing that bothers me is that the number of disks of each brand is different. This will surely affect the results. So far I only consider Seagate (12,765 drives) vs Hitachi (12,956 drives) as being possibly accurate. Take a look at the number of Toshiba drives tested = 58 drives. This is very bad design. What if they got 58 good drives. It's much harder to get 12,000 good drives.

I accidentally mistook Toshiba as the one with 12,000 drives instead of Hitachi ... my bad. I will check Hitachi reviews.

EDIT:
Looking at reviews an Amazon, they seem to have similar reviews to Seagate and WD, so the study might be true after all.

Last edited by metaschima; 01-22-2014 at 12:30 PM.
 
Old 01-22-2014, 01:07 PM   #6
salasi
Senior Member
 
Registered: Jul 2007
Location: Directly above centre of the earth, UK
Distribution: SuSE, plus some hopping
Posts: 4,070

Rep: Reputation: 896Reputation: 896Reputation: 896Reputation: 896Reputation: 896Reputation: 896Reputation: 896
Quote:
Originally Posted by metaschima View Post
One thing that bothers me is that the number of disks of each brand is different.
Well, they can only report on the disks that they have. I suspect that if anyone wants to buy them 10,000 Toshiba drives that they would be happy to include them, but calculate the price before you make them an offer...

And, in comparison, you'd prefer reports from people who only have 1 drive? I only say this because you don't have the choice of many sources of data that are better, so if you reject this source of data, you probably don't have any that are as good (by, more or less, a factor of 58).

Quote:
If you're really woried about long term relability, you'd be better off spending a few more $$$ and getting 'enterprise' drives, or at least the semi-enterprise drives (e.g. WD 'black'HDD get a 5 year warranty).
I'm sorry, but here we go with more articles (largely based on the same data sets, so its chewing over the same data, again and again) than you really want (the first is the one directly relevant to that point):

http://www.zdnet.com/enterprise-disk...ey-7000023904/
https://www.usenix.org/legacy/events...schroeder.html
http://hardware.slashdot.org/story/0...sk-Reliability
http://storagemojo.com/2007/02/19/go...re-experience/
http://www.zdnet.com/how-long-do-dis...st-7000023075/
http://www.zdnet.com/who-makes-the-b...es-7000025375/

As to
Quote:
Newegg 'reviews' are kinda pointless IMO
Well, if there are thousands of answers on one drive and the people haven't just answered about their new drive, then it may well be better than guessing or something, but you can't really know much about the context. Firstly, how many of those reviews are 'unreliable' because they fitted them into a case with a bad power supply or inadequate airflow? Are the workloads comparable (if that makes a differences)?

PS Sorry if this is a highjack! It seems on topic to me, just about, but there has been a little drift from the original post.
 
Old 01-22-2014, 02:22 PM   #7
metaschima
Senior Member
 
Registered: Dec 2013
Distribution: Slackware
Posts: 1,982

Original Poster
Rep: Reputation: 491Reputation: 491Reputation: 491Reputation: 491Reputation: 491
Ok, so I have reworded the question because I mistook Hitachi for Toshiba. Still, I have problems with the study.

Overall, I do find it believable that Hitachi is better than Seagate, and I may buy it next time, or as soon as my Seagate drive fails.

Last edited by metaschima; 01-22-2014 at 02:56 PM.
 
Old 01-22-2014, 03:41 PM   #8
salasi
Senior Member
 
Registered: Jul 2007
Location: Directly above centre of the earth, UK
Distribution: SuSE, plus some hopping
Posts: 4,070

Rep: Reputation: 896Reputation: 896Reputation: 896Reputation: 896Reputation: 896Reputation: 896Reputation: 896
I think you've left it a little late. To quote from the press release:

Quote:
Tokyo, March 9, 2012 --- Hitachi, Ltd. (NYSE:HIT / TSE:6501, “Hitachi”) announced
that it has completed its transfer of Hitachi’s Hard Disk Drive (HDD) business to
Western Digital Corporation (NYSE: WDC, “WD”), effective March 8, 2012 (U.S.time).
which I'd describe as a shame, because I've had good use out of Hitachi/IBM drives, but that's life for you.
 
Old 01-22-2014, 04:55 PM   #9
metaschima
Senior Member
 
Registered: Dec 2013
Distribution: Slackware
Posts: 1,982

Original Poster
Rep: Reputation: 491Reputation: 491Reputation: 491Reputation: 491Reputation: 491
Quote:
Originally Posted by salasi View Post
I think you've left it a little late. To quote from the press release:
which I'd describe as a shame, because I've had good use out of Hitachi/IBM drives, but that's life for you.
That is unfortunate. I'm not sure what this means for the study. Maybe some of the Hitachi drives were actually re-branded WD drives ? I guess I should wait for their next study and then decide what to buy next time. Technically Samsung's HDD branch has also moved to Seagate.

Last edited by metaschima; 01-22-2014 at 04:56 PM.
 
Old 01-23-2014, 11:14 AM   #10
cascade9
Senior Member
 
Registered: Mar 2011
Location: Brisneyland
Distribution: Debian, aptosid
Posts: 3,753

Rep: Reputation: 934Reputation: 934Reputation: 934Reputation: 934Reputation: 934Reputation: 934Reputation: 934Reputation: 934
Quote:
Originally Posted by salasi View Post
I'm sorry, but here we go with more articles (largely based on the same data sets, so its chewing over the same data, again and again) than you really want (the first is the one directly relevant to that point):

http://www.zdnet.com/enterprise-disk...ey-7000023904/
https://www.usenix.org/legacy/events...schroeder.html
http://hardware.slashdot.org/story/0...sk-Reliability
http://storagemojo.com/2007/02/19/go...re-experience/
http://www.zdnet.com/how-long-do-dis...st-7000023075/
http://www.zdnet.com/who-makes-the-b...es-7000025375/
Nice, and thanks for the linkage, though I have my doubts about the data.

IMO its not just 'number of HDDs devided by failure rate over a period = some number you can use to guage relability'. The number of power cycles, heat and load can make a difference.

But I have to admit that I have no 'non-anecdotal' data to support my position.

Quote:
Originally Posted by salasi View Post
Well, if there are thousands of answers on one drive and the people haven't just answered about their new drive, then it may well be better than guessing or something, but you can't really know much about the context. Firstly, how many of those reviews are 'unreliable' because they fitted them into a case with a bad power supply or inadequate airflow? Are the workloads comparable (if that makes a differences)?

PS Sorry if this is a highjack! It seems on topic to me, just about, but there has been a little drift from the original post.
Yeah, I can see youur point(s), and your right. Though I do find it amusing that one of teh links you provided above is to the Google 'Failure Trends in a Large Disk Drive Population' where they claim that (and I quote) "higher temperatures are not associated with higher failure rates"

http://static.googleusercontent.com/...k_failures.pdf

There is another factor to consider- bad handling. If some clutz has dropped the HDDs at any point during shipping, that can make a huge difference. I know of one organisation that had 100% failure rate on 10 abit motherboards, and a person who went though 3 of the same boards in a row from the same store at roughly the same time, all failed.....the store insisted that it was 'because abit makes crap'....I but know of at 5 other boards of the same exact model sourced from different places that ran for years.

Quote:
Originally Posted by metaschima View Post
That is unfortunate. I'm not sure what this means for the study. Maybe some of the Hitachi drives were actually re-branded WD drives ? I guess I should wait for their next study and then decide what to buy next time. Technically Samsung's HDD branch has also moved to Seagate.
I really dont know about Hitachi drives now being 'rebranded' WDs, and I dont have the connections to do more than guess.

If it works out the way that things did when Maxtor got taken over by Seagate, I doubt it. AFAIK, Maxtor kept making drives the same way they had for a while, then Seagate used Maxtor as a 'budget' brand, then dropped the Maxtor branding.

Seagate bought them out in part to aquire some tech, but a lot of it was to do with market share.

Its possible that Seagate made some technical decisions to reduce costs while keeping the external appearance the same (e.g., using the same stepping motors for both its Maxtor and Seagate drives).

If WD keep Hitachi for long enough and dont just drop the name after a while, they might well do the same thing. Or it might have just been another tech and market share acquisition, and Hitachi will disappear. I really dont know....Hitachi actually has a good name, depsite the connection to the old 'deathstar' name. IMO Maxtor never had the rep that Hitachi has.

I still think that an 'enterprise' drive with a 5 year warranty should outlast a 'consumer' grade drive with a 2 years or less warranty. I'm no big fan of Seagate, and I tend to buy WD HDDS, but I'd trust a Seagate Cheetah over a WD 'Blue'.

Last edited by cascade9; 01-23-2014 at 11:15 AM.
 
Old 01-23-2014, 12:55 PM   #11
metaschima
Senior Member
 
Registered: Dec 2013
Distribution: Slackware
Posts: 1,982

Original Poster
Rep: Reputation: 491Reputation: 491Reputation: 491Reputation: 491Reputation: 491
Here's a report from the same site on enterprise vs consumer HDDs:
http://blog.backblaze.com/2013/12/04...e-reliability/

I'm glad I didn't buy an enterprise drive, but I wouldn't have paid twice the price for a promise anyway. I figured even if the consumer one lasts only 2 years, I could just buy another and it would last 2 years and it would cover at least 4 years for the same price. But I know they last longer than 2 years.
 
Old 01-23-2014, 07:38 PM   #12
salasi
Senior Member
 
Registered: Jul 2007
Location: Directly above centre of the earth, UK
Distribution: SuSE, plus some hopping
Posts: 4,070

Rep: Reputation: 896Reputation: 896Reputation: 896Reputation: 896Reputation: 896Reputation: 896Reputation: 896
Quote:
Originally Posted by cascade9 View Post
Nice, and thanks for the linkage, though I have my doubts about the data.

IMO its not just 'number of HDDs devided by failure rate over a period = some number you can use to guage relability'. The number of power cycles, heat and load can make a difference.

But I have to admit that I have no 'non-anecdotal' data to support my position.



Yeah, I can see youur point(s), and your right. Though I do find it amusing that one of teh links you provided above is to the Google 'Failure Trends in a Large Disk Drive Population' where they claim that (and I quote) "higher temperatures are not associated with higher failure rates"

http://static.googleusercontent.com/...k_failures.pdf
You are right, in general, that power cycles, heat and load can make a difference. That's one of the reasons that work like google's can be more valuable. Their work will concern drives which are all in a data centre, with vaguely similar work loads and power provision, unlike just throwing the drives out to a mass of end users, where every one will have different environments, and who knows which of those conditions are the relevant ones, and which aren't?

"higher temperatures are not associated with higher failure rates"

That's the most interesting one. I think if anyone had answered this from 'gut feel' they would have said that higher temperature is the one thing that they 'knew' would be associated with higher failure rates (which tells you something about the usefulness of 'gut feel' in this connection).

Some things to note:
  1. the highest failure rates are associated with low temperatures; now, while this is a surprise, it certainly isn't entirely infeasible. In very particular, if the failure is associated in some way with lubrication of mechanical moving parts, it can be that lubrication is in some way compromised as temperature goes down and viscosity of lubricants increases.
  2. IIRC, there was a small upkick in failure rate at higher temperatures - not to the highest levels, but a rise all the same. It could well be that there are two significant failure mechanisms, one that gets worse at lower temperatures and one that gets worse at higher temperatures and that the result is that there is a 'bathtub' type of characteristic, where over the mid range of temperatures, the fall of one more-or-less compensates for the rise of the other.
  3. (and it does somewhat call into question the advice I'm sure we've all given or heard over the years "that's a hot computer...you need to keep the hard disk cool, or it will fail sooner")
  4. I've seen the theory, somewhere, that the lubrication for the head mechanism dries up, but the mechanism can still keep working unless you stop working the head for a while, and then it won't start again. Now I am sure that this theory isn't adequately backed up by extensive failure analyses, but it certainly does align with experience of those 90s hard disks that always have difficulty re-starting after an extended vacation.

Quote:
If it works out the way that things did when Maxtor got taken over by Seagate, I doubt it. AFAIK, Maxtor kept making drives the same way they had for a while, then Seagate used Maxtor as a 'budget' brand, then dropped the Maxtor branding.

Seagate bought them out in part to aquire some tech, but a lot of it was to do with market share.
Well, Maxtor never had the best rep for reliability, so the name itself did nothing to add to the (perceived) value of a Hard Disk Drive. OTOH, Hitachi have a good rep, so there is more reason to keep the name going. Now, of course, this has little to do with the actual drives or the processes and procedures used to manufacture them, so whether this correlates with anything real, or is just the badge applied at the end of the production line, I don't know.

My suspicion is that one factor is that, with the coming of SSDs, Hitachi came to the conclusion that the business of turning out good but unprofitable hard disks wasn't going to turn around anytime soon, and wasn't worth the heartache (Thai floods and the need to invest/gamble on newer generations).

Quote:
I'm glad I didn't buy an enterprise drive, but I wouldn't have paid twice the price for a promise anyway. I figured even if the consumer one lasts only 2 years, I could just buy another and it would last 2 years and it would cover at least 4 years for the same price. But I know they last longer than 2 years.
Well, I bought a second hard disk to supplement my home server, recently. I went for an enterprise Hitachi Ultrastar, because I could get an obsolescent one cheaply (and who knows how that will turn out, particularly given the limited support I'm likely to get from the drive manufacturer if something does go wrong). It supplements a Maxtor, and I haven't had a moment's problem with that, over the years. F***ing anecdote!
 
Old 01-29-2014, 10:22 PM   #13
metaschima
Senior Member
 
Registered: Dec 2013
Distribution: Slackware
Posts: 1,982

Original Poster
Rep: Reputation: 491Reputation: 491Reputation: 491Reputation: 491Reputation: 491
Well, it looks like the study is flawed after all:
http://hardware.slashdot.org/story/1...y-study-flawed
http://www.enterprisestorageforum.co...esearch-1.html

Until there is a standardized, properly done report, I'm sticking with Seagate. I've been using these for as long as I can remember and they are reliable according to my experience. A badly done report cannot trump that.

Last edited by metaschima; 01-29-2014 at 10:29 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Kernel Longevity - Making the same mistakes again? GazL Slackware 20 02-06-2012 02:11 PM
Un Flipping Believable : ZynAddSubFX as DSSI Plug-in for Rosegarden tpreitzel Slackware 6 01-22-2011 01:38 PM
Linux / BSD laptop longevity Kernel_Sanders General 1 02-09-2006 02:33 PM
Longevity of CompactFlash-based storage? thegnu Linux - Hardware 1 02-03-2005 12:11 PM
iptables longevity? WeNdeL Linux - General 2 03-04-2003 03:18 PM

LinuxQuestions.org > Forums > Non-*NIX Forums > General

All times are GMT -5. The time now is 04:53 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration