LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices

Reply
 
Search this Thread
Old 01-26-2014, 12:26 PM   #1
Ulysses_
Member
 
Registered: Jul 2009
Posts: 645

Rep: Reputation: 42
If OSI layers correct errors, how can an http download still fail md5sum?


Just downloaded a 3 GB iso image using http and tested it with md5sum. It failed. Downloaded the file again and the md5 sum tests ok now

Errors are supposed to be handled by some of the layers of the OSI model and not just the physical layer, in order to provide a reliable service over an unreliable physical medium.

Why might such a thorough scheme have failed to correct the error in the download above?

Last edited by Ulysses_; 01-26-2014 at 03:00 PM.
 
Old 01-26-2014, 04:09 PM   #2
Ser Olmy
Senior Member
 
Registered: Jan 2012
Distribution: Slackware
Posts: 1,996

Rep: Reputation: Disabled
The error most likely occurred at one of the endpoints.

As you say, both TCP data and IP/TCP headers are protected by checksums. While these are not foolproof, the chance of a datagram with a random error generating the same checksum as the uncorrupted version of that same datagram is very slim indeed.

However, bit errors occurring before the web server served the data, or after the data was received by the client, could still go unnoticed. This would include:
  • errors generated by the hard drive(s) on the server (not terribly likely as the data are checksummed, but it has been known to happen)
  • errors on the SATA bus or SATA chipset (the latter is not all that uncommon, while the former is usually caught by parity checking)
  • faulty RAM on the server, especially if non-ECC RAM is used
  • faulty server NIC (especially if the NIC supports TCP checksum offloading)
  • NIC/PCIe bus/RAM/SATA chipset/hard drive errors on the client
  • and last but not least: bugs in the web browser, especially with regards to cache handling
Considering the vast number of components involved in such a seemingly simple transaction, it's really remarkable that we don't see errors like that more often.
 
Old 01-27-2014, 02:19 AM   #3
Ulysses_
Member
 
Registered: Jul 2009
Posts: 645

Original Poster
Rep: Reputation: 42
Thanks. If server hardware produces errors, then wouldn't that be noticed and fixed? Or is a mean time between errors of a few hours acceptable and common?
 
Old 01-27-2014, 02:30 AM   #4
Ser Olmy
Senior Member
 
Registered: Jan 2012
Distribution: Slackware
Posts: 1,996

Rep: Reputation: Disabled
If the errors occur before the data is segmented by the IP stack, and if no other mechanism detect those errors, they won't be noticed nor corrected. The most common scenario is bad non-ECC RAM; unless programs segfault of kernels crash, you'll never know it happened unless you verify the data (as you did).

Buggy or faulty chipsets can also introduce errors. Parity checking on the Hypertransport bus or the PCIe bus is of no use if the data corruption occurs before the data is put on the bus.

No, a mean time of a few hours between errors is not acceptable, it indicates faulty hardware or seriously buggy software. On servers with ECC memory I see perhaps one correctable error per year.
 
Old 01-27-2014, 07:31 AM   #5
Ulysses_
Member
 
Registered: Jul 2009
Posts: 645

Original Poster
Rep: Reputation: 42
Now I'm getting worried about my PC ram and other hardware too. How do I know all existing files are ok? Or whether new ones I create myself are corrupted?
 
Old 01-27-2014, 07:44 AM   #6
Ser Olmy
Senior Member
 
Registered: Jan 2012
Distribution: Slackware
Posts: 1,996

Rep: Reputation: Disabled
Run memtest86+ overnight. If there are no errors, you can be reasonably sure your RAM is OK.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] slackpkg md5sum errors. What to do? catkin Slackware 3 09-18-2010 09:59 AM
Reviewing the OSI layers UndiFineD Linux - Networking 5 04-03-2008 03:28 PM
md5sum errors for disc3 bauer172uw Fedora 1 11-08-2003 03:13 AM
Linux OSI Layers MelLinux Linux - General 8 10-29-2002 07:54 PM
Linux FTP/HTTP Download MD5SUM howzmusic Linux - General 2 05-29-2002 01:18 PM


All times are GMT -5. The time now is 11:50 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration