Slow lan and d/l speed: bad nic or some other problem?

slackhack · 02-26-2008, 08:29 AM

Transferring files FROM my server TO my desktop (scp), I get up to 10MB/s. It starts slower, like at 2-4MB/s, and only hits 10M/s at the end, but maybe that's normal? It takes about 1-2 minutes to transfer a 700MB file, so I think that seems about right (?).

Transferring from my DESKTOP to the SERVER, though, it starts out at 2MB/s for only a couple of seconds, then steadily falls over the next minute or two all the way down to 150K/s and lower, saying the ETA for transferring the file is 1+ hour!

It's the same thing with internet downloads. From the desktop, I get a pretty steady 1M/s (wget .iso file). On the server, it starts at about 1M/s, quickly falls to around 250K, goes back up to around 450K, down to 2K/s etc. --it's up and down like a sine wave, very slow and unstable. It doesn't seem to be the remote server, because I tried it a couple of times on each machine, and every time it was the same. As soon as I connect from the desktop, it goes at a steady 1MB/s, while the server is all over the place (and mostly slow).

Now for the "twist": transferring the file from the desktop to the server using FTP, it goes at normal speeds! (I don't see an indicator of the transfer rate in gFTP, but it took <2 minutes for 700MB). (>>edit: just tried with lftp, and I get a very steady 5M/s -- seems like it should be 10M/s minus some overhead though, right?) but still faster and steadier than with scp or the internet.

The server hard drives seem to be okay:

Code:

root@moe:~ # hdparm -i /dev/hda /dev/hdb

/dev/hda:

 Model=SAMSUNG SP0411N, FwRev=TW100-11, SerialNo=S01JJ60XB60759
 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs }
 RawCHS=16383/16/63, TrkSize=34902, SectSize=554, ECCbytes=4
 BuffType=DualPortCache, BuffSize=2048kB, MaxMultSect=16, MultSect=off
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=78242976
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio1 pio2 pio3 pio4 
 DMA modes:  mdma0 mdma1 mdma2 
 UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5 
 AdvancedPM=no WriteCache=enabled
 Drive conforms to: ATA/ATAPI-7 T13 1532D revision 0:  ATA/ATAPI-1,2,3,4,5,6,7

 * signifies the current active mode

/dev/hdb:

 Model=MAXTOR STM3250820A, FwRev=3.AAE, SerialNo=9QE4P8T5
 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
 BuffType=unknown, BuffSize=8192kB, MaxMultSect=16, MultSect=off
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=268435455
 IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio1 pio2 pio3 pio4 
 DMA modes:  mdma0 mdma1 mdma2 
 UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5 
 AdvancedPM=no WriteCache=enabled
 Drive conforms to: Unspecified:  ATA/ATAPI-1,2,3,4,5,6,7

 * signifies the current active mode

I also ran hdparm on them, and they seemed to be fine:

Code:

root@moe:~ # hdparm -Tt /dev/hda

/dev/hda:
 Timing cached reads:   632 MB in  2.00 seconds = 316.00 MB/sec
 Timing buffered disk reads:  148 MB in  3.02 seconds =  49.00 MB/sec
root@moe:~ # hdparm -Tt /dev/hdb

/dev/hdb:
 Timing cached reads:   638 MB in  2.00 seconds = 319.01 MB/sec
 Timing buffered disk reads:  152 MB in  3.01 seconds =  50.53 MB/sec

Is there anything else I should troubleshoot before getting a new card? This is a 3com 3c509 "boomerang" using 3c59x module. Here's ethtool while I"m at it:

Code:

root@moe:~ # ethtool eth0
Settings for eth0:
	Supported ports: [ TP MII ]
	Supported link modes:   10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	Supports auto-negotiation: Yes
	Advertised link modes:  10baseT/Half 10baseT/Full 
	                        100baseT/Half 100baseT/Full 
	Advertised auto-negotiation: Yes
	Speed: 100Mb/s
	Duplex: Full
	Port: MII
	PHYAD: 24
	Transceiver: internal
	Auto-negotiation: on
	Current message level: 0x00000001 (1)
	Link detected: yes

I don't get what the bottleneck would be since it goes fine in one direction but not the other, internet downloading on the server is slow, but ftp seems stable. I also just opened the FTP port and did a transfer over the internet, and while it was slower than through the lan, it was very steady at 2.xMB/s. So what could be the problem?

farslayer · 02-26-2008, 10:19 AM

Might want to view the network information for the NIC card...

Code:

eth0      Link encap:Ethernet  HWaddr 00:1A:A5:B8:69:D8
          inet addr:10.10.0.4  Bcast:10.10.10.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4283 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2520 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:489932 (478.4 KiB)  TX bytes:202899 (198.1 KiB)
          Interrupt:169

Check if the error numbers are all ZERO as they should be, or if they are climbing due to errors..
If there are errors, different errors will indicate specific problems..

Like you should see ZERO collisions on a switched network, but they are normal when using a HUB.
Collisions on a switched network could possibly indicate a duplex mismatch between the network card and the switch

to gather more detailed ethernet statistics, you can use ethtool

ethtool -S eth0

you can also use ethtool to test your NIC (if this feature is supported by your NIC).

Possible problems could be:
A bad NIC
A bad port on the Switch
A Bad cable

I would swap out the network cable and try a different port on the switch before I swapped the NIC..

ARC1450 · 02-26-2008, 10:38 AM

Another vote for swap out the cable and try a new port.

One thing to also check is the CPU usage during transfers. It could be possible that for whatever reason, it's not using DMA. (Highly unlikely, though).

slackhack · 02-26-2008, 10:58 AM

Quote:

Originally Posted by farslayer

Might want to view the network information for the NIC card...

Possible problems could be:
A bad NIC
A bad port on the Switch
A Bad cable

I would swap out the network cable and try a different port on the switch before I swapped the NIC..

woah - many errors!

Code:

eth0      Link encap:Ethernet  HWaddr 00:10:4b:33:7f:eb  
          inet addr:192.168.0.103  Bcast:192.168.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6587476 errors:99214 dropped:0 overruns:0 frame:101814
          TX packets:8762533 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:4141405168 (3.8 GiB)  TX bytes:3296122522 (3.0 GiB)
          Interrupt:10 Base address:0xc000

I exchanged ports between the two computers, and then used two completely different ports, and I had the same results each time. So I don't think it's the switch. The cable is still a possibility, I guess. I have this cable routed through the basement, then up right near a heating duct - now that I check it it's a little "soft" -- maybe the wires got melted? I hope not, that would be the worst.

here's ethtool after testing the different ports:

Code:

 ethtool -S eth0
NIC statistics:
     tx_deferred: 0
     tx_max_collisions: 0
     tx_multiple_collisions: 0
     tx_single_collisions: 0
     rx_bad_ssd: 17481657

So it's something to do with "rx" errors -- does that give a clue as to the source of the problem?

Both desktop and server are set for "negotiated" 100baseTx-FD, and both are plugged into the same switch. I guess negotiated must be the right setting, since it works on the desktop?

I tried "ethool --test eth0" and it said "Cannot test: operation not supported." Am I out of luck with that, or is there another way to test? I didn't see any other references to testing in the man pages.

If it's hardware, I guess it still could be the card or the mobo -- I bought the mobo used a few years ago from someone on the internet, and one of the PCI slots was burned out a little and never worked. Maybe another one is bad now. I guess the only way I'm going to know is to see if I have a spare card around and try it, and then try a different cable if that doesn't work (changing the card would actually probably be easier than the cable given the setup here). Should that be the next step do you think, or is there something about the RX errors that points somewhere else, like to a configuration problem?

farslayer · 02-26-2008, 06:31 PM

I can't really make heads of tails of this explanation. but the other people that had the issue did resolve it by replacing the NIC. From what I can gather it's a signal sent by the card that it has received the start of a stream (packet) if your's is erring out, that would cause the sender to have to repeatedly re-transmit data until it was successful, causing your slow down..

Quote:

3.3.11 Bad SSD Detection
A Bad Start of Stream Delimiter (Bad SSD) is any transition from consecutive idle code-groups to non-idle code-groups
which is not prefixed by the code-group pair /J/K.

If this condition is detected, the DP83847 will assert RX_ER and present RXD[3:0] = 1110 to the MII for the
cycles that correspond to received 5B code-groups until at least two IDLE code groups are detected. In addition, the
False Carrier Sense Counter register (FCSCR) will be incremented by one.
Once at least two IDLE code groups are detected, RX_ER and CRS become de-asserted.

slackhack · 02-27-2008, 03:58 PM

Hey guys - I think it is the cable! Before switching the card, I took the computer inside and hooked it up with a known good 3ft patch cable -- full speeds!

Strange thing is, after messing around with a couple of used nics I just got to test them out, something went wrong with the network and I couldn't connect to anything. My router slowed to a crawl and I couldn't even ssh in. So I brought that one inside too and hooked up some other cables, and I was back in business again. So it seems that all the cables going out into the garage are going bad.

I wonder if excessive cold can damage network cables? It's pretty cold out there -- unheated garage, so I'd guess around 30-40F if not colder at night. Or is it more likely heat damaging them from being too close to the heating ducts?

farslayer · 02-27-2008, 10:37 PM

Moisture would be a more likely culprit...
Pinching or kinking the wires can have adverse effects.
Cables being terminated improperly.
Not following the EIA/TIA 568 A/B color codes when making cables
Excessive heat.. possibly if it has caused some of the insulation to melt.

There are a lot of ways to damage cable and once it's damaged, the only recourse is to replace it.