LinuxQuestions.org - Strange bottleneck in my local network

- Slackware (https://www.linuxquestions.org/questions/slackware-14/)

- - Strange bottleneck in my local network (https://www.linuxquestions.org/questions/slackware-14/strange-bottleneck-in-my-local-network-4175573388/)

Strange bottleneck in my local network

Hi,

I just tested my local network with IPerf and got some puzzling results. According to lspci, all machines have Gigabit Ethernet cards, so I should get an estimated bandwidth of roughly 900 Mbits/sec.

I'm starting IPerf on leanore.microlinux.lan (my most recent acquisition) and then launch tests from a series of clients as well as the server. Here's what I get:

Code:

[root@leanore:~] # iperf -s

------------------------------------------------------------

Server listening on TCP port 5001

TCP window size: 85.3 KByte (default)

------------------------------------------------------------

[  4] local 192.168.2.9 port 5001 connected with 192.168.2.2 port 43085

[ ID] Interval      Transfer    Bandwidth

[  4]  0.0-10.0 sec  1.09 GBytes  940 Mbits/sec

[  5] local 192.168.2.9 port 5001 connected with 192.168.2.8 port 58073

[  5]  0.0-10.0 sec  1.03 GBytes  885 Mbits/sec

[  4] local 192.168.2.9 port 5001 connected with 192.168.2.1 port 52521

[  4]  0.0-10.1 sec  113 MBytes  94.1 Mbits/sec

Uh oh. Clearly there's a little problem with 192.168.2.1, which happens to be my local server. So let's start IPerf on the server and then run tests from some clients:

Code:

[root@nestor:~] # iperf -s

------------------------------------------------------------

Server listening on TCP port 5001

TCP window size: 85.3 KByte (default)

------------------------------------------------------------

[  4] local 192.168.2.1 port 5001 connected with 192.168.2.9 port 50445

[ ID] Interval      Transfer    Bandwidth

[  4]  0.0-10.1 sec  114 MBytes  94.2 Mbits/sec

[  5] local 192.168.2.1 port 5001 connected with 192.168.2.8 port 43795

[  5]  0.0-10.1 sec  114 MBytes  94.2 Mbits/sec

[  4] local 192.168.2.1 port 5001 connected with 192.168.2.2 port 59429

[  4]  0.0-10.0 sec  112 MBytes  94.1 Mbits/sec

So there's clearly a problem with the server. Can't be the hardware:

Code:

[root@nestor:~] # lspci | grep -i eth

02:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection

03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection

On the LAN side, the server is connected to a small 8-port switch which I had to change recently after the last switch didn't survive a power surge. All the facts seem to point to this piece of hardware as being the culprit. Is it possible that a switch bought new in 2015 is only capable of 100 Mbit/sec? I admit I'm not very proficient in hardware things, I usually learn on the spot and by trial-and-error.

Cheers,

Niki

Check you aren't missing the correct firmware for the ethernet interface on the slow server. Some gigabit interfaces will still work but revert to slower speed if that is the case.

Edit: Looks like the Intel 82574L (e1000e) driver has a history of problems:
http://www.doxer.org/resolved-intel-...work-blipping/

Quote:

Originally Posted by kikinovak (Post 5507000)

Hi,

On the LAN side, the server is connected to a small 8-port switch which I had to change recently after the last switch didn't survive a power surge. All the facts seem to point to this piece of hardware as being the culprit. Is it possible that a switch bought new in 2015 is only capable of 100 Mbit/sec? I admit I'm not very proficient in hardware things, I usually learn on the spot and by trial-and-error.

Cheers,

Niki

If you run "ethtool ethX" what speed does it report for the interface ?

Also, what is the manufacturer and model of the 8port switch ? Are the two working boxes connected to other switches ? If yes, then it is possible that the switch is 100mbit.

Quote:

Originally Posted by imitheos (Post 5507031)

The server has two network cards, showing as eth0 and eth1.

Code:

[root@nestor:~] # ethtool eth0

Settings for eth0:

        Supported ports: [ TP ]

        Supported link modes:  10baseT/Half 10baseT/Full 

                                100baseT/Half 100baseT/Full 

                                1000baseT/Full 

        Supported pause frame use: No

        Supports auto-negotiation: Yes

        Advertised link modes:  10baseT/Half 10baseT/Full 

                                100baseT/Half 100baseT/Full 

                                1000baseT/Full 

        Advertised pause frame use: No

        Advertised auto-negotiation: Yes

        Speed: 100Mb/s

        Duplex: Full

        Port: Twisted Pair

        PHYAD: 1

        Transceiver: internal

        Auto-negotiation: on

        MDI-X: on (auto)

        Supports Wake-on: pumbg

        Wake-on: d

        Current message level: 0x00000007 (7)

                              drv probe link

        Link detected: yes

[root@nestor:~] # ethtool eth1

Settings for eth1:

        Supported ports: [ TP ]

        Supported link modes:  10baseT/Half 10baseT/Full 

                                100baseT/Half 100baseT/Full 

                                1000baseT/Full 

        Supported pause frame use: No

        Supports auto-negotiation: Yes

        Advertised link modes:  10baseT/Half 10baseT/Full 

                                100baseT/Half 100baseT/Full 

                                1000baseT/Full 

        Advertised pause frame use: No

        Advertised auto-negotiation: Yes

        Speed: 100Mb/s

        Duplex: Full

        Port: Twisted Pair

        PHYAD: 1

        Transceiver: internal

        Auto-negotiation: on

        MDI-X: off (auto)

        Supports Wake-on: pumbg

        Wake-on: d

        Current message level: 0x00000007 (7)

                              drv probe link

        Link detected: yes

And the switch is a TPLink TL-SG108, which is 10/100/1000.

So I'm clueless as to why connectivity to my server is ten times slower than between the clients.

And yes, I do have a second switch on my network, it's the exact same model, and connectivity between two clients on that switch is fast.

Edit: oops, I just saw that the speed on the two cards is indeed 100 Mb/s, which is ten times slower as advertised. Is there a way I can change that?

I have similar setup, two gigabit switches. One day the speed between two dropped to 100 Mbit/s. I was drilling holes for TV cable and accidentally damaged the CAT cable ...

Also check that your cable is CAT 5e or 6. CAT 5 wont work.

ethtool --negotiate eth0

will make it renegotiate, there are options to force it, but honestly, if it cant negotiate then forcing it isn't likely to work either.

Hi,

What type of cable are you using? Backbone should be Cat 6 with interconnections for switch also Cat 6. If your cable is correct then I would trade the switches to see if the problem follows the server switch after testing again.

Hope this helps.
Have fun & enjoy!

oops: codeguy beat me for cable type.

EDIT: https://en.wikipedia.org/wiki/1000BASE-T
:hattip:

Quote:

Originally Posted by codeguy (Post 5507097)

Also check that your cable is CAT 5e or 6. CAT 5 wont work.

Quote:

Originally Posted by onebuck (Post 5507099)

What type of cable are you using? Backbone should be Cat 6 with interconnections for switch also Cat 6

@kikinovak As we are talking about cables, though unlikely to be the cause of the trouble in this case, for the future also have in mind that some cheap patch cables (especially those that come bundled with xDSL modems) only contain 2 pairs of wires instead of the full 4 pairs. Once i tried to debug why gigabit didn't work and after trying a ton of things, i noticed that the "new" cable i used, only had the 2 pairs :)

Thanks for the numerous suggestions. There's a chance that the cable is the culprit. The router is on the floor below, and I've installed a 20 meter long cable to my office. I'll look into this and try a different cable. I wonder if besides quality, cable length can also be of importance.

Niki

Hi,

Quote:

Originally Posted by kikinovak (Post 5507119)

From Cat 6

Quote:

When used for 10/100/1000BASE-T, the maximum allowed length of a Cat 6 cable is 100 meters (328 ft). This consists of 90 meters (295 ft) of solid "horizontal" cabling between the patch panel and the wall jack, plus 10 meters (33 ft) of stranded patch cable between each jack and the attached device.

EDIT: One other thing is the bend radius for your cables. Look at the wiki tables for the max bend. Support the cables and allow some sag, do not stretch or place on the floor where foot traffic can cause issues.

From Cat 6

Quote:

Category 6 and 6A cable must be properly installed and terminated to meet specifications. The cable must not be kinked or bent too tightly (the bend radius should be at least four times the outer diameter of the cable[7]). The wire pairs must not be untwisted and the outer jacket must not be stripped back more than 0.5 in (12.7 mm).

Cable shielding may be required in order to improve a Cat 6 cable's performance in high electromagnetic interference (EMI) environments. This shielding reduces the corrupting effect of EMI on the cable's data. Shielding is typically maintained from one cable end to the other using a drain wire that runs through the cable alongside the twisted pairs. The shield's electrical connection to the chassis on each end is made through the jacks. The requirement for ground connections at both cable ends creates the possibility that a ground loop may result if one of the networked chassis is at different instantaneous electrical potential with respect to its mate. This undesirable situation may compel currents to flow between chassis through the network cable shield, and these currents may in turn induce detrimental noise in the signal being carried by the cable.

You should troubleshoot.
Physically, there are 3 variables: network card, switch and network cable.
Since you have an identical one, testing the switch is the easiest.
Next, test the network card (might be easier to switch the box than opening it and switch cards).
Last, test the cable.
If the problem is not physical, you might have a driver problem (boot some live-CDs and test with other distros).

Gigabit ethernet is very fussy about cable type. Check all the cables, patch panels & outlets between NIC and switch port are CAT6 compliant.
CAT6 should be printed on the cable jacket.