Gigabit card with 100Mbits speeds?

darkefge · 03-03-2011, 06:01 PM

From what I can see in mii-tool I should be getting 1000Mbit link but when I transfer files I only get 100mpbs?

Code:

morrow:~# mii-tool
eth0: no link
eth1: no link
eth2: negotiated 1000baseT-HD flow-control, link ok
eth3: negotiated 1000baseT-HD flow-control, link ok
morrow:~# lshw -C Network
  *-network:0
       description: Ethernet interface
       product: NC100 Network Everywhere Fast Ethernet 10/100
       vendor: ADMtek
       physical id: 9
       bus info: pci@0000:00:09.0
       logical name: eth3
       version: 11
       serial: 00:03:6d:##:##:##
       width: 32 bits
       clock: 33MHz
       capabilities: pm bus_master cap_list ethernet physical
       configuration: broadcast=yes driver=tulip driverversion=1.1.15-NAPI latency=32 maxlatency=255 mingnt=255 module=tulip multicast=yes
  *-network:0
       description: Ethernet interface
       product: 82557/8/9/0/1 Ethernet Pro 100
       vendor: Intel Corporation
       physical id: 4
       bus info: pci@0000:02:04.0
       logical name: eth1
       version: 05
       serial: 00:50:8b:##:##:##
       size: 10MB/s
       capacity: 100MB/s
       width: 32 bits
       clock: 33MHz
       capabilities: pm bus_master cap_list ethernet physical tp mii 10bt 10bt-fd 100bt 100bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=e100 driverversion=3.5.23-k4-NAPI duplex=half firmware=N/A latency=32 link=no maxlatency=56 mingnt=8 module=e100 multicast=yes port=MII speed=10MB/s
  *-network:1
       description: Ethernet interface
       product: 82557/8/9/0/1 Ethernet Pro 100
       vendor: Intel Corporation
       physical id: 5
       bus info: pci@0000:02:05.0
       logical name: eth2
       version: 05
       serial: 00:50:8b:##:##:##
       size: 100MB/s
       capacity: 100MB/s
       width: 32 bits
       clock: 33MHz
       capabilities: pm bus_master cap_list ethernet physical tp mii 10bt 10bt-fd 100bt 100bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=e100 driverversion=3.5.23-k4-NAPI duplex=full firmware=N/A latency=32 link=yes maxlatency=56 mingnt=8 module=e100 multicast=yes port=MII speed=100MB/s
  *-network:1
       description: Ethernet interface
       product: VT6102 [Rhine-II]
       vendor: VIA Technologies, Inc.
       physical id: 12
       bus info: pci@0000:00:12.0
       logical name: eth0
       version: 78
       serial: 00:11:5b:##:##:##
       size: 10MB/s
       capacity: 100MB/s
       width: 32 bits
       clock: 33MHz
       capabilities: pm bus_master cap_list ethernet physical tp mii 10bt 10bt-fd 100bt 100bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=via-rhine driverversion=1.4.3 duplex=half latency=32 link=no maxlatency=8 mingnt=3 module=via_rhine multicast=yes port=MII speed=10MB/s
  *-network
       description: Ethernet interface
       physical id: 1
       logical name: br0
       serial: 00:03:6d:##:##:##
       capabilities: ethernet physical
       configuration: broadcast=yes driver=bridge driverversion=2.3 firmware=N/A ip=192.168.1.99 link=yes multicast=yes

andrewthomas · 03-03-2011, 06:53 PM

mii-tool(8) - Linux man page

Quote:

Note
This program is obsolete. Valid media are only 100baseT4, 100baseTx-FD,100baseTx-HD, 10baseT-FD and 10baseT-HD ethernet cards. For replacement check ethtool.

tommylovell · 03-03-2011, 07:32 PM

How are you measuring your 100mbps throughput? Is it on a tcp connection? How many hops between the endpoints? Are you going through just switches or is there a router (or routers) in the path? If you're equipment has them, what do the connected speed LED's say?

Even under the best of circumstances, with jumbo frames, low latency in all the intervening nodes, and with files large enough so that tcp's self tuning "ramps up" the transfer, you'll never get anywhere near 1000mbps...

jefro · 03-03-2011, 09:48 PM

Dooohhh!! What tommylovell says next.

tommylovell · 03-03-2011, 10:17 PM

...and your mii-tool output, as bogus as it is, seems to indicate that eth2 and eth3 are not gigabit cards.

description: Ethernet interface
product: 82557/8/9/0/1 Ethernet Pro 100
vendor: Intel Corporation
physical id: 5
bus info: pci@0000:02:05.0
logical name: eth2

description: Ethernet interface
product: NC100 Network Everywhere Fast Ethernet 10/100
vendor: ADMtek
physical id: 9
bus info: pci@0000:00:09.0
logical name: eth3

darkefge · 03-05-2011, 02:25 AM

According to this thread, and looking at the chips soldered on I have the 82558b Intel dual gigabit card.
http://www.linuxquestions.org/hcl/sh...php?product=55

I thought it might have something to do with all the NICs in this box set into a bridge. But as far as I have read it should auto-negotiate similar to a router.

Network topology is this box up to a gigabit router down to an XP build that for sure linked at gigabit speeds per the router. Link LED orange 100, 1000 green.

Code:

morrow:~# ethtool eth2
Settings for eth2:
        Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
        Advertised auto-negotiation: Yes
        Speed: 100Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: g
        Wake-on: d
        Current message level: 0x00000007 (7)
        Link detected: yes
morrow:~# ifconfig eth2
eth2      Link encap:Ethernet  HWaddr 00:50:8b:##:##:##
          inet6 addr: fe80::250:xxxx:xxxx:xxxx/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:13199973 errors:4 dropped:0 overruns:0 frame:4
          TX packets:4850849 errors:10 dropped:0 overruns:0 carrier:10
          collisions:0 txqueuelen:1000
          RX bytes:1864072945 (1.7 GiB)  TX bytes:1380207126 (1.2 GiB)

Might just be jumbo frames?

Code:

raymond@morrow:~$ rsync -vr --progress --stats /tv/Real.Time/Real.Time.avi ~/skull.e/
sending incremental file list
Real.Time.avi
   574930944 100%   11.48MB/s    0:00:47 (xfer#1, to-check=0/1)

Number of files: 1
Number of files transferred: 1
Total file size: 574930944 bytes
Total transferred file size: 574930944 bytes
Literal data: 574930944 bytes
Matched data: 0 bytes
File list size: 75
File list generation time: 0.002 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 575001251
Total bytes received: 31

sent 575001251 bytes  received 31 bytes  10550482.24 bytes/sec
total size is 574930944  speedup is 1.00
raymond@morrow:~$ ls -lah skull.e/Real.Time.avi
-rwxrwSrwx 1 root root 549M 2011-03-05 02:20 skull.e/Real.Time.avi

tommylovell · 03-05-2011, 02:52 PM

Jumbo frames might help, but they are very hard to set up. Not all pieces of network gear support them, and sometimes even the ones that do don't... Before even trying jumbo frames you need to find the bottlenecks in your network and eliminate them.

Simply because you can get an interface to clock at 1Gbps doesn't mean that you are going to get 1Gbps throughput. There are a lots of factors that can affect throughput. And although I've heard it rumored that it is possible to get utilization up to 50 to 60% of gigabit speed, I don't think I'm likely ever to see that. I don't see that at work where we have some very expensive network gear and I don't see it at home.

I tried a little experiment at home. I transferred some 2GB files from my Macbook to a Linux server over ftp (sftp, that is encrypted ftp). The first ones I did were over wireless and went at about 3.4 or 3.5MB/s, so about roughly 27-28Mbps. That's roughly 50% of 54Mbps wireless speed. The second set of files I sent from the Macbook over the gigabit interface through a cheap router/switch over a gigabit interface to the same Linux server. This second set of files went at about a 25MB/s rate, or roughly 200Mbps. That's much quicker, but now is just 20% utilization of gigabit speed. When I connected the MacBook and the Linux server to a more expensive switch I have, the next set of files went at about 27.5MB/s. I thought I could attribute that to lower latency through the more expensive switch, but when I put the two switches between the two systems I got a slight increase to about 28MB/s. I have no clue as to why. All of this was with no other traffic in the network besides the small amount of babbling the Mac is prone to. And this was all repeatable.

But clearly 10.5MB/s is disappointing and there is probably a good and possibly correctable reason for the low throughput. But it is often hard to see.

I would say that in the absence of expensive analysis software (like Opnet), your best bet is to run a 'tcpdump' packet trace on your Linux system, and open it with Wireshark looking for retransmissions, closed tcp windows, duplicate ACKs, etc.

And, btw, when you say "router" I assume you really mean the switch portion of your home router. The Linux systems and the Windows systems are on the same subnet, so packets are just switched between them, not routed. Right? A router in between would mean more latency and lower transfer rates.

And I'm not exactly sure what you meant by "I thought it might have something to do with all the NICs in this box set into a bridge. But as far as I have read it should auto-negotiate similar to a router."

tommylovell · 03-05-2011, 03:45 PM

Another experiment. I did the same ftp transfers, but this time in clear text - not sftp, so no encryption/decryption.

The transfers all started out at about 38-39MB/s and went down to 33-34MB/s, averaging around 36Mb/s. About 290Mbps, or 29% of a raw gigabit.

My Mac is a dual core 2.66, and the Linux system is a dual core 2.8. They have a decent amount of horsepower, but clearly the encypt/decrypt still has an impact.