Hey guys, thanks in advance for taking a look at my problem.

I'm running a very slimmed down DAQ (data acquisition) server in my lab at school. It's running Slackware with a slimmed down custom kernel (2.6.38.6).

Before slimming down the kernel, I saw that the new Ultra U12-41388 PCI-E ethernet card I put in it was using a JMicron driver via module. I made the kernel module-less, and built in the driver along with another than I know I use, and a couple other common ethernet drivers. Here is my lspci output:


00:00.0 Host bridge: ATI Technologies Inc RX780/RX790 Chipset Host Bridge
00:0a.0 PCI bridge: ATI Technologies Inc RD790 PCI to PCI bridge (PCI express gpp port F)
00:11.0 SATA controller: ATI Technologies Inc SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode] (rev 40)
00:12.0 USB Controller: ATI Technologies Inc SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:12.2 USB Controller: ATI Technologies Inc SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:13.0 USB Controller: ATI Technologies Inc SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:13.2 USB Controller: ATI Technologies Inc SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:14.0 SMBus: ATI Technologies Inc SBx00 SMBus Controller (rev 42)
00:14.1 IDE interface: ATI Technologies Inc SB7x0/SB8x0/SB9x0 IDE Controller (rev 40)
00:14.3 ISA bridge: ATI Technologies Inc SB7x0/SB8x0/SB9x0 LPC host controller (rev 40)
00:14.4 PCI bridge: ATI Technologies Inc SBx00 PCI to PCI Bridge (rev 40)
00:14.5 USB Controller: ATI Technologies Inc SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
00:15.0 PCI bridge: ATI Technologies Inc Device 43a0
00:15.1 PCI bridge: ATI Technologies Inc Device 43a1
00:16.0 USB Controller: ATI Technologies Inc SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:16.2 USB Controller: ATI Technologies Inc SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:18.0 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor HyperTransport Configuration
00:18.1 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor Address Map
00:18.2 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor DRAM Controller
00:18.3 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor Miscellaneous Control
00:18.4 Host bridge: Advanced Micro Devices [AMD] Family 10h Processor Link Control
01:00.0 Ethernet controller: JMicron Technology Corp. JMC250 PCI Express Gigabit Ethernet Controller (rev 03)
02:05.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)
04:00.0 USB Controller: Device 1b21:1042

The Realtek one is definitely the onboard ethernet chip, and it works perfectly. The Ultra card, however, uses the JMC250 driver, and runs at about half of the speed that it should, on my 100 Mbps network. It's important for me to get that speed back.

I have been completely unable to find ANY information on this Ultra U12-41388 card, or it's M-Tek G24101MKG chipset. If anyone knows of a driver that works well with this other than the JMicron one, or if anyone knows any information at all about possible support in linux, please let me know.

Thanks again,
- dBCooper

P.S. Knowing how to contact M-Tek would even be a great help to me.

did you check you ethernet devices using ? Are they all configured on max speed and full duplex?

1 members found this post helpful.

Good idea, and thanks for the reply. I just checked, and it all looks good as far as I can tell. I don't know anything about the message level stuff, though. Here is the output on the problematic card (eth0):

As you can see it is on full duplex and 100 Mb/s (both this card and my onboard ethernet are). It would make sense if it was operating on half-duplex though, since it is very much half the speed that I should be getting (I get transfers around 5 MiB/s to terminals).

Edit: As a side note... Using the same setup and switching cards to the on-board (Realtek) gives me the speeds I expect: between 10 and 12 MiB/s.

See if you can set checksum offload to the card.
An underpowered/overworked cpu with a single thread nic driver could cause such a slowdown.
A poorly written driver could too.
Do the test with only that card in the box.
Never hurts to try different cables and different destination box.

1 members found this post helpful.

If you have two cards, one that works, and one that doesn't, runt ethtool > file on them both and diff the files :-D. We might learn something.

There is pci settings stored on the pci card as to where it's happy, There are also settings on the socket as to what interrupt that socket might get. Usually I leave that stuff on 'auto' in the bios and hope it doesn't bite. A bit of trial & error assigning the slot an irq might help. It's not supposed to any more, but it might.

Lastly, every card sold in the States has an FCC ID and can be traced from that.
Try Here

1 members found this post helpful.

Thank you jefro and business_kid for your input.

First off, the cpu on this server is a 3.1 GHz dual-core Athlon II, and it's load is about 10% on average under normal data acquisition. It's unlikely that offloading would improve the performance, but as per jefro's advice, I have investigated it none-the-less. It turns out that the checksum offload options were already enabled. I disabled them to see the effect, but it was minimal if existent at all (but please note that there may have been a minimal advantage to disabling offload). I agree about the driver, and my next move might be to take a look at the source (I'm a decent amateur c/c++ coder in Linux).

I ran diff on the two ethtool outputs, and the only differences were as I expected; the message levels. I am ignorant about these levels, but I tried replicating the levels of the well-working Realtek chip. This had no effect. I cannot remove the Realtek card since it's an on-board chip, but I tried disabling it. This also had no effect on the performance of the Ultra card. As for messing with the IRQs, I'm not sure my bios is up to the task. Further details about the socket settings might help me.

Thanks again for your insightful help. All of this leads me to believe that the driver is not well matched, or is poorly written (otherwise I may have to concede that the card sucks, and to that extent I'm unwilling!). Like I said, barring better ideas, I may look at the JMicron driver source.

After quickly realizing that I don't have the time or the knowledge to understand any problems with the code, I have put in a request for help from JMicron and the writer of the Linux driver. I also upgraded to the newest driver version (1.0.8) with no improvement.

I will post on this thread if I make any progress.

OK, looking back, 5 Mb/s is a funny place to be.

A 10 Mb(megabit)/s gives around 1 MB(megabyte)/sec. 100 Mb/s gives 10 times that. You are at around 5. I don't take account of the full duplex stuff, as you are hardly sending and receiving in bulk when doing speed trials. So it's doing 100MB/s, saying 'Hey I can't hack this,' dropping to 10 Mb/s and then trying the higher speed again. Am I making sense?

have you tried replacing the cabling? Messing with error checking? There's a flashing red light in my head behind a sign saying "The hardware sucks," but you might as well prove it.

1 members found this post helpful.

I agree.
Do the test with only that card in the box.
Never hurts to try different cables and different destination box.

Why not try a live cd or two also?

1 members found this post helpful.

No no, I'm getting actually 4.5 MiB/s (mebibytes per sec), or 4.7 MBps (megabytes per sec), or 38 Mbps (megabits per sec). In most circles, a megabyte is considered 2^20 bytes, but this is not strictly true. Mebi (Mi) is the technically correct prefix to designate 2^20, and in my work it's important to specify =). Anyway that's an explanation of the units I used, but I did use bytes (B). I am getting about 38 Mbps tx (haven't really tested rx) with this card on my 100 Mbps LAN. You make a fair deduction about the speed switching, though. (EDIT: Oh actually I see that we're on the same page here)

I agree with the duplex comment. Full or half shouldn't really matter here. The transfers are raw binary data in TCP packets going fairly strictly one direction.

I did try everything with the on-board card disabled, which is the best I can do since it can't be physically removed. Also, as a possible alternative to the live cd (partially since this server doesn't have a cd drive), I temporarily switched back to the Slackware "huge" kernel; no effect.

So all that being said, there is one thing that worked, and it's blowing my mind! First, I need to explain my setup in more detail, so I have drawn it (see attachment). Now you see where in cyan the link goes off to the large ASU (Arizona State University) subnet? I accessed the server from a computer next-door on that subnet, and I got my 11 MiBps (92 Mbps)!! But for some reason, computers on either the red switch or hub ("Download Terminal") get 4.5 MiBps! I have checked their adapters with ethtool, and when on the switch, they are at 100 Mbps & full-duplex.

I don't get it. The "ASU subnet" computer goes through the same cabling and same switch+hub, but it consistently goes the full speed. I would definitely like to understand this. I can't imagine it's a problem with the terminal computers, since each one is vastly different from the next.

Attached Thumbnails

 

OK, the hardware sucks all right but it's just finding _which_ piece of hardware. That is , presuming you're not using ssh? Tell me you are not using ssh.

There's a lot of switches, unless you have big distances. I would start with a laptop, and a file on your server. Connect the laptop directly to both cards of the server in turn and fetch the file, noting speed. No switch in between. Can you get 100Mb (=2^20)? Then add in the red switch on the Ultra card & repeat.

The other curved ball I will ask you to check is this: pcie is usually for GPUs which have massive data transfers and we're tiny network talking packets. If there are complicated transfer settings for the pcie card, change them to other complicated settings and see if that helps. Lastly, check what is sharing the irq with that ultra card. That could be an issue.

You're not beaten until you run out of ideas, As you'll notice, that takes me some time :-D.

1 members found this post helpful.

More questions:
How are you testing your speed?
All you seeing the same behavior at all terminal computers or just testing some of them?
Do they use linux?
Is it possible to connect server directly so some of the terminals?
Is in any way the ASU connected switch port different?
What brand/model is the switch you are using for the terminals/ASU?

netstat -niv output shows any error?
netstat -snv output shows retransmissions?
how about ip layer?
netstat -rnv ?
ip addr show ?

At this point it looks to me your problem is at data link.
I can only think of a different framing issue. 802.2/802.3 and so on.

1 members found this post helpful.

SOLVED IT!!!! There is no problem with the card's performance at all. Let me explain since it explains all of what I was seeing.

What I saw:

1. JMicron card transmitted poorly at about 38 Mbps
2. Realtek card in same setup produced great performance
3. Later found that with JMicron card, performance was good from more remote computer (this one leads to the problem of course)

Problem:

subnetting + hub

What was happening:

JMicron card had a static IP allowed by the ASU servers via its MAC address. The Realtek card was not allowed such a static IP since I did not request it. When I tested the Realtek card in the "same" circumstances that the JMicron card was slow in, I had to use a dynamic IP address. This was the confusing part that made me think it was the card. The fact is that since one was dynamic, and one was static, they were given/used difference subnet masks. The terminal computers are also dynamically associated.

So in comes the hub. The hub is of course a half-duplex communication method. Because the server and terminal are on different subnets, the information actually had to be routed out to the gateway, and back in to the terminal, causing the half-duplex hub to shrink the rate to less than half. When I was on the "more remote" computer and got full speed, that was because this collision did not take place.


SO... thanks all for your help. I wasn't able to address a few suggestions since I figured out that it wasn't a problem with the card at all, but I appreciate them all the same. Damn those ASU network admins and their confusing ways!