Hello,

We have 2 of the following boxes directly connected to each other with a CAT6 cable (to eliminate other factors) and we are trying to achieve a highest throughput. Currently, testing with the iperf utility we get 110 - 115 MB TCP & UDP. I was wondering if this can be improved in anyway or that's just the limit? I can provide other information if required.

Appreciate your help and any input.

- RHEL 4 Advanced Server
- Intel Dual Xeon 3.6
- Intel SE7520JR2 Motherboard
- 4G RAM
- Intel NIC (PCI Express Gigabit Adapters - Copper)

That's the limit! A bit is 1/8 of a byte, so a gigabit is equal to 125 megabytes. Theoretically, you could get 125 megabytes per second, but there is a certain amount of overhead in each packet of data.

Thank you very much Isaac Kuo for your prompt response. So basically the limit is the NIC capacity. Now If we need a higher throughput is there any NIC (Intel is preferred) which can handle more data. Price doesn't matter as long as Linux (RHEL) has a good driver for it.

Thanks again.

Note:
We have been using the Intel NIC (e100 & e1000) and never had problem. They work amazingly good.

I have no idea...maybe Fibre Channel?

Thank you Isaac.

I think there are 10gigabit nics available (maybe just fiber). Here is a link from 2003: http://www.gridtoday.com/03/0303/101137.html

You could also look into nic bonding. I cannot remember for sure but I think you would need mode 5 or maybe 6. Some modes are for fail over but at least one mode is for greater bandwidth (I think it has to be done on both ends the same). If you are looking at transferring data to disk make sure you know how fast your disks (I assume raid) can sustain.

Thank you Lazlow for your input. These boxes will be used for connecting 2 locations to each other using VPN over a third party ISP link so there is no disk IO involved. But we will certainly look into 10Gb NIC cards.

It would be great if somebody who has utilized any of these cards with Linux (specially RHEL) shed some light on us.

If say your target was to say double you 1Gb through put I would do as Lazlow said and look at "bonding". 10Gb looks nice but you would have to be serious about "price does not matter". With bonding you just keep adding NICs and bandwidth. Have a look at:

Overview : http://linux-net.osdl.org/index.php/Bonding
Speedtest : http://www.fos.su.se/compchem/jazz/bond.html (showing 100Mb example getting twice the through put with two nics).

If you do some bonding test it would be good to share your resaults.

sbababei

This did not occur to me until this morning. Regular ethernet 100 has more than enough bandwidth to handle a 10meg (common cable isp) connection. What kind of connection do you have that can handle more than GigE and how much does it cost a month?

The link we have between two locations is 1Gb. I was under the impression that it's actually higher.

Ok, this always bites all of us.

1 GB (one GigaByte) = 1,024 MB (approximately 1 billion Bytes)

1 Gbps = 1,000,000,000 bits per second

Gigabit Ethernet [1000Base-T] is capabile of speeds up to 1000 mbps (mega-bits per second), or 1 gbps

Here is a handy cheat sheet: http://www.lyberty.com/encyc/articles/kb_kilobytes.html

Thats why I always cheat and call my connection 5meg (which really does not mean anything). I think my actual connection is 5mbps(if I remember correctly). The other thing to whatch out for is GB does not equal Gb (GigaByte vs Gigabit) and all the caps/noncaps variations (PITA).

Thank you Lazlow for info. It's definitely more clear now!