Maximizing 10GbE throughput

fantasygoat · 07-04-2011, 10:14 AM

I'm in the process of setting up an NFS-based file share via Broadcom 57710 10GbE interfaces and a 10GbE switch, and I was wondering if anyone had any experience in configuring it for optimal performance in transferring smaller files (mostly in the 1-2.5GB range).

A lot of the optimizations I see is for large file transfer, which will often impact small file transfer speeds.

The OS is Centos 5.6.

16pide · 07-04-2011, 11:36 AM

Quote:

smaller files (mostly in the 1-2.5GB range)

That's not what I call small files!

Your bottlenecks will probably be the disks. 16 SAS 15Krpm/s disks would be good.
Some things to consider:

What bandwidth is your network and client devices ready to absorb?
What's the disk controller?
How many disks do you have?
What type of disks is it?
What speed is the PCI slot that you're connecting the network card to?
What speed is the bus driving the PCI slots?

With a standard Linux kernel, you'll probably be limited to around 7Gb/s which is good though.

Dani1973 · 07-04-2011, 01:21 PM

16pide is right 1-2.5GB are BIG files.
For network communication a small file would be like a few KB (and even less than that).

Don't forget 10 gigabit per second is 1250 megabytes per second!!!
Even 16 SAS 15k drives in RAID0 will have big troubles to sustain that rate and your RAID controller, ... etc ...

I think that nearly all points that 16pide mentioned will be a bottleneck compared to such fast network connection.

I think if you get 4-5Gbps on out of it you should be happy.