LinuxQuestions.org
Review your favorite Linux distribution.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices

Reply
 
Search this Thread
Old 07-27-2011, 06:25 AM   #1
JonasKunze
LQ Newbie
 
Registered: Jul 2011
Posts: 3

Rep: Reputation: Disabled
UDP vs TCP and tcp offload issues


Hey folks,

I have two PCs with a 10G Nic (Intel 82599EB). Within my diploma thesis I need to do some performance comparison between udp and tcp. My first results showed me that with both protocols the speed is quite comparable as long as you aren't using jumbo frames.
What made me pensive was that using udp the CPU load was at 100% but using TCP the load was at about 20% independent on the mtu! As I knew about tcp offload I tried to find out if the udp checksums aren't generated by the hardware. So I checked the offload state by following:

# ethtool -k eth5
Offload parameters for eth5:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off

Well, that sounds for me like tcp offload is on and udp offload is off. Is that right?
So I tried to enable udp offload:

ethtool -K eth5 ufo on
Cannot set device udp large send offload settings: Operation not supported

Does that mean that my NIC can only generate tcp checksums but not udp checksums? That sound pretty senseless for me! Or did I misunderstand ethtool -k?

So my next decision was to disable tcp offload to get some comparable results (meaning the CPU load should rise to 100% for tcp too). So I typed following:

ethtool -K eth5 rx off tx off sg off tso off gso off gro off lro off

Afterwards nothing changed at all! The transmission speeds and the CPU loads were perfectly reproduced.

So here are my questions as a summary:
1) Are there really NICs with tcp offload but without udp offload?
2) How do I disable the tcp offload, or why didn't anything change after my ethtool-K?

Thanks in advanced!
Jonas Kunze

Last edited by JonasKunze; 07-28-2011 at 09:56 AM.
 
Old 07-27-2011, 09:50 PM   #2
jefro
Guru
 
Registered: Mar 2008
Posts: 11,098

Rep: Reputation: 1362Reputation: 1362Reputation: 1362Reputation: 1362Reputation: 1362Reputation: 1362Reputation: 1362Reputation: 1362Reputation: 1362Reputation: 1362
I don't get this. "using udp the CPU load was at 100% but using UDP the load was at about 20% independent on the mtu"

I'd think that to find out what ethtool could be able to read and change would be command like ethtool eth5 .

Might post that.

What a card could do and what a tool might find and what a driver might have been written with/for are all different thoughts.

What the card "could" do may be in the technical pages at intel. If not there the card designer would/should know what would be possible.

From what I have seen, what the card can do my be quite different than what my installed OS can do.

I think you are using the command wrong to disable but post the exact command.
 
Old 07-28-2011, 11:27 AM   #3
JonasKunze
LQ Newbie
 
Registered: Jul 2011
Posts: 3

Original Poster
Rep: Reputation: Disabled
Sorry I meant "using udp the CPU load was at 100% but using TCP the load was at about 20% independent on the mtu"!!!

I've posted the whole command I used to disable the tcp offload!
"ethtool -K eth5 rx off tx off sg off tso off gso off gro off lro off"

As i don't have segmentation "ethtool -K eth5 rx off tx off" would be enough I would say, right?


I've had a look at the Datasheet and it looks like UDP and TCP checksum offload is implemented.

I have scientific linux 6 running. Kernel 2.6.32. Do you think it is possible that the kernel cannot use the udp offload but the tcp offload works?

Or does somebody have another idea why my tcp communication generates much less CPU load?
You can see a diagram at http://butler.physik.uni-mainz.de/~k...to_perf-2p.pdf
 
Old 07-28-2011, 10:02 PM   #4
jefro
Guru
 
Registered: Mar 2008
Posts: 11,098

Rep: Reputation: 1362Reputation: 1362Reputation: 1362Reputation: 1362Reputation: 1362Reputation: 1362Reputation: 1362Reputation: 1362Reputation: 1362Reputation: 1362
I think the command ethtool on that nic would show most of the possible choices that result from the driver and nic and OS.

I guess I'd run wireshark on some packets to see if the checksum is wrong. If it reports that it usually means that offload is on. If they show up as correct then usually that means the checksum is off.

Just for grins I'd repeat that on some way different distro, maybe a debian live dvd or opensuse or even opensidianna if possible to double check results.

Just too many issues to decide. tcp and udp traffic could be handled in so many different ways by various parts of the system.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] TCP segmentation offload (TSO) code in Linux network kernel cameral Linux - Networking 2 08-30-2009 08:30 PM
TCP and UDP aatwell Programming 4 11-07-2007 08:47 AM
tcp/udp and c++ Kroenecker Programming 1 05-10-2005 11:56 AM
TCP vs. UDP mikeshn Linux - Networking 5 05-17-2003 04:14 PM
TCP/IP Offload reschaedl Linux - Networking 0 01-10-2002 03:34 PM


All times are GMT -5. The time now is 12:56 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration