Hi network experts,

I have a very simple test system running.
A microcontroller (mc) with 100Mbit/s ethernet sending 500bytes packets at a 2000Hz rate (so one packet every 500µs).
The mc is directly connected with an 1m ethernet cable to a dedicated intel ethernet adapter in the PC. The PC runs Ubuntu 9.10 (the same problems with 9.04) has 2GB ram, a quad core cpu and is idle.

My single threaded PC program just receives the packets with "recvfrom" and checks the packet number and writes an info to the screen - nothing more. The mc writes a 16 bit number n=n+1 to every packet, so the PC knows, which packet number to expect next.
The PC program reports no problem as long as I don't touch the PC. If I switch between the 4 desktops - the program receives some more packets but then quits with an error because it received packet#630 but expected packet#621 (just an example).
So it seems, a buffer is filling up and then dropping packets.

Why this is so strange to me:
- the PC has 4 cores, enough ram and cannot handle 1kbytes/s (the correct number is 1Mbytes/s) data???
- wireshark runs on the PC and sees ALL the packets - even the ones that my program did not receive. (wireshark is not the reason for the problem - the packets get lost even without running wireshark)

Can somebody explain to me what is happening here?

Thank you,
mibo

If I understand this correctly you are generating 1 MBytes/seconds (500 * 2000) or about 8 mBits/sec. When you set up to read UDP packets from a given port, the system creates a little buffering for incoming packets. The TCP/IP stack receives UDP packets and stuffs them into the buffer for any processing listening for packets on that port.

Your reader task is probably being interrupted for a few milliseconds as part of your desktop switch and during that time it is not reading the UDP packets and enough new ones have arrived that some get thrown away. There is no flow control for UDP packets and you are getting them at a pretty high rate.

I have not done anything quite this high rate, but in Java you can get and set the receive buffer size for a DatagramSocket. In C I think this is called the SO_RCVBUF option and it is manipulated via equivalent library calls. I suggest to read the value from your socket and try to set it to a higher value. This will increase the amount of buffering to something that you are pretty sure cannot be filled up while your application is not running.

Good luck.

In addition to the advice given by dckump, I'd add that your program should be able to cope with dropped UDP packets. This might be obvious, but I think it needs saying - if you don't design it to be able to cope with lost UDP packets, it *will* crash & burn sooner or later due to network "happenings", regardless of how big a buffer the receiver has and how quickly it can process packets.

If you need to guarantee that you see all the data coming in, TCP is a better option.

One additional note: maybe some of these so-called "dropped packets" are actually be "out of order" packets. Your program appears to be failing to handle either case. UDP, of course, guarantees neither receipt nor ordering.

SUGGESTION:
1. Write a test program that simply receives the packets

2. Count the %/packets received out of order

3. Count the %/packets never received at all

4. Post back the results.

We're curious what you find

Thank you for the good ideas.

dckump, you are right - it's 1MByte/s.

I will check, if I can make the receive buffer bigger. I will also check if the packets are just in the wrong order. ATM my program quits as soon as the first packet with a wrong number arrives.

The mc sends "realtime" data and the PC needs to handle it with a low latency. So, there is no option to resend data. I have to build in a (non quitting) packet loss detection and a "data good" flag/counter.
In the end the receiving PC will run a realtime linux but its easier to start just with an Ubuntu PC in my office.
About the network traffic - it will never happen, as the PC will always directly be connected to the mc (just one network cable between them).

mibo

Sorry for not responding lately. The mc-device has some problems, so atm I can not make the tests.
But, I tried the SO_RCVBUF option and found out that the buffer size was set to something about 100000 bytes. I set it to 1Mbytes but the kernel set it automatically to something more than 200000 bytes. Hopefully, next week I can test if the bigger buffer size helps.

Cheers,
mibo

PS: I found an interesting website discussing UDP buffers and packet loss
http://www.29west.com/docs/THPM/udp-buffer-sizing.html