[SOLVED] receiving UDP packets

mibo · 06-15-2011, 11:07 AM

Hi network experts,

I have a microcontroller sending a 500 bytes UDP packet every millisecond to my PC (direct cable connection, no other device or software on the dedicated eth2 interface).
With tcpdump I see the packets arriving with latencies (time between two packets) of 1000µs (+/- 11µs) - this is what I want.

BUT, my C program sees latencies between 10µs and 3500µs (when executed as a user). When I run it as root with "nice -n -20" the latencies get better - between 10µs and 2000µs, but still not good.
Most of the measured latencies are still around 1000µs, but there are packets that arrive with a hefty delay. If that delayed packet is received, my program receives immediately (10µs) the next packet.

So, what is the reason for the delay of the packets? I expected it to be the Linux scheduler, but with "nice -n -20" I hoped it would be nicer to me.
Any idea how to get rid of the delay?

The program uses blocking I/O recvfrom() to receive the UDP packet.
The machine is a 2x dual core Opteron running 2.6.34-gentoo-r6-cs x86_64

Cheers,
mibo

smeezekitty · 06-15-2011, 02:18 PM

I doubt the networking system is designed to send packets 1/ms.

theNbomr · 06-15-2011, 05:10 PM

An example of how Linux is not a real-time OS. For that matter, neither TCP/IP or ethernet are deterministic. If you need hard real-time, consider an OS that is built for it, such as vxWorks.
--- rod.

orgcandman · 06-15-2011, 05:15 PM

tcpdump uses packet_mmap interface. Try that out with a realtime priority+scheduler config. Also, you should make sure that your system doesn't have the NO_HZ option enabled.

mibo · 06-16-2011, 04:27 AM

Thank you very much for the hints.
Especially thank you orgcandman. packet_mmap seems to be exactly the key word I was looking for.

Changing the operating system is no option because there is already running some special real-time stuff on other cpu cores (there are digital control loops with response times well below 10µs - the code is locked to it's own cpu core and "shut down" for the Linux system). I can't do this for the UDP packet receiving code because I want to use sockets and other kernel functionality.

All the best,
mibo

mibo · 06-22-2011, 10:33 AM

The worst latency for the packets went down from 1000µs to 500µs by setting my process to real-time scheduling priority using the code below.
But, I don't understand the PF_PACKET raw packet sending/receiving - that I need for packet_mmap :-(
I will open a new thread for this.

Thanks for your help,
mibo

Code:

  // set process to highest priority
  struct sched_param *param;
  int prio_min, prio_max;

  prio_min = sched_get_priority_min( SCHED_FIFO );
  if ( prio_min == -1 ) {
    printf ( "sched_get_priority_min error: %d: %s\n",errno, strerror(errno));
    exit(-1);
  }
  printf ( "sched_get_priority_min = %d\n", prio_min );
  prio_max = sched_get_priority_max( SCHED_FIFO );
  if ( prio_max == -1 ) {
    printf ( "sched_get_priority_max error: %d: %s\n",errno, strerror(errno));
    exit(-1);
  }
  printf ( "sched_get_priority_max = %d\n", prio_max );

  param->sched_priority = prio_max;	// real-time priority
  rc = sched_setscheduler( 0, SCHED_FIFO, param);
  if ( rc != 0 ) {
    printf ( "sched_setscheduler error: %d: %s\n",errno, strerror(errno));
    exit(-1);
  }
  printf( "set priority to SCHED_FIFO, %d\n", param->sched_priority );