I am developing a UDP based client on Redhat Linux 8 that needs to communicate as quickly as possible with a server (using an embedded real time OS). This product is only destined for local LAN's where I can assume no packets will come in out of order and will not be dropped by a router.

Basic client/server communications work fine but when I increase the speed at which the server responds to a request from the client occasionally I get an EAGAIN error on the recvfrom call at which point I call recvfrom until it recovers. The problem is that when this happens, the packet I was supposed to receive is dropped. I know it came into the linux box (I have ethereal running which indicates it was received okay) but the packet never gets all the way to the recvfrom call.

Additionally, packets are sometimes dropped without any indication at all (again I have ethereal running so I know that the packet was received).

The client program consists of three threads using the pthreads library. A main thread which sends and waits for packets, a send thread which sends packets (no problems occur sending packets), and a receive thread which gets the packets and notifies the main thread when info arrives via a pthread condition.

I am compiling with the -D_REENTRANT and -D_THREAD_SAFE options. I have set the socket to be non-blocking and use a select / recvfrom pair to receive the packets. I have tried using a blocking recvfrom call without a select but this also drops packets.

I am using a RealTek RTL-8139, SMC EZ card but also have tried an Intel EtherExpressPro 100B.

I have tried increasing /proc/sys/net/core/rmem_default and rmem_max to 256k (is this only for tcp anyway?).
I have also increased /proc/sys/net/core/netdev_max_backlog to 2000.

I am only getting responses to packets that I have sent out so there is synchronization going on and it is about 1.2 - 1.7 millisecs between each loop of two sent packets and two received packets (each less than 200 bytes a piece).

I am planning on making TCP an option as well for applictions not requiring as much speed which I know will solve these problems but will be less efficient (I need it to be as efficient as possible). Can anyone give me any other suggestions as to what I can do to increase reliability?

I have had one time a similar problem: the program was not so complex, but I lost packets I solved the problem by modifying the length of the driver reception queue... It's very simple to do: just a #define value to modify and a driver to recompile
Hope this'll help you