I'm experiencing little glitches when I try to send/receive datas with local sockets under heavy loads.

under normal load it behaves normally, but with load increasing, I get latency peaks once every second.
an Image is worth a thousand words:
http://img255.imageshack.us/img255/3...lottsy7.th.png

X: time elapsed from beginning of execution
Y: call latency

red: under heavy load
green: no load at all

those peeks of 200ms really disturbs me as I'm using the sockets for RPC calls and 200ms (and far more with load increasing) is really too much.

I've been testing various things
enabling/disabling kernel preempt : no effects
active waiting (doing some stuff that consumes cpu) between RPC calls: no effects (even worse)
non blocking sockets doesn't show any improvements and never yield for a EWOULDBLOCK

setting policy to SCHED_FIFO solves the problem: http://img47.imageshack.us/img47/444...fifoxh5.th.png
also, adding a usleep(0) between each call (still with SCHED_NORMAL policy) removes the peaks
from my understanding, usleep(0) puts the task in sleeping mode until the next TICK is emitted and may cause a context switch if there's another runnable task
sched_yield()'ing once every 1000 calls helps also greatly (some peaks still appears here and there though)

upgrading to 2.6.23:

h00rray it solves everything: http://img371.imageshack.us/img371/7...3lldnl6.th.png
still the mean time is a bit higher and provokes a 30% overhead at running the test

but my problem is that I can't upgrade my kernel (yet) and need to find a solution on 2.6.17
I couldn't reproduce the behavior of the 2.6.17 with the 2.6.23, no matter the kernel config

what has changed and could impact on that 'glitch' between the 2 kernels:
-lock classes of AF_UNIX domain has became bh-unsafe :: seems out of suspicion since the peaks hasn't shown up with AF_INET sockets
-scheduler for SCHED_NORMAL tasks has been completely rewritten :: seems to be guilty of the new (improved?) behavior details following: http://kerneltrap.org/node/8059

from now on;
it must be the scheduler, but if an expert eye could confirm.
and if possibly there's a way with the 2.6.17 kernel to get the expected behaviour without hacking/patching (or not too much) the kernel and staying on SCHED_NORMAL policy

or maybe we can discuss about setting my tasks to SCHED_{FIFO,RR}
but I don't think it's a good idea since my tasks are just doing RPCs and much more important things are running on top of that.