Setting up a very low-latency router using linux.
 

Heya,

I've been dabbling in Linux, UX derivatives such as BSD and HP-UX for a while now and was in need of a bit of help.
I've had this small side-project going on of creating a linux router out of an old 2013 personal computer at home.


The specs of the machine:

i5-3570k
8GB 1600Mhz XMP
OCZ Vertex4 SSD
z77a-g45

I'm using OEL8.4 and doing IPv4 routing and firewalling + NAT on this machine.
On its outbound interface I also configure CAKE: https://www.bufferbloat.net/projects/codel/wiki/Cake/
By default you cannot do this because the OEL kernel in use is too "old" and without cake.
So I'm using elrepo to install the latest longterm-branch kernel version. 5.4.#

I'm currently trying to set up a "real" low-latency OS configuration where I isolate a few cores and configure networking to run on only the isolated cores.
The problem I have is, I don't know exactly which kernel threads do the networking.
My guess is I need to move the IRQs to the isolated cores but I don't know what else needs to be moved. Any help here, is appreciated.


tuned-adm has a profile for partitioning the CPU: https://access.redhat.com/documentat...ence-tuned_adm

The plan I have is to utilize it and isolate cores 3 and 4, move the device IRQs to them using tuna: https://access.redhat.com/documentat...-of-using-tuna
via a systemd startup service.

The system has 2 ethernet adapters. eth0 for WAN, eth1 for LAN.
Intel Gigabit CT desktop adapter: https://www.intel.com/content/www/us...fications.html

The onboard is not used.


Does anyone know what else needs to be assigned to the isolated cores and perhaps what extra steps I'd need to take to build this lowest and stablest possible latency router?

Thank you in advance.

Just some thoughts, maybe others have ideas.

I might run it from ram. Might switch packet transfers so that it doesn't wait for the entire packet to route. Forget what the cisco name for that is. Usually the true enterprise level devices get up wire speeds. It may be hard to compete even with a 7 year old used device.

Not sure trying to isolate cores is best. Would need to do test. Moving to near real time kernel may help if the drivers for nic are also nrt. Best would be a full real time system that had all drivers in real time. QNX and some commercial RTL tout that a lot. One could build a custom kernel too and consider how the scheduler works.