I have a Debian dedicated firewall that is (among other things) sitting between three external networks and a LAN on which we host multiple servers running an application. All the external network IPs are terminated on the firewall, and the internal servers are at 192.168.x.x addresses. We use standard IPTables SNAT/DNAT and source-based routing to ensure that traffic to/from specific external addresses is mapped to specific 192.168.x.x addresses. The application involves a large number of distributed units (>100000) using TCP to connect to these servers.

Everything works fine, but I am concerned at the amount of conntrack activity that this generates, and would rather simply route the traffic. As far as I can see, this would involve invoking NOTRACK in the RAW table to ensure that connection tracking does not take place; but this would also require that the destination address of inbound packets/source address of outbounfd packets be modified in the PREROUTING phase, and I cannot find any way of doing this, as NOTRACK switches off the NAT/PREROYITING call - or am I missing something obvious?

Maybe I'm the one missing something obvious, but don't DNAT and SNAT require tracking so that returning packets can have addresses modified appropriately?

Absolutely. The [fairly-standard] way we do it at the moment works (using DNAT, source-based routing and implicitly requiring a conntrack entry).

1) Inbound packet arrives via external IP x.x.x.x
2) Iptables DNAT re-directs to internal 192.168.x.x (and implicitly creates new conntrack entry for SYN packet)
3) Normal routing takes place and packet goes to 192.168.x.x
4) replies from internal 192.168.x.x get modified by conntrack entry
5) source-based routing ensures packet gets routed back out via external IP x.x.x.x

Works fine, however each connection requires a conntrack entry, and this doesn't scale quite so well for 100k connections.

What I've been trying to do is simply to do this by a combination of mangling the packet in RAW and MANGLE, e.g. a scenario something like this

1) Inbound packet arrives via external IP x.x.x.x
2) Iptables recognised the external IP/destination port combination and unceremoniously <<mangles>> the packet to set the modified destination IP to 192.168.x.x
3) Tell Iptables NOT to conntrack the packet (using NOTRACK in RAW/PREROUTING)
3) Normal routing takes place and packet goes to 192.168.x.x
4) replies from internal 192.168.x.x/source port <<somehow>> mangled to substitute the original external[source] IP for 192.168.x.x, and mark the packet using MANGLE/MARK
5) routing using ip rule fwmark ensures packet gets routed back out via external IP x.x.x.x, using MARK put in packet above.

Achieves the same but <<without>> the overhead of connection tracking.

Problem is - I know how to stop the conntrack entry being created (by using the RAW/PREROUTING table and NOTRACK); but I can't find a way to mangle the IP address in the packet except by using DNAT which implicitly creates, and requires, a conntrack entry - catch 22 - hence my post :-)

Thanks for the clarification ... and sorry I didn't pick that up from your first post. Perhaps you've hit a use case that wasn't anticipated by the respective developers and you need to push for an extenstion of the current options?

Yes . . . thank you for that . . . the reason for my original post is that it seems like a fairly regular thing one might want to do (i.e. mangle the source and destination addresses of packets directly - true redirection prior to routing), and I thought I simply MUST have missed something . . .

One could, in theory, get round the conntrack issue by turning connection tracking off at the kernel level, but without the other bit, that doesn't help and, besides, it's a bit drastic.

I'll make a few enquiries along the lines you suggest.