Multiple default gateways - weird side-effect.
 

Hi all,

Centos 5.7 i386 multihomed.

I'm moving from one ISP to another and I can't allow our services to be unavailable for any length of time, so to cater for the DNS propagation window I have defined an extra routing table as described here and here, the effect of which is that any internet traffic coming in on an interface will be answered on that interface. It works extremely well. I am also taking this opportunity to resubnet the LAN, so currently each interface is on a completely different subnet.

I've hit an interesting side effect, however, and I think it could be useful in a DMZ situation.

I have two servers on the new network which are on the same switch & subnet - they can ping each other and server2 can SSH to server1. However, server1 cannot SSH to server2.

After much messing about with iptables and logging, I believe that what has happened is that ALL traffic on the interface on the new network is being forwarded to the gateway regardless of whether the target host is available on the same switch & subnet. I have confirmed this with a comparison of the traceroutes between the hosts. Server2 shows two hops to server1 via the the gateway, server1 shows one hop to server2.

In this case, the gateway is a stateful firewall. I think the response traffic (ACK SYN) is being discarded by the firewall because it has no knowledge of the original SYN. From testing, traceroute succeeds however tcptraceroute fails.

It strikes me that this would be useful in absence of VLANs in a DMZ situation where I would like to isolate each host in the DMZ from others. If every host had a route like this defined, then the firewall could protect each machine from the others. Even if a particular machine were compromised and the routes changed to allow subnet traffic, the target machine would not be able to reply because of the route in place pointing all traffic back to the firewall. This would limit attacks to flooding, given that a stateful connection between the compromised machine and the target could not be established and hence no interactive process could be leveraged. This, obviously, would be used in tandem with a local firewall running on each host.

Am I correct in my synopsis? Are there repercussions that I have not considered?

With thanks and regards,

S.