Route configuration problem

wimnat · 09-07-2009, 12:19 AM

Sorry to hijack but I am faced with a similar problem.

I have two NICs. Eth0 and Eth1.

One is connected to my private network and the other to my public network.

If I specify my default gw in my eth0 config then all is well on the private side but the box is publically inaccessible. Vice versa if I set the gw in my eth1 config.

I can ping both interfaces.

So, time to crack out the route command.

I tried adding a default gw for both networks...

Code:

[root@yum ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
203.38.38.38  0.0.0.0         255.255.255.224 U     0      0        0 eth1
10.0.8.0        0.0.0.0         255.255.255.0   U     0      0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 eth1
0.0.0.0         203.38.38.38  0.0.0.0         UG    0      0        0 eth1
[root@yum ~]# route add default gw 10.0.8.1 eth0
[root@yum ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
203.38.38.38  0.0.0.0         255.255.255.224 U     0      0        0 eth1
10.0.8.0        0.0.0.0         255.255.255.0   U     0      0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 eth1
0.0.0.0         10.0.8.1        0.0.0.0         UG    0      0        0 eth0
0.0.0.0         203.38.38.38  0.0.0.0         UG    0      0        0 eth1

Now there is no metric differnce but I thought that the interface value would be enough to distinguish the traffic.

However, this is not the case. With the above route table I can reach services on my public IP (like SSH) but not my private. Again, both are pingable.

I have checked config files of services and they are configured to bind to all interfaces.

jschiwal · 09-07-2009, 06:30 AM

I moved your post to it's own thread. Please don't highjack someone else's thread. It can cause confusion keeping track which answer is for which post, and may reduce the number of replies as a result.

jschiwal · 09-07-2009, 06:52 AM

Having two default gateways doesn't make sense. Does eth1 connect to a router or your ISP's modem?

Code:

203.38.38.38  0.0.0.0         255.255.255.224 U     0      0        0 eth1

The destination address should be a network address, unless you are subnetting the 203.38.38.36 network into several subnets. Something like 203.38.38.32. I just masked 38 with 224 to obtain 32.

Given that you have a host address (the default gateway) of the same value, you entered the gateway IP address instead of the network for the first route.

evilted · 09-07-2009, 09:19 AM

you need something called policy routing. 2 default gateways doesnt work. metric's are not failover, but a measure of distance (hops) and are obselete (apparently) in newer versions of linux.

we have 2 defaults

Code:

0.0.0.0         10.0.8.1        0.0.0.0         UG    0      0        0 eth0
0.0.0.0         203.38.38.38  0.0.0.0         UG    0      0

but if you run a trace route you will see that your are only using the gateway at 10.0.8.1.
remember that the default gateway is also called the gateway of last resort. so when your machine looks up a route in the routing table, it will go to the default when the route doesnt exist. having 2 defaults defeats the purpose. only one default will be used - if the last entered gateway (in your case the 10.0.8.1) is considered/marked a dead gateway will you use the 2nd (first entered gateway 203.38.38.38)

Code:

echo 200 to_local >> /etc/iproute2/rt_tables
ip rule add from 10.8.0.1 table to_local
ip route add default via 10.0.8.1 dev eth0 table to_local

you can leave your gateway for internet access as the default(203.38.38.38) and add a gateway for your local (if you really need it). this should make access to both networks.. (you may need routes for your public net in the routing table for the local net.. you should be able to do this from the code above..

most people dont realize that they are using policy routing. if you type 'ip rule show table main' you will see the entries in the main (default) table. these are created when you make routing in the normal manner.. route add -net (some net)/(some subnet) gw (some gateway)

read here -> http://lartc.org/howto/
and here -> http://lartc.org/howto/lartc.rpdb.html

wimnat · 09-07-2009, 11:28 PM

Thanks, this works after one small adjustment. I had to add the whole subnet...

Code:

ip rule add from 10.8.0.0/24 table to_local

rather than

Code:

ip rule add from 10.8.0.1 table to_local

How can I make these rules stay in place on a 'service network restart'?