Hi All,

I have a linux router, configured with two internet connections and two lan segments. I've setup multipath routing as described in the linux advanced routing howto (lartc.org)

My problem (I think) is that somehow the router will randomly choose incorrect routing paths for different hosts, for example: on my workstation (192.168.1.20), I ssh to a server I have on an external network (157.238.135.60), and my connection locally hangs. On the router, I search the routing cache:

# ip route show cache | grep 157.238.135.60
157.238.135.60 via 207.180.31.137 dev eth0 src 207.180.31.140
157.238.135.60 from 192.168.1.20 tos 0x10 via 10.14.1.1 dev ppp0 src
192.168.1.1
157.238.135.60 from 192.168.1.20 via 207.180.31.137 dev eth0 src
192.168.1.1
192.168.1.20 from 157.238.135.60 dev eth3 src 207.180.31.140

Compare this to cache entries for a host that does work (157.238.135.90):
# ip route show cache | grep 157.238.135.90
192.168.1.20 from 157.238.135.90 tos 0x10 dev eth3 src 207.180.31.140
157.238.135.90 via 10.14.1.1 dev ppp0 src 151.203.160.233
192.168.1.20 from 157.238.135.90 dev eth3 src 207.180.31.140
157.238.135.90 from 192.168.1.20 via 10.14.1.1 dev ppp0 src 192.168.1.1
157.238.135.90 from 192.168.1.20 tos 0x10 via 10.14.1.1 dev ppp0 src
192.168.1.1

My question is: why does this happen? what can I do to fix it?

Thanks in advance!

Here's some information from my router:

# ip route
10.14.1.1 dev ppp0 scope link src 151.203.160.233
207.180.31.136/29 dev eth0 scope link src 207.180.31.140
192.168.1.0/24 dev eth3 scope link src 192.168.1.1
10.0.0.0/16 dev eth2 scope link src 10.0.0.1
127.0.0.0/8 dev lo scope link
default
nexthop via 207.180.31.137 dev eth0 weight 2
nexthop via 10.14.1.1 dev ppp0 weight 1

# ip rule show
0: from all lookup local
32764: from 151.203.160.233 lookup T2
32765: from 207.180.31.140 lookup T1
32766: from all lookup main
32767: from all lookup 253

# ip route show table T1
207.180.31.136/29 dev eth0 scope link src 207.180.31.140
192.168.1.0/24 dev eth3 scope link src 192.168.1.1
10.0.0.0/16 dev eth2 scope link src 10.0.0.1
127.0.0.0/8 dev lo scope link
default via 207.180.31.137 dev eth0 src 207.180.31.140

# ip route show table T2
10.14.1.1 dev ppp0 scope link src 151.203.160.233
192.168.1.0/24 dev eth3 scope link src 192.168.1.1
10.0.0.0/16 dev eth2 scope link src 10.0.0.1
127.0.0.0/8 dev lo scope link
default via 10.14.1.1 dev ppp0 src 151.203.160.233

Thanks Again,

-Josh

Hi Josh,

Can you post your `ifconfig -a` listing? Or maybe just post something like which networks are going through which interfaces? That would help me to more comprehend your networking setup. I'll definitely help out if I can.

Thanks,

Greg

Hey Greg, sure thing

eth1 is slaved to ppp0, so that's why it dosen't have an address.

I also chopped out the stuff with how much data was sent over which interface, as it was unrelated.

# ifconfig -a
eth0 Link encap:Ethernet HWaddr 00:04:5A:82:98:95
inet addr:207.180.31.140 Bcast:207.180.31.143 Mask:255.255.255.248
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

eth1 Link encap:Ethernet HWaddr 00:04:5A:82:99:05
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

eth2 Link encap:Ethernet HWaddr 00:04:5A:82:98:E2
inet addr:10.0.0.1 Bcast:10.0.255.255 Mask:255.255.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

eth3 Link encap:Ethernet HWaddr 00:04:5A:82:99:29
inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1

ppp0 Link encap:Point-to-Point Protocol
inet addr:151.203.160.233 P-t-P:10.14.1.1 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1492 Metric:1

Is there no route between 207.180.31.140 and 157.238.135.60?

If anything, you can add a static route to 157.238.135.60 through dev ppp0. I know this is probably not the solution you're looking for, but it should be a quick fix for the time being.

I'm sure you've tried to clear the routing cache? Are you running any kind of dynamic discovery protocol like RIP? With two Internet interfaces, you may want to look into that.

Greg

actually 157.238.135.60 was routable through the 'default' multipath route of eth0 and ppp0. I've actually recieved a pretty good solution from Julian Anastasov (he's written some patches for the linux kernel to help with routing) I'll copy and paste his reply here:

Hello Joshua,

On Tue, 4 May 2004, Joshua Szmajda wrote:

>> I have a question I thought you could answer. I have a linux router,
>> configured with two internet connections and two lan segments. I've
>> setup multipath routing as described in
>> http://lartc.org/howto/lartc.rpdb.multiple-links.html
>>
>> My problem (I think) is that somehow the router will randomly choose
>> incorrect routing paths for different hosts, for example:
>> on my workstation (192.168.1.20), I ssh to a server I have on an
>> external network (157.238.135.60), and my connection locally hangs. On
>> the router, I search the routing cache:

Yes, Netfilter NAT does not use correctly the routing
and does not bind each NAT connection to particular path (nexthop).
As result, a routing cache flush or entry expiration leads to
rerouting which can select another path. ssh can further exploit
this by providing different TOS values during the connection life
causing different packets to use different paths.

So, all current kernels are unusable in their default code, you
need the extra CONNMARK netfilter module or the "routes" patch.

>> My question is: why does this happen? what can I do to fix it? Would
>> your patches help?

Yes, the patches can help, you can check nano.txt, you
will need some user space script(s) that will monitor the gateways'
status.

Regards

--
Julian Anastasov


So I'm going to try his patches to the kernel and see if that helps. Thanks for your help!