Routing problem?

eqxro · 03-12-2005, 05:27 PM

I've got a router with 3 (three NICs), 2 that connect to 2 different ISPs and one for the LAN. I let the lan users choose their preferred ISP (speed/reliability trade-off). A few days ago I had a problem, that was that anyone going out on eth1 (eth2 is LAN) would get timeouts, but when switching to the second ISP (on eth0) would work all right...

Somehow I got it to work again by doing some interface restart... Today, I stumbled upon the same problem and this time it won't go away... If the router's default route is via eth1, I can ping out wherever I want from the server. Any LAN user would get timeouts and this would be reported by tcpdump -i eth1 'icmp':

Code:

01:07:09.469349 IP localhost > thorin.mediasat.ro: icmp 64: echo request seq 84
01:07:09.481696 IP thorin.mediasat.ro > localhost: icmp 64: echo reply seq 84
01:07:09.481784 IP localhost > thorin.mediasat.ro: icmp 92: time exceeded in-transit
01:07:10.469194 IP localhost > thorin.mediasat.ro: icmp 64: echo request seq 85
01:07:10.483333 IP thorin.mediasat.ro > localhost: icmp 64: echo reply seq 85
01:07:10.483434 IP localhost > thorin.mediasat.ro: icmp 92: time exceeded in-transit
01:07:11.468981 IP localhost > thorin.mediasat.ro: icmp 64: echo request seq 86
01:07:11.479700 IP thorin.mediasat.ro > localhost: icmp 64: echo reply seq 86
01:07:11.479791 IP localhost > thorin.mediasat.ro: icmp 92: time exceeded in-transit
01:07:12.468799 IP localhost > thorin.mediasat.ro: icmp 64: echo request seq 87
01:07:12.498854 IP thorin.mediasat.ro > localhost: icmp 64: echo reply seq 87
01:07:12.498952 IP localhost > thorin.mediasat.ro: icmp 92: time exceeded in-transit
01:07:13.468633 IP localhost > thorin.mediasat.ro: icmp 64: echo request seq 88
01:07:13.480465 IP thorin.mediasat.ro > localhost: icmp 64: echo reply seq 88
01:07:13.480558 IP localhost > thorin.mediasat.ro: icmp 92: time exceeded in-transit
01:07:14.468896 IP localhost > thorin.mediasat.ro: icmp 64: echo request seq 89
01:07:14.486021 IP thorin.mediasat.ro > localhost: icmp 64: echo reply seq 89
01:07:14.486117 IP localhost > thorin.mediasat.ro: icmp 92: time exceeded in-transit
01:07:15.469653 IP localhost > thorin.mediasat.ro: icmp 64: echo request seq 90
01:07:15.477948 IP thorin.mediasat.ro > localhost: icmp 64: echo reply seq 90
01:07:15.478041 IP localhost > thorin.mediasat.ro: icmp 92: time exceeded in-transit

...while tcpdump -i eth2 'icmp' (LAN) would return this...

Code:

01:07:09.469282 IP Alexx > thorin.mediasat.ro: icmp 64: echo request seq 84
01:07:10.469099 IP Alexx > thorin.mediasat.ro: icmp 64: echo request seq 85
01:07:11.468911 IP Alexx > thorin.mediasat.ro: icmp 64: echo request seq 86
01:07:12.468726 IP Alexx > thorin.mediasat.ro: icmp 64: echo request seq 87
01:07:13.468560 IP Alexx > thorin.mediasat.ro: icmp 64: echo request seq 88
01:07:14.468828 IP Alexx > thorin.mediasat.ro: icmp 64: echo request seq 89
01:07:15.469572 IP Alexx > thorin.mediasat.ro: icmp 64: echo request seq 90

I've got a gentoo router with a shorewall firewall, and I'd like to know what can cause the "time exceeded in-transit" message...

eqxro · 03-14-2005, 12:23 PM

Okay, I solved this, it seems my ISP sent back al the packets with TTL=1 and they couldn't be forwarded anymore from my router (it would die on the server). I had to patch my kernel to be able to do something like iptables -t mangle -A PREROUTING -i eth1 -j TTL --ttl-set 64. The patch is patch-o-matic-ng, the TTL part only.