LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices

Reply
 
Search this Thread
Old 12-16-2010, 02:04 PM   #1
fragtion
LQ Newbie
 
Registered: Dec 2010
Posts: 5

Rep: Reputation: 0
Exclamation split routing - traceroute not displaying unrouted hops


Hi guys,
Firstly, I'm new here... I've used linuxquestions.org from time to time, but never needed to register until now... hopefully someone can help...

An in-depth explanation of this issue is already described over here: http://newyork.ubuntuforums.org/show...4#post10241054 , so I will merely quote it again here - it seems nobody knows the solution over at Ubuntu's forums...:
Quote:
I recently switched my home server from debian lenny to ubuntu maverick. I've managed to port all my configs and stuff and so far I'm very happy.
There's one tiny thing that's griping me, that I never experienced before with debian's (older) packages/configuration...

Here's the situation:
My server dials up 2 pppoe (adsl) interfaces (different isp's) with split internet routes.

If I run a general traceroute to an internet IP, all the hops which are not routed via the same interface as the destination host/IP, will appear as "* * *" in the traceroute. This was never the case before and it would be preferred to see the IP's of all routers along the way regardless of whether they are routed... (I used to be able to see IP's like 10.0.0.x before through INTERNET traceroutes if they were hops along the way [IP's which would be unreachable if traced directly], and that's no longer the case) - isn't this kinda defeating the point of traceroute?

Here's an example:
Only ppp0 up, it is default gateway
Code:
root@pappasfw:/etc/ppp/peers# traceroute 196.24.33.35
traceroute to 196.24.33.35 (196.24.33.35), 30 hops max, 60 byte packets
 1  dsl-242-108-01.telkomadsl.co.za (41.242.108.1)  5.233 ms  6.335 ms  7.786 ms
 2  196.43.11.202 (196.43.11.202)  9.428 ms  10.642 ms  11.826 ms
 3  196.43.25.138 (196.43.25.138)  13.242 ms  14.213 ms  15.622 ms
 4  internet-solutions-gw.telkom-ipnet.co.za (196.25.46.90)  16.569 ms  18.000 ms  18.957 ms
 5  196.26.0.10 (196.26.0.10)  82.462 ms 168.209.1.170 (168.209.1.170)  82.237 ms 196.26.0.10 (196.26.0.10)  82.483 ms
 6  * * *
 7  jnb2-vl700-jnb1-vl700.net.tenet.ac.za (155.232.6.18)  21.466 ms  7.138 ms  6.714 ms
 8  196.24.33.35 (196.24.33.35)  8.092 ms * *
root@pappasfw:/etc/ppp/peers#
now ppp1 goes up, and I route 196.43.11.202 through it, arbitrarily for the sake of demonstration, as it was the second hop before
Code:
root@pappasfw:/etc/ppp/peers# ip route add 196.43.11.202 dev ppp1
root@pappasfw:/etc/ppp/peers# traceroute 196.24.33.35
traceroute to 196.24.33.35 (196.24.33.35), 30 hops max, 60 byte packets
 1  dsl-242-108-01.telkomadsl.co.za (41.242.108.1)  5.842 ms  7.152 ms  7.226 ms
 2  * * *
 3  196.43.25.138 (196.43.25.138)  13.169 ms  14.593 ms  15.553 ms
 4  internet-solutions-gw.telkom-ipnet.co.za (196.25.46.90)  17.221 ms  18.239 ms  19.673 ms
 5  168.209.1.170 (168.209.1.170)  51.166 ms 196.26.0.10 (196.26.0.10)  51.356 ms  51.574 ms
 6  * * *
 7  jnb2-vl700-jnb1-vl700.net.tenet.ac.za (155.232.6.18)  13.866 ms  13.241 ms  13.493 ms
 8  196.24.33.35 (196.24.33.35)  12.997 ms * *
root@pappasfw:/etc/ppp/peers#
Perhaps if I explicitly specified that the trace should be done from a specific interface, then this behaviour could be expected, but if it's just a general straightforward traceroute <ip> then you'd expect to see all the IP addresses if they are actually reachable at all and they constitute the hops

Also, all computers who use this server as a gateway (including windows machines) also experience this 'firewall effect' since the switch from debian 5 to ubuntu 10.10, so I doubt it's a traceroute-specific problem but rather a routing or kernel configuration?
Any pointers or feedback about this would be great... I'm guessing this could be resolved using iptables? Well I'm not all that clued up with iptables, hence my pleading for assistance =)

Thanks!
 
Old 12-16-2010, 03:36 PM   #2
kbp
Senior Member
 
Registered: Aug 2009
Posts: 3,758

Rep: Reputation: 643Reputation: 643Reputation: 643Reputation: 643Reputation: 643Reputation: 643
Seeing "*"'s during a traceroute usually indicates that ICMP is filtered, I wouldn't necessarily expect to see every hop
 
Old 12-17-2010, 06:09 AM   #3
fragtion
LQ Newbie
 
Registered: Dec 2010
Posts: 5

Original Poster
Rep: Reputation: 0
Thanks for the response
This is not a problem with external filtering then, the filtering must logically be taking place on my local machine if another distro doesn't show a 'filtered' output on the same exact connection? I'd like to know how to change this behaviour if anybody has any idea...

Last edited by fragtion; 12-17-2010 at 06:10 AM.
 
Old 12-17-2010, 04:15 PM   #4
kbp
Senior Member
 
Registered: Aug 2009
Posts: 3,758

Rep: Reputation: 643Reputation: 643Reputation: 643Reputation: 643Reputation: 643Reputation: 643
You also need to remember that ICMP is a low priority protocol, if a router is under heavy load it may choose not to respond to ICMP packets
 
Old 12-20-2010, 01:58 PM   #5
fragtion
LQ Newbie
 
Registered: Dec 2010
Posts: 5

Original Poster
Rep: Reputation: 0
Thanks again for the feedback, but I don't know why you consider that a relevant explanation if you've read and understood the described problem...
I explained that the hops are only 'non-responsive' when that particular hop IP is routed elsewhere or is unrouted, and that this was not the case with previous linux distributions that I've used

Let me try explain again... The behaviour goes explicitly as follows: If any of the hops is not routed via the same gateway/interface as the destination IP, no information about that hop will be displayed at all (asterix's will be displayed in that hop's place). That's all I'm saying, and that this was not the case previously for me (when using debian lenny) - before I'd even see the ISP's Private Class internal router IP's (which are not routed from my end) show up in the traceroute, and that's no longer the case. This is annoying because traceroutes take a long time now because as a result of my split routing, some of the intermediary hops are part of a subnet that routes via another interface and therefore are not displaying (and hence in turn, hanging/delaying the rest of the trace) when they would have before...

I'm sure there must be a simple (iptables-based?) solution... I'm just intruiged that nobody has any idea yet, neither here or on the ubuntu forums. As I said, this is not a traceroute-specific problem, but a distribution issue, as all clients that lie behind this server (router), including windows machines, are now also experiencing this undesired 'phantom firewall' [so to speak] effect on traces, seemingly regardless of whether it's a UDP or ICMP trace

Last edited by fragtion; 12-20-2010 at 02:02 PM.
 
Old 12-20-2010, 04:44 PM   #6
kbp
Senior Member
 
Registered: Aug 2009
Posts: 3,758

Rep: Reputation: 643Reputation: 643Reputation: 643Reputation: 643Reputation: 643Reputation: 643
Could you go through the process again but show the routing table at each stage ?
 
Old 12-20-2010, 06:35 PM   #7
fragtion
LQ Newbie
 
Registered: Dec 2010
Posts: 5

Original Poster
Rep: Reputation: 0
Sure, although surely there's already enough information provided? Well, as requested:

We start off with just one pppoe (internet) interface up - ppp0
Quote:
ppp0 Link encap:Point-to-Point Protocol
inet addr:41.144.73.X P-t-P:41.144.72.1 Mask:255.255.255.255
Quote:
# ip route
41.144.72.1 dev ppp0 proto kernel scope link src 41.144.73.X
172.16.6.0/27 dev eth0 proto kernel scope link src 172.16.6.1
default dev ppp0 scope link
default via 172.16.6.2 dev eth0 metric 100
Let's do a traceroute...
Quote:
# traceroute www.tenet.ac.za
traceroute to www.tenet.ac.za (155.232.135.33), 30 hops max, 60 byte packets
1 dsl-144-72-01.telkomadsl.co.za (41.144.72.1) 5.750 ms 6.854 ms 7.797 ms
2 rrba-ip-lir-1-gig-3-0-0.telkom-ipnet.co.za (196.43.11.134) 9.732 ms 10.983 ms 11.888 ms
3 196.43.25.138 (196.43.25.138) 13.054 ms 14.551 ms 15.407 ms
4 internet-solutions-gw.telkom-ipnet.co.za (196.25.46.90) 17.334 ms 18.286 ms 19.712 ms
5 196.26.0.10 (196.26.0.10) 22.875 ms 23.339 ms 168.209.1.170 (168.209.1.170) 23.508 ms
6 tenet.jinx.net.za (198.32.142.33) 24.479 ms 19.577 ms 20.605 ms
7 cpt1-t101-bfn1-t100.net.tenet.ac.za (155.232.6.38) 44.041 ms 42.524 ms 43.018 ms
8 unknown.uni.net.za (155.232.137.249) 43.503 ms virtsys2.tenet.ac.za (155.232.135.10) 29.700 ms 30.136 ms
#
Great. Now let's put up ppp1, an account with another ISP
Quote:
# ifup ppp1
Plugin rp-pppoe.so loaded.
RP-PPPoE plugin version 3.8p compiled against pppd 2.4.5
ssh stop/waiting
ssh start/running, process 2697
Quote:
ppp0 Link encap:Point-to-Point Protocol
inet addr:41.144.73.X P-t-P:41.144.72.1 Mask:255.255.255.255
ppp1 Link encap:Point-to-Point Protocol
inet addr:196.209.9.Y P-t-P:196.209.9.1 Mask:255.255.255.255
Quote:
# ip route
196.209.9.1 dev ppp1 proto kernel scope link src 196.209.9.Y
41.242.108.1 dev ppp0 proto kernel scope link src 41.242.118.X
172.16.6.0/27 dev eth0 proto kernel scope link src 172.16.6.1
default dev ppp0 scope link
default via 172.16.6.2 dev eth0 metric 100
Ok, now let's say I route a subnet via ppp1 which could be reachable via both ppp0 or ppp1, but that I'd hypothetically have routed via ppp1 because it belongs to that respective ISP, or because reaching it would take a shorter path via this ISP.
Quote:
# ip route add 196.26.0.0/24 dev ppp1
^ Well what do you know! Notice that the IP of the 5th hop of the first traceroute (196.26.0.10) is part of the subnet I just added to ppp1
Quote:
# ip route
196.209.9.1 dev ppp1 proto kernel scope link src 196.209.9.Y
41.144.72.1 dev ppp0 proto kernel scope link src 41.144.73.X
172.16.6.0/27 dev eth0 proto kernel scope link src 172.16.6.1
196.26.0.0/24 dev ppp1 scope link
default dev ppp0 scope link
default via 172.16.6.2 dev eth0 metric 100
Now I do a traceroute to the very same IP as before, which will again be routed via default (ppp0), this time with one of the hops (hop 5) being routed via another interface (ppp1)
Quote:
# traceroute www.tenet.ac.za
traceroute to www.tenet.ac.za (155.232.135.33), 30 hops max, 60 byte packets
1 dsl-144-72-01.telkomadsl.co.za (41.144.72.1) 6.254 ms 7.429 ms 8.614 ms
2 rrba-ip-lir-1-gig-3-0-0.telkom-ipnet.co.za (196.43.11.134) 10.567 ms 11.612 ms 12.972 ms
3 196.43.25.138 (196.43.25.138) 13.941 ms 15.390 ms 16.585 ms
4 internet-solutions-gw.telkom-ipnet.co.za (196.25.46.90) 18.006 ms 19.212 ms 20.430 ms
5 * * *
6 tenet.jinx.net.za (198.32.142.33) 25.505 ms 20.111 ms 20.805 ms
7 cpt1-t101-bfn1-t100.net.tenet.ac.za (155.232.6.38) 43.798 ms 42.267 ms 42.947 ms
8 virtsys2.tenet.ac.za (155.232.135.10) 42.967 ms 29.634 ms 30.157 ms
#
Notice how hop 5 shows as filtered/unreachable -- this is the behaviour I'm describing

This may arguably be intended or expected behaviour now, but the point I'm trying to make is that it was not like this before. In fact, as I said, I'd often even see private LAN class subnets appear in the middle of a traceroute indicating that the ISP is internally routing via private class subnets (ie, something like 10.5.2.1) which, if I explicitly tried to trace those hops directly, I'd get an instant unreachable error. Of course now that IP wont even display in the first place, like it used to, because it's not typically routed/reachable on this fussy ubuntu server! xD

In other words, it seems more natural that if the destination is routed and reachable via one interface, that traceroute should show the whole hop path irrespective of whether some/all of the constituent intermediary hops may be routed via another interface. It's as if it's taking the entire routing table into account for each hop, even if it's not going to end up using/routing via those other interfaces to reach the destination anyway in reality, and even though the hops are still an integral part of actual IP traffic via that route... so it should show these hops anyway

Thanks

Last edited by fragtion; 12-20-2010 at 09:32 PM.
 
Old 12-20-2010, 09:30 PM   #8
kbp
Senior Member
 
Registered: Aug 2009
Posts: 3,758

Rep: Reputation: 643Reputation: 643Reputation: 643Reputation: 643Reputation: 643Reputation: 643
I see ... it may be because the 5th hop via ppp1 is not the 5th hop via ppp0 - you're basically creating a "shortcut" which sends packets for hop 5 out a different interface. You could try shutting down ppp0 and seeing what the hop count is .. but really, this is expected behaviour. Unless 196.26.0.10 happens to be 5 hops or less via ppp1 you won't get an answer.

hth
 
Old 01-21-2011, 12:37 AM   #9
fragtion
LQ Newbie
 
Registered: Dec 2010
Posts: 5

Original Poster
Rep: Reputation: 0
If anyone else stumbles upon this little pest, I found the solution (for my case at least)!:

rp_filter must be disabled for the relevant interfaces

echo "0" > /proc/sys/net/ipv4/conf/all/rp_filter

edit /etc/sysctl.conf:
Quote:
# Uncomment the next two lines to enable Spoof protection (reverse-path filter)
# Turn on Source Address Verification in all interfaces to
# prevent some spoofing attacks
net.ipv4.conf.default.rp_filter=0
net.ipv4.conf.all.rp_filter=0
and then reboot
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
rsync, keychain and ssh going 2 hops. drenalin23 Linux - Server 2 01-20-2010 05:50 PM
554 5.4.6 Too many hops spangler Linux - Server 2 09-30-2008 12:13 PM
scanner hops over between sg0 and sg2 hetOrakel Linux - Hardware 0 01-21-2007 04:34 PM
Scoll wheel not working properly hops to the right zipo13 Linux - Hardware 2 07-04-2005 03:40 PM
Hops counter in JAVA(urgent) nedian123 Programming 1 08-01-2004 04:30 PM


All times are GMT -5. The time now is 03:40 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration