RHEL 7.0, IPV6

Scenario:
I have routed specific network using network scripts.
1. "ip -6 route show" shows that route has been added. ( with metric 1024)
2. Ping of the specific IP through that route is successful.
3. Now after few days, for some reason, we see that cache route appears for that given destination IP with lower metric ( which is actually trying to take it through interface/bond of the default route.)Ping to that destination IP fails then because packet is going out of wrong interface.
4. If I delete that cache route, again the packet starts going out of the correct interface ( which is not interface of the default route) and ping is successful.

Has anyone faced similar issue with REHL. I am using RHEL 7
Questions:
Q1. In what situation OS will try to route through default, even if static route for specific IP/network exists.
Q2. How to configure the route so that OS does not fall back on default route. Please note, we can not delete the default route.
Q3. Even if OS is creating cache route for some temporary situation, why is it not detecting that situation has normalized and why it is not deleting the automatically created cache route then. ( assuming that some temporary network side failure caused this issue )

Please note this issue observed is only for IPV6. We did not observe this issue for IPV4.
Can anyone suggest on how to permanently resolve this situation ?

Adding further specific details related to the problem.

Please see below output of ip -6 route show.

default route is

Code:

default via 2405:200:370:237::1 dev bond1 metric 1


specific route for network that was added:

Code:

2405:200:370:10::/64 via 2405:200:370:236::1 dev bond0.236 metric 1024


cache route added automatically by OS with metric 0 was as below.

Code:

2405:200:370:10::20 via 2405:200:370:237::1 dev bond1 metric 0 cache


This cache route caused packet to go out of default route instead of the specific route. Destination IP is not reachable from the default route. cache route had to be deleted manually for normal traffic to resume.
--------------------------------------------------------------------------



Code:

ip -6 route show
unreachable ::/96 dev lo metric 1024 error -101
unreachable ::ffff:0.0.0.0/96 dev lo metric 1024 error -101
unreachable 2002:a00::/24 dev lo metric 1024 error -101
unreachable 2002:7f00::/24 dev lo metric 1024 error -101
unreachable 2002:a9fe::/32 dev lo metric 1024 error -101
unreachable 2002:ac10::/28 dev lo metric 1024 error -101
unreachable 2002:c0a8::/32 dev lo metric 1024 error -101
unreachable 2002:e000::/19 dev lo metric 1024 error -101
2405:200:201:400:172:16:21:156 via 2405:200:370:243::1 dev bond3.243 metric 1024
2405:200:201:400:172:16:21:157 via 2405:200:370:243::1 dev bond3.243 metric 1024
2405:200:310:15::2a via 2405:200:370:240::1 dev bond2.240 metric 0 cache
2405:200:310:15::56 via 2405:200:370:240::1 dev bond2.240 metric 0 cache
2405:200:310:15::/64 via 2405:200:370:240::1 dev bond2.240 metric 1024
2405:200:311:230::11 via 2405:200:370:230::1 dev bond3.230 metric 0 cache
2405:200:311:230::/64 via 2405:200:370:230::1 dev bond3.230 metric 1024
2405:200:311:247::11 via 2405:200:370:243::1 dev bond3.243 metric 0 cache
2405:200:311:247::12 via 2405:200:370:243::1 dev bond3.243 metric 0 cache
2405:200:311:247::13 via 2405:200:370:243::1 dev bond3.243 metric 0 cache
2405:200:311:247::/64 via 2405:200:370:243::1 dev bond3.243 metric 1024
2405:200:314:170::/64 via 2405:200:370:236::1 dev bond0.236 metric 1024
2405:200:360:10::/64 via 2405:200:370:236::1 dev bond0.236 metric 1024
2405:200:370:10::20 via 2405:200:370:237::1 dev bond1 metric 0 cache
2405:200:370:10::/64 via 2405:200:370:236::1 dev bond0.236 metric 1024
2405:200:370:16::2a via 2405:200:370:240::1 dev bond2.240 metric 0 cache
2405:200:370:16::56 via 2405:200:370:240::1 dev bond2.240 metric 0 cache
2405:200:370:16::/64 via 2405:200:370:240::1 dev bond2.240 metric 1024
2405:200:370:228::/64 dev bond3.228 proto kernel metric 256
2405:200:370:230::/64 dev bond3.230 proto kernel metric 256
2405:200:370:231::/64 dev bond4 proto kernel metric 256
2405:200:370:234::/64 dev bond3.234 proto kernel metric 256
2405:200:370:236::/64 dev bond0.236 proto kernel metric 256
2405:200:370:237::/64 dev bond1 proto kernel metric 256
2405:200:370:238::/64 dev bond0.238 proto kernel metric 256
2405:200:370:239::/64 dev bond0.239 proto kernel metric 256
2405:200:370:240::/64 dev bond2.240 proto kernel metric 256
2405:200:370:242::/64 dev bond3.242 proto kernel metric 256
2405:200:370:243::13 dev bond3.243 metric 0 cache
2405:200:370:243::/64 dev bond3.243 proto kernel metric 256
2405:200:370:244::/64 dev bond2.244 proto kernel metric 256
2405:200:a10:fc82::/64 via 2405:200:370:243::1 dev bond3.243 metric 1024
unreachable 3ffe:ffff::/32 dev lo metric 1024 error -101
fe80::/64 dev bond0 proto kernel metric 256
fe80::/64 dev bond1 proto kernel metric 256
fe80::/64 dev bond2 proto kernel metric 256
fe80::/64 dev bond3 proto kernel metric 256
fe80::/64 dev bond4 proto kernel metric 256
fe80::/64 dev eno4 proto kernel metric 256
fe80::/64 dev ens1f3 proto kernel metric 256
fe80::/64 dev bond0.236 proto kernel metric 256
fe80::/64 dev bond0.238 proto kernel metric 256
fe80::/64 dev bond0.239 proto kernel metric 256
fe80::/64 dev bond2.240 proto kernel metric 256
fe80::/64 dev bond2.244 proto kernel metric 256
fe80::/64 dev bond3.228 proto kernel metric 256
fe80::/64 dev bond3.230 proto kernel metric 256
fe80::/64 dev bond3.234 proto kernel metric 256
fe80::/64 dev bond3.242 proto kernel metric 256
fe80::/64 dev bond3.243 proto kernel metric 256
fe80::/64 dev ens6f3 proto kernel metric 256
default via 2405:200:370:237::1 dev bond1 metric 1


----------------------------------------------------------
Configuration for the bond is as follows.


Code:

/etc/sysconfig/network-scripts ] cat route6-bond0.236
2405:200:370:10::/64 via 2405:200:370:236::1 dev bond0.236
2405:200:314:170::/64 via 2405:200:370:236::1 dev bond0.236
2405:200:360:10::/64 via 2405:200:370:236::1 dev bond0.236



Code:

/etc/sysconfig/network-scripts ] cat ifcfg-bond0.236
DEVICE=bond0.236
BOOTPROTO=none
ONBOOT=yes
VLAN=yes
IPV6INIT=yes
IPV6_AUTOCONF=no
IPV6ADDR=2405:200:370:236::12/64



Code:

/etc/sysconfig/network-scripts ] cat route6-bond1
via 2405:200:370:237::1 dev bond1



Code:

/etc/sysconfig/network-scripts ] cat ifcfg-bond1
DEVICE=bond1
BONDING_MASTER=yes
BONDING_OPTS="miimon=100 mode=1"
BOOTPROTO=none
ONBOOT=yes
IPADDR=10.71.167.102
NETMASK=255.255.255.240
IPV6INIT=yes
IPV6_AUTOCONF=no
IPV6ADDR=2405:200:370:237::12/64
IPV6_DEFAULTGW=2405:200:370:237::1

I'm not a networking pro. but route(1) should take the more specific route first and defaults after.

if it is not doing so, you may have found a bug (i.e., use a different route cashier). however first make sure you have a more specific route that works, test that it works

some daemons might re-order routes by "fastest route". if that's so, then "why is it the wrong route?" maybe it's the right one.

I have no clue about ipv6 but the first part of the address is hardware routing (decides which physical interface gets to the destination, used by telephone switches or by linux), and the second is "interpreted" (what interface the packet should arrive on)

for ipv6 if the physical part of the address is the same, then there's no rule which it should take (take the "fastest" that is up). that is unless routers are instructed to choose particular interfaces - which you already should have done if that's your goal.

for ipv6 there is some way request specific routing path along the whole circuit (from home to other place) as well as QoS (scheduling of timing and delivery size) - but I'm unsure what tools are used to set up the chain of ipv6 requests needed to do so.