Hello everyone.
I have a weird issue with source routing on a linux box.
The plan goes like this:
I have 2 internet providers at my office and i want to use a single router to route them both ( i don't need load balancing or failover). I just want access to either provider based on the ip i use on my pc.

The first provider, let's call it RDS, is simple: i've got an RDS_IP, RDS_MASK and RDS_GW.
The second provider is complicated, we'll call it INES. I have a INES_IP, INES_MASK, INES_GW and they also gave me a subnet of public ip's: ILAN_NET which i have to route myself through INES_IP.

I also have a third nic with an local ip: LAN_NET and an alias for the INES subnet: ILAN_NET

the router has dhcp enabled, giving by default ip's from LAN_NET and using the default gw, RDS

I have snat for the LAN_NET to go through RDS.

If i enter an ip from the ILAN_NET, instead of routing in through the INES_GW, it also goes through RDS_GW.

the routing i've used for about 5 months has worked perfectly untill one day, when it just stopped. this is my setup:

ip route add $RDS_MASK dev $RDS_IF src $RDS_IP table rds
ip route add default via $RDS_GW table rds

ip route add $INES_MASK dev $INES_IF src $INES_IP table ines
ip route add default via $INES_GW table ines

ip route add default via $RDS_GW

ip rule add from $RDS_IP table rds
ip rule add from $INES_IP table ines

ip route add $LAN_NET dev $LAN_IF table rds
ip route add 127.0.0.0/8 dev lo table rds

ip route add $ILAN_NET dev $ILAN_IF table ines
ip route add 127.0.0.0/8 dev lo table ines

what puzzles me the most is that this setup has worked, and now it doesn't .... without any changes on the router.
I've tried everything save for a format/reinstall.

Is there a better/different way to do this?
Thanks.

do you have any traceroute, ping, arp, route information available to us? Is the box outputting anything (tcpdump)?

Can you reach any interface on the hardware router from the linux box? Are the link lights on?

The fact that it just stopped working out of nowhere would force me to check the obvious physical connections and such (cables, links, ports, etc...)

well, i have checked the hardware stuff.
The cables, the connection to both isp, the NIC's are all ok because the RDS uplink works fine, the INES uplink works if i plug it in another computer.
I can ssh on both eth0 and eth1 ip's from outside.

As i've said, if i traceroute from my pc, in which i've put a INES ip from the subnet i have, the pacakges go throuh rds gateway.

Let me give you the figures ( for obvious reasons i will use generic ip's ):

auto eth0 (this is the rds uplink)
iface eth0 inet static
address 1.1.1.97
netmask 255.255.255.0

auto eth1 (this is the ines uplink)
iface eth1 inet static
address 2.2.2.234
netmask 255.255.255.128

auto eth2 (this is the LAN interface)
iface eth2 inet static
address 192.168.1.1
netmask 255.255.255.0

auto eth2:1 (this is the INES subnet routed through the ines uplink)
iface eth2:1 inet static
address 3.3.3.145
netmask 255.255.255.248

The ip routes are in the first post.

router:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
3.3.3.144 0.0.0.0 255.255.255.248 U 0 0 0 eth2
2.2.2.128 0.0.0.0 255.255.255.128 U 0 0 0 eth1
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2
1.1.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
0.0.0.0 1.1.1.1 0.0.0.0 UG 0 0 0 eth0

router:~# ip route show
3.3.3.144/29 dev eth2 proto kernel scope link src 3.3.3.145
2.2.2.128/25 dev eth1 proto kernel scope link src 2.2.2.234
192.168.1.0/24 dev eth2 proto kernel scope link src 192.168.1.1
1.1.1.0/24 dev eth0 proto kernel scope link src 1.1.1.97
default via 1.1.1.1 dev eth0

Could this problem be related to a cache of the routes?

The fast solution would be to reinstall the whole system and be done with it. But if this happens again after 5-6 months... i don't want to make a habbit out of it