Networking Dilemma involving multiple interfaces on different subnets
 

I have recently configured a server with 2 network interfaces. (eth0, eth1) These interfaces are each configured for a different vlan. Eth0 is configured to use a default gateway, while eth1 is configured to use a static route for the gateway. I can communicate with eth0 just fine, all day long. I have issues communicating with eth1. I thought I found the solution by disabling reverse path filtering, but that didn't quite do it. This made eth1 responsive, but not all the time. Eth1 is unresponsive unless I communicate with the machine over eth0. Something simple as a ping to eth0 will cause eth1 to respond. I'm pretty stumped here, and looking for help. I shouldn't have to ping eth0 to cause eth1 to respond. I'm scratching my head and looking for help.

I have other linux servers with a single interface on vlan1 that have no problem pinging the troubled servers with the eth1 interface configured on vlan2. The problem seems to occur from outside the switch housing the vlans. I have access to the network via Cisco VPN, and the problem is evident there. Research I've done has led to disabling reverse path filtering and enabling arp filtering. These haven't alleviated the problem. I suspect the issue is outside the linux box, but I'm more into software than networking, and am looking for guidance.

Default gateway.

Which 'gateway'? Another gateway? or the one used by eth0?

How many routers are there and how many vlans? Unless short words will sketch the topography any one can hardly help.

Default gateway.

Which 'gateway'? Another gateway? or the one used by eth0?

How many routers are there and how many vlans? Unless short words will sketch the topography any one can hardly help.

The machines are located on a network that I don't have control of, or even a diagram yet, so I apologize for the lack of information. There are 2 gateways being used. Eth0 has an address on the 10.200.16.0 network and is configured to use the default gateway of 10.200.16.1. Eth1's interface has an address on the 10.200.15.0 network and has a gateway configured with a static route using the following route command. route add -net 10.200.15.0 netmask 255.255.255.0 gw 10.0.0.1 eth1. Both of these networks are vlans on a cisco switch of some sort. I connect to the network using vpn which gives me an address on a different network which is probably another vlan.

I did some research last night regarding multi-homing and may have found a solution. It's my understanding now that even though a gateway route is configured for each interface, there is no guarantee that traffic will come in or go out the same interface without policy routing in place. I followed some instructions from lartc to create a routing table and some rules to force traffic coming in and out of the 10.200.15.0 network to use the eth1 interface and correct gateway. Things seem to be working with this change.