I am running fedora 12 (kernel-2.6.31.5-127) on a VM that is acting as my boundary router. It is connected via 3 interfaces, eth0 connects to my firewall, eth1 to IPS1 and eth2 to ISP2. I advertise my internal network using quagga (bgp) to both ISP1 and ISP2.

eth1 is the default route.

I have a scenario where traffic to host A leaves via eth1 and returns via eth2. All packets should be forwarded to eth0. But they are not. It seems that something blocks the return packets. If I disable the eth2 route, packets enter and leave via eth1.

I am not running any packet filtering on the router so can not work out what I have to do to get this to work. The whole point of being multi-homed was to allow traffic to use either ISP. Having the firewall on another server attached to eth0 was to avoid state issues etc and allow asymmetry to work.

Any help would be appreciated.

TCP probable never will work like that, UDP may be.
But any way, when you (any) packets leaves NIC, they enter ROUTER, which remember from what IP it received packet for any connection. I do not think it will be possible to tell it when packets comes from IP A, send back to IP B. You can do it only if you change sender IP in packet header.

Unfortunately, you haven't actually understood what I wrote ( or I didn't explain it correctly ). I have been involved with networking and routers for years and it is supposed to work this way.

A packet comes from one of my servers located on the other side of my firewall and enters the router on eth0, the default route will then send the packet out on eth1. Depending on where the destination is and how they are connected, the reply may come back on either eth1 or eth2 ( it isn't supposed to matter ). The router is then supposed to forward the packed back to the server on eth0.

For some reason, this isn't happening. If the reply comes back on eth2 it doesn't get forwarded to eth0 when I believe it should. The router should accept a packet on any of its interfaces, it should apply the routing rules and retransmit the packet on the inetrface that will get the packet closer to their destination.

Well this one turned out to be caused by the open-vm-tools vxnet driver as well. Having reverted to the default PCnet32 driver, the packet flows have returned to what the had been - asymmetric, and are working fine.

Nevertheless, all of the debugging and investgation was a good exercise and I think I will migrate to Vyatta in the new year.