I'm not sure what i'm attempting is possible but from what i've read so far it's plausable

what i'm attempting is to route all traffic going out on port 62010 via eth2 and all traffic to out on port 34010 on eth3. The destination ip is the same (10.71.241.121) (and reachable by both interfaces if you set the route)

eth0 10.175.29.52 MGMT
eth2 10.71.231.2/25 GW 10.71.231.222
eth3 10.71.241.3/25 GW 10.71.241.254
dst IP:10.71.241.121

What i've done is setup a routing table as follows

IP rules
[root@hyperion ~]# cat /etc/iproute2/rt_tables
#
# reserved values
#
255 local
254 main
253 default
0 unspec
#
# local
#
#1 inr.ruhep
2 eth2OUT
3 eth3OUT

[root@hyperion ~]# ip rule list
0: from all lookup local
32762: from all fwmark 0x2 lookup eth3OUT
32763: from all fwmark 0x1 lookup eth2OUT
32766: from all lookup main
32767: from all lookup default

[root@hyperion ~]# ip route show table eth3OUT
10.71.241.121 via 10.71.241.254 dev eth3
default via 10.71.241.254 dev eth3
[root@hyperion ~]# ip route show table eth2OUT
10.71.241.121 via 10.71.231.222 dev eth2
default via 10.71.231.222 dev eth2


IPtables
789 iptables -A OUTPUT -p tcp --dport 62010 -t mangle -j MARK --set-mark 1
790 iptables -A OUTPUT -p tcp --dport 34010 -t mangle -j MARK --set-mark 2

[root@hyperion ~]# iptables -t mangle -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination

Chain INPUT (policy ACCEPT)
target prot opt source destination

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
MARK tcp -- anywhere anywhere tcp dpt:62010 MARK set 0x1
MARK tcp -- anywhere anywhere tcp dpt:34010 MARK set 0x2

Chain POSTROUTING (policy ACCEPT)
target prot opt source destination

pretty sure thats all the config i need but still not joy:-



[root@hyperion ~]# telnet 10.71.241.121 62010
Trying 10.71.241.121...
telnet: connect to address 10.71.241.121: Connection timed out
[root@hyperion ~]# telnet 10.71.241.121 34010
Trying 10.71.241.121...
telnet: connect to address 10.71.241.121: Connection timed out

TCPDUMP
Still out via the default route i.e the MGMt address
07:45:34.161371 IP 10.175.29.52.44567 > 10.71.241.121.62010: Flags [S], seq 2992185250, win 14600, options [mss 1460,nop,nop,TS val 3536716705 ecr 0,nop,wscale 10], length 0
07:46:12.440551 IP 10.175.29.52.45850 > 10.71.241.121.34010: Flags [S], seq 2237464079, win 14600, options [mss 1460,nop,nop,TS val 3536754984 ecr 0,nop,wscale 10], length 0

Can anyone see what i'm missing here?

ok i've found the bit of config i was missing :-

/etc/sysctl.conf

net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.all.rp_filter = 2
net.ipv4.ip_forward = 1

for f in /proc/sys/net/ipv4/conf/*/rp_filter; do echo 0 >| $f ; done

So it appears to be working now however the source always appears to be the eth0 MGMT address

Go read up on POSTROUTING / SNAT

Also, keep in mind that you should set the 'src' in your routing tables.

Thanks, I've added the following lines to resolve the final issue

iptables -A POSTROUTING -p tcp -m tcp --dport 34010 -j SNAT --to-source 10.71.241.121
iptables -A POSTROUTING -p tcp -m tcp --dport 64010 -j SNAT --to-source 10.71.241.121

Can I ask a question, what do you plan on accomplishing with this setup? You should never have 2 interfaces on the same network unless they are port-channeled somehow. the switch spanning-tree should shut one of those ports off otherwise.

i dont have 2 interfaces on the same network

You are correct sir. Sorry.

What was the fix since you marked this as resolved?