Port-based routing

dcarrington · 05-25-2018, 02:43 PM

Hello,

I'm trying to do some port-based routing on a RHEL5 system that we still haven't been able to decommission.

The system has 2 physical NIC's. One is used for the application and the other is used for management (SSH, DNS, NTP, LDAP, etc.)

We have a monitoring software that tests server metrics by connecting via SSH and gathering stats on cpu, memory, storage and that sort of thing. This will connect over the management NIC.

The same monitoring server also connects over ports used by the application to verify that the application is responding properly.

Since the monitor comes from the same IP, I want to route SSH through the management NIC and all other traffic through the application NIC.

What I've done so far is this:

1. Default route is the gateway for the application network as found via 'ip route show all'

2. Added new route table ('mgmt') to /etc/iproute2/rt_tables

3. Added route entries to new 'mgmt' table

Code:

# ip route add 10.0.10.10 dev eth1 table mgmt
# ip route add default via 192.168.2.1 dev eth1 table mgmt

4. Added rule to route traffic to 'mgmt' table

Code:

# ip rule add fwmark 0x1 table mgmt

5. Mark packets going out over tcp 22 to my target system

Code:

# iptables -t mangle -A OUTPUT -p tcp -d 10.0.10.10 --sport 22 -j MARK --set-mark 0x1

6. Copy all but the DEFAULT route from the 'main' routing table into the new 'mgmt' table

Code:

# ip route show table main | grep -Ev ^default | while read ROUTE ; do ip route add table mgmt $ROUTE ; done

I've seen several sites that have indicated that this is the solution to what I'm trying to do. However, what I'm seeing is that ALL traffic going to 10.0.10.10 is going out through the management NIC rather than just SSH traffic.

Am I missing something? I don't get why all traffic is being routed when I'm only marking port 22 packets.

Thanks in advance!

AwesomeMachine · 05-29-2018, 04:00 PM

I'm sorry you haven't received a response. If you only use one client for management, you could set up a route to that specific host.

dcarrington · 05-29-2018, 05:15 PM

@AwesomeMachine

Thanks for the reply. I agree, that would certainly make this a lot easier. Unfortunately, I'm constrained by certain security practices that don't seem to allow for that.

I think we've actually gotten a little bit further by using the application IP address instead of hostname for non-SSH connections. Since the hostname resolves to the management address, I think that was part of the issue. There are some other monitors that are not working, but I'm wondering if they're still using the hostname. I'll update after the powers-that-be convert those to use IP address and let you know if that resolves the issue. I suspect that might be the issue.