UID based routing doesn't work

eranga · 05-26-2013, 10:53 PM

I have an Ubuntu 12.04 LTS server with two NICs. They are connected to different networks.

Code:

# ip route list table main
default via 192.168.1.254 dev eth0 
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.30 
192.168.2.0/24 dev eth1  proto kernel  scope link  src 192.168.2.30

I created another table for the other network (VPN)

Code:

# ip route list table VPN
default via 192.168.2.1 dev eth1 
192.168.2.0/24 dev eth1  proto kernel  scope link  src 192.168.2.30

Then configured iptables to mark the packets from UID 108.

Code:

#iptables -t mangle -A OUTPUT -m owner --uid-owner 108 -j MARK --set-mark 1
#iptables-save > /etc/iptables.up.rules
#iptables-restore < /etc/iptables.up.rules
#iptables -t mangle -nvL OUTPUT
Chain OUTPUT (policy ACCEPT 18544 packets, 3026K bytes)
pkts bytes target prot opt in  out  source   destination         
17139 2734K MARK  all  --  *  *   0.0.0.0/0  0.0.0.0/0   owner UID match 108 MARK set 0x1

Finally created a rule as below.

Code:

#ip rule add fwmark 1 pri 201 table VPN
#ip rule list
0:  from all lookup local 
201:    from all fwmark 0x1 lookup VPN 
32766:  from all lookup main 
32767:  from all lookup default

But the packets from UID 108 still go through eth0! Why?

Code:

$tracepath -n google.com
1:  192.168.1.30                                          0.088ms pmtu 1500
1:  192.168.1.1                                           0.623ms 
1:  192.168.1.1                                           0.560ms 
2:  192.168.1.1                                           0.362ms pmtu 1400

Please help.

Ygrex · 05-28-2013, 08:05 AM

first of all it should work:

Code:

$ iptables -t mangle -A OUTPUT -m owner --uid-owner 33 -j MARK --set-mark 1
$ echo '8 www-data' >>/etc/iproute2/rt_tables
$ ip rule add fwmark 1 table www-data
$ ip route add unreachable 173.194.71.138/32 table www-data
$ sudo -u www-data ping -c1 173.194.71.138
PING 173.194.71.138 (173.194.71.138) 56(84) bytes of data.
ping: sendmsg: Operation not permitted
$ ping -c1 173.194.71.138
PING 173.194.71.138 (173.194.71.138) 56(84) bytes of data.
64 bytes from 173.194.71.138: icmp_seq=1 ttl=52 time=5.33 ms

the second, source address is not necessarily modified to 192.168.2.30 when rerouting after mangle/OUTPUT, it is better to set source address explicitly

then, tracepath is not the best utility to trace outgoing interface; can you check it with some sniffer tool?