Summary
I want to setup policy routing using fwmark on my router machine, and route some selected traffic over a vpn connection.
I use iptables to mark packets in the mangle chain both in OUTPUT and PREROUTING tables, in order to affect both local traffic and traffic incoming from the intranet.
I define a new routing table and rule so that all marked traffic will go via the vpn.
Everything works as expected when routing traffic generated from the intranet, but I have a problem with traffic generated by local processes, which is not rerouted after the mark is set in the mangle OUTPUT table.
More info
On my router machine I have 3 interfaces:
- lan0 facing the intranet
- wan0 facing the internet
- tun0 which is a VPN tun device
Code:
# ip addr
...other stuff omitted for simplicity
5: lan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether ea:10:be:99:32:94 brd ff:ff:ff:ff:ff:ff
inet 192.168.1.1/24 brd 192.168.1.255 scope global lan0
valid_lft forever preferred_lft forever
6: wan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether ea:10:be:99:32:94 brd ff:ff:ff:ff:ff:ff
inet 202.156.44.182/21 brd 202.156.47.255 scope global dynamic wan0
valid_lft 11216sec preferred_lft 11216sec
8: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN group default qlen 100
link/none
inet 10.8.8.6 peer 10.8.8.5/32 scope global tun0
valid_lft forever preferred_lft forever
For testing purpose, currently I am marking only traffic directed to a specific web site. Here below I only show the content of the tables associated with traffic generated by local processes, which is the problematic one. As you can see I also insert some logging to trace the packets.
Code:
# iptables -nvL OUTPUT -t mangle
Chain OUTPUT (policy ACCEPT 11075 packets, 1951K bytes)
pkts bytes target prot opt in out source destination
1 252 MARK all -- * * 0.0.0.0/0 37.9.239.33 MARK or 0x4
1 252 LOG all -- * * 0.0.0.0/0 37.9.239.33 LOG flags 0 level 4 prefix "MANGLE "
# iptables -nvL OUTPUT -t nat
Chain OUTPUT (policy ACCEPT 361 packets, 25074 bytes)
pkts bytes target prot opt in out source destination
1 84 LOG all -- * * 0.0.0.0/0 37.9.239.33 LOG flags 0 level 4 prefix "NAT "
# iptables -nvL OUTPUT -t filter
Chain OUTPUT (policy ACCEPT 11131 packets, 1961K bytes)
pkts bytes target prot opt in out source destination
1 252 LOG all -- * * 0.0.0.0/0 37.9.239.33 LOG flags 0 level 4 prefix "FILTER "
The routing table and rules are:
Code:
# ip rule list
0: from all lookup local
198: from all fwmark 0x4/0x4 lookup vpn
32766: from all lookup main
32767: from all lookup default
# ip route list table main
default via 202.156.40.1 dev wan0 proto dhcp src 202.156.44.182 metric 1024
10.8.8.5 dev vpn-nord-hk5 proto kernel scope link src 10.8.8.6
192.168.1.0/24 dev lan proto kernel scope link src 192.168.1.1
202.156.40.0/21 dev wan0 proto kernel scope link src 202.156.44.182
202.156.40.1 dev wan0 proto dhcp scope link src 202.156.44.182 metric 1024
# ip route list table vpn
default via 10.8.8.5 dev tun0 proto static src 10.8.8.6
192.168.1.0/24 dev lan0 proto static scope link src 192.168.1.1
When I ping 37.9.239.33 from the router machine, I get no reply. I would expect the packet to follow this path:
MANGLE:OUTPUT (to wan0) => NAT:OUTPUT (to wan0) => reroute to tun0 => FILTER:OUTPUT (to tun0)
Analysing the log I see the packet enter the mangle table routed for wan0 (as expected), but when it traverses the filter table it is still routed for wan0 (which is not expected). It seems that setting the mark does not trigger rerouting.
Code:
# dmesg --notime | tail -n 3
MANGLE IN= OUT=wan0 SRC=202.156.44.182 DST=37.9.239.33 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=27744 DF PROTO=ICMP TYPE=8 CODE=0 ID=2801 SEQ=1 MARK=0x4
NAT IN= OUT=wan0 SRC=202.156.44.182 DST=37.9.239.33 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=27744 DF PROTO=ICMP TYPE=8 CODE=0 ID=2801 SEQ=1 MARK=0x4
FILTER IN= OUT=wan0 SRC=202.156.44.182 DST=37.9.239.33 LEN=84 TOS=0x00 PREC=0x00 TTL=64 ID=27744 DF PROTO=ICMP TYPE=8 CODE=0 ID=2801 SEQ=1 MARK=0x4
The mark rule in the mangle output table is actually hit, as you can see from the counter in the iptables output above.
The routing rules seems to be working correctly, because everything works fine for traffic incoming from the intranet.
Questions
Any idea why rerouting is not triggered?
Is this the correct way to do policy based routing for traffic generated by local processes?
I am using a custom build kernel 4.2-rc8. Is there any special kernel compilation configuration flag I need to set to enable rerouting (I already have CONFIG_ADVANCED_ROUTING=y)?
Thanks
