LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Security (https://www.linuxquestions.org/questions/linux-security-4/)
-   -   IPTables NAT weirdness (https://www.linuxquestions.org/questions/linux-security-4/iptables-nat-weirdness-833872/)

codevyper 09-22-2010 05:20 PM
IPTables NAT weirdness
 
I have setup an OpenVPN gateway behind our SonicWall UTM device at our office and have resolved most everything except one last hurdle.

Here's the deal:

The DMZ segment is 10.0.10.0/24
The LAN segment is 10.0.11.0/24
The OpenVPN Server/Gateway sits at 10.0.10.2
The OpenVPN tunnel segment is 172.20.0.0/24
SonicWall is using X0 interface for LAN and X8 interface for DMZ
SonicWall has a permanent static route to 10.0.10.2 for all 172.20.0.0/24 bound packets coming from Firewalled Subnets.

I have the following IPTables rules currently on the OpenVPN Server:

# iptables -t filter -nvL
Chain INPUT (policy ACCEPT 12186 packets, 896K bytes)
pkts bytes target prot opt in out source destination
307K 63M ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED

Chain FORWARD (policy ACCEPT 42665 packets, 18M bytes)
pkts bytes target prot opt in out source destination
30475 2652K ACCEPT all -- eth0 tun0 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
25496 1074K ACCEPT all -- eth0 tun0 0.0.0.0/0 0.0.0.0/0

Chain OUTPUT (policy ACCEPT 203K packets, 23M bytes)
pkts bytes target prot opt in out source destination



# iptables -t nat -nvL
Chain PREROUTING (policy ACCEPT 5 packets, 309 bytes)
pkts bytes target prot opt in out source destination

Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
5 304 MASQUERADE all -- * eth0 172.10.0.0/24 0.0.0.0/0
4 288 MASQUERADE all -- * eth0 172.20.0.0/24 0.0.0.0/0
116 7352 MASQUERADE all -- * tun0 10.0.11.0/24 0.0.0.0/0
0 0 MASQUERADE all -- * tun0 10.0.10.0/24 0.0.0.0/0

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination


Now, if I try to access any of the machines connected via VPN from the 10.0.11.0/24 network, I get through and can access any service I like.

If I try to access any machine from the 10.0.10.0/24 network The connection never establishes.

Here's what I get with tcpdump at tun0 when I do ssh from a 10.0.11.0/24 terminal:

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun0, link-type RAW (Raw IP), capture size 65535 bytes
15:10:25.576776 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [S], seq 3861254507, win 65535, options [mss 1460,nop,wscale 3,nop,nop,TS val 994580580 ecr 0,sackOK,eol], length 0
15:10:26.487197 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [S], seq 3861254507, win 65535, options [mss 1460,nop,wscale 3,nop,nop,TS val 994580589 ecr 0,sackOK,eol], length 0
15:10:26.914063 IP 172.20.0.5.22 > 172.20.0.113.53246: Flags [S.], seq 1585038574, ack 3861254508, win 5792, options [mss 1352,sackOK,TS val 329954920 ecr 994580580,nop,wscale 6], length 0
15:10:26.914792 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [.], ack 1, win 65535, options [nop,nop,TS val 994580594 ecr 329954920], length 0
15:10:27.827913 IP 172.20.0.5.22 > 172.20.0.113.53246: Flags [S.], seq 1585038574, ack 3861254508, win 5792, options [mss 1352,sackOK,TS val 329955148 ecr 994580580,nop,wscale 6], length 0
15:10:27.828763 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [.], ack 1, win 65535, options [nop,nop,TS val 994580603 ecr 329955148], length 0
15:10:27.936341 IP 172.20.0.5.22 > 172.20.0.113.53246: Flags [P.], seq 1:33, ack 1, win 91, options [nop,nop,TS val 329955176 ecr 994580594], length 32
15:10:27.937443 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [.], ack 33, win 65535, options [nop,nop,TS val 994580604 ecr 329955176], length 0
15:10:27.937487 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [P.], seq 1:22, ack 33, win 65535, options [nop,nop,TS val 994580604 ecr 329955176], length 21
15:10:28.358993 IP 172.20.0.5.22 > 172.20.0.113.53246: Flags [.], ack 22, win 91, options [nop,nop,TS val 329955281 ecr 994580604], length 0
15:10:28.359772 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [P.], seq 22:814, ack 33, win 65535, options [nop,nop,TS val 994580608 ecr 329955281], length 792
15:10:28.360029 IP 172.20.0.5.22 > 172.20.0.113.53246: Flags [P.], seq 33:817, ack 22, win 91, options [nop,nop,TS val 329955281 ecr 994580604], length 784
15:10:28.360824 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [.], ack 817, win 65535, options [nop,nop,TS val 994580608 ecr 329955281], length 0
15:10:28.464241 IP 172.20.0.5.22 > 172.20.0.113.53246: Flags [.], ack 814, win 136, options [nop,nop,TS val 329955308 ecr 994580608], length 0
15:10:28.465023 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [P.], seq 814:838, ack 817, win 65535, options [nop,nop,TS val 994580609 ecr 329955308], length 24
15:10:28.484324 IP 172.20.0.5.22 > 172.20.0.113.53246: Flags [.], ack 838, win 136, options [nop,nop,TS val 329955312 ecr 994580609], length 0
15:10:28.486021 IP 172.20.0.5.22 > 172.20.0.113.53246: Flags [P.], seq 817:969, ack 838, win 136, options [nop,nop,TS val 329955313 ecr 994580609], length 152
15:10:28.486735 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [.], ack 969, win 65535, options [nop,nop,TS val 994580609 ecr 329955313], length 0
15:10:28.491713 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [P.], seq 838:982, ack 969, win 65535, options [nop,nop,TS val 994580610 ecr 329955313], length 144
15:10:28.526565 IP 172.20.0.5.22 > 172.20.0.113.53246: Flags [P.], seq 969:1689, ack 982, win 161, options [nop,nop,TS val 329955323 ecr 994580610], length 720
15:10:28.527396 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [.], ack 1689, win 65535, options [nop,nop,TS val 994580610 ecr 329955323], length 0
15:10:28.533702 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [P.], seq 982:998, ack 1689, win 65535, options [nop,nop,TS val 994580610 ecr 329955323], length 16
15:10:28.591881 IP 172.20.0.5.22 > 172.20.0.113.53246: Flags [.], ack 998, win 161, options [nop,nop,TS val 329955340 ecr 994580610], length 0
15:10:28.593000 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [P.], seq 998:1046, ack 1689, win 65535, options [nop,nop,TS val 994580611 ecr 329955340], length 48
15:10:28.617541 IP 172.20.0.5.22 > 172.20.0.113.53246: Flags [.], ack 1046, win 161, options [nop,nop,TS val 329955346 ecr 994580611], length 0
15:10:28.617836 IP 172.20.0.5.22 > 172.20.0.113.53246: Flags [P.], seq 1689:1737, ack 1046, win 161, options [nop,nop,TS val 329955346 ecr 994580611], length 48
15:10:28.618615 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [.], ack 1737, win 65535, options [nop,nop,TS val 994580611 ecr 329955346], length 0
15:10:28.618926 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [P.], seq 1046:1110, ack 1737, win 65535, options [nop,nop,TS val 994580611 ecr 329955346], length 64
15:10:28.646605 IP 172.20.0.5.22 > 172.20.0.113.53246: Flags [P.], seq 1737:1801, ack 1110, win 161, options [nop,nop,TS val 329955353 ecr 994580611], length 64
15:10:28.647434 IP 172.20.0.113.53246 > 172.20.0.5.22: Flags [.], ack 1801, win 65535, options [nop,nop,TS val 994580611 ecr 329955353], length 0


Now here's what I get from a 10.0.10.0/24 terminal:

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tun0, link-type RAW (Raw IP), capture size 65535 bytes
15:11:51.356033 IP 172.20.0.113.34157 > 172.20.0.5.22: Flags [S], seq 3655813192, win 5840, options [mss 1460,sackOK,TS val 2465250269 ecr 0,nop,wscale 9], length 0
15:11:52.180448 IP 172.20.0.5.22 > 172.20.0.113.34157: Flags [S.], seq 2927277448, ack 3655813193, win 5792, options [mss 1352,sackOK,TS val 329976235 ecr 2465250269,nop,wscale 6], length 0
15:11:52.181121 IP 10.0.10.222.34157 > 172.20.0.5.22: Flags [R], seq 2382746838, win 0, length 0
15:11:54.354929 IP 172.20.0.113.34157 > 172.20.0.5.22: Flags [S], seq 3655813192, win 5840, options [mss 1460,sackOK,TS val 2465251019 ecr 0,nop,wscale 9], length 0
15:11:55.152177 IP 172.20.0.5.22 > 172.20.0.113.34157: Flags [S.], seq 2927277448, ack 3655813193, win 5792, options [mss 1352,sackOK,TS val 329976978 ecr 2465250269,nop,wscale 6], length 0
15:11:55.152615 IP 10.0.10.222.34157 > 172.20.0.5.22: Flags [R], seq 2382746838, win 0, length 0
15:11:56.378263 IP 172.20.0.5.22 > 172.20.0.113.34157: Flags [S.], seq 2927277448, ack 3655813193, win 5792, options [mss 1352,sackOK,TS val 329977285 ecr 2465250269,nop,wscale 6], length 0
15:11:56.378980 IP 10.0.10.222.34157 > 172.20.0.5.22: Flags [R], seq 2382746838, win 0, length 0
15:12:00.355235 IP 172.20.0.113.34157 > 172.20.0.5.22: Flags [S], seq 3655813192, win 5840, options [mss 1460,sackOK,TS val 2465252519 ecr 0,nop,wscale 9], length 0
15:12:01.565932 IP 172.20.0.5.22 > 172.20.0.113.34157: Flags [S.], seq 2927277448, ack 3655813193, win 5792, options [mss 1352,sackOK,TS val 329978582 ecr 2465250269,nop,wscale 6], length 0
15:12:01.566510 IP 10.0.10.222.34157 > 172.20.0.5.22: Flags [R], seq 2382746838, win 0, length 0
15:12:02.377760 IP 172.20.0.5.22 > 172.20.0.113.34157: Flags [S.], seq 2927277448, ack 3655813193, win 5792, options [mss 1352,sackOK,TS val 329978785 ecr 2465250269,nop,wscale 6], length 0
15:12:02.378527 IP 10.0.10.222.34157 > 172.20.0.5.22: Flags [R], seq 2382746838, win 0, length 0
15:12:12.355297 IP 172.20.0.113.34157 > 172.20.0.5.22: Flags [S], seq 3655813192, win 5840, options [mss 1460,sackOK,TS val 2465255519 ecr 0,nop,wscale 9], length 0
15:12:13.661826 IP 172.20.0.5.22 > 172.20.0.113.34157: Flags [S.], seq 2927277448, ack 3655813193, win 5792, options [mss 1352,sackOK,TS val 329981605 ecr 2465250269,nop,wscale 6], length 0
15:12:13.662247 IP 10.0.10.222.34157 > 172.20.0.5.22: Flags [R], seq 2382746838, win 0, length 0
15:12:14.379418 IP 172.20.0.5.22 > 172.20.0.113.34157: Flags [S.], seq 2927277448, ack 3655813193, win 5792, options [mss 1352,sackOK,TS val 329981785 ecr 2465250269,nop,wscale 6], length 0
15:12:14.379929 IP 10.0.10.222.34157 > 172.20.0.5.22: Flags [R], seq 2382746838, win 0, length 0

It seems that the 3rd packet sent from 10.0.10.222 scraps MASQUERADING and from there on in the connection is foobared. I thought it may just be an issue with SSH but I get the same results with HTTP as well.

Can anyone see what I am missing? Thanks in advance for any help!

Codevyper

szboardstretcher 09-24-2010 11:37 AM
You aren't getting a full handshake.. the connection is [R]esetting before this happens.

What is your actual IPTables config?

codevyper 09-26-2010 03:12 AM
Quote:
Originally Posted by szboardstretcher (Post 4107920)
You aren't getting a full handshake.. the connection is [R]esetting before this happens.

What is your actual IPTables config?
Hey thanks for the response. I have been messing with it the last few days unsuccessfully, but the situation is the same. Here's the printout of the ipTables.

Thanks again,

# iptables -nvL
Chain INPUT (policy ACCEPT 46878 packets, 3520K bytes)
pkts bytes target prot opt in out source destination
1260K 278M ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED

Chain FORWARD (policy ACCEPT 209K packets, 54M bytes)
pkts bytes target prot opt in out source destination
208K 10M ACCEPT all -- eth0 tun0 10.0.0.0/8 172.20.0.0/24

Chain OUTPUT (policy ACCEPT 845K packets, 145M bytes)
pkts bytes target prot opt in out source destination

# iptables -nvL
Chain INPUT (policy ACCEPT 46878 packets, 3520K bytes)
pkts bytes target prot opt in out source destination
1260K 278M ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED

Chain FORWARD (policy ACCEPT 209K packets, 54M bytes)
pkts bytes target prot opt in out source destination
208K 10M ACCEPT all -- eth0 tun0 10.0.0.0/8 172.20.0.0/24

Chain OUTPUT (policy ACCEPT 845K packets, 145M bytes)
pkts bytes target prot opt in out source destination
root@otcdebtest:~# iptables -t nat -nvL
Chain PREROUTING (policy ACCEPT 49512 packets, 3389K bytes)
pkts bytes target prot opt in out source destination

Chain POSTROUTING (policy ACCEPT 19902 packets, 1517K bytes)
pkts bytes target prot opt in out source destination
129 5426 MASQUERADE all -- * eth0 172.10.0.0/24 0.0.0.0/0
8 576 MASQUERADE all -- * eth0 172.20.0.0/24 0.0.0.0/0

Chain OUTPUT (policy ACCEPT 19895 packets, 1516K bytes)
pkts bytes target prot opt in out source destination

codevyper 10-08-2010 06:43 PM
I had to put this issue on hold to deal with more pressing issues over the past couple of weeks, but I finally had a chance to get back to this today and here's what I've discovered. If I run tcpdump on either the tun0 or eth0 interfaces on the OpenVPN gateway as you see in my dumps above, everything appears to be fine. If I do tcpdump on the server I am trying to get to, it also appears fine. But if I do a TCP dump from the 10.0.10.x based client machine I get the real story:

TCPDUMP FROM THE CLIENT:

listening on eth1, link-type EN10MB (Ethernet), capture size 96 bytes
16:18:06.718930 IP 10.0.10.225.37684 > 172.20.0.105.22: S 2652232679:2652232679(0) win 5840 <mss 1460,sackOK,timestamp 24044662 0,nop,wscale 7>
16:18:06.880740 IP 172.20.0.105.22 > 10.0.10.225.37684: S 4018503789:4018503789(0) ack 2962149961 win 5792 <mss 1352,sackOK,timestamp 2334324269 24044662,nop,wscale 7>
16:18:06.880759 IP 10.0.10.225.37684 > 172.20.0.105.22: R 2962149961:2962149961(0) win 0


TCPDUMP FROM OPENVPN GATEWAY LOCAL INTERFACE:

listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
16:18:05.372414 IP 10.0.10.225.37684 > 172.20.0.105.22: Flags [S], seq 2962149960, win 5840, options [mss 1460,sackOK,TS val 24044662 ecr 0,nop,wscale 7], length 0
16:18:05.534102 IP 172.20.0.105.22 > 10.0.10.225.37684: Flags [S.], seq 4018503789, ack 2962149961, win 5792, options [mss 1352,sackOK,TS val 2334324269 ecr 24044662,nop,wscale 7], length 0
16:18:05.534224 IP 10.0.10.225.37684 > 172.20.0.105.22: Flags [R], seq 3272067242, win 0, length 0


The ACK sequence number coming back into the client machine is way off from the one the client sends on the SYN packet. The sequence numbers get shifted as they go to the OpenVPN gateway on SYN and SYN+ACK sends back the correct sequence for the shifted sequence number, which the client doesn't know about and rightfully sends an RST.

The iptables rules involved here are:

Chain FORWARD (policy ACCEPT 6166 packets, 589K bytes)
pkts bytes target prot opt in out source destination
119K 12M ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
52460 2202K ACCEPT all -- eth0 tun0 10.0.0.0/8 172.20.0.0/24
0 0 ACCEPT all -- tun0 eth0 0.0.0.0/0 10.0.10.0/24

Chain POSTROUTING (policy ACCEPT 17183 packets, 1343K bytes)
pkts bytes target prot opt in out source destination
17 1020 SNAT all -- * * 10.0.10.225 0.0.0.0/0 to:10.0.10.225
5004 301K SNAT all -- * tun0 10.0.0.0/8 172.20.0.0/24 to:172.20.0.113
0 0 SNAT all -- * eth0 172.20.0.0/24 10.0.10.0/24 to:10.0.10.2

The only reason I have the first -t nat rule in place is to prevent it from getting to the second two for testing purposes. My -t mangle tables are all empty as is PREROUTING.

Any ideas/help is greatly appreciated.

Thanks

codevyper 10-09-2010 03:48 PM
I fixed it. For the record here's what happened.

I have a static route setup on my SonicWall so that everything in the "Firewalled Subnets" (10.0.0.0/8) knows to go to the OpenVPN gateway, when they want to communicate with any of the external servers. But the OpenVPN gateway sits on the DMZ subnet along with the other machines that were causing me my problems. It turned out, that when I tried to initiate communication with an external site, the client on the 10.0.10.x subnet asked SonicWall to route the package, and SonicWall changed the sequence number to act as an intermediary. But when the packet came back through the OpenVPN gateway, it routed it straight to the 10.0.10.x client and not back through SonicWall so that freaked out the client because of the changed sequence and it reset the connection.

So I added direct routes to the machines on the 10.0.10.x subnet to point them straight at the OpenVPN gateway for the external network ranges and it all works now. I could have also pointed the OpenVPN gateway back to the SonicWall for incoming packets destined for any 10.0.10.x server and only had to make one change but I want to limit the traffic through the SonicWall where it's not needed, and since I trust the traffic between the OpenVPN gateway and the other 10.0.10.x machines I decided to do a straight route through.

Plus I am filtering traffic on the OpenVPN gateway with IPTables anyway.


All times are GMT -5. The time now is 04:47 AM.