[SOLVED] Iptables: how to redirect locally-generated packets to a remote server?

briwood · 03-23-2010, 12:54 PM

Thanks eantoraz,

First I setup my rules again:

Code:

[root@adm-10-cms ~]# iptables -F
[root@adm-10-cms ~]# iptables -F -t nat
[root@adm-10-cms ~]# iptables -X
[root@adm-10-cms ~]# iptables -t nat -A PREROUTING -p tcp --dport 3306 -j DNAT --to 128.xxx.xxx.xxx:3197
[root@adm-10-cms ~]# iptables -A FORWARD -p tcp -d 128.xxx.xxx.xxx --dport 3197 -j ACCEPT
[root@adm-10-cms ~]#  iptables -t nat -A OUTPUT -p tcp -o lo --dport 3306 -j DNAT --to 128.xxx.xxx.xxx:3197
[root@adm-10-cms ~]# 
[root@adm-10-cms ~]# iptables -t nat -A POSTROUTING  -j MASQUERADE

Here's tcpdump of a a successfull connection: 'telnet adm-10-cms 3306' initiated from dev-10-cms

Code:

[root@adm-10-cms ~]# tcpdump -i eth0 -p tcp and port 3197 -n -v
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
10:27:20.745788 IP (tos 0x10, ttl  63, id 52105, offset 0, flags [DF], proto: TCP (6), length: 60) 169.xxx.xxx.xxx.48658 > 128.xxx.xxx.xxx.embrace-dp-s: S, cksum 0x5f9a (correct), 3997077676:3997077676(0) win 5840 <mss 1460,sackOK,timestamp 1904696101 0,nop,wscale 7>
10:27:20.746836 IP (tos 0x0, ttl  63, id 0, offset 0, flags [DF], proto: TCP (6), length: 60) 128.xxx.xxx.xxx.embrace-dp-s > 169.xxx.xxx.xxx.48658: S, cksum 0x3d23 (correct), 4207250702:4207250702(0) ack 3997077677 win 5792 <mss 1380,sackOK,timestamp 2221807272 1904696101,nop,wscale 2>
10:27:20.747693 IP (tos 0x10, ttl  63, id 52106, offset 0, flags [DF], proto: TCP (6), length: 52) 169.xxx.xxx.xxx.48658 > 128.xxx.xxx.xxx.embrace-dp-s: ., cksum 0x8209 (correct), ack 1 win 46 <nop,nop,timestamp 1904696104 2221807272>
10:27:20.748452 IP (tos 0x8, ttl  63, id 64331, offset 0, flags [DF], proto: TCP (6), length: 112) 128.xxx.xxx.xxx.embrace-dp-s > 169.xxx.xxx.xxx.48658: P 1:61(60) ack 1 win 1448 <nop,nop,timestamp 2221807274 1904696104>
10:27:20.748957 IP (tos 0x10, ttl  63, id 52107, offset 0, flags [DF], proto: TCP (6), length: 52) 169.xxx.xxx.xxx.48658 > 128.xxx.xxx.xxx.embrace-dp-s: ., cksum 0x81ca (correct), ack 61 win 46 <nop,nop,timestamp 1904696105 2221807274>
10:27:25.749780 IP (tos 0x8, ttl  63, id 64333, offset 0, flags [DF], proto: TCP (6), length: 52) 128.xxx.xxx.xxx.embrace-dp-s > 169.xxx.xxx.xxx.48658: F, cksum 0x68c6 (correct), 61:61(0) ack 1 win 1448 <nop,nop,timestamp 2221812275 1904696105>
10:27:25.752042 IP (tos 0x10, ttl  63, id 52108, offset 0, flags [DF], proto: TCP (6), length: 52) 169.xxx.xxx.xxx.48658 > 128.xxx.xxx.xxx.embrace-dp-s: F, cksum 0x5ab3 (correct), 1:1(0) ack 62 win 46 <nop,nop,timestamp 1904701109 2221812275>
10:27:25.752594 IP (tos 0x8, ttl  63, id 64335, offset 0, flags [DF], proto: TCP (6), length: 52) 128.xxx.xxx.xxx.embrace-dp-s > 169.xxx.xxx.xxx.48658: ., cksum 0x5536 (correct), ack 2 win 1448 <nop,nop,timestamp 2221812278 1904701109>

8 packets captured
8 packets received by filter
0 packets dropped by kernel

When I try to capture the unsuccessful connection from localhost I don't get any output. I do:

Code:

[root@adm-10-cms ~]# tcpdump -i lo -p tcp and port 3197 -n -v
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 96 bytes

then:

Code:

[root@adm-10-cms ~]# telnet localhost 3306
Trying 127.0.0.1...

Nothing appears in the tcpdump console, so I ^C and get:

Code:

0 packets captured
0 packets received by filter
0 packets dropped by kernel

Next I try to listen on 3306 on the lo interface, but nothing comes across when I telnet localhost 3306

Code:

[root@adm-10-cms ~]# tcpdump -i lo -p tcp and port 3306 -n -v
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 96 bytes

0 packets captured
0 packets received by filter
0 packets dropped by kernel

Trying the same thing to 3307 (no service there) I get this tcpdump output:

Code:

[root@adm-10-cms ~]# tcpdump -i lo -p tcp and port 3307 -n -v
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 96 bytes
10:37:06.874458 IP (tos 0x10, ttl  64, id 10105, offset 0, flags [DF], proto: TCP (6), length: 60) 169.xxx.xxx.xxx.47587 > 127.0.0.1.opsession-prxy: S, cksum 0xd07d (correct), 2294676478:2294676478(0) win 32792 <mss 16396,sackOK,timestamp 2478038151 0,nop,wscale 7>
10:37:06.874670 IP (tos 0x10, ttl  64, id 0, offset 0, flags [DF], proto: TCP (6), length: 40) 127.0.0.1.opsession-prxy > 127.0.0.1.47587: R, cksum 0x62f9 (incorrect (-> 0x663a), 0:0(0) ack 2294676479 win 0

2 packets captured
4 packets received by filter
0 packets dropped by kernel

For the last test I remove the -t nat OUTPUT rule and listen on 3306 while attempting the same telnet localhost 3306. This yields expected output.

Code:

[root@adm-10-cms ~]#  iptables -t nat -D OUTPUT -p tcp -o lo --dport 3306 -j DNAT --to 128.xxx.xxx.xxx:3197

[root@adm-10-cms ~]# tcpdump -i lo -p tcp and port 3306 -n -vtcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 96 bytes
10:50:19.579767 IP (tos 0x10, ttl  64, id 18940, offset 0, flags [DF], proto: TCP (6), length: 60) 169.xxx.xxx.xxx.34570 > 127.0.0.1.mysql: S, cksum 0xf892 (correct), 3120938229:3120938229(0) win 32792 <mss 16396,sackOK,timestamp 2478830857 0,nop,wscale 7>
10:50:19.580301 IP (tos 0x10, ttl  64, id 0, offset 0, flags [DF], proto: TCP (6), length: 40) 127.0.0.1.mysql > 127.0.0.1.34570: R, cksum 0xa39c (incorrect (-> 0xa6dd), 0:0(0) ack 3120938230 win 0

2 packets captured
4 packets received by filter
0 packets dropped by kernel

eantoranz · 03-23-2010, 01:49 PM

With the rules you set in place, do the tcpdump on -i lo and try the telnet localhost 3306 and tell us what comes out.

eantoranz · 03-23-2010, 01:52 PM

And is the counter of the -t nat OUTPUT rule you set increasing when you do the telnet?

eantoranz · 03-23-2010, 02:04 PM

Oh, I saw you tried the tcpdump listening on -i lo so skip that test for now. Tell me about the counter, because that traffic must be going somewhere, right?

eantoranz · 03-23-2010, 02:29 PM

I would think that the problem here is that there's code on the network stack doing checkups after OUTPUT that when looking at the packet (source address: 127.0.0.1, dest addres: something not in loopback) drops it for not being "consistent".... perhaps someone knows about this? And perhaps could sysconf be used to disable such checkup?

briwood · 03-23-2010, 03:10 PM

Quote:

Originally Posted by eantoranz

With the rules you set in place, do the tcpdump on -i lo and try the telnet localhost 3306 and tell us what comes out.

When I do

Code:

[root@adm-10-cms ~]# tcpdump -i lo -v

and then 'telnet localhost 3306' in a different console on the same server nothing is captured.

The counter is incrementing. I see:

Code:

[root@adm-10-cms ~]# iptables -L -nvx -t nat
...snip...
Chain OUTPUT (policy ACCEPT 108 packets, 7372 bytes)
    pkts      bytes target     prot opt in     out     source               destination         
       6      360 DNAT       tcp  --  *      lo      0.0.0.0/0            0.0.0.0/0           tcp dpt:3306 to:128.XXX.XXX.XXX:3197 
...snip...

I do another telnet localhost and then I see it's at 8:

Code:

Chain OUTPUT (policy ACCEPT 108 packets, 7372 bytes)
    pkts      bytes target     prot opt in     out     source               destination         
       8      480 DNAT       tcp  --  *      lo      0.0.0.0/0            0.0.0.0/0           tcp dpt:3306 to:128.XXX.XXX.XXX:3197

I really appreciate all your help thus far!

briwood · 03-23-2010, 03:16 PM

Quote:

Originally Posted by eantoranz

I would think that the problem here is that there's code on the network stack doing checkups after OUTPUT that when looking at the packet (source address: 127.0.0.1, dest addres: something not in loopback) drops it for not being "consistent".... perhaps someone knows about this? And perhaps could sysconf be used to disable such checkup?

What do you think of my theory here:
http://www.linuxquestions.org/questi...21#post3909121

Since this is a locally-generated packet it is never going to hit this rule

Code:

iptables -A FORWARD -p tcp -d 128.xxx.xxx.xxx --dport 3197 -j ACCEPT

which does the actual forwarding. I *think* that this rule:

Code:

iptables -t nat -A OUTPUT -p tcp -o lo --dport 3306 -j DNAT --to 128.xxx.xxx.xxx:3197

only rewrites the destination of the packet. We need the forward rule above to actually send the packet on...

eantoranz · 03-23-2010, 05:22 PM

Well, FORWARD is not touched by packets that are going out from OUTPUT so don't worry cause it doesn't affect you.

I think it's because of the source address inconsistency, I think. Check out this article I just found (a little dated, by the way):

http://lists.netfilter.org/pipermail...er/040104.html

So, how about other tricks? Why do you need locally generated connections on local port 3306 to get connected to a remote host in the first place? In case it's a _must_, wouldn't a SSH tunnel (or a simpler approach) work for you?

In case you want to try the ssh runnel trick, remove the OUTPUT rule doing the DNAT to remote:3197 and run this command on that same host:

ssh -nNT -L 3306:remote-server:3197 user@localhost

After the connection is established (you will know because after a few seconds the command won't return), try telnet localhost 3306

Maybe there are simpler approaches but _at least_ I bet that one will work.

eantoranz · 03-23-2010, 05:30 PM

It _has_ to be the IP address inconsistency.

With all your rules in place, try to do a telnet to your local IP address on the intranet (instead of localhost). That way, it did work with your DNAT trick.

eantoranz · 03-23-2010, 05:31 PM

At least, it did work for me.

briwood · 03-23-2010, 07:44 PM

Quote:

Originally Posted by eantoranz

wouldn't a SSH tunnel (or a simpler approach) work for you?

Thought of that. The remote mysql server doesn't allow ssh and I don't have control over that server.

Quote:

(or a simpler approach)

Is there a non ssh option for setting up a simple tunnel to 3197 on the remote server?

Will try telnet to ipaddr tomorrow and report back. If that works, I may look into snat.

Thanks.;

eantoranz · 03-24-2010, 09:13 AM

The ssh tunnel as I told you to use it yesterday requires no ssh on the mysql server but on the host you are working instead (the one where you want local connections to port 3306 to be sent to a remote server port 3197).

briwood · 03-24-2010, 10:20 AM

Thanks again for all the help!

You are right. Telnetting to anything other than localhost works:

Code:

[root@adm-10-cms ~]# telnet 169.xxx.xxx.xxx 3306
Trying 169.xxx.xxx.xxx...
Connected to adm-10-cms.example.com (169.xxx.xxx.xxx).
Escape character is '^]'.
8
5.0.67-log��Mw.}OP%e,�!QDAYz{|{v^R`^]
telnet> quit
Connection closed.
[root@adm-10-cms ~]# telnet adm-10-cms 3306
Trying 169.xxx.xxx.xxx...
Connected to adm-10-cms.example.com (169.xxx.xxx.xxx).
Escape character is '^]'.
8
5.0.67-log��q%])SE@R,�![SnO!Sbo+%s:Connection closed by foreign host.
[root@adm-10-cms ~]# telnet localhost 3306
Trying 127.0.0.1...
(hangs...^C)

So if you snatted the localhost packets correctly, this could probably be made to work from localhost via the loopback interface.

Thanks for enlightening me on the ssh tunnel. I assumed that you needed sshd running on the target server. Since you don't I'm pursuing this option since it is way simpler. Always good to know more about iptables though!

fredericgermain · 02-16-2016, 09:35 PM

I think your problem is really solved here (I mean, solved the iptables way) :

http://unix.stackexchange.com/questi...s-to-127-0-0-1

you need to activate local routing on your outbound interface. if eth0 :

sysctl -w net.ipv4.conf.eth0.route_localnet=1

it seems to be like security feature

hope it helps some people coming to this question !