[SOLVED] iptables masquerading, can establish connections, cannot transfer data.

kremlar · 10-04-2012, 03:01 AM

I've reached a point of great frustration trying to set up a box to work as a router on my home network, any insight would be appreciated.

The network setup is very straightforward.
-eth0 goes to the modem, ip address assigned by dhcpcd. From this box the internet as a whole works properly and I've ruled it out as an issue.

-eth1 goes to the lan.
It is manually assigned 192.168.1.1, dnsmasq is set up to listen to this interface and serve up 192.168.1.50 through 192.168.1.100. Computers on the lan are able to successfully connect to the network, properly look up ip addresses by name, and able to access http/ftp servers offered up through 192.168.1.1 as well. So this part is all hunky dory.

Relevant lines from the init script, just imagine them with the leading /sbin jazz, typing it over manually from other monitor.
ip link set lo up
ip link set eth0 up
ip link set eth1 up
dhcpcd eth0
ip addr add 192.168.1.1/24 dev eth1 broadcast +

Super advanced dnsmasq.conf:
bind-interfaces
interface=eth1
dhcp-range=192.168.1.50,192.168.1.100,12h

The current iptables commands being run, the first two being the ones actually necessary for the routing, the third i've been adding/removing and paring down(usually you'll see it with the -m state --state ESTABLISHEDetcetc stuff), just to double check if it'll change anything.
/usr/sbin/iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
/usr/sbin/iptables -A FORWARD -i eth1 -j ACCEPT
/usr/sbin/iptables -A FORWARD -o eth1 -j ACCEPT

Ok, so the network is setup, everybody on the lan can see eachother and communicate properly. everything is able to properly use dnsmasq for dns. everything can ping and traceroute to anywhere on the internet. a webpage served by the router is fully accessible, ssh works fine both from lan side and internet side. all things generally normal.

Here's the kicker, everything on the network can Establish a connection to anywhere on the internet....and then that's where it stops. Some example situations to further explain the behavior:

1: If I try to load up a website in a browser on 192.168.1.80: tcpdump -i eth1 on the router will show the initial communication back and forth, netstat on the 192.168.1.80 machine will then show an established connection, and then it just stops. No data is ever transferred after the connection is established.
2: Opening a telnet connection to a server(mud~) on the internet will give me the server's intro blurb, ask me for my username, ask me for my password, and then I'll receive the appropriate invalid username/pass or login message from the server. And then nothing else will transfer.
3: I can login to steam on my windows machine, I can browse and search the server list in tf2, i can start the loading process and then it will stop.

Everything is working in all respects except for that one minor detail of actually transferring the damn data. It's like the router just up and forgets about the connections after they've been established, possible possession by evil spirit, please advise.

GlennsPref · 10-04-2012, 03:44 AM

Hi, not tooo sure, but you haven't mentioned the sysctl.conf file requires a portforward line for the kernel.

Add this line, if it's not there (=1)
/etc/sysctl.conf

Code:

net.ipv4.ip_forward=1

My 2cents.

Regards Glenn

kremlar · 10-04-2012, 10:58 AM

sysctl.conf most definitely contains that line and the related file in the /proc tree properly holds the '1'. valid suggestion since i had forgotten to mention it in the op.

kremlar · 10-05-2012, 04:47 AM

Ok, I have slightly more information. I was playing around with the fact that i could perform the authentication on a remote telnet server from the lan side of this router box I'm setting up. For some reason I didn't realize sooner that as far as the networking system is concerned the connection is already legit and the authentication is just part of an established connection from it's perspective, it was simply choking on the initial login complete message that's around a page long.

I'm able to maintain a normal working connection indefinitely through the router as long as the server doesn't try to send more than a few lines of text in a single packet. Running tcpdump -i eth0 and -i eth1 side by side give a line for line matchup if i grep it to only show messages involving the remote telnet server. This should pretty much rule out issues with the setup of the masquerading part because it is indeed flipping things back and forth between lan and wan correctly.

But when the server tries to send more than a few lines of text.. Suddenly everything except acks stop coming in from the remote server according to both tcpdumps. Which is frustrating because I can still see my telnet commands going out, I can still see the acknowledgement that the remote server is getting them, but all the text that the server is sending back to me is vanishing into the aether somewhere between the remote server and whereever tcpdump is latched on to.

So, the question still remains the same: 'wtf?', except now I've got no more theories to test. The internet is 100% functional using a terminal on the router, everything in the lan can connect to the router perfectly and use all of its services, and now I can see that indeed, everything on the lan can establish fully working connections and can use them as long as they don't exceed a certain packet size. Possible possession by evil dietician's spirit?

GlennsPref · 10-05-2012, 06:40 PM

Hmm, my iptables script has these lines for forwarding...

I'm using a local squid server, on port 10.0.0.15:3128 (eth1) "$INT_IF"
and on the same system (eth0), port 10.0.0.16 to household router. "$EXT_IF"
The squid only serves me.(this machine.)

example lines from my iptables startup script...

Code:

## --- FORWARD CHAIN --- ##
#

	# Stateful inspection -- Forward in connections already established

	$IPTABLES -A FORWARD -i $EXT_IF -o $INT_IF -s $ANY -d $INT_NET -m state	--state ESTABLISHED,RELATED -j ACCEPT

.......
.......

	# Forward out all traffic

	$IPTABLES -A FORWARD -i $INT_IF -d $ANY -j ACCEPT

#
## --- OUTPUT CHAIN --- ##
#

	# Follows policy

#
## --- NAT --- ##
#

	# Enable masquerade

	$IPTABLES -A POSTROUTING -t nat -o $EXT_IF -j MASQUERADE

#
## -- Transparent proxy to Squid --- ##
#

	$IPTABLES -A PREROUTING -t nat -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128

	$IPTABLES -t nat -A PREROUTING -i $INT_IF -p tcp --dport 80 -j REDIRECT --to-port 3128

So, I'm still on the forwarding.... That's all I got.

'cept, http://easyfwgen.morizot.net/gen/

I've used this to write me a new script, then diff to see where I was different to the software generated one from this site.

Maybe.(?)...

Regards Glenn

kremlar · 10-05-2012, 07:10 PM

Very cool website, thank you for the link, will play around with it after work and see if a more robust iptables configuration will get this working. Setting up squid would also probably let me fake things out enough to get http traffic through I suppose, but still wouldn't address the more pressing issue of vidya games

kremlar · 10-07-2012, 09:08 PM

Well, I still remain uncertain as to what the actual cause was, but the issue has now been resolved after reformatting and doing a fresh install with pretty much the same exact everything except maybe i hit some magical switch under menuconfig when setting up the new kernel.

Before the reinstall my box was silently hucking out any packets over 578~ or so and then ignoring the connection afterwards, whereas now it was correctly reporting an mtu of 578 on the eth0 interface under ip link and after manually running ip link set eth0 mtu 1500 everything began flowing normally. Previously it was behaving as if the mtu was 578 but reporting it as 1500 and failing to change its behavior after manually adjusting it.

So, nothing to do with iptables, dnsmasq, or sysctl, but the actual culprit remains a mystery.