Slow Linux NAT

get00x · 06-01-2009, 07:07 PM

Hiya,

I got myself a new cable internet connection with a downstream of 50Mbps and 10Mbps upstream. I realized that my D-Link home router wouldn't be able to handle that traffic (WAN<->LAN), so I set up a computer with two NIC's and Slackware 12.2 (kernel 2.6.29.4) to use as a router instead.

My problem is that I only get a throughput of about 20Mbps down and 10Mbps up (the upstream speed is fine though of course) with the linux router. Any computer connected directly to the cable modem get full speed (50/10)... any as in my usual desktop, as well as the linux router. I'm also able to send and recieve files in 100Mbps over the LAN between my router and desktop. But I just don't get full speed through the NAT routing to or from the internet.

The two NIC's:
eth0: 01:08.0 Ethernet controller: 3Com Corporation 3c905B 100BaseTX [Cyclone] (rev 30)
eth1: 01:06.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10)

eth0 is connected to cable modem and eth1 is for LAN.

I've tried two different sets of iptables rules with same results:

Code:

iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -A FORWARD -i eth0 -o eth1 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT

and

Code:

iptables -I INPUT 1 -i eth1 -j ACCEPT
iptables -I INPUT 1 -i lo -j ACCEPT
iptables -A INPUT -p TCP -i ! eth1 -d 0/0 --dport 0:1023 -j DROP
iptables -A INPUT -p UDP -i ! eth1 -d 0/0 --dport 0:1023 -j DROP
iptables -I FORWARD -i eth1 -d 192.168.0.0/255.255.255.0 -j DROP
iptables -A FORWARD -i eth1 -s 192.168.0.0/255.255.255.0 -j ACCEPT
iptables -A FORWARD -i eth0 -d 192.168.0.0/255.255.255.0 -j ACCEPT
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

The computer has a pentium 4 cpu and 1GB of ram.. cpu-usage is around 0-1% when I download from the Internet to my desktop in 20Mbps.

Any ideas what might be the cause of this..?

grepmasterd · 06-02-2009, 10:32 AM

iptables NAT is fast, so don't look too hard at that part of the configuration. I assure you, people use iptables NAT for gigabit networks with very little impact.

Quote:

Any computer connected directly to the cable modem get full speed (50/10)

Except for your router, you mean. My first guess is that there are more likely some link issues between eth0 (linux router) and your cable modem. Are there transmission errors on eth0? Try some download tests to the linux router directly to isolate the point of slowness. If there appears to be errors, try placing a switch (not a hub) between eth0 and the cable modem - that can sometimes normalize link connectivity.

get00x · 06-02-2009, 11:47 AM

Quote:

Originally Posted by grepmasterd

Except for your router, you mean. My first guess is that there are more likely some link issues between eth0 (linux router) and your cable modem. Are there transmission errors on eth0? Try some download tests to the linux router directly to isolate the point of slowness. If there appears to be errors, try placing a switch (not a hub) between eth0 and the cable modem - that can sometimes normalize link connectivity.

Actually, not except for my router.

# wget ftp://ftp.port80.se/100M
(4.80 MB/s) - `100M' saved [104857600]

That was downloaded to the router itself. If I try to download the same file to my desktop or laptop in the LAN I get speeds of around 2MB/s.

eth0:
RX packets:228112 errors:0 dropped:0 overruns:699 frame:0
TX packets:127707 errors:0 dropped:0 overruns:0 carrier:0

eth1:
RX packets:91983 errors:0 dropped:0 overruns:0 frame:0
TX packets:151671 errors:0 dropped:0 overruns:1 carrier:0

No errors and nothing dropped, don't know what overruns might be though? I recently rebooted it btw.

With the router active my network looks like:
cable modem <-> linux router <-> 100Mbps switch <-> client
The router gets good speeds from the Internet, but the LAN clients doesn't.

If I plug one of the LAN clients like this, it get goods speeds:
cable modem <-> 100Mbps switch <-> client

grepmasterd · 06-02-2009, 12:26 PM

hm, weird indeed. overruns means generally that packets are being received too fast, but that shouldn't be happening on a 50Mb line. Do you get the same overruns count when downloading directly to the router, or only when forwarding/routing? What drivers are being used by the cards? (try 'ethtool -i eth0')

also 1GB sounds like plenty of RAM, but I'm curious, what does the output of 'free' show?

get00x · 06-02-2009, 12:59 PM

# ethtool -i eth0
driver: 3c59x
version:
firmware-version:
bus-info: 0000:01:08.0

# ethtool -i eth1
driver: 8139too
version: 0.9.28
firmware-version:
bus-info: 0000:01:06.0

# free -m
total used free shared buffers cached
Mem: 1002 45 957 0 0 30
-/+ buffers/cache: 14 988
Swap: 0 0 0

I used watch ifconfig, and the overruns only occured when downloading some big file on a lan client from the internet, not when downloading directly to the router. Just like it can't route it quick enough?

grepmasterd · 06-02-2009, 01:42 PM

ok, just checking to see that they're using the standard drivers, and that something unexpected wasn't consuming memory, but that looks good too.

hm, try just the masquerading w/out the filter rules, since the filter rules aren't quite correctly written.

Code:

iptables -t filter -F
iptables -t mangle -F
iptables -t nat -F
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

see if there's any improvement.

as a separate test and just for kicks, try making eth1 your WAN and eth0 your LAN interface, to see if the behavior changes.

get00x · 06-02-2009, 02:24 PM

Quote:

Originally Posted by grepmasterd

Code:

iptables -t filter -F
iptables -t mangle -F
iptables -t nat -F
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

Only using that didn't work at all. No connectivity.

grepmasterd · 06-02-2009, 02:43 PM

it should have, it does for me. here's the output from iptables-save on my system (running a similar setup as yours)

Code:

iptables-save 
# Generated by iptables-save v1.4.1.1 on Tue Jun  2 12:37:44 2009
*nat
:PREROUTING ACCEPT [216:46295]
:POSTROUTING ACCEPT [5:812]
:OUTPUT ACCEPT [27:2712]
-A POSTROUTING -o eth1 -j MASQUERADE 
COMMIT
# Completed on Tue Jun  2 12:37:44 2009

the only real difference besides hardware and drivers is that eth0 is my LAN and eth1 is my WAN (shouldn't matter, just pointing it out for the sake of clarity).

get00x · 06-02-2009, 02:47 PM

Well, no traffic got through with only that rule for some reason.

Anyway I changed eth1 to be wan and eth0 to be lan. It actually gave my clients on the lan a boost of around 15Mbps. Weird.

grepmasterd · 06-02-2009, 02:55 PM

maybe you have a DROP policy in place, which would explain things. iptables-save would disclose that.

so swapping interfaces fixed you up? if so, then you probably have a hardware or driver issue there somewhere.

get00x · 06-03-2009, 04:11 PM

And not only did it give a 15Mbps boost. I now have the speeds I'm supposed to be having, 50/10. Thanks for the help.