Hi,

I am running an authentication load on my Linux server with 4GB RAM which acts as a gateway for 2000 concurrent users who are accessing internet. I was trying to simulate a scenario which will increase the number of TCP connections in the gateway when end users use stuffs like Bittorrent. I was using iperf for the same. I was running an iperf client in one server and an iperf server in another which will be opening 5000 ports concurrently and sending traffic through out. But that helps me to create only 5000 connections which is no where when compared to my requirement which is 70000 connections. Can some one help me to increase the connections?

Thanks in advance....

-Sushitha

Most parameters are set in /etc/sysctl.conf.

If you run "sysctl -a" it shows you all possible parameters. Those that don't appear in sysctl.conf are at default settings.

Running "sysctl -a |grep contrack" should show something like:

net.ipv4.ip_conntrack_max = 49608
net.ipv4.netfilter.ip_conntrack_tcp_max_retrans = 3
net.ipv4.netfilter.ip_conntrack_tcp_be_liberal = 0
net.ipv4.netfilter.ip_conntrack_tcp_loose = 1
net.ipv4.netfilter.ip_conntrack_tcp_timeout_max_retrans = 300
net.ipv4.netfilter.ip_conntrack_log_invalid = 0
net.ipv4.netfilter.ip_conntrack_generic_timeout = 600
net.ipv4.netfilter.ip_conntrack_icmp_timeout = 30
net.ipv4.netfilter.ip_conntrack_udp_timeout_stream = 180
net.ipv4.netfilter.ip_conntrack_udp_timeout = 30
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close = 10
net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait = 120
net.ipv4.netfilter.ip_conntrack_tcp_timeout_last_ack = 30
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close_wait = 60
net.ipv4.netfilter.ip_conntrack_tcp_timeout_fin_wait = 120
net.ipv4.netfilter.ip_conntrack_tcp_timeout_established = 432000
net.ipv4.netfilter.ip_conntrack_tcp_timeout_syn_recv = 60
net.ipv4.netfilter.ip_conntrack_tcp_timeout_syn_sent = 120
net.ipv4.netfilter.ip_conntrack_checksum = 1
net.ipv4.netfilter.ip_conntrack_buckets = 6201
net.ipv4.netfilter.ip_conntrack_count = 3
net.ipv4.netfilter.ip_conntrack_max = 49608

These are the parameters you can set/use with sysctl.

At command line you run:
sysctl -w net.ipv4.ip_conntrack_max=70000
It will bump that parameter up to 70000.
Also above output suggests the netfilter parameter for this should be the same so to update that:
sysctl -w net.ipv4.netfilter.ip_conntrack_max=70000

Note that running the above two commands only sets it for the currently running system. You'd lose the settings on a reboot so to make them permanent you'd need to add to the /etc/sysctl.conf file.

One way of doing this is to add them to the file first then just run "sysctl -p /etc/sysctl.conf" which makes it load in the parameters for the file. It's a good way of verifying it will do the right setup after a reboot.

P.S. I have no idea what the effect of raising these parameters will be on your system.

Thank you so much. I will try this.

More slots for remembering connections means more memory used for it. Where /proc/net sysctls are concerned people often think bigger values are "better" (regardless of reasoning or mistaking symptoms for causes), while the docs usually say it isn't.

Hi jlightner,

First of all let me thank you once again for the support. I tried your suggestion. In my gateway which is running on Ubuntu has the conntrack_max as 65536 which is almost 70000. SO I did not change the value. That means the gateway supports 65536 number of connections concurrently. This makes sure that gateway can handle connections till 65000. In my testbed, when almost 2000 users are authenticated and browsing the internet and all these traffic goes through my gateway, I am getting only 8000 connections when I check /proc/net/ip_conntrack|wc -l in my gateway. How can I increase this value to 60000? Can you suggest something?

Right. It always annoys me when people troubleshooting an issue say "double that parameter" as it lets me know they've done no research at all. My experience has been that, except in situations where you have some idea of why you're constrained, increasing parameters to fix issues typically has the effect of simply delaying WHEN you'll have the issue rather than fixing it. Often it is a sign of some sort of runaway process and the real fix is to resolve the bad code.

What I meant was I hadn't used the specific parameters I was telling the OP about so couldn't give guidance as to whether increasing them was a good idea or not. My intent was to emphasize my answer was for "how to" rather than "should he". I had temporarily increased values on a test system before posting and it didn't crash and burn right away but that was hardly an exhaustive test.

When you cat /proc/net_ipconntrack you're seeing how many are connected NOT how many are allowed to connect. Since your parameters are higher than 8000 it would seem your issue is something else. For example if your connections were each using a pty then you might be having an issue with that.

The best way to troubleshoot this would be to first look at /var/log/messages (and other logs in /var/log) to see if you're getting any messages indicative of an issue. You could also look at dmesg but remember that what you see there may be from the last boot rather than from today though it could e from today. What message occurs on the system that tries to connect once your server has hit 8000? Have you tried to run tcpdump to see what happens when a test system hits this after you've hit 8000? Is your CPU overloaded? Is memory constrained?

That's a nice way of putting it. BTW the OP's 5000 connlimit and the "something else" part of your reply makes me think about default sysctls. How about local port range, maximum network sockets, maximum sockets per process et cetera?..