LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Server (https://www.linuxquestions.org/questions/linux-server-73/)
-   -   ip_conntrack table full (https://www.linuxquestions.org/questions/linux-server-73/ip_conntrack-table-full-581142/)

csdhiman 08-31-2007 12:42 AM

ip_conntrack table full
 
Dear All

I have Fedora Core 1 on Server which has postfix with mysql running on it.
Hardware details are
Dual Intel Xeon CPU 3 Ghz ,
1 Gb Memory
Swap is 3 GB

i get the error messeges on the screen as: ( from syslog )

Aug 30 13:25:46 mailsswl kernel: ip_conntrack: table full, dropping packet.
Aug 30 13:25:50 mailsswl kernel: NET: 66 messages suppressed.
Aug 30 13:25:51 mailsswl kernel: ip_conntrack: table full, dropping packet.
Aug 30 13:25:55 mailsswl kernel: NET: 39 messages suppressed.
Aug 30 13:25:56 mailsswl kernel: ip_conntrack: table full, dropping packet.
Aug 30 13:26:01 mailsswl kernel: NET: 36 messages suppressed.
Aug 30 13:26:02 mailsswl kernel: ip_conntrack: table full, dropping packet.

Check the output of
# cat /proc/net/ip_conntrack | wc -l is
62222

cat /proc/sys/net/ipv4/ip_conntrack_max
65496

What can be wrong pls suggest me ... As the messege said dropping packets is it is dropping mails.

felosi 09-01-2007 10:14 AM

Look like you are getting syn flooded. You need to find a way to limit connections like using a firewall like csf or apf with dos deflate and you can increase the syn_backlog and contrack_table according to your hardware in /etc/sysctl.conf
You can do sysctl -a | grep contrack
and then add the line to sysctl.conf with the new limit. You may wanna google adjusting that stuff to get the right equation for limits to use per certain amount of memory, etc;

csdhiman 09-03-2007 12:08 AM

re
 
Quote:

Originally Posted by felosi (Post 2877814)
Look like you are getting syn flooded. You need to find a way to limit connections like using a firewall like csf or apf with dos deflate and you can increase the syn_backlog and contrack_table according to your hardware in /etc/sysctl.conf
You can do sysctl -a | grep contrack
and then add the line to sysctl.conf with the new limit. You may wanna google adjusting that stuff to get the right equation for limits to use per certain amount of memory, etc;

There is no entry for ip_conntrack in sysctl
sysctl -a | grep contrack
no output

what should i use to solve the problem

rameshshihora 09-03-2007 08:48 AM

Hi,

Just enable tcp_syncookies will solve your problem.

net.ipv4.tcp_syncookies = 1

sysctl -p

Best Regards,
Ramesh Shihora

felosi 09-03-2007 11:26 PM

mispelled conntrack
sysctl -a | grep contrack
should be
sysctl -a | grep conntrack

win32sux 09-05-2007 08:04 PM

How did you guys reach the conclusion that this was a SYN flood and not just excessive normal traffic? Strikes me as odd to have had concluded that so fast with only the information given. I've seen several cases where Netfilter's state table is overloaded from normal (non-DoS attack) usage. Tweaking maxes and timeouts almost always proved to be the solution in these situations.

csdhiman 09-07-2007 06:50 AM

Dear All
The output of sysctl -a | grep conntrack is

net.ipv4.ip_conntrack_max = 65496
net.ipv4.netfilter.ip_conntrack_generic_timeout = 600
net.ipv4.netfilter.ip_conntrack_icmp_timeout = 30
net.ipv4.netfilter.ip_conntrack_udp_timeout_stream = 180
net.ipv4.netfilter.ip_conntrack_udp_timeout = 30
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close = 10
net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait = 120
net.ipv4.netfilter.ip_conntrack_tcp_timeout_last_ack = 30
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close_wait = 259200
net.ipv4.netfilter.ip_conntrack_tcp_timeout_fin_wait = 120
net.ipv4.netfilter.ip_conntrack_tcp_timeout_established = 432000
net.ipv4.netfilter.ip_conntrack_tcp_timeout_syn_recv = 60
net.ipv4.netfilter.ip_conntrack_tcp_timeout_syn_sent = 120
net.ipv4.netfilter.ip_conntrack_max = 65496

When the problem occur it stop responding the ping request or n/w activity
i found that on restarting iptables service , the server works fine and it give output
cat /proc/net/ip_conntrack |wc -l
4421
which was 60000 before restarting iptables service


now i added the line in sysctl.conf
net.ipv4.tcp_syncookies = 1

sysctl -p

is it ok or again i will get the problem

csdhiman 09-19-2007 11:16 PM

The Problem Is still There
I have to restart the iptables service once in a day .
Can i disable firewall of this server . Because this server is behind a windows firewall serever ..

Or i have to update the kernel

pls suggest

win32sux 09-20-2007 08:28 AM

Change this:
Code:

net.ipv4.ip_conntrack_max = 65496
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close_wait = 259200
net.ipv4.netfilter.ip_conntrack_tcp_timeout_established = 432000

To this:
Code:

net.ipv4.ip_conntrack_max = 98000
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close_wait = 240
net.ipv4.netfilter.ip_conntrack_tcp_timeout_established = 28800

This way you slightly increase the capacity of your state table, while at the same time reducing the amount of time for established connections to timeout from 5 days (432000 seconds) to 8 hours (28800 seconds). You are also changing the amount of time connections will be kept in TIME_WAIT states to something sane. I believe these settings represent the bulk of your problem, but let us know how it goes, please.

PS: You might also wanna disable TCP SYN cookies unless you are sure you are under attack.

ittec 02-04-2008 05:01 AM

The number of bytes per connection
 
Hi

very helpful thread. But i have a doubt about the real number of bytes used per tcp/ip connection. I did

#grep conntrack messages

and the last line output was:

Feb 3 16:35:58 dns kernel: ip_conntrack version 2.4 (8192 buckets, 65536 max) - 228 bytes per conntrack

So i understand that each tpc connection to my host uses 228 bytes of RAM actually. But how can i confirm that information? I read before another howtos that speaks of another values of bytes per connection(350 bytes f.e)

Thanks

ittec 03-14-2008 10:11 AM

Experimenting troubles
 
The worst problem of this issue is that althought you increase the value of your ip_contrack_max value, this value is more and more big and it never flushes. So one day you reach the conntrack_max value. These are all values about time of open connections:

Code:

net.ipv4.netfilter.ip_conntrack_tcp_timeout_close = 30
net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait = 30
net.ipv4.netfilter.ip_conntrack_tcp_timeout_last_ack = 120
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close_wait = 30
net.ipv4.netfilter.ip_conntrack_tcp_timeout_fin_wait = 60
net.ipv4.netfilter.ip_conntrack_tcp_timeout_established = 30
net.ipv4.netfilter.ip_conntrack_tcp_timeout_syn_recv = 30
net.ipv4.netfilter.ip_conntrack_tcp_timeout_syn_sent = 30

Is a very restrictive setup and now I need to watch and keep the performance of server. But Im not sure ....


All times are GMT -5. The time now is 11:39 AM.