package loss - ping: sendmsg: Operation not permitted
 

hello,
i ran into a problem. my HP ProLiant ML310 G5 server tends to loose packages. That happens with both interfaces. it's a proxy server and browsing gets impossible when that happens. here's some ping output:

~# ping google.com
PING google.com (74.125.53.100) 56(84) bytes of data.
64 bytes from pw-in-f100.google.com (74.125.53.100): icmp_seq=1 ttl=46 time=228 ms
ping: sendmsg: Operation not permitted
64 bytes from 74.125.53.100: icmp_seq=3 ttl=46 time=232 ms
ping: sendmsg: Operation not permitted
ping: sendmsg: Operation not permitted
64 bytes from 74.125.53.100: icmp_seq=6 ttl=46 time=235 ms
ping: sendmsg: Operation not permitted
64 bytes from pw-in-f100.google.com (74.125.53.100): icmp_seq=8 ttl=46 time=232 ms
ping: sendmsg: Operation not permitted
ping: sendmsg: Operation not permitted
ping: sendmsg: Operation not permitted
ping: sendmsg: Operation not permitted
64 bytes from 74.125.53.100: icmp_seq=13 ttl=46 time=235 ms
64 bytes from pw-in-f100.google.com (74.125.53.100): icmp_seq=14 ttl=46 time=228 ms

--------------------------------

:/# ping 192.168.0.2
PING 192.168.0.2 (192.168.0.2) 56(84) bytes of data.
ping: sendmsg: Operation not permitted
64 bytes from 192.168.0.2: icmp_seq=2 ttl=64 time=0.129 ms
64 bytes from 192.168.0.2: icmp_seq=3 ttl=64 time=0.166 ms
64 bytes from 192.168.0.2: icmp_seq=4 ttl=64 time=0.146 ms
64 bytes from 192.168.0.2: icmp_seq=5 ttl=64 time=0.129 ms
ping: sendmsg: Operation not permitted
ping: sendmsg: Operation not permitted
ping: sendmsg: Operation not permitted
64 bytes from 192.168.0.2: icmp_seq=9 ttl=64 time=0.138 ms
64 bytes from 192.168.0.2: icmp_seq=10 ttl=64 time=0.107 ms
64 bytes from 192.168.0.2: icmp_seq=11 ttl=64 time=0.141 ms
64 bytes from 192.168.0.2: icmp_seq=12 ttl=64 time=0.130 ms

maybe someone has any idea why this happens? after restart ping is ok for about 1 day to 1 week.
i'll be grateful for any help.

Post the output of the following,

Did you install any firewall such as Shorewall or any iptables rules ?

# ifconfig
eth0 Link encap:Ethernet HWaddr 00:18:71:ea:e5:e3
inet addr:89.117.106.137 Bcast:89.117.106.143 Mask:255.255.255.248
inet6 addr: fe80::218:71ff:feea:e5e3/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:23834380 errors:0 dropped:0 overruns:0 frame:0
TX packets:19644488 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:20072667967 (18.6 GiB) TX bytes:3189597876 (2.9 GiB)
Memory:fdee0000-fdf00000

eth1 Link encap:Ethernet HWaddr 00:1e:0b:5c:e7:3c
inet addr:192.168.0.1 Bcast:192.168.0.255 Mask:255.255.255.0
inet6 addr: fe80::21e:bff:fe5c:e73c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1300 Metric:1
RX packets:23395135 errors:30172 dropped:0 overruns:0 frame:30172
TX packets:29501081 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:3548612778 (3.3 GiB) TX bytes:20376501079 (18.9 GiB)
Interrupt:16

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:289305 errors:0 dropped:0 overruns:0 frame:0
TX packets:289305 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:39804613 (37.9 MiB) TX bytes:39804613 (37.9 MiB)

tun0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 -00
inet addr:10.1.0.1 P-t-P:10.1.0.2 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:17551 errors:0 dropped:0 overruns:0 frame:0
TX packets:19894 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:3270747 (3.1 MiB) TX bytes:3770380 (3.5 MiB)

tun1 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00 -00
inet addr:10.69.23.62 P-t-P:10.69.23.61 Mask:255.255.255.255
UP POINTOPOINT RUNNING NOARP MULTICAST MTU:1500 Metric:1
RX packets:173622 errors:0 dropped:0 overruns:0 frame:0
TX packets:173586 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:10067286 (9.6 MiB) TX bytes:10002126 (9.5 MiB)


# iptables -L
Chain INPUT (policy DROP)
target prot opt source destination
ACCEPT all -- anywhere anywhere
LOG_DROP all -- loopback/8 anywhere
ACCEPT all -- anywhere 89.117.106.137 state RELATED,ESTABLISHED
ACCEPT udp -- anywhere 89.117.106.137 udp dpt:domain
ACCEPT tcp -- anywhere 89.117.106.137 tcp dpt:ssh
ACCEPT all -- aii.alna.lt 89.117.106.137
ACCEPT all -- mail.veikia.net 89.117.106.137
ACCEPT all -- lan-84-240-5-213.vln.skynet.lt 89.117.106.137
ACCEPT all -- ird.vrm.lt 89.117.106.137
ACCEPT all -- mail2.officeday.lt 89.117.106.137
ACCEPT all -- 89.117.128.5 89.117.106.137
ACCEPT all -- libis.kretvb.lt 89.117.106.137
ACCEPT udp -- aii.alna.lt 89.117.106.137 udp dpt:openvpn
ACCEPT udp -- lan09.maxi.lt 89.117.106.137 udp dpt:openvpn
ACCEPT all -- anywhere anywhere
ACCEPT icmp -- anywhere 89.117.106.137
DROP all -- anywhere anywhere
ACCEPT all -- anywhere 192.168.0.1

Chain FORWARD (policy ACCEPT)
target prot opt source destination

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Chain LOG_CT (0 references)
target prot opt source destination

Chain LOG_DROP (1 references)
target prot opt source destination
LOG all -- anywhere anywhere LOG level warning
DROP all -- anywhere anywhere

Chain LOG_REJE (0 references)
target prot opt source destination
LOG all -- anywhere anywhere LOG level warning
REJECT all -- anywhere anywhere reject-with icmp-port-unreachable

=================

no firewall. security based on iptables.

i don't think that the problem is with firewall, because there are two equal machines with same OS and same configuration. they are sharing the load, but i have no problem with other machine.

the same ping problem is when pinging 127.0.0.1 :(

I too was having this problem. It turned out that conntrack's number of connections to track was set too low for the amount of connections I had. You can see if this is the case by looking at /var/log/syslog (or wherever syslog is on your system.) If you see the message "ip_conntrack: table full, dropping packet" you can know that is the problem.

To fix this, do something like "echo 200000 > /proc/sys/net/ipv4/netfilter/ip_conntrack_max". Then set something like "net.ipv4.netfilter.ip_conntrack_max = 200000" in you /etc/sysctl.conf so the setting will be applied on every restart.

Hopefully that helps some very confused/frustrated people like me. Thanks to blog.rackcorp.com/?p=19 where I finally found the answer!