Network connectivity problem - Debian 6 server
 

I'm having a problem with a Debian 6 server where it looses network connectivity and doesn't come back (server does not respond to pings, and can't connect to any services). The problem happens sporadically. Some times it will go a month without the problem occurring, while other times it will happen within a few days of being rebooted.

When logged into the console, I confirm that the server doesn't seem to have network conectivity (can't ping 8.8.8.8 for example). All services are running though (mysql, Apache, SSH, etc).

When the problem happens, the network interfaces are up (I checked ifconfig, all interfaces looked normal). I also check iptables, and verify there were no rules that would block anything. I also try running /etc/init.d/networking/restart, but that does not fix the problem. The only thing I've found that fixes the problem is rebooting.

The server is a virtual machine with a wired connection in a data centre. The company I rent the server from thinks the problem is due to high RAM usage. They said that the server was frozen due to resource exhaustion and the server stopped responding to ICMP requests, referencing this:

======
root@vm2:~# free -m
total used free shared buffers cached
Mem: 8181 7816 365 0 364 6201
-/+ buffers/cache: 1250 6931
Swap: 8191 0 8191
======

Do you think that high RAM usage could have anything to do with this problem? I think it is something else, but I don't know what else to check.

A low memory condition should eventually trigger the kernel oomkiller, but that will only affect processes running on the server, and never something as basic as network connectivity.

Did you try pinging the gateway? Was there an entry for the gateway in the ARP cache when this happened?

Anything in the logs immediately prior to and after network connectivity was lost?

I didn't try pinging the gateway. I'll try that the next time the problem happens, thanks for the idea.

The last thing in /var/log/messages was some apache entries like this:

Mar 8 23:49:57 hostname drupal: http://www.hostname.com|1394347797|m...||0||Retrieved ne
w CAPTCHA#012#012Request: POST http://rest.mollom.com/v1/captcha#012 type = 'image'#012 ssl = 0#012#012Request headers:#012 Accept = 'application/x
ml, application/json;q=0.8, */*;q=0.5'#012 Content-Type = 'application/x-www-form-urlencoded'#012 Authorization = 'OAuth oauth_consumer_key="079943c
bb7a9d6141e0d49fb8700de78", oauth_version="1.0", oauth_nonce="58fda8a94d1f9ed0d9f26c864f0aab4b", oauth_timestamp="1394347796", oauth_signature_method=
"HMAC-SHA1", oauth_signature="Ica0u1rFipmFj%2FctiNrWHpoXjrw%3D"'#012#012Response: 200 #012 code = '200'#012 captcha =#012 id = '14030919d19a4dbbc
6'#012 url = 'http://173.192.235.120:80/v1/captcha/14030919d19a4dbbc6.png'#012#012Retrieved new CAPTCHA

After the network connection is lost, there is not much in the logs, but I can see that things are still running. Cron is still processing (runs an ISPConfig update script every 2 minutes), named is complaining that the network is unreachable when trying to resolve hostnames, and the postfix mailq is being run (all attempts from previously queued mail gets set back to deferred due to domain names not being found).