Server crash troubleshooting

imadsani · 08-10-2015, 02:33 AM

Hey,

I'm running a couple of high traffic servers for client's with the following:

PHP-FPM
Nginx
Percona 5.6
Varnish
CentOS 6.7
Plesk 12

The servers randomly crash, I don't have KVM access to the servers so don't know at which state they stand when the crash occurs.
By crash I mean that the websites stop loading, cannot ssh into the server neither do pings succeed

Where can I start troubleshooting all this? I am fairly certain this not hardware related, i've since swapped servers multiple times.

auge · 08-10-2015, 05:19 AM

Look at the logfiles in "/var/log/", start with messages, then go through the others you find. When the whole system freezes and needs a hard reboot most likely you find some trace to faulty hardware or really bad configs in your logs.

voleg · 08-10-2015, 05:32 AM

Swap servers to same hardware ? I had experience with server lost network, looks similar to your case. Do you sure if it crashed or lost network ? In my case problem was with NIC driver.

imadsani · 08-10-2015, 05:45 AM

That's the thing, because I don't have KVM access and the servers are in a datacenter in Germany I have no idea whether it's a kernel panic, reboot, shutdown or what.

voleg · 08-10-2015, 06:18 AM

Then start to read logs, as auge said.