Dedicated server crashes every 24 hours; Linux amateur cannot diagnose
 

Hello, my name is David, I am a writer and part time freelance web developer with only the most minor understanding of servers and how they work. I recently started renting from 1and1 a dedicated server but for a while now have been having trouble with server crashes, about once every 24 hours, which require me to go into the control panel and reboot.

I have added code to my .htaccess, started using a script from zbblock
to intercept bots from China, added some IP ranges to my blacklist to prevent access to my server, etc. but am still having trouble. I'm using a Drupal 7 script to run perhaps 7 sites, and in a separate directory, a Drupal 6 script to run 2 sites.

I have read a lot of material on the forums but do not feel competent to try to diagnose what is going on.

If anyone is able to help me out, please tell me what information you need. I have images of all processes, and can provide screen shots of the SSH screen. Also have all specs.

Thanks very much for any assistance. regards

Trying to understand this readout from top -b
 

Hello, I am trying to understand why my server crashes once a day. Thought I'd start out by sharing this. Does anyone see anything glaringly wrong?

Please forgive my ignorance, I have some websites on a dedicated server running CentOS 6.2, and Plesk panel 11.0.

Thank you for any assistance.

Showing top output won't much help, because memory usage keep changing every moment.

Do you have system's sar reports or did you notice anything special in dmesg and server logs (/var/log/...)?
Also what does server crash mean, does http process get killed unexpectedly?

And what is reported about the crash? or did it just hang, requiring a hard reboot?

Hello, thanks for your reply. I've installed Sysstat. Is there a way to collect all reports at once? I can copy and paste the results individually. Here is the output from pidstat (below). Please tell me what else I can provide. PS: Yes, the server stops unexpectedly, as does the http process. Nothing is reported in the logs about the crash, and yes, it requires a reboot from the 1&1 control panel. It happens fairly regularly. I can do screenshots of what appears on the Plesk panel as well.

Please advise, and many thanks.

Since both physical as well as swap is enough to run https process successfully (as you can see most of your cpu is idle and https process is taking only few hundreds mb of memory). However to me, it sounds some pb with BIOS or might be some H/W failt.

If you noticed in above output of dmesg:-
And one more thing to check, that's /var/log/httpd. What does it say?

Thank you very much for your help, greatly appreciated. Here are some excerpts from the error log in var/log/httpd; are there other logs from which you'd like me to post excerpts?

A sampling from the January 13 error log:
A sampling from the January 19 error log (thousands of these sorts of messages):
Another sampling from the January 19 error log (thousands of these sorts of messages):
NOTE: the SIGTERM issue was addressed in this thread:
A sampling from the January 20 error log (thousands of these sorts of messages):

Not sure if this will help, but I took this screenshot this morning (attached). The empty spaces are the duration of the crash. If you look carefully you will notice a drop from a high of 2.4g to 2.2g.

Yesterday, I disabled in my Plesk control panel several domain forwards that I had set up (i.e., forward traffic from domain1.com to domain2.com).

That made an immediate impact on memory usage, but during the course of the day that use climbed again.

I also reduced in my.conf the number of max visitors from 500 down to 200.

If the BIOS has an issue, could this represent the fact that I have more or less permanently lost 1 gig or so of memory?

Also, the server has not crashed as of this morning since I made the changes above.

Thanks again for any help / insights you can provide.

Hello,

As per logs and all above stuff, I can conclude it's possibly a problem with BIOS chip, not just physically, but with it's configuration/setup. I have googled the error also and almost every discussion points to problem with BIOS. But I am not sure, to be honest, where exactly the problem is and how to solve it. However further searching and will get back if found something helpful.

Thank you very much for your help. I have emailed tech support with the BIOs issue and will let you know what I hear back.

Response back from the tech department:

"This message indicates that the BIOS uses a range of memory addresses that is generally used by the Linux kernel, and that the kernel recognizes this and will not attempt to use that memory range. This will not cause any errors in the operation or your server."

1. what was the last thing you changed before the crashes started

2. can you show your httpd.conf and the .htaccess files (contents)
NB: best practice is to not use .htaccess files, but put the directives inside the relevant httpd.conf Dir section.
Better performance (httpd.conf is cached by Apache, .htaccess isn't). Also security; htaccess is in Document root; possibly hackable.

3. show OS type
4. re crashes; have you looked at /var/log/messages ?