Hi,

We have been running a website on a linux system for more than a year now. However all of sudden we started getting out of memory errors. What happens is that these errors kill every single program and we basically have to reboot the system. Now we have been trying to debug this issue for very long but without any success. The only findings we have are as follows.

Just before the machine is going down, kswapd moves far up in the process list obtained by running top. The most intriguing part is that never ever has more than 1.6 Megs out our 3 Megs of RAM been used. Also, just when the machine is going down, there is absolutely no activity on the swap and absolutely zero swap usage.

What can be the issue and how can we find it?

I would greatly appreciate any help on this.

Regards

I would also like to add couple of points to this.

We are using Tux content accelerator. This supposedly runs in the Kernel.
Our site experiences peaks of traffic.

Have your run memtest to verify your RAM? If not, that's the first place I'd look. Along those lines, have you physically made any changes to the box - adding new devices, moving hard drives, etc? In theory those changes may be a factor -- J.W.

Hi,
Thanks for the reply.

The machine is located at our Web Hosting company's facility in Texas. I believe we need access to the machine to run memtest. Please correct me if I am wrong.

Regards

Yes, at least as far as I'm aware. The typical way I run memtest is to put my handy Knoppix CD into the drive, then reboot into the memtest mode and let it run for at least several full cycles.

In any case, does the machine belong to you, or are you running your website off someone else's equipment? If it's your machine which you've shipped to the hosting company, then you will need their help in analyzing the problem and/or replacing any defective parts. Personally, based on the symptoms you've described, I'd guess that the RAM may be the most likely culprit, followed by a bad mobo, followed by a bad controller. To be honest the cheapest solution to this issue may be to just build a new machine at your site, test it out, and then ship it to the Texas location and have them ship the other one back. Obviously that's just a suggestion, and it may not be practical but I'll toss it out there anyway.

Alternatively, if you are paying a hosting company to run your website, and they are having hardware problems which interfere with your website's availability, then if I were you I'd demand that they move the site to an entirely different machine. In other words, hardware failures/errors are the responsibility of the hardware owner to fix, not the customer who happens to be using that hardware. The way things are right now, you are paying the penalty for their inability to diagnose and/or correct the problem, and that's a bad deal for you. If the hosting company cannot get their act together, I'd take my business elsewhere. Good luck with it -- J.W.