Hi all.

I have a Nagios server with a lot of hosts and services; around 400 services (in all) and 150 hosts. Most of these services are programmed in bash language. The problem is the server has a high load average; between 5 and 11. The server has the next features:

- Intel Xeon 2.66GHz Dual Core
- 4MB cache memory
- 1GB RAM memory
- 50GB hard disk

Is this load average normal? Should I programme the plugins with C?

Thanks for all.

you didnt mention your distro?? and load average between 5 and 11 is too much, i think 1 GB RAM is not enough, you should have atleast 2 GB RAM for your scenario.

Besides RAM, A complete list of all the things you should look for at this point would be large. Initially, see if there are common problems that may have been overlooked:

• Performance always degrades when the system starts swapping.
• Network performance can be poor due to bad hardware or a bad negotiation.
• Common applications errors that will result in poor Linux performance include such things as polling for I/O instead of using poll/select (or using these calls with a zero timeout!).

There is a command vmstat available in linux to identify the possible problems concerning performance.
The vmstat utility will show how many processes are in the run queue. Look Under "r" --> Processes waiting for CPU. More the count we observe, more processes waiting to run. If we just observe a spike in the count, we shouldn’t treat them as bottleneck. If value is constantly high (most people treats 2 * CPU count as high), it hints that CPU is the bottleneck.

If the vmstat utility shows many processes in the blocked statistic (.b. under .procs.), then the system may be I/O bound. This could be because of slow disks or slow Ethernet cards.

Sorry.

The distro is Debian 5.


I've tried the vmstat command. The processes in the run queue is variable; it goes between 1, 16 and 32. I think the problem is that Nagios does a lot of asynchronous process call to obtain the plugins information. Therefore, in a determined momment the process grows up. I've seen that the cpu is around 5%-7% (except in the determined momments, before commented).

I think that this behavior is the way that Nagios works. This behavior is also produced with the CPU.

It's difficult to improve this behavior. I want to try two options:

- Programme the plugins with c (this will go faster).
- Reduce several apache options like MaxKeepAliveRequests,MaxClients, MaxSpareThreads, ... (this server is only for Nagios).

What do you think about?

Thanks,
David

I previously described that CPU is a bottleneck if no. of processes in run queue is continuously high, and in your case its variable, so i think its normal.


Its obvious that compiled code often runs faster (sometimes a lot faster) than bytecode-interpreted languages such as Perl or Java, but it totally depends on how the code is written. So may be, you will get some runtime efficiency by having the plugins in C.

I think tuning apache wont do a lot because Nagios has least to do with Apache, you should try some tuning of Nagios itself, follow this article

http://nagios.sourceforge.net/docs/2_0/tuning.html

I hope it will improve the performance a bit.