Hello,

We have noticed a strange behaviour of the OpenVZ server that we have. One of the machines on the node is highly loaded. All of it resources are taken, but there is many free resources on the node, as you can see on the below results.

[root@ovz01 ~]# top
top - 18:26:13 up 28 days, 9:02, 1 user, load average: 14.04, 11.09, 7.37
Tasks: 315 total, 21 running, 292 sleeping, 0 stopped, 2 zombie
Cpu(s): 16.7%us, 1.8%sy, 0.0%ni, 81.5%id, 0.0%wa, 0.0%hi, 0.1%si, 0.0%st
Mem: 8031308k total, 6993140k used, 1038168k free, 478168k buffers
Swap: 10485752k total, 0k used, 10485752k free, 5125828k cached

- This is the laverage of all ctids :

[root@ovz01 ~]# vzlist -o ctid,laverage
CTID LAVERAGE
102 0.00/0.00/0.00
105 14.31/11.68/7.90
106 0.00/0.00/0.00
107 0.00/0.00/0.00
108 0.00/0.00/0.00
109 0.00/0.00/0.00

- The CPU utilisation :

[root@ovz01 ~]# vzcpucheck
Current CPU utilization: 1042
Power of the node: 480017

- The kernel version :

[root@ovz01 ~]# uname -a
2.6.32-042stab079.5 #1 SMP Fri Aug 2 17:16:15 MSK 2013 x86_64 x86_64 x86_64 GNU/Linux

My question is - Can we trust the TOP command and only the VM resources are taken, but not the resources of the whole node, or there is another better way to monitor the node resources ?

Thank you in advance!

Hello

the "top" command indicates the cpu usage (%) in real time, which is not relevant is some cases. Especially if many processes are using the processor for a short period of time but very often... This counter is usefull when a big huge process is saturating the computation capabilities of your node, or when you have a bottleneck with the interruptions.

I believe that you should pay more attention to the load average. Compare the the load average given by "top" (or rather /proc/loadavg) with the one given by "vzlist -o ctid,laverage". In your case, we see that vmid 105 is consuming all the cpu resource and that the node is not loaded by its own processes.

You can also build a script which calculates a dedicated load average with ps by removing the openvz processes. Something like :
ps -eo state,cmd | grep -v "Something which identifies your vm in the cmd" | grep "^R" | wc -l
However, you will have to execute this each seconds within 5 minutes and aggregate the results minus the number of executions (because "ps" will find itself)
That's just an idea, hope this helps !

bests
julien

I am also having very strange CPU behavior from OpenVZ clients. Can you post some information about your OpenVZ server and clients so we can compare notes? I have setup a server with one client to keep things simple. This is an issue I am willing to put some work into.

Promox Server kernel 2.6.32-26-pve

processor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 31
model name : AMD Athlon(tm) 64 Processor 3000+
stepping : 0
cpu MHz : 1808.291
cache size : 512 KB
fpu : yes
fpu_exception : yes
cpuid level : 1
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt lm 3dnowext 3dnow up rep_good extd_apicid lahf_lm
bogomips : 3616.58
TLB size : 1024 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp

The Client is CentOS release 6.2 hosting Asterisk