Some processes use 99% CPU but overall CPU usage is 20-30%?

Vita · 07-15-2013, 07:23 AM

Hi,

I'm troubled with this question for a long time now. I don't know how to solve that. I'm hosting few HLDS game servers on my server.

I've noticed that top command shows that some users are using 90-100% of CPU usage, but the overall CPU usage is 20-30%. Those HLDS servers are lagging and have low FPS rate, so PINGS for all players are way too high 100ms+ and they should be around 5-30ms.

http://img824.imageshack.us/img824/2945/arqv.jpg

So is there a way to somehow increase users CPU usage limit, so the game servers (HLDS) can run smoother with higher FPS?

I'm using the default preempt kernel that came with Debian 6 (i686).

Thanks in advance!

bmrtin · 07-15-2013, 10:12 AM

It looks like this is a single processor application meaning that it can only use one processor at a time. Your limiting factor here is how fast the single core speed is. If there is a multicore version or setting this may help spread out the process. What you see in top for the cpu% for the process is how much of the cpu you are using. For example if I had a multicore application that is using 100% of 2 of the 4 cores in a system it would show 200% (core 1 100% + core 2 100% + core 3 0% + core 4 0% = 200%)

Vita · 07-15-2013, 10:16 AM

Is there a way to balance the processes to all cores, or at least start specific processes on specific cores?

The output of mpstat -P ALL shows this:

Code:

Linux 2.6.33.2-100Hz (premium2)         07/15/2013      _i686_  (8 CPU)

04:47:24 PM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest   %idle
04:47:24 PM  all   23.24   17.74    4.80    0.04    0.13    0.52    0.00    0.00   53.54
04:47:24 PM    0   22.40   16.61    4.65    0.05    1.03    2.98    0.00    0.00   52.29
04:47:24 PM    1   23.48   17.87    4.85    0.06    0.00    0.18    0.00    0.00   53.56
04:47:24 PM    2   23.40   17.84    4.82    0.06    0.00    0.18    0.00    0.00   53.71
04:47:24 PM    3   23.41   17.84    4.82    0.05    0.00    0.18    0.00    0.00   53.70
04:47:24 PM    4   23.80   17.43    4.84    0.03    0.00    0.16    0.00    0.00   53.75
04:47:24 PM    5   23.20   18.11    4.80    0.03    0.00    0.17    0.00    0.00   53.69
04:47:24 PM    6   23.14   18.11    4.80    0.03    0.00    0.17    0.00    0.00   53.76
04:47:24 PM    7   23.09   18.09    4.79    0.03    0.00    0.17    0.00    0.00   53.83

Thanks in advance!

bmrtin · 07-15-2013, 11:24 AM

You can http://www.cyberciti.biz/tips/settin...r-process.html

In your case I don't think it would help to much because all core being the same it would be like pouring all the water out of one solo cup into a different (but same size) solo cup because the first one was too full

Vita · 07-15-2013, 08:40 PM

So you think there is no way of balancing the load somehow? System has 4 CPUs with 2 threads per CPU?
I've tried the taskset but it didn't help much. On the other side I haven't tried to put for example all game servers on one CPU and only one HLDS game server on the other.

My other question is can kernel be the problem for poor performance?

Thanks in advance!

suicidaleggroll · 07-15-2013, 08:50 PM

From the output of your mpstat command, the load is already balanced...almost exactly 23% on each processor. How do you expect to balance the load any more than this?

If all threads are doing the same thing though, you may get a performance improvement by disabling hyperthreading. Most of my setups involve doing one task (or multiple similar tasks) simultaneously, and in that kind of setup I typically see a 10-30% improvement in processing speed by disabling hyperthreading and dropping back to 1 thread per physical cpu. In one specific application the system is WAY too slow to keep up with real time with hyperthreading enabled, it hits 150+% CPU (on a single core machine) and starts dropping data almost immediately, yet with hyperthreading disabled it hovers around 85% CPU usage and has no problem keeping up with the demands for days on end.

I've made it a habit to disable hyperthreading on every system I build, because I have yet to run across a use-case in my own environment in which HT does anything but make the system slower.

Vita · 07-15-2013, 09:09 PM

Thanks for posting, but the main problem with this is that I want to increase the CPU utilization, so processes that I run can get more CPU power. My CPU is at 23% load but I want it to go higher and increase performance of the game servers, is there a way to that? I don't currently have access to the server, so disabling HT is only possible I guess by compiling a new kernel without HT enabled, am I correct?

Thanks in advance!

jefro · 07-17-2013, 07:51 PM

Performance of a system for some use has little to do with cpu load.

If you want to utilize the server more then you need to use more tasks.

One might be able to use other vm's or other non-related tasks to help offset the purchase price. It would however result in higher energy costs and most likely reduced length of service and permit failure of parts sooner calender time.

Vita · 07-17-2013, 08:30 PM

So I guess it's not possible or it's not that easy.

Thanks all for your help!

selfprogrammed · 07-18-2013, 11:15 AM

From appearances, that 53% idle time is due to some waiting within the program.
It could be communication handshaking, or VSYNC, or just frame syncing. Deal with that to increase your utilization. This involves tweaking your program settings.
If desperate then instrument the program or use a profiling tool to find where it is waiting.

zhjim · 07-18-2013, 01:36 PM

As the output of systat looks alright to me. Specialy the (io)wait i would check for some network bound hog ups. Check out traffic shaping with token buckets or some to have the hlds server to get enough bandwith. tldp.org has some nice things about this.