LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 10-09-2015, 05:32 AM   #1
angel115
Member
 
Registered: Jul 2005
Location: France / Ireland
Distribution: Debian mainly, and Ubuntu
Posts: 537

Rep: Reputation: 79
High CPU load, top not showing any high CPU process


Hi There,

I've got an issue today with one of our server.
We experience a very high load but top doesn't show wich PID is using all the resources.


Code:
top - 12:27:46 up 47 min,  2 users,  load average: 30.15, 24.36, 21.24
Tasks: 163 total,   1 running, 162 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.2%us,  0.0%sy,  0.0%ni, 33.3%id, 66.5%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   4054488k total,   495592k used,  3558896k free,    15432k buffers
Swap:  2099196k total,        0k used,  2099196k free,   193844k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 7528 root      20   0 19296 1420 1016 R    0  0.0   0:01.45 top
    1 root      20   0 23772 1924 1252 S    0  0.0   0:02.13 init
    2 root      20   0     0    0    0 S    0  0.0   0:00.00 kthreadd
    3 root      20   0     0    0    0 S    0  0.0   0:00.02 ksoftirqd/0
    4 root      20   0     0    0    0 S    0  0.0   0:00.00 kworker/0:0
    5 root      20   0     0    0    0 S    0  0.0   0:00.50 kworker/u:0
    6 root      RT   0     0    0    0 S    0  0.0   0:00.00 migration/0
    7 root      RT   0     0    0    0 S    0  0.0   0:00.00 migration/1
    8 root      20   0     0    0    0 S    0  0.0   0:00.00 kworker/1:0
    9 root      20   0     0    0    0 S    0  0.0   0:00.02 ksoftirqd/1
   10 root      20   0     0    0    0 S    0  0.0   0:00.18 kworker/0:1
   11 root      RT   0     0    0    0 S    0  0.0   0:00.00 migration/2
Any idea where I can look to check which process is eating all the resources?

Best regards,
 
Old 10-09-2015, 06:05 AM   #2
Emerson
LQ Guru
 
Registered: Nov 2004
Location: Saint Amant, Acadiana
Distribution: Gentoo ~arch
Posts: 6,317

Rep: Reputation: Disabled
High load average is really not hogging up your resources, it is merely showing there are lots of processes waiting. You should look at processes in D state.
 
Old 10-09-2015, 06:09 AM   #3
angel115
Member
 
Registered: Jul 2005
Location: France / Ireland
Distribution: Debian mainly, and Ubuntu
Posts: 537

Original Poster
Rep: Reputation: 79
Thanks, checking...
 
Old 10-09-2015, 06:13 AM   #4
angel115
Member
 
Registered: Jul 2005
Location: France / Ireland
Distribution: Debian mainly, and Ubuntu
Posts: 537

Original Poster
Rep: Reputation: 79
Ok...

apart of all the apache2 process (which is normal I guess as this a web server) I don't see much things

By the way what is the D state?

Code:
ps -aux | grep "        D    "
Warning: bad ps syntax, perhaps a bogus '-'? See http://procps.sf.net/faq.html
root      1696  0.0  0.0      0     0 ?        D    11:40   0:00 [kjournald]
root     10789  0.0  0.0  84636  3440 ?        D    12:52   0:00 sendmail: MTA: ./t97468dQ016145 from queue
www-data 11340  0.0  0.2 203792  9304 ?        D    12:57   0:00 /usr/sbin/apache2 -k start
root     11932  0.0  0.0  84672  3404 ?        D    13:02   0:00 sendmail: MTA: ./t98BG8AD001049 from queue
www-data 12303  0.0  0.2 203028  8612 ?        D    13:04   0:00 /usr/sbin/apache2 -k start
www-data 12412  0.0  0.2 203720  9256 ?        D    13:05   0:00 /usr/sbin/apache2 -k start
www-data 12448  0.0  0.2 203412  8828 ?        D    13:06   0:00 /usr/sbin/apache2 -k start
www-data 12523  0.0  0.2 203380  8804 ?        D    13:06   0:00 /usr/sbin/apache2 -k start
www-data 12525  0.0  0.2 203156  8760 ?        D    13:06   0:00 /usr/sbin/apache2 -k start
www-data 12597  0.0  0.2 203404  9016 ?        D    13:07   0:00 /usr/sbin/apache2 -k start
www-data 12600  0.0  0.2 207852 10640 ?        D    13:07   0:00 /usr/sbin/apache2 -k start
www-data 12606  0.0  0.2 203692  9152 ?        D    13:07   0:00 /usr/sbin/apache2 -k start
www-data 12609  0.0  0.2 203276  9216 ?        D    13:07   0:00 /usr/sbin/apache2 -k start
www-data 12614  0.0  0.2 203384  8884 ?        D    13:07   0:00 /usr/sbin/apache2 -k start
www-data 12626  0.0  0.2 203300  8920 ?        D    13:07   0:00 /usr/sbin/apache2 -k start
www-data 12628  0.0  0.2 203408  8684 ?        D    13:07   0:00 /usr/sbin/apache2 -k start
www-data 12631  0.0  0.2 203300  9116 ?        D    13:07   0:00 /usr/sbin/apache2 -k start
www-data 12760  0.0  0.2 203356  8356 ?        D    13:08   0:00 /usr/sbin/apache2 -k start
www-data 12842  0.0  0.2 203356  8408 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12843  0.0  0.1 202988  7900 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12844  0.0  0.1 202100  5780 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12845  0.0  0.2 204272  9240 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12846  0.0  0.2 203356  8504 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12848  0.0  0.1 202496  7664 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12850  0.0  0.2 203356  8372 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12851  0.0  0.1 202100  5780 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12854  0.0  0.1 202100  5780 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12855  0.0  0.1 202628  7664 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12856  0.0  0.1 202100  5780 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12858  0.0  0.1 202100  5780 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12861  0.0  0.1 202620  6560 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12865  0.0  0.2 203356  8360 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12866  0.0  0.1 202988  7896 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12867  0.0  0.1 202484  6524 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12868  0.0  0.1 202988  7896 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12869  0.0  0.1 202484  6524 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12870  0.0  0.1 202100  6032 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12871  0.0  0.2 203356  8356 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12876  0.0  0.1 202100  6032 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12877  0.0  0.1 202100  6032 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12878  0.0  0.2 202972  8652 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12879  0.0  0.1 202100  6032 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12880  0.0  0.1 202100  6032 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12881  0.0  0.1 202100  6032 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12882  0.0  0.2 203356  8412 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12883  0.0  0.1 202988  7896 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12884  0.0  0.2 203356  8412 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12885  0.0  0.1 202620  6536 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12886  0.0  0.1 202988  7896 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12887  0.0  0.2 204272  9240 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12889  0.0  0.1 202988  7896 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
www-data 12890  0.0  0.1 202988  7896 ?        D    13:09   0:00 /usr/sbin/apache2 -k start
 
Old 10-09-2015, 06:19 AM   #5
Emerson
LQ Guru
 
Registered: Nov 2004
Location: Saint Amant, Acadiana
Distribution: Gentoo ~arch
Posts: 6,317

Rep: Reputation: Disabled
From man top:
Code:
20. S  --  Process Status
           The status of the task which can be one of:
               D = uninterruptible sleep
               R = running
               S = sleeping
               T = stopped by job control signal
               t = stopped by debugger during trace
               Z = zombie
 
Old 10-09-2015, 07:01 AM   #6
angel115
Member
 
Registered: Jul 2005
Location: France / Ireland
Distribution: Debian mainly, and Ubuntu
Posts: 537

Original Poster
Rep: Reputation: 79
Ok, thanks.

And do you know what could possibly create this issue?
As nothing was touch on the server.

Does HDD latency could create this type of issue?

Thanks for your insight
 
Old 10-09-2015, 08:33 AM   #7
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,237

Rep: Reputation: 1654Reputation: 1654Reputation: 1654Reputation: 1654Reputation: 1654Reputation: 1654Reputation: 1654Reputation: 1654Reputation: 1654Reputation: 1654Reputation: 1654
This has been discussed: http://www.linuxquestions.org/questi...ry-low-315210/
 
Old 10-11-2015, 08:31 PM   #8
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 7.7 (?), Centos 8.1
Posts: 17,782

Rep: Reputation: 2538Reputation: 2538Reputation: 2538Reputation: 2538Reputation: 2538Reputation: 2538Reputation: 2538Reputation: 2538Reputation: 2538Reputation: 2538Reputation: 2538
I'd want to know what those apache processes are actually trying to do - DB access by any chance ? Maybe nfs mount access?
 
Old 10-12-2015, 12:58 AM   #9
angel115
Member
 
Registered: Jul 2005
Location: France / Ireland
Distribution: Debian mainly, and Ubuntu
Posts: 537

Original Poster
Rep: Reputation: 79
Hi All,

Problem solved.

This was a latency issue on our SAN.
After migrating the VM on an other SAN storage the problem is now solved.

I've notice that with the high "wa" on the top command which was on average at 80 +.

Now the "wa" is beween 0 and 2%.

Thank you all for your precious help.

Angel.

Last edited by angel115; 10-12-2015 at 01:00 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Very high CPU load, but nothing significant in top rjbathgate Linux - Server 11 06-06-2014 03:04 PM
[SOLVED] High CPU load, but low CPU usage (high idle CPU) baffy Linux - Newbie 5 03-13-2013 09:24 AM
Load is very high but CPU usage is almost zero in top! mam2 Linux - Server 3 12-18-2009 03:53 PM
CPU load high, top processes very low? Thinking Linux - Software 12 03-19-2007 12:59 AM
X Process: Occasional High CPU Load Fahrion Linux - Software 6 07-16-2005 06:47 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 06:28 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration