Very high load and memory usage?

newbie14 · 04-04-2012, 09:38 AM

Dear All,
Below is the value of the command. This machine have no any raid setup. How to decide on how large is the data set that the accesses are scattered across?

Code:

iostat -kdx
Linux 2.6.27.5-117.fc10.x86_64 (localhost.localdomain)  04/04/2012

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu        -sz   await  svctm  %util
sda               0.73   115.31   32.86   17.52   379.73   532.96    36.23     1        .93   38.26   0.46   2.31
sda1              0.00     0.00    0.00    0.00     0.00     0.00    17.93     0        .00    4.43   3.98   0.00
sda2              0.01     0.00    0.00    0.00     0.01     0.00    15.69     0        .00    2.80   2.02   0.00
sda3              0.72   115.31   32.86   17.52   379.72   532.96    36.23     1        .93   38.27   0.46   2.31
dm-0              0.00     0.00   33.58  133.24   379.71   532.96    10.94    14        .11   84.53   0.14   2.31
dm-1              0.00     0.00    0.00    0.00     0.00     0.00     8.00     0        .00    7.69   1.22   0.00

johnsfine · 04-04-2012, 10:17 AM

If that info was captured at a representative time, then my guess was wrong. The disk utilization shown there is light and does not explain why the load numbers significantly exceed the CPU load.

But please use CODE tags rather than QUOTE tags for posting such info, because the way you posted it makes it very hard to read.

Edit: Thanks for changing to CODE tags. Now I see that 14.11 for avgqu-sz for dm-0.

Does anyone here have any clue what that means? I don't use LVM, so I think there is some key detail I'm missing.

It appears that the disk use is essentially all dm-0 and dm-0 (at least the part being accessed) is actually part or all of sda3 and sda3 is, of course, part of sda. So the 2.31 %util of each of those is three different measures of the same actual disk activity. Similarly, the 1.93 for avgqu-sz on sda and sda3 are two different measures of the same queue size. But that makes the 14.11 for dm-0 a total mystery to me.

pantdk · 04-04-2012, 10:55 AM

Hi

i am focusing on these "svctm %util" but it seems ok

can you tell, this is the output of iostat when the load is high ?

when load is high i need only "top" command output

newbie14 · 04-04-2012, 11:55 AM

Dear John,
Sorry I already change it be code tag. Based on which column do you did that the hard disk is very light. I would love to learn and improvise myself to be better to handle this situation next time.

newbie14 · 04-04-2012, 11:56 AM

Dear Pantdk,
Sorry I did not capture it at that moment the iostat. I have put my top results in my first post here.

pantdk · 04-04-2012, 01:00 PM

actually the row of "CPU" in top i am not seen, becz that will also give me the rough idea about i/o wait

example:

Tasks: 125 total, 1 running, 124 sleeping, 0 stopped, 0 zombie

Quote:

Cpu(s): 0.0%us, 0.3%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st

Mem: 249512k total, 235500k used, 14012k free, 31600k buffers
Swap: 0k total, 0k used, 0k free, 103948k cached

newbie14 · 04-04-2012, 08:32 PM

Dear Pantdk,
I do not get you cant see the row or that row gives you some information? How many % of the wa is considered good or safe?

pantdk · 04-05-2012, 01:01 PM

actually i just explaining you that if you will get the out put of "top" command then the cpu row also be there, so i just give you the
output of the top command of my machine & also try to explain the "cpu" row which i required from the command generally it will came after executing the command "top"

when i check your top output in your first post. I didn't see it, so that's why i have given that example,

i/o wait is depending upon the read/write of data in the disk & what is the time it will take to perform these task.

if i/o wait is high, then the cpu load increase, because data take time to write in the disk & also in read. so the number of queues increase the load .

if the HDD is physical then may be one case from hdd that all depend upon the digging the root cause

johnsfine · 04-05-2012, 01:39 PM

Quote:

Originally Posted by pantdk

when i check your top output in your first post. I didn't see it, so that's why i have given that example,

You seem to be asking for the line giving the total across all cores. So far as I know, top gives EITHER the total across all cores OR one line per core as seen in post #1 of this thread. It doesn't give both.

You could easily compute the total from the individual lines if you really want to know the total.

Quote:

i/o wait is depending upon the read/write of data in the disk & what is the time it will take to perform these task.

I'm not certain, but I think you are mistaken. I think i/o wait means that at least one thread assigned to that core is in an I/O wait state at the same moment that no thread assigned to that core can actually use that core.

That would rarely be a useful stat and in this case I think it wouldn't be useful.

Quote:

if the HDD is physical then may be one case from hdd that all depend upon the digging the root cause

Could you translate that to English please

newbie14 · 04-07-2012, 07:38 AM

Dear John,
What indication could give it be when the hdd is having some problem?

johnsfine · 04-07-2012, 12:21 PM

Quote:

Originally Posted by newbie14

What indication could give it be when the hdd is having some problem?

I don't know what you are asking or why with that question.

I think we have pretty much ruled out hard drive accesses as the source of the unexplained difference you reported between load average and CPU use.

newbie14 · 04-07-2012, 07:15 PM

Dear John,
Ok now just asking for extra knowledge. So I guess the final guess for the problem should be then heavy traffic is it.