OS: Red Hat Enterprise Linux ES release 4 (Nahant Update 4) (no support planned to upgrade to CentOS 5.5)
Linux AIR06 2.6.9-42.ELsmp #1 SMP Wed Jul 12 23:32:02 EDT 2006 x86_64 x86_64 x86_64 GNU/Linux
Intel(R) Xeon(R) CPU 5160 @ 3.00GHz x 4
SWAP=/dev/sda6
Want to know the cause for high load average and iowait.
Want to confirm if I am running out of memory or the cause is something different?
Code:
top - 17:00:41 up 1057 days, 10:48, 4 users, load average: 24.11, 16.06, 8.45
Tasks: 740 total, 1 running, 739 sleeping, 0 stopped, 0 zombie
Cpu(s): 3.2% us, 5.6% sy, 6.3% ni, 33.0% id, 51.8% wa, 0.0% hi, 0.2% si
Mem: 8166092k total, 8125536k used, 40556k free, 3752k buffers
Swap: 8385888k total, 4264540k used, 4121348k free, 43000k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
422 root 15 0 0 0 0 D 0 0.0 115:21.88 kjournald
1998 root 15 0 0 0 0 D 0 0.0 67:25.16 kjournald
28473 root 16 0 3628 596 488 D 0 0.0 0:00.00 syslogd
9740 root 15 0 0 0 0 D 0 0.0 0:00.00 pdflush
25344 root 15 0 0 0 0 D 0 0.0 0:00.00 pdflush
27329 root 18 0 5356 944 796 D 0 0.0 0:00.00 run-parts
Total status D: 6
Code:
free -o
total used free shared buffers cached
Mem: 8166092 8147336 18756 0 3752 43644
Swap: 8385888 4281388 4104500
Code:
iostat -kd 1
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 472.73 1074.75 0.00 1064 0
sda6 255.56 1022.22 0.00 1012 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 7.07 48.48 0.00 48 0
sda2 1.01 4.04 0.00 4 0
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 472.16 919.59 98.97 892 96
sda6 218.56 874.23 0.00 848 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 24.74 37.11 74.23 36 72
sda2 8.25 8.25 24.74 8 24
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 455.10 457.14 0.00 448 0
sda6 114.29 457.14 0.00 448 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 407.14 424.49 363.27 416 356
sda6 91.84 367.35 0.00 360 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 61.22 57.14 216.33 56 212
sda2 36.73 0.00 146.94 0 144
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 446.94 1228.57 0.00 1204 0
sda6 303.06 1212.24 0.00 1188 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 1.02 16.33 0.00 16 0
sda2 0.00 0.00 0.00 0 0
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 413.27 1077.55 8.16 1056 8
sda6 267.35 1069.39 0.00 1048 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 3.06 8.16 4.08 8 4
sda2 1.02 0.00 4.08 0 4
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 364.65 2533.33 404.04 2508 400
sda6 628.28 2513.13 0.00 2488 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 59.60 20.20 222.22 20 220
sda2 45.45 0.00 181.82 0 180
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 510.20 6942.86 0.00 6804 0
sda6 1734.69 6938.78 0.00 6800 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 1.02 4.08 0.00 4 0
sda2 0.00 0.00 0.00 0 0
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 468.04 6103.09 0.00 5920 0
sda6 1525.77 6103.09 0.00 5920 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 476.77 6339.39 0.00 6276 0
sda6 1584.85 6339.39 0.00 6276 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 425.51 5648.98 179.59 5536 176
sda6 1412.24 5648.98 0.00 5536 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda2 44.90 0.00 179.59 0 176
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 182.65 3085.71 412.24 3024 404
sda6 766.33 3065.31 0.00 3004 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 108.16 20.41 412.24 20 404
sda2 0.00 0.00 0.00 0 0
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 18.81 277.23 0.00 280 0
sda6 69.31 277.23 0.00 280 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 7.14 69.39 0.00 68 0
sda6 17.35 69.39 0.00 68 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 16.33 265.31 0.00 260 0
sda6 66.33 265.31 0.00 260 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 135.35 2319.19 226.26 2296 224
sda6 579.80 2319.19 0.00 2296 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda2 56.57 0.00 226.26 0 224
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 47.96 102.04 465.31 100 456
sda6 25.51 102.04 0.00 100 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 116.33 0.00 465.31 0 456
sda2 0.00 0.00 0.00 0 0
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 4.08 57.14 0.00 56 0
sda6 14.29 57.14 0.00 56 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda1 0.00 0.00 0.00 0 0
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda 9.18 142.86 0.00 140 0
sda6 35.71 142.86 0.00 140 0
sda5 0.00 0.00 0.00 0 0
sda4 0.00 0.00 0.00 0 0
sda3 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
sda1 0.00 0.00 0.00 0 0
High iowait%. How to find the cause?
