LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
Reload this Page monitoring i/o usage
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 10-21-2015, 02:13 PM   #1
Bashed
Member
 
Registered: Jul 2015
Posts: 133

Rep: Reputation: Disabled
monitoring i/o usage

I'm having a hard time understanding some of the i/o monitoring tools available in Linux. I'm running Centos 6 here, a few vps servers within (OpenVZ). Trying to figure out why some of the VM's are slow and occasionally logging into ssh on the man node itself is sluggish, randomly. RAM is usually around 50-605 usage, even at the time of sluggishness, so is CPU (3%). Dual processors, Xeon with 12GB RAM.

Can someone please translate the below in laymen terms for me?

Code:
[root@vps ~]# sar
Linux 2.6.32-042stab083.2 (vps.somehost.com)    10/21/2015      _x86_64_        (8 CPU)

12:00:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:10:02 AM     all      4.86      0.56      4.27     14.92      0.00     75.40
12:20:01 AM     all      6.10      0.57      3.98     12.18      0.00     77.17
12:30:01 AM     all      4.97      0.55      4.10      6.13      0.00     84.26
12:40:01 AM     all      4.76      0.37      4.01      9.28      0.00     81.58
12:50:02 AM     all      5.28      0.56      4.34     10.95      0.00     78.87
01:00:01 AM     all      5.68      0.40      4.25      6.26      0.00     83.41
01:10:13 AM     all      6.62      0.57      4.31     20.63      0.00     67.87
01:20:01 AM     all      5.77      0.40      3.97     14.63      0.00     75.24
01:30:01 AM     all      5.21      0.54      4.14      6.47      0.00     83.64
01:40:01 AM     all      5.07      1.33      4.28      9.63      0.00     79.69
01:50:08 AM     all      4.77      1.69      4.67     29.28      0.00     59.59
02:00:01 AM     all      6.78      2.31      4.46     25.37      0.00     61.08
02:10:02 AM     all      6.38      1.08      4.73     30.46      0.00     57.35
02:20:01 AM     all      5.97      0.39      4.41      8.45      0.00     80.77
02:30:01 AM     all      7.56      0.56      5.06      5.74      0.00     81.07
02:40:01 AM     all      4.14      1.38      3.74      5.93      0.00     84.82
02:50:01 AM     all      4.80      0.56      4.14     15.33      0.00     75.17
03:00:01 AM     all      4.95      0.38      3.97      9.48      0.00     81.22
03:10:01 AM     all      6.12      0.55      4.47     17.69      0.00     71.16
03:20:01 AM     all      6.81      1.14      5.14      6.18      0.00     80.74
03:30:01 AM     all      7.03      4.94      5.33      6.51      0.00     76.19
03:40:01 AM     all      5.96      3.60      4.93     10.67      0.00     74.85
03:50:01 AM     all      8.46      1.46      4.67      5.94      0.00     79.47
04:00:01 AM     all      6.65      0.38      4.04      8.32      0.00     80.61
04:10:06 AM     all      9.22      0.58      6.39     19.46      0.00     64.35
04:20:01 AM     all      4.52      0.38      3.97     11.88      0.00     79.24
04:30:03 AM     all      6.11      0.65      4.44     15.32      0.00     73.47
04:40:02 AM     all      6.84      0.59      5.59     25.70      0.00     61.28
04:50:02 AM     all      6.96      0.54      5.21     36.45      0.00     50.83
05:00:01 AM     all      4.35      0.39      4.40     15.03      0.00     75.82
05:10:01 AM     all      6.76      0.57      4.94     21.71      0.00     66.02
05:20:08 AM     all      5.29      0.39      4.88     33.48      0.00     55.95
05:30:05 AM     all      7.29      0.79      4.77     17.61      0.00     69.53
05:40:02 AM     all      4.85      0.63      4.17      9.07      0.00     81.28

Code:
[root@vps ~]# iostat -x 1
Linux 2.6.32-042stab083.2 (vps.somehost.com)    10/21/2015      _x86_64_        (8 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.52    0.00    1.16   74.64    0.00   23.67

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.00     0.00    0.00    0.00     0.00     0.00     0.00   162.00    0.00   0.00 100.00
sdb               0.00     1.00    0.00    4.00     0.00    40.00    10.00     0.00    1.00   0.25   0.10
dm-0              0.00     0.00    0.00    0.00     0.00     0.00     0.00    73.00    0.00   0.00 100.00
dm-1              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
dm-2              0.00     0.00    0.00    5.00     0.00    40.00     8.00    96.00    1.00 200.00 100.00

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.25    0.00    4.79   71.04    0.00   23.93

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.00     0.00    0.00    0.00     0.00     0.00     0.00   162.00    0.00   0.00 100.00
sdb               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
dm-0              0.00     0.00    0.00    0.00     0.00     0.00     0.00    73.00    0.00   0.00 100.00
dm-1              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
dm-2              0.00     0.00    0.00    0.00     0.00     0.00     0.00    96.00    0.00   0.00 100.00

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           0.25    0.00    3.38   72.09    0.00   24.28

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.00     0.00    0.00    0.00     0.00     0.00     0.00   162.00    0.00   0.00 100.00
sdb               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
dm-0              0.00     0.00    0.00    0.00     0.00     0.00     0.00    73.00    0.00   0.00 100.00
dm-1              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
dm-2              0.00     0.00    0.00    0.00     0.00     0.00     0.00    96.00    0.00   0.00 100.00

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           1.92    0.13    3.07   64.83    0.00   30.05

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.00     2.00    3.00   41.00    32.00   512.00    12.36   153.84 20399.64  22.73 100.00
sdb               0.00   431.00    1.00   59.00     8.00  5120.00    85.47     0.39    6.38   1.48   8.90
dm-0              0.00     0.00    0.00    4.00     0.00    32.00     8.00    65.15 148481.25 250.00 100.00
dm-1              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
dm-2              0.00     0.00    3.00  495.00    24.00  5416.00    10.92   100.09  710.40   2.01 100.00

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           5.13    0.26    5.26   53.15    0.00   36.20

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.00     5.00   32.00  157.00   920.00  3336.00    22.52    90.81 13068.50   5.29 100.00
sdb              89.00   393.00  113.00  120.00  5592.00  5256.00    46.56     0.80    3.48   1.30  30.40
dm-0              0.00     0.00    0.00   22.00     0.00   248.00    11.27    30.09 53207.05  30.45  67.00
dm-1              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
dm-2              0.00     0.00  245.00  501.00  7392.00  5296.00    17.01    64.86 1802.80   1.34 100.10

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          16.22    4.85    4.34   36.78    0.00   37.80

Device:         rrqm/s   wrqm/s     r/s     w/s   rsec/s   wsec/s avgrq-sz avgqu-sz   await  svctm  %util
sda               7.00     0.00   15.00   15.00   720.00   416.00    37.87    15.79 6977.77  33.33 100.00
sdb               0.00    90.00   46.00   31.00  2064.00   992.00    39.69     0.05    0.60   0.49   3.80
dm-0              0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
dm-1              0.00     0.00    8.00    0.00    64.00     0.00     8.00     0.77    0.00  12.00   9.60
dm-2              0.00     0.00   55.00  120.00  2272.00   992.00    18.65    15.88 1197.41   5.71  99.90
Some Specs

Code:
[root@vps ~]# cat /sys/block/sda/queue/rotational
1
[root@vps ~]# cat /sys/block/sdb/queue/rotational
0

Code:
[root@vps ~]# fdisk -l

Disk /dev/sda: 250.0 GB, 250000000000 bytes
255 heads, 63 sectors/track, 30394 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00021bc2

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1       30273   243162112   8e  Linux LVM

Disk /dev/sdb: 512.1 GB, 512110190592 bytes
255 heads, 63 sectors/track, 62260 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0007fed0

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *           1          64      512000   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sdb2              64       62261   499594240   8e  Linux LVM

Disk /dev/mapper/VolGroup-lv_root: 53.7 GB, 53687091200 bytes
255 heads, 63 sectors/track, 6527 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/VolGroup-lv_swap: 4160 MB, 4160749568 bytes
255 heads, 63 sectors/track, 505 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/mapper/VolGroup-lv_home: 702.7 GB, 702730469376 bytes
255 heads, 63 sectors/track, 85435 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
 
Old 10-21-2015, 09:10 PM   #2
Rinndalir
Member
 
Registered: Sep 2015
Posts: 733

Rep: Reputation: Disabled
Something's hitting sda hard. Figure out what that is and you may get on the right track. Maybe you have a disk problem.
 
Old 10-21-2015, 09:37 PM   #3
Bashed
Member
 
Registered: Jul 2015
Posts: 133

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by Rinndalir View Post
Something's hitting sda hard. Figure out what that is and you may get on the right track. Maybe you have a disk problem.
The purpose of my thread is to figure out what's going on and that's the point I posted here. Also, it's sdb (the SSD drive is primary) not the sda.

I mentioned the low memory/cpu usage for a reason.

I see nothing unusual...

Code:
[root@vps ~]# iotop
Total DISK READ: 13.03 K/s | Total DISK WRITE: 1178.97 K/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND                                                                                                                                                                   
  940 be/4 root        0.00 B/s  127.02 K/s  0.00 %  0.00 % [flush-253:2]
141968 be/4 nobody      0.00 B/s    6.51 K/s  0.00 %  0.00 % httpd -k start
556333 be/4 root        0.00 B/s    6.51 K/s  0.00 %  0.00 % supervise tinydns----.---.254.28
556335 be/4 root        0.00 B/s    6.51 K/s  0.00 %  0.00 % supervise axfrdns----.---.254.25
556351 be/4 root        0.00 B/s    6.51 K/s  0.00 %  0.00 % supervise axfrdns----.---.254.26
556363 be/4 root        0.00 B/s    6.51 K/s  0.00 %  0.00 % supervise tinydns----.---.254.30
95207 be/5 lighttpd    0.00 B/s    0.00 B/s  0.00 %  0.00 % pmtad 
38294 be/4 root        0.00 B/s   94.45 K/s  0.00 %  0.00 % python /usr/bin/yum -d 2 -y --disablerepo=rpmforge,ius,remi install postfix mysql-serv~ython-lxml mod_wsgi fail2ban unzip bzip2 acl patch tmpwatch crontabs dos2unix logwatch
556341 be/4 root        0.00 B/s    6.51 K/s  0.00 %  0.00 % supervise tinydns----.---.254.26
556343 be/4 root        0.00 B/s    6.51 K/s  0.00 %  0.00 % supervise axfrdns----.---.254.30
556349 be/4 root        0.00 B/s    6.51 K/s  0.00 %  0.00 % supervise axfrdns----.---.254.27
556353 be/4 root        0.00 B/s    6.51 K/s  0.00 %  0.00 % supervise tinydns----.---.254.27
556355 be/4 root        0.00 B/s    6.51 K/s  0.00 %  0.00 % supervise axfrdns----.---.254.28
556359 be/4 root        0.00 B/s    6.51 K/s  0.00 %  0.00 % supervise tinydns----.---.254.25
556365 be/4 root        0.00 B/s    6.51 K/s  0.00 %  0.00 % supervise axfrdns----.---.254.29
556369 be/4 root        0.00 B/s    6.51 K/s  0.00 %  0.00 % supervise tinydns----.---.254.29
556419 be/4 321         0.00 B/s    3.26 K/s  0.00 %  0.00 % multilog t ./main
556425 be/4 321         0.00 B/s    6.51 K/s  0.00 %  0.00 % multilog t ./main
    1 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % init
    2 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [kthreadd]
    3 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/0]
    4 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [ksoftirqd/0]
    5 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/0]
    6 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [watchdog/0]
    7 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/1]
    8 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/1]
    9 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [ksoftirqd/1]
   10 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [watchdog/1]
   11 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/2]
   12 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/2]
   13 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [ksoftirqd/2]
   14 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [watchdog/2]
   15 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/3]
   16 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/3]
   17 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [ksoftirqd/3]
   18 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [watchdog/3]
   19 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/4]
   20 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [migration/4]
   21 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [ksoftirqd/4]
   22 rt/4 root        0.00 B/s    0.00 B/s  0.00 %  0.00 % [watchdog/4]
Last edited by Bashed; 10-21-2015 at 09:46 PM.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Monitoring SWAP Usage instrumentpilot Linux - General 5 04-10-2012 07:51 AM
[SOLVED] Monitoring Usage Konphine Linux - Newbie 14 09-24-2011 03:54 AM
Monitoring Flash usage Shamis Linux - Embedded & Single-board computer 2 04-25-2011 01:16 PM
Monitoring System Usage dunmarie Linux - Newbie 2 11-11-2003 12:37 AM
Monitoring processor usage bananaman Linux - General 3 04-22-2003 06:21 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 09:21 AM.

Contact Us - Advertising Info - Rules - Privacy - LQ Merchandise - Donations - Contributing Member - LQ Sitemap -
Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration