Script to find Disk utilized processes.

pinga123 · 05-25-2011, 11:13 PM

I have written a script that triggers a mail if the server load average goes beyond a specific value.

The mail contains following field
Current Load average.
Top 10 CPU utilized processes.

Code:

ps -auxf | sort -nr -k 3 | head -10

Top 10 Memory Utilized processes.

Code:

ps -auxf | sort -nr -k 4 | head -10

But the problem is that whenever there is any disk related activity happens the load gets high and the command (for ex.mkfs.ext3 /dev/sdb1, dd,scp,cp)which are the main cause behind the load average doesnt get displayed in top 10 CPU/Memory Utilized processes.

Is there any way of finding top 10 processes for Disk related activity?

macemoneta · 05-25-2011, 11:30 PM

Code:

iotop -b -n1 -o -P -qqq | head -n 10

The iotop command is like top, but lists processes doing I/O.

pinga123 · 05-26-2011, 12:07 AM

Quote:

Originally Posted by macemoneta

Code:

iotop -b -n1 -o -P -qqq | head -n 10

The iotop command is like top, but lists processes doing I/O.

iotop not installed in the distribution.

macemoneta · 05-26-2011, 12:23 AM

Most major distributions have it available for installation. You can also get it here.

syg00 · 05-26-2011, 12:46 AM

iotop Taskstats requires taskstats to be:
a) available in the kernel, and
b) enabled.

If the distro doesn't ship iostats, it's likely they don't (yet) have a kernel that supports it. Try a later release of the distro.

I have grave concerns about using loadavg as a trigger like this in Linux. It's quite common to see high loadavg with little or no CPU usage and little (or no) I/O.
But, it's Linux - the choice is yours.

pinga123 · 05-26-2011, 02:22 AM

Quote:

Originally Posted by syg00

iotop Taskstats requires taskstats to be:
a) available in the kernel, and
b) enabled.

If the distro doesn't ship iostats, it's likely they don't (yet) have a kernel that supports it. Try a later release of the distro.

I have grave concerns about using loadavg as a trigger like this in Linux. It's quite common to see high loadavg with little or no CPU usage and little (or no) I/O.
But, it's Linux - the choice is yours.

I would like to hear what causes the High Loadavg if its not CPU or I/O?

16pide · 05-26-2011, 06:14 AM

iotop works great.
It was not installed by default, but I used yum to get it and I love it.

@pinga123: which distro do you use, and doesn't it have iotop in its repository?

pinga123 · 05-26-2011, 06:48 AM

Quote:

Originally Posted by 16pide

iotop works great.
It was not installed by default, but I used yum to get it and I love it.

@pinga123: which distro do you use, and doesn't it have iotop in its repository?

I checked the repository but no package starting from iotop* found.

Code:

# lsb_release -a
LSB Version:    :core-3.1-ia32:core-3.1-noarch:graphics-3.1-ia32:graphics-3.1-noarch
Distributor ID: EnterpriseEnterpriseServer
Description:    Enterprise Linux Enterprise Linux Server release 5.2 (Carthage)
Release:        5.2
Codename:       Carthage

Tried downloading and installing iotop from external sources but the iotop seems to be made for Linux >= 2.6.20

Code:

# rpm -ivh python-ctypes-1.0.2-2.el5.i386.rpm
warning: python-ctypes-1.0.2-2.el5.i386.rpm: Header V3 DSA signature: NOKEY, key ID 217521f6
Preparing...                ########################################### [100%]
   1:python-ctypes          ########################################### [100%]
# rpm -ivh iotop-0.4.1-2.el5.noarch.rpm
warning: iotop-0.4.1-2.el5.noarch.rpm: Header V3 DSA signature: NOKEY, key ID 217521f6
Preparing...                ########################################### [100%]
   1:iotop                  ########################################### [100%]
# iotop
Could not run iotop as some of the requirements are not met:
- Linux >= 2.6.20 with
  - I/O accounting support (CONFIG_TASKSTATS, CONFIG_TASK_DELAY_ACCT, CONFIG_TASK_IO_ACCOUNTING)
# uname -r
2.6.18-92.el5

stress_junkie · 05-26-2011, 06:50 AM

iotop is in the sysstat package.

Edit: correction. iostat is in the sysstat package. iotop is all by itself in the Ubuntu (my system's) repository.

pinga123 · 05-26-2011, 06:59 AM

is iotop and iostat does the same? I dont have iostat too is there easier way to enable it?
edited:
Enabled iostat by installing sysstat package.
Does iostat tells which process is causing high io utilization. by simply giving iostat results in total read/write on every disk.

16pide · 05-27-2011, 03:08 AM

I am guessing you are using redhat 5.2 from your kernel name.
I did a bit of googling, andhere is what I found:
The EPEL repository could be what you need to get the iotop rpm:
http://www.cyberciti.biz/faq/rhel-fe...ble-epel-repo/

http://download.fedora.redhat.com/pu...r_i.group.html

pinga123 · 05-30-2011, 10:38 PM

Quote:

Originally Posted by 16pide

I am guessing you are using redhat 5.2 from your kernel name.
I did a bit of googling, andhere is what I found:
The EPEL repository could be what you need to get the iotop rpm:
http://www.cyberciti.biz/faq/rhel-fe...ble-epel-repo/

http://download.fedora.redhat.com/pu...r_i.group.html

Installed iotop from the link you mentioned but now i m confused on how to fulfill its requirement Do i need to change the Kernel parameter as mentioned in below output? How?.

Code:

# iotop
Could not run iotop as some of the requirements are not met:
- Linux >= 2.6.20 with
  - I/O accounting support (CONFIG_TASKSTATS, CONFIG_TASK_DELAY_ACCT, CONFIG_TASK_IO_ACCOUNTING)

16pide · 06-06-2011, 07:52 AM

to check your kernel version, can you post output of:

Code:

uname -a

And can someone comment on the flags pinga123 mentions in his latest post?

pinga123 · 06-06-2011, 11:20 PM

Quote:

Originally Posted by 16pide

to check your kernel version, can you post output of:

Code:

uname -a

And can someone comment on the flags pinga123 mentions in his latest post?

Its 2.6.18.

syg00 · 06-06-2011, 11:29 PM

(just noticed this thread again).
As I stated above, you must be on a kernel that has taskstats support. At 2.6.18, no (userspace) tool can provide the data as it doesn't exist.

Upgrade is the only answer.