'time' for disk io

bigearsbilly · 12-09-2013, 04:15 AM

Is there a way of testing the disk i/o of a process like time does for timing?

I know of pidstat but that only works for running processes.

syg00 · 12-09-2013, 04:56 AM

collectl is also cognisant of taskstats and maintains history, try that.
Or maybe blktrace depending on what you really want.

bigearsbilly · 12-09-2013, 07:39 AM

what I have is a 24x7 daemon with heavy i/o.

I have found now that I need to try and limit the disk i/o using my brain.
So I want to try different optimisations.

I have partial data files which I can run "one-shot"
I want to test different iterations and compare the disk stats
on different runs.

sundialsvcs · 12-10-2013, 08:12 AM

If at all possible, try to adjust the algorithms used by a "disk-I/O intensive" process. If not, look hard at cacheing. Hardware purchases are often called for ... caching controllers, faster drives, solid-state.

bigearsbilly · 12-10-2013, 12:55 PM

Quote:

Originally Posted by sundialsvcs

If at all possible, try to adjust the algorithms used by a "disk-I/O intensive" process.

That's what I am doing all I need is the metrics to prove it! accton looked a good candidate but no disk stats on linux.

Quote:

Originally Posted by sundialsvcs

purchases are often called for ... caching controllers, faster drives, solid-state.

I am on a virtual host, off-site, so it's easy to mess about, but can change only at weekends when it's off line, so then I have to sit on it for a week.

I've had some success, I split disk to a double SSD to spread the load and it keeps it generally below the red line but still revs a bit high on occasion for my taste. I will up the CPU this weekend see if more timeslices will help.

Then the boss wants things done and the bloody customers!

chrism01 · 12-12-2013, 04:45 AM

This looks interesting

Quote:

Well, starting from 2.6.20, Linux Kernel has built-in capability of doing per process I/O accounting. Although, it's not so much easy on other UNIX variants, you can take a look at /proc/PID/io file of a process to determine the I/O activity of that in Linux.

http://jotdownux.blogspot.in/2012/03...ccounting.html

suicidaleggroll · 12-12-2013, 09:02 AM

If these are temporary files, then you could set up a ramdisk to hold them.

bigearsbilly · 12-12-2013, 03:42 PM

Quote:

Originally Posted by chrism01

This looks interesting

http://jotdownux.blogspot.in/2012/03...ccounting.html

yes chris, that might do it.

no can't do ramdisk as the data has to be written and I share it over NFS and you can't share ramdisk.

Heart transplant surgery is very easy, it's keeping the patient alive,
that's the trick

markseger · 12-24-2013, 07:41 AM

one of the tricks with collectl is to run it for awhile or better yet continously and then play back the data with '--top iokb'. to see all the top options try 'collectl --showtopopt' as there are lots of things you can sort by. you can also play back the overall diskio and/or plot it to correlate the two.

only gottcha is to see process i/o stats you need to run collectl as root OR with the processes's uid otherwise collectl can't read the io stats.

-mark