Correct syntax - random read benchmark with iozone
 

Running iozone in some ways works, and in almost identical ways it gives me strange errors.

my mount is at /my/mount and /my/mount/test does not exist (should be created by iozone)

This is the command I was ideally trying to run something like this, that tests all the main block sizes, from small to large:
iozone -i 5 -i 9 -F /my/mount/test -b /tmp/results.xls
invalid options: must specify -t N before -F

strangely these commands work:
iozone -l 1 -u 1 -F /my/mount/test
iozone -t 1 -F /my/mount/test
iozone -f /my/mount/test
iozone -a -f /my/mount/test

commands that don't work:
iozone -i 5 -F /my/mount/test
invalid options: must specify -t N before -F

iozone -a -F /my/mount/test
invalid options: must specify -t N before -F

iozone -a -l 1 -u 1 -F /my/mount/test
Can not mix throughput mode and auto-mode flags.

iozone -t 1 -F /my/mount/test
Can not mix throughput mode and auto-mode flags.

iozone -a -f /my/mount/test -i 5
Can not open temporary file for read
open: No such file or directory

Interestingly this worked
iozone -i 5 -x -t 1 -l 1 -u 1 -F /my/mount/test

But now it only seems to be testing 512KB blocks

ARG such a PITA
iozone -i 5 -x -t 1 -l 1 -u 1 -f /my/mount/test -y 4k
Can not mix throughput mode and auto-mode flags.

I've gotten almost every error that I can imagine from iozone
Can not run throughput tests with unmount & remounts.

Alternatively can anyone suggest a tool that just-works?

I haven't tried iozone. vdbench requires java, but will loop through different sizes and patterns of I/O and will create your mountpoints and directories if asked properly. It has a 100-page pdf document and comes with example configurations. Because it is flexible, it is also complex. Instead of command line args, you give it a config file in which you define what you want to do.

To install vdbench (which was written for Solaris) on RedHat what I do is:

Thanks smallpond.

But I would prefer a simple commandline tool that does not require java. I don't mind using iozone. If someone can offer a command that does the following:
* Run test 5 and test 9
* on a specific file/mountpoint (preferably a file)
* test a full range of block sizes
* output the result to ods or xls

Edit:
I've gotten it to behave reasonably close to what would work for me:
iozone -a /dev/sda1 -y 4k -q 2m -g 2m -b /tmp/sda1.xls

But I still can't select the tests (with or without -a)
iozone -i 5 /dev/sda1 -y 4k -q 2m -g 2m -b /tmp/sda1.xls

Using Minimum Record Size 4 KB
Using Maximum Record Size 2048 KB
Using maximum file size of 2048 kilobytes.
Command line used: iozone -a -i 5 -y 64k -q 2m -g 2m -b /tmp/sda1_cheaplaptop.xls /dev/sda1
Can not open temporary file for read
open: No such file or directory

What does running a specific test have to do with a temporary file?
Even if I specify a temporary file. It gives the same error
iozone -a -i 5 /dev/sda1 -y 64k -q 2m -g 2m -b /tmp/sda1.xls -f /tmp/foo.bar
Can not open temporary file for read
open: No such file or directory

Using a capital F gives complicated errors
iozone -a -i 5 /dev/sda1 -y 64k -q 2m -g 2m -b /tmp/sda1.xls -F /tmp/foo.bar
invalid options: must specify -t N before -F

So I specify -t to try make it happy.
iozone -a -i 5 /dev/sda1 -y 64k -q 2m -g 2m -b /tmp/sda1_cheaplaptop.xls -t 1 -F /tmp/foo.bar
Can not mix throughput mode and auto-mode flags.
Same old problem.

Running the following command, the results are useless to me, because I'm interested in random reads. And the reads are all cached by the time it does the random read test.
iozone -a /dev/sda1 -y 4k -q 2m -g 2m -b /tmp/sda1.xls

Adding +r does not cure the cache problem. Adding +r and +D also does not.

If I had just written my own application specific benchmark I would have been finished by now :/

Lope,

Wow.. you've been busy.

-F is for use only when using throughput mode ( -t )
-f is for use when using auto mode ( -a )
-i 5 is a test that needs a file to be able to work. So, it must be preceded by -i 0
or a combination of -w and a previous execution with -i 0
(The -w tells Iozone to leave its test files behind)
Example:
Iozone -r 64 -s 1m -t 1 -F /mnt/mymount/test -i 0 -w
Iozone -r 64 -s 1m -t 1 -F /mnt/mymount/test -i 5 -w

-a Auto mode is for testing with a single stream of I/O operations. This is used
to answer the question, "How fast and a single stream application perform I/O ?"

-t Throughput mode is for testing with multiple processes and multiple streams
of I/O operations. It is used to answer the question "What is the maximum aggregate
throughput that this system can achieve?"

-+m in combination with -t can be used to expand the multiple processes and streams
of I/O operations across multiple physical client nodes.

-u sets the upper limit on the number of processes that will be tested. This is
for use in throughput mode, when one is scaling the number of I/O streams
from 1 to N and monitoring how well the system I/O performs as the workload
is increasing.

-x (disable stonewalling) is a special case where one is wishing to relax the testing
of multiple streams such that the measurement is not confined to only the region
of time when all of the processes were performing I/O at the same time.

Combining throughput and auto mode features is not permitted because humans
don't like to process 4 dimensional data in their brains. :-) If one were to permit
the combination of features then one would be able to produce 4D results:
Kbytes/sec on one axis
File size on another axis
Transfer size on another axis
Number of streams on another axis
Thus, Iozone limits the possible combinations such that the output is at most
a three dimensional data set.
Auto mode (3D): File size, Transfer size, Kbytes/sec.
Throughput mode(2D): Number of streams, Kbytes/sec.

-U un-mount between tests... This is limited to only working in auto mode. Getting
this to work reliably in throughput mode, where there may be hundreds of physical
clients participating in the testing, is a bit too painful.

Note: If one doesn't specify a transfer size, or a file size, then Iozone will try to
pick some rational defaults for the testing. If these are not good choices, then
one may wish to use -r and -s to choose more reasonable values for their
system under test.

Note: If one is wishing to control the caching behavior of the client, one may
wish to explore using -I ( this enables O_DIRECT, bypassing caches )
There are also many other options that Iozone supports that affect caching.
(O_SYNC, O_RSYNC, O_DSYNC, file size, aggregate data set size, unmounting
between tests, file locking mode (disables caching on NFS clients ) and
the ability to engage multiple physical clients to present load on a server under
test.

Iozone is a very flexible tool. But because it has such extreme flexibility, it
also can be complex to use. Such is the nature of modern systems.
It can take a while to learn the 52 factorial possible command line combinations.. :-(

BTW, I wrote most of this 20 years ago... my more recent works have considerably
more flexibility... :-)

See:
https://dl.dropboxusercontent.com/u/...sers_Guide.doc

Thank you,
Don Capps
capps@iozone.org

Hi Don,

Wow, thanks for taking the time to write that extensive reply.

I'm trying out the examples you gave, but I couldn't get it doing what I need or cooperate as much as I'd like. Perhaps I'll spend some more time on this in future but for now I need to move on.

Hopefully the guidance you've provided here can help others as well :)

Would be glad to help you with any problems you are having. I would
need a bit more descriptive text than:

"I couldn't get it doing what I need or cooperate as much as I'd like"

More details would make it possible for me to assist you with your configuration
and testing :-)

If you wish to take this to email, (should the exchanges be more focused
on your config and not as valuable to the forum members) you can
reach me at: capps at iozone dot org