LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 07-30-2014, 04:39 PM   #1
Lop3
Member
 
Registered: Aug 2013
Posts: 48

Rep: Reputation: Disabled
Question Correct syntax - random read benchmark with iozone


Running iozone in some ways works, and in almost identical ways it gives me strange errors.

my mount is at /my/mount and /my/mount/test does not exist (should be created by iozone)

This is the command I was ideally trying to run something like this, that tests all the main block sizes, from small to large:
iozone -i 5 -i 9 -F /my/mount/test -b /tmp/results.xls
invalid options: must specify -t N before -F

strangely these commands work:
iozone -l 1 -u 1 -F /my/mount/test
iozone -t 1 -F /my/mount/test
iozone -f /my/mount/test
iozone -a -f /my/mount/test

commands that don't work:
iozone -i 5 -F /my/mount/test
invalid options: must specify -t N before -F

iozone -a -F /my/mount/test
invalid options: must specify -t N before -F

iozone -a -l 1 -u 1 -F /my/mount/test
Can not mix throughput mode and auto-mode flags.

iozone -t 1 -F /my/mount/test
Can not mix throughput mode and auto-mode flags.

iozone -a -f /my/mount/test -i 5
Can not open temporary file for read
open: No such file or directory


Interestingly this worked
iozone -i 5 -x -t 1 -l 1 -u 1 -F /my/mount/test

But now it only seems to be testing 512KB blocks

ARG such a PITA
iozone -i 5 -x -t 1 -l 1 -u 1 -f /my/mount/test -y 4k
Can not mix throughput mode and auto-mode flags.

I've gotten almost every error that I can imagine from iozone
Can not run throughput tests with unmount & remounts.

Alternatively can anyone suggest a tool that just-works?

Last edited by Lop3; 07-31-2014 at 03:04 AM.
 
Old 07-30-2014, 05:17 PM   #2
smallpond
Senior Member
 
Registered: Feb 2011
Location: Massachusetts, USA
Distribution: CentOS 6 (pre-systemd)
Posts: 2,761

Rep: Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743Reputation: 743
I haven't tried iozone. vdbench requires java, but will loop through different sizes and patterns of I/O and will create your mountpoints and directories if asked properly. It has a 100-page pdf document and comes with example configurations. Because it is flexible, it is also complex. Instead of command line args, you give it a config file in which you define what you want to do.

To install vdbench (which was written for Solaris) on RedHat what I do is:
Code:
mkdir vdbench502
download the vdbench502.zip file into vdbench502
cd vdbench502
unzip vdbench502.zip
ln /bin/bash /bin/csh
mv vdbench vdbench.csh
cp vdbench.bash vdbench
yum install java-1.7.0-openjdk
./vdbench -t
 
Old 07-31-2014, 02:30 AM   #3
Lop3
Member
 
Registered: Aug 2013
Posts: 48

Original Poster
Rep: Reputation: Disabled
Thanks smallpond.

But I would prefer a simple commandline tool that does not require java. I don't mind using iozone. If someone can offer a command that does the following:
* Run test 5 and test 9
* on a specific file/mountpoint (preferably a file)
* test a full range of block sizes
* output the result to ods or xls

Edit:
I've gotten it to behave reasonably close to what would work for me:
iozone -a /dev/sda1 -y 4k -q 2m -g 2m -b /tmp/sda1.xls

But I still can't select the tests (with or without -a)
iozone -i 5 /dev/sda1 -y 4k -q 2m -g 2m -b /tmp/sda1.xls

Using Minimum Record Size 4 KB
Using Maximum Record Size 2048 KB
Using maximum file size of 2048 kilobytes.
Command line used: iozone -a -i 5 -y 64k -q 2m -g 2m -b /tmp/sda1_cheaplaptop.xls /dev/sda1
Can not open temporary file for read
open: No such file or directory


What does running a specific test have to do with a temporary file?
Even if I specify a temporary file. It gives the same error
iozone -a -i 5 /dev/sda1 -y 64k -q 2m -g 2m -b /tmp/sda1.xls -f /tmp/foo.bar
Can not open temporary file for read
open: No such file or directory

Using a capital F gives complicated errors
iozone -a -i 5 /dev/sda1 -y 64k -q 2m -g 2m -b /tmp/sda1.xls -F /tmp/foo.bar
invalid options: must specify -t N before -F

So I specify -t to try make it happy.
iozone -a -i 5 /dev/sda1 -y 64k -q 2m -g 2m -b /tmp/sda1_cheaplaptop.xls -t 1 -F /tmp/foo.bar
Can not mix throughput mode and auto-mode flags.
Same old problem.

Running the following command, the results are useless to me, because I'm interested in random reads. And the reads are all cached by the time it does the random read test.
iozone -a /dev/sda1 -y 4k -q 2m -g 2m -b /tmp/sda1.xls

Code:
Random Read Report						
	64	128	256	512	1024	2048
64	2662899					
128	3759450	2420455				
256	2943325	2607399	2210228			
512	2512247	2391924	2039772	1546280		
1024	2337008	2397005	2015654	1410125	605538	
2048	2376319	2322353	2033695	1216221	472888	814999
Adding +r does not cure the cache problem. Adding +r and +D also does not.

If I had just written my own application specific benchmark I would have been finished by now :/

Last edited by Lop3; 07-31-2014 at 04:01 AM.
 
Old 07-31-2014, 08:44 AM   #4
iozone
LQ Newbie
 
Registered: Jul 2014
Location: Texas
Distribution: One of everything, on one of every platform type.
Posts: 2

Rep: Reputation: Disabled
Lope,

Wow.. you've been busy.

-F is for use only when using throughput mode ( -t )
-f is for use when using auto mode ( -a )
-i 5 is a test that needs a file to be able to work. So, it must be preceded by -i 0
or a combination of -w and a previous execution with -i 0
(The -w tells Iozone to leave its test files behind)
Example:
Iozone -r 64 -s 1m -t 1 -F /mnt/mymount/test -i 0 -w
Iozone -r 64 -s 1m -t 1 -F /mnt/mymount/test -i 5 -w

-a Auto mode is for testing with a single stream of I/O operations. This is used
to answer the question, "How fast and a single stream application perform I/O ?"

-t Throughput mode is for testing with multiple processes and multiple streams
of I/O operations. It is used to answer the question "What is the maximum aggregate
throughput that this system can achieve?"

-+m in combination with -t can be used to expand the multiple processes and streams
of I/O operations across multiple physical client nodes.

-u sets the upper limit on the number of processes that will be tested. This is
for use in throughput mode, when one is scaling the number of I/O streams
from 1 to N and monitoring how well the system I/O performs as the workload
is increasing.

-x (disable stonewalling) is a special case where one is wishing to relax the testing
of multiple streams such that the measurement is not confined to only the region
of time when all of the processes were performing I/O at the same time.

Combining throughput and auto mode features is not permitted because humans
don't like to process 4 dimensional data in their brains. :-) If one were to permit
the combination of features then one would be able to produce 4D results:
Kbytes/sec on one axis
File size on another axis
Transfer size on another axis
Number of streams on another axis
Thus, Iozone limits the possible combinations such that the output is at most
a three dimensional data set.
Auto mode (3D): File size, Transfer size, Kbytes/sec.
Throughput mode(2D): Number of streams, Kbytes/sec.

-U un-mount between tests... This is limited to only working in auto mode. Getting
this to work reliably in throughput mode, where there may be hundreds of physical
clients participating in the testing, is a bit too painful.

Note: If one doesn't specify a transfer size, or a file size, then Iozone will try to
pick some rational defaults for the testing. If these are not good choices, then
one may wish to use -r and -s to choose more reasonable values for their
system under test.

Note: If one is wishing to control the caching behavior of the client, one may
wish to explore using -I ( this enables O_DIRECT, bypassing caches )
There are also many other options that Iozone supports that affect caching.
(O_SYNC, O_RSYNC, O_DSYNC, file size, aggregate data set size, unmounting
between tests, file locking mode (disables caching on NFS clients ) and
the ability to engage multiple physical clients to present load on a server under
test.

Iozone is a very flexible tool. But because it has such extreme flexibility, it
also can be complex to use. Such is the nature of modern systems.
It can take a while to learn the 52 factorial possible command line combinations.. :-(

BTW, I wrote most of this 20 years ago... my more recent works have considerably
more flexibility... :-)

See:
https://dl.dropboxusercontent.com/u/...sers_Guide.doc

Thank you,
Don Capps
capps@iozone.org
 
Old 07-31-2014, 10:24 AM   #5
Lop3
Member
 
Registered: Aug 2013
Posts: 48

Original Poster
Rep: Reputation: Disabled
Hi Don,

Wow, thanks for taking the time to write that extensive reply.

I'm trying out the examples you gave, but I couldn't get it doing what I need or cooperate as much as I'd like. Perhaps I'll spend some more time on this in future but for now I need to move on.

Hopefully the guidance you've provided here can help others as well
 
Old 07-31-2014, 11:05 AM   #6
iozone
LQ Newbie
 
Registered: Jul 2014
Location: Texas
Distribution: One of everything, on one of every platform type.
Posts: 2

Rep: Reputation: Disabled
Would be glad to help you with any problems you are having. I would
need a bit more descriptive text than:

"I couldn't get it doing what I need or cooperate as much as I'd like"

More details would make it possible for me to assist you with your configuration
and testing :-)

If you wish to take this to email, (should the exchanges be more focused
on your config and not as valuable to the forum members) you can
reach me at: capps at iozone dot org
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
need to run iozone with read+ write parameters !! danibhai Linux - Newbie 1 11-23-2013 08:58 AM
[SOLVED] correct lftp syntax mrmnemo Linux - Software 1 10-09-2010 02:31 PM
What is the correct syntax for MTR? abefroman Linux - Server 1 08-15-2009 11:20 AM
Correct syntax for cu jowa45 Slackware 10 11-01-2007 09:26 AM
what is the correct syntax shanenin Linux - Software 2 10-19-2003 12:31 PM


All times are GMT -5. The time now is 09:50 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration