LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 05-08-2008, 10:45 AM   #1
BradRalston
LQ Newbie
 
Registered: May 2008
Posts: 10

Rep: Reputation: 0
Performance Script to track runtime and I/O information


Hey all, I am trying to build a performance tracking script that will tell me information about how fast I can write to a metalun or logical volume in linux. So far I have just created some large files and written a script that captures the time it takes to write. I was hoping for something more sophisticated that will track i/o and performance.
This is what is have so far :

root@isp5599:/root $ cat run_bigfile.s
#!/bin/ksh

#FILELIST="file1 file2 file3"
FILELIST="file1"

#sleeptime=5

for i in ${FILELIST}
do
# record the value of SECONDS before processing begins
start=$SECONDS

print "Processing $i..."

cp /root/bigfile.file /etc/bigfile.file1
cp /root/bigfile.file /etc/bigfile.file2
cp /root/bigfile.file /etc/bigfile.file3
cp /root/bigfile.file /etc/bigfile.file4


# record the value of SECONDS after processing ends
stop=$SECONDS

# calculate processing time
(( runtime=stop-start ))
print "Processing time for $i was $runtime seconds.\n"
echo Processing time was $runtime seconds > /root/bigfile.log
# increase the sleep time for next iteration of the loop
#(( sleeptime=sleeptime*2 ))

rm /etc/bigfile.file*

done
 
Old 05-08-2008, 10:55 AM   #2
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,053
Blog Entries: 14

Rep: Reputation: 1226Reputation: 1226Reputation: 1226Reputation: 1226Reputation: 1226Reputation: 1226Reputation: 1226Reputation: 1226Reputation: 1226
There is already a utility that will measure this for you.

It is called "time" on Linux (timex on some UNIX variants).

You can run a time on each of your copies:

Code:
time cp /root/bigfile.file /etc/bigfile.file1
time cp /root/bigfile.file /etc/bigfile.file2
time cp /root/bigfile.file /etc/bigfile.file3
time cp /root/bigfile.file /etc/bigfile.file4
Then add the metrics together or you could do something like create your copies as a function or separate script then run time against that.

Also you might want to look at the iostat command for performance metrics for writing to devices.
 
Old 05-08-2008, 11:37 AM   #3
BradRalston
LQ Newbie
 
Registered: May 2008
Posts: 10

Original Poster
Rep: Reputation: 0
hmmm. okay. I have a metalun mounted over external storage (san), how do I get the device name for that logical volume in order to get iostat information by device?
 
Old 05-08-2008, 01:43 PM   #4
MensaWater
LQ Guru
 
Registered: May 2005
Location: Atlanta Georgia USA
Distribution: Redhat (RHEL), CentOS, Fedora, CoreOS, Debian, FreeBSD, HP-UX, Solaris, SCO
Posts: 7,053
Blog Entries: 14

Rep: Reputation: 1226Reputation: 1226Reputation: 1226Reputation: 1226Reputation: 1226Reputation: 1226Reputation: 1226Reputation: 1226Reputation: 1226
That question is a little involved. You say metalun then say logical volume then talk about SAN. This isn't surprising as different vendors and/or technologies use the same name for things.

On Linux "generally" a Logical Volume is part of the LVM setup. A meta device is part of the mdadm (Software RAID setup). I suspect you don't mean both or either of those things.

The first thing to do to determine what you want to measure:
df -h /pathtofile
Where pathtofile is a file on the device you want to measure
e.g. df -h /root/bigfile.file OR df -h /etc.

This will tell you both the "device" the filesystem is on as well as the "mountpoint" it has been mounted on.

Most of the time df -h would show /etc is on / (unless you have for some reason made /etc a separate filesystem which would be very unusal). e.g.

df -h /etc
Filesystem Size Used Avail Use% Mounted on
/dev/md0 7.8G 5.6G 1.8G 77% /

-OR-

df -h /etc
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LogVol00
5.0G 1.2G 3.6G 25% /

Notice on both systems /etc does indeed show it is mounted on /. However, on the first system the "device" is /dev/md0 which is a metadisk. On the second it is on something completely different. Ignoring the mapper part of the name the important information is that this is Logical Volume named LogVol00 on Volume Group named VolGroup00.

To determine the "disks" associated with each you can use:
vgdisplay -v VolGroup00 (see physical volumes at end of output).
-OR-
mdadm -D /dev/md0 (see the entries at end of output).

"disks" in Linux are generally /dev/sda, /dev/sdb, /dev/sdc etc... (the a-z are the disks in order discovered)

Note that you'll also see things like /dev/sda1, /dev/sda2, /dev/sda3. These numbers denote partions on the disk /dev/sda (for example).

To make things even more fun multipathing tools (such as PowerPath for EMC) create "luns" (logical units) names that should be used instead of the underlying device. That is to say a device like /dev/emcpowera on my system that uses PowerPath is actually the multipath device. The tools I use with PowerPath or Navisphere (another EMC product) would show me the underlying "disks" (e.g. emcpowera might have 4 paths which might be /dev/sda, /dev/sde, /dev/sdi & /dev/sdn).

To determine THAT relationship depends a lot on what Multipathing tool you are using. I'm guessing you're using one from Hitachi although Linux itself does have its own multipathing tool. Since I haven't used either of those I can't offer guidance on their use but hopefully what I've given you above at least gets you started on looking for the right ones.

Of course you may NOT need the underlying devices. iostat should provide information the multipath device itself as well as the underlying devices as for example:

iostat
Linux 2.6.18-8.1.14.el5 (hostname.dom.com) 05/08/2008

avg-cpu: %user %nice %system %iowait %steal %idle
0.73 0.00 1.07 0.08 0.00 98.13

Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sda 19.69 0.02 0.00 147325 3747
sdb 1.20 0.01 0.00 130396 3728
sdc 0.00 0.00 0.00 2486 0
sdd 0.00 0.00 0.00 2044 0
sde 0.00 0.00 0.00 2044 0
sdf 0.00 0.00 0.00 2042 0
sdg 0.00 0.00 0.00 1972 0
sdh 0.00 0.00 0.00 3585 0
sdi 0.00 0.00 0.00 2506 0
sdj 0.00 0.00 0.00 2370 0
sdk 0.00 0.00 0.00 2367 0
sdl 0.00 0.00 0.00 2370 0
sdm 0.00 0.00 0.00 2278 0
sdn 0.00 0.00 0.00 1836 0
sdo 0.00 0.00 0.00 1836 0
sdp 0.00 0.00 0.00 1834 0
sdq 0.00 0.00 0.00 1836 0
sdr 0.00 0.00 0.00 3457 0
sds 0.00 0.00 0.00 2378 0
sdt 0.00 0.00 0.00 2378 0
sdu 0.00 0.00 0.00 2375 0
sdv 0.00 0.00 0.00 2378 0
dm-0 10.19 0.69 20.34 6550512 191948114
dm-1 0.01 0.02 0.01 203134 122430
dm-2 1.90 0.00 3.79 20216 35807602
dm-3 2.65 1.76 5.09 16599792 48060020
dm-4 33.33 0.44 66.62 4135930 628634174
dm-5 0.00 0.01 0.03 100736 241360
dm-6 0.00 0.00 0.00 696 0
dm-7 0.01 1.07 0.00 10119734 5374
dm-8 5.37 1.57 42.85 14776114 404360744
emcpowerd 1.62 289.62 121.11 2733038605 1142880384
emcpowere 1.00 286.66 108.58 2705094391 1024673910
emcpowerc 2.14 117.58 46.98 1109557418 443367130
emcpowerb 4.40 173.22 58.94 1634654858 556197647
emcpowera 3.04 4.05 1.01 38232870 9548603

Last edited by MensaWater; 05-08-2008 at 01:55 PM.
 
Old 05-08-2008, 07:51 PM   #5
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,411

Rep: Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397
incidentally, there is a GNU cmd \time which gives you more info than std time eg:

time ./t.sh

real 0m0.002s
user 0m0.000s
sys 0m0.000s


\time ./t.sh
0.00user 0.00system 0:00.00elapsed 200%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+404minor)pagefaults 0swaps
 
Old 05-09-2008, 09:19 AM   #6
BradRalston
LQ Newbie
 
Registered: May 2008
Posts: 10

Original Poster
Rep: Reputation: 0
Thanks for all the great information.
 
Old 05-09-2008, 04:03 PM   #7
BradRalston
LQ Newbie
 
Registered: May 2008
Posts: 10

Original Poster
Rep: Reputation: 0
Any ideas on how you would accurately test the read speed off of the filesystem ?
 
Old 05-12-2008, 10:00 PM   #8
imadoofus
LQ Newbie
 
Registered: May 2008
Posts: 6

Rep: Reputation: 0
I'd start with iozone and sar.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Skype runtime error: libstdc++.so.6: no version information billion Linux - Software 1 01-21-2008 12:43 PM
Looking for php script to keep track of payments received michaeljoser Linux - Software 3 11-05-2007 07:26 AM
Amarok (v 1.4.4): Track information etc. not obtained on Debian system PeteD Linux - Software 2 09-04-2007 06:36 PM
What is script PID information? Stepaj47 Linux - Newbie 7 11-15-2005 03:50 PM
strange request: script to track logged in time hamster Linux - General 3 05-08-2003 01:52 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 01:39 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration