LinuxQuestions.org
Latest LQ Deal: Latest LQ Deals
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware
User Name
Password
Linux - Hardware This forum is for Hardware issues.
Having trouble installing a piece of hardware? Want to know if that peripheral is compatible with Linux?

Notices


Reply
  Search this Thread
Old 11-17-2010, 06:30 PM   #1
digdogger
Member
 
Registered: Mar 2007
Distribution: CentOS
Posts: 32

Rep: Reputation: 2
RAID, stride, and stripe-size, oh my!


Hello. I've been Google searching for ways to improve the performance of our webhosting server, as disk I/O latency and throughput are currently huge bottlenecks in the performance of our servers. I've found:

http://storageadvisors.adaptec.com/2...t-stripe-size/
http://wiki.centos.org/HowTos/Disk_Optimization
http://osdir.com/ml/linux.file-syste.../msg00045.html
http://old.nabble.com/Calculating-st...d10979268.html
http://uclibc.org/~aldot/mkfs_stride.html

But reading through all this just leaves me with more questions than answers.

1. According to http://wiki.centos.org/HowTos/Disk_Optimization : "Sadly, the 'stripe-width=' extended-option has disappeared from man mkfs.ext3 as of CentOS 5.3" ... anybody have any idea why this is? Perhaps the "stripe-width" option is not valid for ext3, and it was a bug to include it in the first place, so that is why it was removed? It appears to be available again in mkfs.ext4 .

2. What is the effect of using tune2fs (or tune4fs) on an existing filesystem to change the values of stride and stripe-size ? Does it just effect files that are newly created or modified after the changes are made, so that existing files on the filesystem are not optimized?

3. How do you query the current values of stride and stripe-size on an existing filesystem? dumpe2fs -h doesn't appear to have the info, nor does dumpe4fs -h or tune4fs -l .

4. Does using LVM have any effect on the effectiveness of the stripe-width or stride options?

Thanks.
 
Old 11-19-2010, 08:49 PM   #2
Electro
LQ Guru
 
Registered: Jan 2002
Posts: 6,042

Rep: Reputation: Disabled
It seems you have set up a RAID-0 or a RAID-5. This is fine for database and file servers, but not the right one for web hosting. Web hosting requires latency, but not throughput. You should setup a RAID-1 or RAID-10. I have not used a RAID, but from experience 16 kilobyte is best for hard drives although do a test on your own. Setting the stripe and the ext spec stride is a trial and benching is only way to get the best performance.

The utility tune2fs only will make changes on newly creating files.

LVM2 or EVMS will not provide you any performance. It gives you higher capacity.

If you think you have a bottleneck for your server, it is best to look at the software first. If using MySQL, is the queries too complex? If so any server will be slow and it does not matter how fast the server is. Also if the server opens up too many connections, the bandwidth will choke and it will seem it is behind a phone modem. There is the kernel you can optimize by using the utility sysctl. Not using GUI will help a lot. If the scripts allow it, setup caching. Caching can help with repetitive tasks.

From reading some sites, "dumpe2fs -h" does in fact provide information. You should pipe it through grep.
 
Old 11-26-2010, 05:49 PM   #3
digdogger
Member
 
Registered: Mar 2007
Distribution: CentOS
Posts: 32

Original Poster
Rep: Reputation: 2
Thanks for your reply, Electro. We were previously using raid 6, but when hosting around 1000 ish sites you really start to notice the IO wait lag. Using the 'sar' command provided by the 'sysstat' RPM package, we're seeing 10 minute %iowait shooting up to 30% or 40% at times. So we're switching to raid 10. According to dbench and bonnie++, the throughput is higher and latency is much lower using raid 10 on our hardware (IBM x3650).

When we setup a new machine with raid 10, I took the time to do dbench tests with and without stride and stripe-width settings set. The system is hardware raid 10, 6 disks, 256 KB stripe size, and a block size of 4096 . According to http://uclibc.org/~aldot/mkfs_stride.html , we should be using stride=64 and stripe-width=192 . I ran dbench 10 times in a row and saved the output into text files (with and without stide/stripe-width set, for a total of 20 times). The exact command was: dbench 10 -t 60 -c /usr/local/share/client.txt . I then created a Perl script to find the minimum, average, and maximum of the throughput and latency of all of the 'execute' lines in each of the dbench runs. The results were:

Code:
RAID10 and noatime turned on:

                     Minimum              Average              Maximum
         Throughput: 100.26               174.163508474576     222.91     (MB/sec)
            Latency: 46.232               231.828379661017     1176.877   (ms)
Code:
RAID10 and noatime turned on and stride / stripe-width set:

                     Minimum              Average              Maximum
         Throughput: 77.13                158.045508474576     194.74     (MB/sec)
            Latency: 39.927               251.216361016949     1432.605   (ms)
So it appears to have made very little difference. In fact, it appears to have made things a bit worse. Which is why I wonder if stride / stripe-width does anything at all in ext3? I was asking about LVM as I was wondering if using LVM has any effect on the stride / stripe-width calculations, since LVM creates various data structures that will consume disk space, and perhaps create some sort of 'offset' that needs to be taken into account when doing the stride / stripe-width calculations.

And you're saying "From reading some sites, "dumpe2fs -h" does in fact provide information. You should pipe it through grep." Do you happen to have any links to those sites handy? I have run "dumpe2fs -h", "dumpe4fs -h", and "tune4fs -l" with "grep -i stride" and "grep -i stripe" and didn't get any results.

Last edited by digdogger; 11-26-2010 at 05:54 PM.
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
lvm: PE size, number of stripes and stripe size considerations tramni1980 Slackware 1 08-07-2009 11:19 AM
Installing onto Raid 0 Stripe? Dxvid Linux - Newbie 2 03-13-2008 08:59 AM
Suse 10 raid stripe rozzo Linux - Newbie 2 12-14-2005 11:39 AM
raid-0 XFS stripe size and stripe width ganja_guru Linux - Software 1 09-04-2005 09:15 AM
RAID0 Stripe size? ganja_guru Linux - Hardware 3 08-18-2005 05:46 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware

All times are GMT -5. The time now is 05:14 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration