LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 06-26-2010, 03:23 PM   #1
Sum1
Member
 
Registered: Jul 2007
Distribution: Slackware and Gentoo
Posts: 227

Rep: Reputation: 21
Advice - ext4 tuning and software raid 1


I have a server running Slackware64 ver. 13.
The SATA controller on the motherboard is only SATA I.
The 2 hard drives are 500GB SATA II Western Digital ABYS.

I setup /dev/md0 (in raid 1) on a 100 gig. partition using default settings with ext4 file system. When I do dd write and read testing using 4k, 8k, 16k byte-sizes, I'm only getting a write speed of 22-25 MB/sec. and read speed of 70-72 MB/sec. Not very impressive.

Example tests --

time sh -c "dd if=/dev/zero of=bigfile bs=4k count=_blocks_ && sync"

time dd if=bigfile of=/dev/null bs=4k

I've created 6 additional /dev/md* raid 1 arrays to experiment with and I'd like to get some advice about tuning ext4 on 2 raid1 devices:
block-size
stride
fragment-size
journaling levels - journal, ordered, writeback

Your thoughts and comments are greatly appreciated.
 
Old 06-26-2010, 04:07 PM   #2
Sum1
Member
 
Registered: Jul 2007
Distribution: Slackware and Gentoo
Posts: 227

Original Poster
Rep: Reputation: 21
I should add the environmental usage is probably 75% read and 25% write. 90% percent of file reading/writing is done on files of less than 1 meg.; however, I make .tar archives for storage and move them from partition to partition, and ultimately to other servers for storage. Those .tar files range from 15 - 30 gigs.
 
Old 06-27-2010, 01:25 AM   #3
Sum1
Member
 
Registered: Jul 2007
Distribution: Slackware and Gentoo
Posts: 227

Original Poster
Rep: Reputation: 21
Latest testing showing very slow write speeds:

root@b1:/mnt/arc_sat# time sh -c "dd if=/dev/zero of=/mnt/arc_wed/bigfile bs=4k count=1000000 && sync"
1000000+0 records in
1000000+0 records out
4096000000 bytes (4.1 GB) copied, 383.042 s, 10.7 MB/s

real 6m33.257s
user 0m0.250s
sys 0m18.345s
root@b1:/mnt/arc_sat# time dd if=/mnt/arc_wed/bigfile of=/dev/null bs=4k
1000000+0 records in
1000000+0 records out
4096000000 bytes (4.1 GB) copied, 58.4375 s, 70.1 MB/s

real 0m58.479s
user 0m0.142s
sys 0m6.467s

- - - - - - - - - - - - - - - - - - -
Here's the dumpe2fs and mdadm details of the 2-drive raid1 array:

root@b1:/mnt/arc_sat# dumpe2fs -h /dev/md4
dumpe2fs 1.41.8 (11-July-2009)
Filesystem volume name: <none>
Last mounted on: <not available>
Filesystem UUID: 7a679b0d-ddf4-4591-9b06-5b79236114ae
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file
Filesystem flags: signed_directory_hash
Default mount options: journal_data
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 3940352
Block count: 15731616
Reserved block count: 157316
Free blocks: 14437272
Free inodes: 3940340
First block: 0
Block size: 4096
Fragment size: 4096
Reserved GDT blocks: 1020
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 8192
Inode blocks per group: 512
Filesystem created: Sat Jun 26 18:57:10 2010
Last mount time: Sat Jun 26 21:12:52 2010
Last write time: Sat Jun 26 21:12:52 2010
Mount count: 1
Maximum mount count: 50
Last checked: Sat Jun 26 18:57:10 2010
Check interval: 15552000 (6 months)
Next check after: Thu Dec 23 17:57:10 2010
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 256
Required extra isize: 28
Desired extra isize: 28
Journal inode: 8
Default directory hash: half_md4
Directory Hash Seed: 7583ae16-4e52-495e-b7d3-c8226c367d96
Journal backup: inode blocks
Journal size: 128M

and

root@b1:/mnt/arc_sat# mdadm --detail /dev/md4
/dev/md4:
Version : 00.90.03
Creation Time : Sat Jun 26 15:04:23 2010
Raid Level : raid1
Array Size : 62926464 (60.01 GiB 64.44 GB)
Used Dev Size : 62926464 (60.01 GiB 64.44 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 4
Persistence : Superblock is persistent

Update Time : Sun Jun 27 02:16:48 2010
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

UUID : 412c14ca:8cfb89a3:7cdeb91c:0ba389e0
Events : 0.6

Number Major Minor RaidDevice State
0 8 6 0 active sync /dev/sda6
1 8 22 1 active sync /dev/sdb6
 
Old 06-27-2010, 01:32 AM   #4
Sum1
Member
 
Registered: Jul 2007
Distribution: Slackware and Gentoo
Posts: 227

Original Poster
Rep: Reputation: 21
Does anyone have any suggestions on how to get better performance out of this older SATA I onboard controller?
This is the mainboard:
Arima HDAMA 40-CMO120-A800
http://www.arimacorp.com/ViewProduct.asp?View=86

I've disabled the onboard Silicon Image Sil3114 raid controller.
The raid1 array is mdadm software raid.
 
Old 06-27-2010, 08:48 PM   #5
Sum1
Member
 
Registered: Jul 2007
Distribution: Slackware and Gentoo
Posts: 227

Original Poster
Rep: Reputation: 21
I unmounted all 6 raid1 arrays and reinstalled the ext4 filesystem using the following command:

mke2fs -t ext4 -b 4096 -O has_journal,dir_index /dev/md*

Write speeds have improved to 25MB/sec.
Read speeds are about the same as before 65-70MB/sec.

It's good to see the write speed improvement; however, it still seems much slower than it should be. Seems like these hard drives should be able to sustain 50MB/sec. writes.
 
Old 06-29-2010, 10:45 AM   #6
Sum1
Member
 
Registered: Jul 2007
Distribution: Slackware and Gentoo
Posts: 227

Original Poster
Rep: Reputation: 21
I'm marking this thread solved.
I've tested ext4 with settings of: journal_data, journal_data_writeback, and journal_ordered.
I've increased commit="seconds" to 30, 60, 90, 120
I've tried "noatime" parameter.
And several different mixtures of all three above.

I'm getting the same performance on this hardware: 70MB/s. Read & 24-25MB/s. Write.

I'm speculating, but I'll posit the largest factor affecting performance is the older SATA 1 onboard controller working with Raid1. The cpu(s) are 2 dual-core opterons running at 2.2Ghz with 4 gigs. of ECC RAM, so I don't think there's a performance problem there. When I tested single hard-drive performance on the WD 500Gig. ABYS drives, I was getting 50 MB/s. Writes and 75-85 MB/s. Reads.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Advice: RAID or software data storage? MicahCarrick Linux - Hardware 4 01-11-2008 05:18 PM
Disk I/O Bottlekneck. Need tuning advice MiniMoses Linux - Hardware 2 03-17-2007 12:19 AM
Advice on software RAID solution mr_scary Linux - Server 2 02-18-2007 12:22 AM
Some advice with software raid on an old server? Guttorm Debian 2 07-11-2006 04:38 AM
Filesystem/RAID Performance Tuning/Optimization comsatcat Linux - General 1 06-05-2004 09:54 AM


All times are GMT -5. The time now is 03:40 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration