LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Hardware
User Name
Password
Linux - Hardware This forum is for Hardware issues.
Having trouble installing a piece of hardware? Want to know if that peripheral is compatible with Linux?

Notices

Reply
 
Search this Thread
Old 03-27-2007, 12:01 AM   #1
Yalla-One
Member
 
Registered: Oct 2004
Location: Norway
Distribution: Slackware, CentOS
Posts: 635

Rep: Reputation: 35
Extremely poor disk performance on SCSI RAID


All,

I'm getting extremely low disk performance on my new server, and suspect I have some incorrect kernel options or hardware configurations. Troughput as follows:
Code:
[189] SUPERUSER@spirou:/tmp#dd if=/dev/sda7 of=/dev/null bs=64k count=100
100+0 records in
100+0 records out
6553600 bytes (6.6 MB) copied, 1.50139 s, 4.4 MB/s
[190] SUPERUSER@spirou:/tmp#dd if=/dev/sda7 of=/dev/null bs=64k count=1000
1000+0 records in
1000+0 records out
65536000 bytes (66 MB) copied, 18.0209 s, 3.6 MB/s
[191] SUPERUSER@spirou:/tmp#dd if=/dev/sda7 of=/dev/null bs=64k count=10000
10000+0 records in
10000+0 records out
655360000 bytes (655 MB) copied, 179.74 s, 3.6 MB/s
As you can see, 3.6MB/s is appallingly low...

Some more system info:
Code:
00:00.0 Host bridge: Broadcom CMIC-WS Host Bridge (GC-LE chipset) (rev 13)
00:00.1 Host bridge: Broadcom CMIC-WS Host Bridge (GC-LE chipset)
00:00.2 Host bridge: Broadcom CMIC-LE
00:01.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
00:0f.0 Host bridge: Broadcom CSB5 South Bridge (rev 93)
00:0f.1 IDE interface: Broadcom CSB5 IDE Controller (rev 93)
00:0f.2 USB Controller: Broadcom OSB4/CSB5 OHCI USB Controller (rev 05)
00:0f.3 ISA bridge: Broadcom CSB5 LPC bridge
00:11.0 Host bridge: Broadcom CIOB-X2 PCI-X I/O Bridge (rev 03)
00:11.2 Host bridge: Broadcom CIOB-X2 PCI-X I/O Bridge (rev 03)
01:01.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07)
02:01.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5703X Gigabit Ethernet (rev 02)
02:02.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5703X Gigabit Ethernet (rev 02)
Kernel is 2.6.20.4 with SMP - full kernel config can be found here:
http://pastebin.ca/411667

dmesg output can be found here:
http://pastebin.ca/411670

The server in question is an IBM x335 with an LSI Logic 53c1030 SCSI-RAID controller which has two Ultra320 SCSI disks in RAID-1 set up in hardware. Processor is Dual-XEON 2.4Ghz which gives 4 cores in 2.6 Linux kernels. Memory is 2Gb.

Since these are 2x 10kRPM 36GB Ultra320 SCSI disks, something is clearly wrong given the ultra-low read and write data.

Any insight on this would be more than greatly appreciated!

-Y1
 
Old 03-27-2007, 07:52 PM   #2
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 12,057

Rep: Reputation: 971Reputation: 971Reputation: 971Reputation: 971Reputation: 971Reputation: 971Reputation: 971Reputation: 971
Try copying disk to disk - i.e. don't use /dev pseudo device, and see what your numbers look like then.

Edit: Ooops, missed the underlined bit above. I've seen similar numbers from using /dev/urandom to create test files. Copying them disk to disk gets "real" numbers.

Last edited by syg00; 03-27-2007 at 08:15 PM.
 
Old 03-29-2007, 09:10 AM   #3
Yalla-One
Member
 
Registered: Oct 2004
Location: Norway
Distribution: Slackware, CentOS
Posts: 635

Original Poster
Rep: Reputation: 35
Hello, and thank you for responding.

I now did as you recommended and got even worse throughput:

Code:
[495] root@spirou:/root#dd if=testfile.out of=/tmp/test.ttt bs=128k count=1000
1000+0 records in
1000+0 records out
131072000 bytes (131 MB) copied, 86.5747 s, 1.5 MB/s

[496] root@spirou:/root#dd if=testfile.out of=/tmp/test.yalla bs=64k count=1000
1000+0 records in
1000+0 records out
65536000 bytes (66 MB) copied, 12.5143 s, 5.2 MB/s
However, doing the exact same copy once over gives 210MB/s, obviously since the controller (or OS) works in a cache rather than disk. Changing the filename brings me back to the cold, hard reality though...

Any further suggestions would be warmly welcome!

-Y1
 
Old 03-29-2007, 07:12 PM   #4
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 12,057

Rep: Reputation: 971Reputation: 971Reputation: 971Reputation: 971Reputation: 971Reputation: 971Reputation: 971Reputation: 971
I have a quad zeon with hardware raid5, using Adaptec AIC7899 Ultra160 SCSI. Reboot shows my (initial) copy of 80-odd Meg at 8k blocksize getting (variously) 25-30 MB/s. Bumping it to 64k made no discernable difference.
 
Old 03-29-2007, 07:14 PM   #5
Yalla-One
Member
 
Registered: Oct 2004
Location: Norway
Distribution: Slackware, CentOS
Posts: 635

Original Poster
Rep: Reputation: 35
Could the problem perhaps be that I have compiled multiple SCSI-controllers into the kernel, so that they are competing for attention?

Have you compiled only the Adaptec-specific RAID driver, or have you also included the md/raid section?
 
Old 03-29-2007, 08:04 PM   #6
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 12,057

Rep: Reputation: 971Reputation: 971Reputation: 971Reputation: 971Reputation: 971Reputation: 971Reputation: 971Reputation: 971
Couldn't say. In my case, this is an Arch build. i686 optimised, and it loads modules for the hardware it finds - guess it must have the AIC7999 or somesuch in the initrd as I don't recall having to rebuild it.
As it happens I have a test kernel I'm using for cpusets and resource container patch testing. It's all compiled in and as lean as I could sensibly make it.
Testing that shows similar numbers.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Extremely slow disk read access software raid 5 Magsimoe Linux - Software 2 12-18-2006 04:59 PM
SCSI/software RAID performance inadequate stefan_nicolau Linux - Hardware 0 07-19-2006 09:48 PM
Extremely poor SATA performance (Inappropriate ioctl for device) kyletriggs Linux - Hardware 7 02-14-2006 01:14 AM
[Fedora Core 3]Poor write performance with raid controller TomG22 Linux - Hardware 1 09-22-2005 09:25 AM
Extremely poor cedega performance king111 Linux - Games 1 07-15-2005 08:48 PM


All times are GMT -5. The time now is 03:35 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration