LinuxQuestions.org - Extremely poor disk performance on SCSI RAID

- Linux - Hardware (https://www.linuxquestions.org/questions/linux-hardware-18/)

- - Extremely poor disk performance on SCSI RAID (https://www.linuxquestions.org/questions/linux-hardware-18/extremely-poor-disk-performance-on-scsi-raid-540917/)

Extremely poor disk performance on SCSI RAID

All,

I'm getting extremely low disk performance on my new server, and suspect I have some incorrect kernel options or hardware configurations. Troughput as follows:

Code:

[189] SUPERUSER@spirou:/tmp#dd if=/dev/sda7 of=/dev/null bs=64k count=100

100+0 records in

100+0 records out

6553600 bytes (6.6 MB) copied, 1.50139 s, 4.4 MB/s

[190] SUPERUSER@spirou:/tmp#dd if=/dev/sda7 of=/dev/null bs=64k count=1000

1000+0 records in

1000+0 records out

65536000 bytes (66 MB) copied, 18.0209 s, 3.6 MB/s

[191] SUPERUSER@spirou:/tmp#dd if=/dev/sda7 of=/dev/null bs=64k count=10000

10000+0 records in

10000+0 records out

655360000 bytes (655 MB) copied, 179.74 s, 3.6 MB/s

As you can see, 3.6MB/s is appallingly low...

Some more system info:

Code:

00:00.0 Host bridge: Broadcom CMIC-WS Host Bridge (GC-LE chipset) (rev 13)

00:00.1 Host bridge: Broadcom CMIC-WS Host Bridge (GC-LE chipset)

00:00.2 Host bridge: Broadcom CMIC-LE

00:01.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)

00:0f.0 Host bridge: Broadcom CSB5 South Bridge (rev 93)

00:0f.1 IDE interface: Broadcom CSB5 IDE Controller (rev 93)

00:0f.2 USB Controller: Broadcom OSB4/CSB5 OHCI USB Controller (rev 05)

00:0f.3 ISA bridge: Broadcom CSB5 LPC bridge

00:11.0 Host bridge: Broadcom CIOB-X2 PCI-X I/O Bridge (rev 03)

00:11.2 Host bridge: Broadcom CIOB-X2 PCI-X I/O Bridge (rev 03)

01:01.0 SCSI storage controller: LSI Logic / Symbios Logic 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 07)

02:01.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5703X Gigabit Ethernet (rev 02)

02:02.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5703X Gigabit Ethernet (rev 02)

Kernel is 2.6.20.4 with SMP - full kernel config can be found here:
http://pastebin.ca/411667

dmesg output can be found here:
http://pastebin.ca/411670

The server in question is an IBM x335 with an LSI Logic 53c1030 SCSI-RAID controller which has two Ultra320 SCSI disks in RAID-1 set up in hardware. Processor is Dual-XEON 2.4Ghz which gives 4 cores in 2.6 Linux kernels. Memory is 2Gb.

Since these are 2x 10kRPM 36GB Ultra320 SCSI disks, something is clearly wrong given the ultra-low read and write data.

Any insight on this would be more than greatly appreciated!

-Y1

Try copying disk to disk - i.e. don't use /dev pseudo device, and see what your numbers look like then.

Edit: Ooops, missed the underlined bit above. I've seen similar numbers from using /dev/urandom to create test files. Copying them disk to disk gets "real" numbers.

Hello, and thank you for responding.

I now did as you recommended and got even worse throughput:

Code:

[495] root@spirou:/root#dd if=testfile.out of=/tmp/test.ttt bs=128k count=1000

1000+0 records in

1000+0 records out

131072000 bytes (131 MB) copied, 86.5747 s, 1.5 MB/s



[496] root@spirou:/root#dd if=testfile.out of=/tmp/test.yalla bs=64k count=1000

1000+0 records in

1000+0 records out

65536000 bytes (66 MB) copied, 12.5143 s, 5.2 MB/s

However, doing the exact same copy once over gives 210MB/s, obviously since the controller (or OS) works in a cache rather than disk. Changing the filename brings me back to the cold, hard reality though...

Any further suggestions would be warmly welcome!

-Y1

I have a quad zeon with hardware raid5, using Adaptec AIC7899 Ultra160 SCSI. Reboot shows my (initial) copy of 80-odd Meg at 8k blocksize getting (variously) 25-30 MB/s. Bumping it to 64k made no discernable difference.

Could the problem perhaps be that I have compiled multiple SCSI-controllers into the kernel, so that they are competing for attention?

Have you compiled only the Adaptec-specific RAID driver, or have you also included the md/raid section?

Couldn't say. In my case, this is an Arch build. i686 optimised, and it loads modules for the hardware it finds - guess it must have the AIC7999 or somesuch in the initrd as I don't recall having to rebuild it.
As it happens I have a test kernel I'm using for cpusets and resource container patch testing. It's all compiled in and as lean as I could sensibly make it.
Testing that shows similar numbers.