Hard Drive performance issue
 

Hi Guys,

I have nearly two identical servers, the only difference between the
two is the version of Linux and the Hard Drive. Server 1 had (what I
think) is a better drive, therefore the performance should be better
then what I'm seeing compared to Server 2. Most notably is the
difference between the dd test on server 1 and 2 as seen below.

Any suggestions on what to check and what would cause this?

Server 1:
Drive is a Barracuda 7200.12 SATA 6Gb/s 1TB Hard Drive w/64 MB Cache
Running XenServer, test performed @ the non-guest level.

hdparm -i /dev/sda

/dev/sda:

Model=ST31000524AS , FwRev=JC4B ,
SerialNo= 5VP9RWSG
Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs RotSpdTol>.5% }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
BuffType=unknown, BuffSize=0kB, MaxMultSect=16, MultSect=?16?
CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=268435455
IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2
UDMA modes: udma0 udma1 udma2
AdvancedPM=no WriteCache=enabled
Drive conforms to: unknown: ATA/ATAPI-4 ATA/ATAPI-5 ATA/ATAPI-6 ATA/ATAPI-7

hdparm -tT /dev/sda

/dev/sda:
Timing cached reads: 43912 MB in 1.99 seconds = 22043.28 MB/sec
Timing buffered disk reads: 344 MB in 3.00 seconds = 114.64 MB/sec
[root@nycxen01 ~]# hdparm /dev/sda


dd if=/dev/zero of=/tmp/output.img bs=8k count=256k
262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 28.0981 seconds, 76.4 MB/s


Server 2:
Drive is a Hitachi Deskstar 7200 SATA 3Gb/s 1TB Drive w/32 MB Cache
Running Debian 6

hdparm -i /dev/sda

/dev/sda:

Model=Hitachi HDS721010CLA332, FwRev=JP4OA3MA, SerialNo=JP2940HZ3L08GC
Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=56
BuffType=DualPortCache, BuffSize=29999kB, MaxMultSect=16, MultSect=16
CurCHS=65535/1/63, CurSects=4128705, LBA=yes, LBAsects=1953525168
IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2
UDMA modes: udma0 udma1 udma2 udma3 udma4 udma5 *udma6
AdvancedPM=yes: disabled (255) WriteCache=enabled
Drive conforms to: unknown: ATA/ATAPI-2,3,4,5,6,7


dd if=/dev/zero of=/tmp/output.img bs=8k count=256k
262144+0 records in
262144+0 records out
2147483648 bytes (2.1 GB) copied, 13.0055 s, 165 MB/s

/dev/sda:
Timing cached reads: 24350 MB in 2.00 seconds = 12191.54 MB/sec
hdparm -tT /dev/sda Timing buffered disk reads: 418 MB in 3.00
seconds = 139.22 MB/sec

I don't think dd is a very good test. I'd use some file transfer or copy.

Use a live cd or live usb to take OS out of the deal.

Wish I could, these are dvdless dedicated servers.

First, you should use the --direct option for hdparm when doing read tests. This way hdparm tries harder to actually read the data from the drive, and not use the page cache. Also, you could clear the page cache first.

Here are the performance results for my two Samsung Spinpoint F1 1TB (HD103UJ) drives:
and on /dev/md2 (a soft-RAID0 of two matching partitions above),
For real-world testing, create a large test file. For example,
This yields a four-gigabyte random file named file .

Using a software RAID0 (striping) on the above two disks (and LVM), a real-world read yields
Using a software RAID1 (mirroring), I get about half that for a single process, using two different files at the same time on RAID1,
yields
i.e. a combined read speed of over 210 MB/s.

Pure write speed is difficult to measure. I use a program that pregenerates a megabyte of random data, then permutates it while writing. Let me know if you wish me to post it, so you can test that too. A copy test, which includes both reading and writing on the same partition, is still useful. Just remember that each byte of data is actually both read and written.

Copy test on RAID1 (mirrored):
yields
and on RAID0 (striped):
As you can see, using software RAID is definitely worth it. Experiences discussed on the linux kernel mailing list indicate that this scales just like this at least up to six drives; twelve (over 1 gigabyte per second real world read speed) if you use a separate PCI-E card for the SATA controllers (so that the bus bandwidth to the SATA controllers is not a bottleneck).

I've also used 4k block size in the tests, so the results should be close to real-world experience. (I use ext4 and 4096 byte blocks.)

I'm seriously bummed out that I did not get four Samsung F3 1TB drives (for about 60 EUR each, 240 EUR total) before the prices doubled. I manipulate large amounts of data, and having that kind of throughput and storage capacity is very, VERY useful. For a server, you should consider pairing the hard drives (preferably with identical ones, otherwise the slower one will dominate), even considering the current prices. It is definitely easily noticeable in the performance.

While these test just one feature of the drives, and cannot therefore be relied on in comparing the disks, I think you'll find that the Hitachi one is the faster one, if you run the commands above on your servers. And, like I said, disk speed (or lack thereof) is easily noticeable in the performance.

A final note: For some reason, many people did not consider Samsung disks high-end, and Samsung ended up selling the division to Seagate. I've used a lot of disks, and these are the ones I like best. They've been in use for 11334 hours (several years, but not 24/7), with almost 700 power cycles. No reallocated sectors, absolutely no hardware problems. Reliable but cheap, and as you can see in the above results, quite fast too.