Share your knowledge at the LQ Wiki.
Go Back > Forums > Linux Forums > Linux - Server
User Name
Linux - Server This forum is for the discussion of Linux Software used in a server related context.


  Search this Thread
Old 10-05-2012, 12:20 PM   #1
LQ Newbie
Registered: Sep 2012
Location: Canada
Posts: 2

Rep: Reputation: Disabled
iSCSI write performance very poor while read performance is excellent

I've setup a Gentoo system with 15x3TB SATA drives on two Supermicro AOC-SAT2-MV8 controllers. Individually each drive is capable of ~160MB/s read and write (tested with hdparm and dd on the Gentoo system). After building a simple RAID6 array on all 15 drives (39TB effective) I find that the PCI-X bus is saturated at 200MB/s. I see 200MB/s read and 200MB/s write. It's an older Supermicro P4SCi motherboard so that's expected. Each drive maxes out at 40MB/s when they are all being accessed in the array.

The same system has 2 gigabit ethernet adapters. One is connected to the PCI bus and the other is connected to the northbridge. I've setup the system as an iSCSI target, and assigned both ethernet adapters an IP address on separate gigabit networks. Using netperf I can saturate both ports at ~110MB/s (full gigabit speeds) simultaneously.

Now for my problem... I've setup a Debian initiator with multipathing on both networks. I can easily saturate either of the two paths with read tests. But when I attempt to write to the iSCSI target I can never get above 7MB/s. The write performance is horrible.

I'm confident it isn't a hardware issue, because on the iSCSI target I can use DD to read from the array at 157MB/s, while saturating eth0 at 117MB/s and eth1 at 62MB/s all at the same time.

I would expect to see 157MB/s for sequential reads and 157MB/s for sequential writes to/from the array from the initiator.

Using atop I can see my target CPU usage is ~25% system and 75% idle while reading at ~80MB/s and my IRQ usage is 5%. But while writing to the target I can see the same CPU usage of 25% but the IRQ usage jumps to 75%!

Why does my IRQ usage jump to 75% while writing to the target from the initiator? What can I do to prevent that? Is that even the cause the problem?

Last edited by dinominant; 10-05-2012 at 12:22 PM.
Old 10-10-2012, 11:51 AM   #2
LQ Newbie
Registered: Sep 2012
Location: Canada
Posts: 2

Original Poster
Rep: Reputation: Disabled
Due to time constraints I will have to revisit this problem at a later date. I did have better performance and I believe the issue was an MTU mismatch between the target and the initiator but I cannot verify that this was the case until I can spend some more time on this system.


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
Software Raid 6 - poor read performance / fast write performance Kvothe Linux - Server 0 02-28-2011 04:11 PM
Problem with GFS read and write performance rbh123 Linux - Enterprise 1 10-06-2008 05:24 AM
Software RAID5 - poor write performance & freezing Nitrowolf Linux - Hardware 3 09-25-2005 10:49 PM
[Fedora Core 3]Poor write performance with raid controller TomG22 Linux - Hardware 1 09-22-2005 10:25 AM
poor java socket read performance bobwall Linux - Networking 0 01-21-2005 10:55 PM > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 07:26 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration