Multi-thread execution efficiency problem
 

Hello,

I met a puzzle on coding a zero data program for many disks under a multi-thread case. The low-level component is written by C && SCSI WRITE and it is called by a JAVA mulit-thread program(One Disk one thread).

I found that 15 threads around can reach to 60MB/Sec Zero Speed whereas 30 threads around could only get 30MB/Sec or so. That's to say, the more threads involved, the slower speed it could get. On the same time, CPU load was sticked to 32-37% and Mem at 0.6% only no matter 15 or 30 threads running[TOP Showed]. It seems that there is still some room to improve. But I have no idea where I SHOULD fix. Could anyone shed a light on it:)

Thanks in advance,

cdcshu

"On the same time, CPU load was sticked to 32-37% and Mem at 0.6% only no matter 15 or 30 threads running[TOP Showed]. It seems that there is still some room to improve. But I have no idea where I SHOULD fix."

The fact that the CPU maxed at 32-37% means that the CPU is not the critical resource. The fact that the Mem maxed at 0.6% means that the Mem is not the critical resource. Check the other components that you are using to see if one of them hits 100% usage. Some possible critical resources are disk arm movement, network speed, and video buffer access.

-----------------
Steve Stites

there are various scheduling algorithms to choose from in the kernel config.
some grant a lower level of latency in getting to interupts others give faster throughput and a higher level of latency. You can also try different levels of hardware latency "setpci".
different size buffers might effect burst size as well.
also remember you have realtime threads if you need it to controll scheduling.

Good explaination:-)
 

Hello jailbait,

I am clearer about TOP result from your notes. One of the critical resources is Disk itself, I bet. In some degree, SCSI WRITE(10) I used have stressed the disk to reach its max write speed because SCSI Command works on disk directly and there is no other barrier to be involved. If really have, PCIe HBA card(Disk Channel) could be considered. However, I have no clue on it.

Like studioj suggested, for the moment, I can give it a shot on kernel config and choose another scheduling algorithms. In my case, IO process performance is more important.

Thanks for both of your nice reply

cdcshu