Im going to be running a rather large computation/simulation on an HP DL560 G9 box which needs approx 150Tb of storage and about 4GByte/sec throughput for random read/write.
Looking for suggestions which don't break the bank. The computation is going to take about 1 year to complete, even with this HW so speed is important.

4 Gb/s is a very high data troughput.
Modern PCIe SSDs deliver ~2 GB/s.
These PCIe SSDs have 2TB write volume.
E.g.: Intel 1,2 TB PCIe 3.0 with 2,4 GB/s read and 1,2 GB/s write speed.

But the data transfer method from storage to computer might be a problem.

I was thinking of a 16-bay 8TB SAS config, using 2x 8i LSI SAS controllers.
That would give me access to 2x pcie 3.0 x4 lanes.
What im not sure of is the disks being fast enough, even in a raid 5 config.
The 8tb and 4tb ssd's are probably too expensive, so I might have to trade off some performance for cost.
What kind of throughput could I expect from a pair of Raid-5 7+1 arrays using the 7.2K 8TB SAS drives?

You can use normal SATA drives, since SATA = SAS.

Western Digital RE4 or SE Enterprise. (7.200 1/min)

You can calculate the number of drive according to the overall capacity.

150 TB * 3 / 2 = 225 TB.

225 TB / 4 TB = 56,25 drives

225 TB / 6 TB = 37,5 drives

The IBM solution is a total overkill.

Rack for all 3,5 " drives.
https://www.ctt.de/.1087512215-Super...R1K28LPB-Black

or search for server case

Power supply: 600 W high efficiency
E.g.: Be Quiet Power Dark Power Pro 1200 W: 255 €


Then a mainboard for each rack
E.g.: Gigabyte X99P SLI mainboard: 245 €
Maybe the computing power of the storage unit must be increased.
I have no experience with that.
If the data troughput is not sufficient, then a 2 or 4 CPU server mainboard must be used.
Like this: https://www.snap-hunter.de/de/Server...2600-V3-789479

Processor
Intel Xeon E5-2620 V3 WOF: 430 €
or similar

Cooler
Thermalright True spirit 120M: 34 €

RAM DDR4-2133
32 GB RAM ( Kingston Hyper X Savage 32 GB Kit, 4 x 8 GB DDR4-2133): 152 €
With Dual processor mainboard you need Reg ECC RAM.

Graphics card
Gainward GTX970 4GB: 270 €

SSD
SSD for operating system for the racks
Samsung 850 Pro 512 GB: 215 €

RAID controller
LSI Mega Raid SAS 9260 16i
9260 16i: 740 € (2x)

Drives for the storage
WD Gold (up to 8 TB) or WD RE (up to 4 TB)
WD Gold 8 TB: 551 €
WD RE 4 TB: 195 €
Then 29 WD Gold/8 TB are needed or 56 WD RE/4 TB are needed.

And the data transfer solution from the racks to the main computer: fibre channel
HP 81 Q: 320 €

If you are unsure, how many computing power the storage solutio needs, the ask a internet provider nearby.

Im looking at the 8Tb ST8000NM0075 SAS drive or the ST10000NM0016 10Tb SATA drive.
From my prior tests using smaller drives, SAS will significantly out-perform SATA. I believe this is due in part to the scsi queue re-ordering and the higher RPM.
However, I don't have any actual throughput numbers for these 2 specific drives.
I would be interested in their random read performance on medium to large block sizes (32k-256k).
However, as noted above, I do see the benefit of using multiple arrays of smaller drives to improve the number of FC or iSCSI ports being used. That would increase throughput to the host.

Is anyone aware of any metrics being reported for either of these drives? Im looking for a 7+1 Raid-5 or possibly a 6+2 Raid-6. Sustained random read throughput is going to be most important metric for my needs.

I have seen 10k drives.
They run very hot.

SATA does also have reordering queue.

SAS is much more expensive.

AMD dual CPU mainboard: Supermicro H8DGi-F and two AMD Opteron 6344 or 6348.
Intel dual CPU mainbaord: Asus Z10PE-D8 WS and two Intel Xeon E5-2620v3 or E5-2630v4.

The two CPU solution from AMD and Intel do need REG ECC RAM. E.g.: from Kingston.

Assuming for such a large project you have a decent budget, I'd be thinking of calling your HP supply partner and asking them for options, or looking at EMC for their range of storage devices. I'd certainly not be trying to "roll my own".

If have no experience in assembling computers, then you can ask the computer store, which sells these parts to a mount the computer.
And make a burn in test of 24 h.

Costs: ~ 200 €.

Agree the 10K drives run hot, but I was thinking of the ST8000NM0075, which is a 7.2K, 8TB SAS drive. Only about 20-30% more than the SATA equilivant, and you get the benefits of SAS over SATA, which show up in the iostat monitoring I have done. Much less wait/seek times, lower %util, and higher throughput when compared to SATA 6Tb drives.
Yes, I did look at HP, specifically the P2000 and MSA arrays. I like them, but they are un-clear about non HP drives working in them...

How have you measured?

Then the drives a connected to a Raid controller, then there should be no difference in CPU util.

I don´t know how long the re-ordering queue is - SATA / SAS.

When you calculate 29 8 TB (one drives give 200 MB/s data through) = 5,8 GB /s.

The SAS drives gives 250 MB/s and the SATA one gives 205 MB/s.

Seek time of SATA drive: not measured.

Just search a bit deeper in the internet....

I setup a Raid10 mirror using 2 drives of the 6Tb SATA and 8TB SAS.
I ran random read-tests (this is the metric im most interested in) using vdbench and monitored using iostat at the Unix layer.
What I saw was that I got about 30-40 MB/s for the SATA pair, but had %util near 100% and double digit (20-40ms) wait states.
On the SAS pair, I got 75 MB/s with only 80% utilization and around 10ms wait states