LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - General (http://www.linuxquestions.org/questions/linux-general-1/)
-   -   Disk performance slower after upgrade to 4GB RAM. Help. (http://www.linuxquestions.org/questions/linux-general-1/disk-performance-slower-after-upgrade-to-4gb-ram-help-839109/)

dbrazeau 10-19-2010 02:18 PM

Disk performance slower after upgrade to 4GB RAM. Help.
 
I am running openSuse 11.2 (32-bit), my CPU only supports 32-bits. I have a hardware RAID device. My system has 4GB of RAM. When I configure my system to only use 3GB, 2GB, or even 1GB, using mem=1024M in grub, my RAID performance is much better then when letting my system use the default 4GB available.

Can anyone explain to me why this is? Is there anything I can do, i.e. kernel configuration, that will help performance when running with all 4GB enabled?

johnsfine 10-19-2010 03:26 PM

How are you measuring disk performance? The results may not be consistent with actual use.

What performance results are you seeing? (I'm especially curious whether the results are so good that they imply you are just testing the cache of the RAID rather than the actual disk performance).

gjarboni 10-19-2010 03:53 PM

johnsfile's suggestion is a good one. If you want to quantitatively measure disk performance, bonnie++ is a good tool.

With that said, it's possible that the driver for your raid card doesn't handle PAE well. PAE is the architecture that allows a 32-bit processor to access more than 4GB of ram. You need this because the PCI bus uses memory address space (but not actual memory) to send and receive information. So with 4GB of ram you're really using 4+ GB of address space.

johnsfine 10-19-2010 04:08 PM

I'm pretty sure PAE vs. non PAE is a kernel build time decision for Linux, not a decision at kernel load time.

A 32 bit Linux needs PAE to support a full 4GB. It probably doesn't need it for 3GB (on some unusual motherboards you need PAE for 3GB). So if the OP tested with 4GB enabled and then used mem= to test with lower memory size, I think that means the kernel build time choice of PAE or not wasn't changed across that testing.

Assuming the test was high speed transfers from the RAID cache, rather than slower accesses to the actual disks, a small difference in the efficiency of the driver (such as PAE vs. non PAE) might be a significant fraction of the artificially small transfer time tested. But that doesn't explain the OP's results, because there doesn't seem to be a comparison of PAE vs. non PAE in those results.

gjarboni 10-19-2010 04:14 PM

Quote:

Originally Posted by johnsfine (Post 4132857)
I'm pretty sure PAE vs. non PAE is a kernel build time decision for Linux, not a decision at kernel load time.

Right, but if the total memory space is less than 4GB, then no PAE requests will be needed. So you'll be using a PAE kernel, but not making any PAE calls. When using all 4GB of memory PAE calls will be needed to change pages to access all of the memory. My theory was that the driver was doing something remarkably stupid that required lots of switching between pages which caused the poor performance.

dbrazeau 10-19-2010 04:18 PM

Quote:

Originally Posted by johnsfine (Post 4132857)
I'm pretty sure PAE vs. non PAE is a kernel build time decision for Linux, not a decision at kernel load time.

A 32 bit Linux needs PAE to support a full 4GB. It probably doesn't need it for 3GB (on some unusual motherboards you need PAE for 3GB). So if the OP tested with 4GB enabled and then used mem= to test with lower memory size, I think that means the kernel build time choice of PAE or not wasn't changed across that testing.

Correct. I didn't rebuild the kernel for the different memory size test cases, only used "mem=" kernel argument option.

I have primarily used IOMeter for testing the raw block device. I wrote my own benchmark application that shows a similar difference in performance that I see with IOMeter. I have also wrote a scsi target emulator driver that simulates a scsi block device using a RAM buffer, so that I can take the drives/RAID card/RAID card driver out of the equation, but I still see the performance difference using the emulator as well.

From my SCSI emulator testing it looks like the periods with no outstanding SCSI requests are much longer when running with 4GB than with 3GB or less during the performance test. Is there any reason why the SCSI layer would have a longer delay between submitting commands when running 4GB of RAM?


All times are GMT -5. The time now is 05:43 PM.