Max addressable memory for 64bit JVM Process

snoop234 · 02-04-2012, 12:01 PM

Hi,
Im running ubuntu 10.04 64bit on an machine with 12gb physical memory (2x4, 2x2. I wanna upgrade to 32gb of memory because its cheap right now and i need it (don't ask why

).

My mainboard is the "Gigabyte GA-MA790FXT-UD5P"
CPU is : AMD Phenom II x4 955BE

before I buy the ram, i wanna be sure that my machine can actually handle the 32gig and that i can use it with a programm running on an 64bit jvm.

I think the cpu and mainboard should be able to handle it.
Are there any software limitations that i'm not aware of, should i check any system settings before i upgrade?

thanks

johnsfine · 02-05-2012, 08:49 AM

Quote:

Originally Posted by snoop234

My mainboard is the "Gigabyte GA-MA790FXT-UD5P"
CPU is : AMD Phenom II x4 955BE

before I buy the ram, i wanna be sure that my machine can actually handle the 32gig and that i can use it with a programm running on an 64bit jvm.

I think you are ignoring the big difference between physical memory and virtual memory.
You want to know how much physical ram your motherboard can use. But asking how much physical ram a 64bit jvm can use is pretty much meaningless.

A process uses virtual memory. It does not directly use physical memory. The kernel uses physical memory as a resource in implementing the virtual memory.

You might reasonably want to know how much virtual memory a 64bit jvm can use. I think that is controlled by parameters you can set, but I don't recall the details. An ordinary 64 bit process can use a nearly unlimited amount of virtual memory, but I think the jvm has limits designed into it.

Before buying more ram, you could increase your swap space and then try whatever jvm based operation you think should use more ram and distinguish between four likely results.
1) The JVM refuses to allocate enough virtual memory and the operation fails. So you need to identify and adjust jvm parameters before more ram would be usable.
2) The large operation works correctly with acceptable performance using swap space. Even if much more than 12GB of virtual memory is used, the access pattern might be localized enough that paging overhead is insignificant and swap space is almost as good as ram.
3) The large operation works correctly using swap space, but is unacceptably slow, but you look at swap in stats and see that paging is a not a significant component of the slow down. Then you know you need to improve the algorithm of whatever big jvm based operation you are running before more ram would be useful.
4) The large operation works correctly using swap space, but is unacceptably slow, and you look at swap in stats to verify that paging is a significant component of the slow down. Then you know that more ram would actually be useful.

As for how much ram a Gigabyte GA-MA790FXT-UD5P can support, I haven't a clue. I was burned years ago by a Gigabyte motherboard which was advertised with many misrepresentations of its actual capabilities, including overstating the amount of physical ram it supported. I haven't bought any Gigabyte products since then and so know nothing about the capabilities of their current products.

snoop234 · 02-05-2012, 01:04 PM

Hi thanks for your detailed answer.

I know that the jvm works with -Xmx16g so i guess it should work with 32g too.

I tried to increase my swap space before with a swap file, but that made everything even slower. Seems like a swap partition is faster then a file. But my swap partition is to small and i can't increase it.

I haven't looked at swap stats yet but i guess that point 4 is the case because the algorithm is pretty simple and runs fine for about 30-40minutes until all physical memory is used and it starts swapping.

johnsfine · 02-05-2012, 03:06 PM

Quote:

Originally Posted by snoop234

Seems like a swap partition is faster then a file.

Not significantly. A swap file is perfect for the test you should run.

I think you ought to look at swap in statistics (and CPU time stats), so you can determine for sure that swapping is the problem rather than the algorithm doesn't scale well and gets slower when it needs more memory regardless of swapping.

When it starts swapping do both user and system CPU time go down (indicating it is waiting for page swap in)? If user CPU time stays high, the problem is much more likely algorithm. If system CPU time goes up as user CPU time goes down, you may have a soft faulting issue that might or might not be improved by more physical ram.

snoop234 · 02-06-2012, 09:24 AM

hey I did a testrun with -Xmx32g (I created a new swap partition with 32g by deleting an old unused vista partition on my pc)

thats the output of

Code:

vmstat 20

when the pc starts to swap:

Code:

procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 1  0      0  70140    608 362816    0    0   947    11 11372 22566 21  6 72  0
 1  0      0  70884    616 363188    0    0   960     7 11599 23116 20  7 73  0
 1  0      0  68604    744 353608    0    0  1165    14 11138 22188 22  6 72  1
 1  0      0  69344    820 301512    0    0   856    40 9243 18276 22  6 72  1
 1  0      0  74160    836 293988    0    0   781    11 11043 21925 20  7 73  0
 1  0      0  71268    772 284868    0    0   947     9 10836 21561 22  6 72  0
 2  0      0  74916    768 280720    0    0   979    19 10257 20412 20  5 74  0
 1  0      0  75048    480 239012    0    0   883     8 10617 21012 22  6 72  0
 2  0      0  76684    496 237208    0    0   915    13 10917 21675 21  6 73  0
 1  0      0  76772    460 223860    0    0   947    13 10643 21199 21  6 73  0
 2  0      0  79144    460 221852    0    0   947    11 11289 22515 20  7 73  0
 1  0      0  72668    332 180560    0    0   915     7 10906 21449 22  6 67  5
 2  0     80  70548    320 167856    0    4   992    25 11073 21976 21  6 68  5
 1  0    464  72204    304 167496    0   19   938    26 11546 22531 20  6 63 10
 1  1    740  74456    236 112468    0   14   996    26 11260 22155 21  7 62 10
 2  1   1108  66728    232 119688    0   18   993    32 10929 21422 20  6 66  8
 1  2   2004  69532    528 104312    0   49  1193    58 10895 20497 22  6 50 23
 1  0   3228  69432    680 105936    0   62   894    78 11114 21502 20  6 65  8
 1  1   3840  69704   1084  48868    3   31  1639    73 10620 20641 21  6 56 17
 1  0   5048  71264    656  48244   40   66  1125    77 10902 21289 20  6 64 10
 1  0   6992  70176    232  37824  160  146  1487   186 10525 20664 22  6 59 13
 2  0   9616  68804    232  40312   14  136  1479   153 10851 21527 20  6 64 10
 1  0  61972  70580    212  25172   12 2635  5384  2660 7620 13982 16  5 45 34
 1  1  71408  70988    200  34064   18  473  1760   488 10719 21180 20  6 65  9
 2  0  77972  68548    212  29172   56  348  1512   372 10417 20577 21  6 65  8
 1  0  86196  70876    200  34440    0  413  1811   429 10587 20882 20  6 60 13

later on it looks like this:

Code:

 0  2 3389420  67684    228  32908   54   81   348   113   51  889 21  5 67  7
 0  3 3348648  65568    204  29808 3268  774  7435   828 2597 3214  5  3 38 55
 0  2 3365540  71180    468  32124 3063 3195  4812  3292 2699 3053  7  2 42 49
 1  2 3376612  66136    952  30228 3900 3866  6795  3992 2851 2835  6  2 39 53
 1  5 3375448  66192    208  24776 4874 3972  7231  4065 3805 3573  6  3 46 45
 0  3 3386776  65680    284  28140 3905 3861  7803  3943 3224 2816  4  3 41 53
 0  3 3413972  65376    464  33580 3279 3694  5927  3765 2739 2411  5  3 36 57
 0  4 3403396  65984    216  20176 4770 2807  6286  2882 3191 2960  5  3 41 51
 0  2 3356132  67244    616  30276 4209 1242  7262  1287 2376 3335  5  2 47 45
 1  0 3361604  82248    232  33408 1565  535  2213   560 3093 4909 10  3 58 29
 0  2 3357868  65764    228  33800 1561   26  2184    74 3132 5783 10  2 57 31
 0  7 3359224  65280    292  32764 1203  200  2529   246 3671 5372 10  3 52 35
 1  2 3364904  66028    180  30188 1027  340  3198   407 3685 4750  8  3 46 43
 0  3 3374324  65348    280  34076  910  510  3511   588 2871 3936  7  2 44 47
 0  5 3378396  65352    324  27260 1166  251  2730   339 3848 5034  9  2 48 41

cpu is waiting for io a lot: 30-50%
So i guess i can speed this up with more ram

johnsfine · 02-06-2012, 10:34 AM

Quote:

Originally Posted by snoop234

cpu is waiting for io a lot: 30-50%
So i guess i can speed this up with more ram

That very clearly indicates a need for more physical ram.

Sorry I couldn't help you with the original question of how much your motherboard really support.