Hi, I am certain this has been asked some where else but the LQ search isn't allowing me to search with terms less than 4 characters long...

I just installed ubuntu gutsy 32 bit and I have 4 Gig ram, and he only sees 3 Gig. I understand that 32 bits can address 4 Gig, why does he not use the full 4 Gig? What can I do to fix? I want to fix in such a way that doesn't impede future apt updates.

Thanks

4 Gig of address space is not enough for 4 Gig of ram because the BIOS must reserve some address space for PCI I/O and access to the memory inside the display adapter. Usually the BIOS reserves far more than those things ought to need so typically only 3.25 Gig of ram fits in 4 Gig of address space. You can see the details of the ram mapping near the begining of the dmesg output.

With the PAE CPU feature, a 32 bit kernel can use ram beyond 4 Gig of addressing.

But you would need to install a different kernel to make that work. I don't know how that would interact with future kernel updates from the ubuntu repository.

I think the 32 bit kernel (build-time) option to turn on PAE support is called "hugemem"

You didn't say but if you have an integrated mobo video that will reserve memory as well -- at least 64MB. To be honest, with a 1GB desktop system I almost never use any swap space. This would imply that more memory would little or no impact on performance.

Probably a completely ignorant question, but why do we see such a large performance boost in a windows machine with added memory past 1gig?

If you're seeing 3 (plus) Gig, the kernel options are as required.
Different motherboards produce different numbers. 64-bit kernels avoid most of this issue.

In Linux or Windows, the amount that the swap file is used is not as accurate a measure of possible performance improvement from more memory as the many posts on this subject seem to indicate.

1) Depending on the programs you are running, a large fraction of your virtual memory might be backed by the original shared objects and executable images on disk rather than by the swap file.

The swap file is only used to swap out pages (4096 byte chunks) of memory that were created or modified by the running process. Pages that were read in from an executable or .so file and not modified are just discarded when the system is short of memory and read back in from their original source when needed again. That type of paging is half as expensive as paging with the swap space, but still significant for performance and reduced by having more memory.

2) Some programs do memory mapped I/O so there may be data (read only or read/write) paged directly to and from data files the same way item 1 described for read only executable image content. Again, the swap file wouldn't be touched even if low memory were causing significant extra disk accesses.

3) The file caching system in the OS effectively converts most disk I/O that the program does with non memory mapped functions into memory mapped functions using kernel memory. If the access pattern of your ordinary I/O is such that a large file cache improves performance, then that translates into a large performance boost from adding memory. Again, that is even if before adding memory there was no significant uses of swap space.

4) All the above are true of Linux and Windows. For Windows alone, there is the extra factor of rather large speculative use of swap space. Windows will use the swap file even when it doesn't really need to. Sometimes that makes system performance better, sometimes worse. But either way it creates the situation in which having more memory would make performance better.

Could you explain that please.

Do you mean that seeing 3 plus Gig indicates the kernel already has hugemem? I don't think that is correct.

Do you mean that hugemem wouldn't let you see a larger fraction of the 4 gig of ram? I don't think that is correct either.

Maybe you mean the performance cost of hugemem (tlb misses will be a little slower) outweighs the performance benefits of having nearly 4 gig vs. about 3.25 gig. I don't think you meant that and I don't know how to estimate whether it would be true for any given system. Certainly not true for any system I use (where .7 gig of extra file caching would have significant performance impact).

That's because it's not a Windows machine -- it's Ubuntu Linux. With Vista it would appear that 2GB is the minimum. More memory might further improve the performance of Linux but replacing access to swap space on disk with access to memory is a huge gain that is apparent to users without having to run benchmarks to see it.

I think the "HUGEMEM kernel" was a Redhat-ism. Never did know what it was specifically, but then I don't use RH, and I don't run 2.4 kernels.

For 32-bit 2.6 kernels, selecting CONFIG_HIGHMEM4G is sufficient to see the memory up to 4 Gig. CONFIG_HIGHMEM64G gets you addressing to the rest. Without either, you are limited to 1 Gig. (talking real memory addressing, not virtual). Linus has just recently sounded off (again) against people using HIGHMEM and large(-ish) memory. He seems to think it was a hack he would have preferred never happened.
Short answer, go 64-bit.
Large page (i.e. HUGETLB) is almost irrelevant in this discussion. For userland that mainly affects storage taken up in PTEs - for most (home) users that is minor unless they are something like an Oracle developer.

I'm not at all sure of the naming of kernel build options, even in the distributions I use. I understand the meaning of the build option that turns on the CPU's PAE feature. I just am uncertain of its name.

Have you tried that yourself on a machine with 4 Gig of Ram? I'm sure you would find only about 3.25 Gig of Ram is usable.

So CONFIG_HIGHMEM64G is the name (at least at the point where you are looking at option names) of the PAE support and that option must be turned on to get access to the full 4 Gig on a system that has 4 Gig.

So am I. But even in real memory addressing there is a level of mapping that puts about .75 Gig of the first 4 Gig of address space somewhere higher up. You can see all that in the memory list early in dmesg in any system with 4 Gig or more of ram.

I haven't seen any data from any systematic measurement of benefits/costs of 64-bit vs. 32-bit with PAE.

So maybe you're right. But I'm not ready to conclude that yet.

Right. Did I miss that being mentioned somewhere earlier?

You're missing the point of that feature. The 0.2% maximum possible memory savings from the PTEs wouldn't be interesting to anyone. Physical memory fragmentation issues and reduced VM flexibility from using that feature will cost more physical memory than the PTEs save anyway.

The point of that feature is the CPU time of the TLB misses. In some applications, the CPU time cost of TLB misses is the largest share of total CPU time. In some of those applications, using huge pages would dramatically increase performance.

TLB misses are a little slower in 32-bit with PAE than without. They are a little slower in 64-bit than in 32-bit with PAE. Usually those things don't matter because TLB misses usually aren't a significant share of total time.

When large pages are effective they eliminate most of the TLB misses. They also slightly speed up some of the TLB misses that they don't eliminate, but that aspect is not significant.