Linux, 64 bit and >3Gb of memory
 

Hello all,

For the last few days I've been trying to install Linux on several PCs, without any success however.

All the systems I use are fine (seen from a hardware point of view). However, I cannot install any Linux version on them, as soon as they are equipped with more than 3Gb of internal memory.

I have tried several distributions: Ubuntu 7.10 AMD64, CentOS 5.1 AMD64, Debian 4.0 AMD64 etc. All the hardware I use does support >3Gb of memory, without using PAE / memory remapping (!). Used CPUs are Intel Core 2 series (E6420, E6600, Q6600). All the systems run fine with 64 bit XP or Vista, so I figure that if a Microsoft product can run fine, Linux should work like a charm :-).

All the Linux versions I tried work on a 8Gb dual Xeon system.

I guess there's some difference in the memory addressing between the desktop and the server systems.

Still, I need to get the desktop systems to work with a reasonable amount of memory (8Gb); how can this be done?

Best, Robin

do you get any error messages? if so.. please list it.

without a bit more details about the problem it is hard to actually figure out what is actually going on.

Does this mean you are adding memory as part of this investigation ???.Nope.
Different uses maybe, same architecture. This laptop is trundling along quite nicely on 4 Gig - either Ubuntu 7.10 or 8.04 Beta; both AMD64. I also have both of those plus Archlinux and Vista on a q6600 with 4 Gig - all run 64 bit happily.

Sounds like you have a memory mismatch - even if they say the same thing on the outside. Linux is much less tolerant of memory - and unfortunately even memtest doesn't always show the same issue.

Actually, the error messages are pretty random. For example, Ubuntu and Debian fail while installing packages with an error such as "Debootstrap failed...". CentOS also fails while installing packages, producing some random Python error.

Sadly enough, the error messages seem to be a bit random, at best.

Well more or less. As the systems didn't function properly I started fiddling with (amongst others) memory. Then I noticed that the system could run quite fine with just 3Gb installed.

I know... I use matching pairs of Kingston memory (kit of 2) which are actually identical. Memtest indeed doesn't reveal any issues.

How should I proceed from here?

Dunno - you seem to have done what you could test.

well..
i have to say your problem is very confusing.

however.. a kit of 2 that gives you 3 gb memory in total?...
also.. you might try to switch to 1 gb or 2.. (just one memory) see what it does then...

as without a mismatching chip or a broken memory.. this problem seems.. even more odd.

No I don't have a kit-of-2 3Gb set... as there's no such thing :)

I actually found out that 2Gb works fine, while 4Gb doesn't. Then I started testing with numerous configurations (we've got quite some memory in stock, so that makes life a bit easier ;)).

Anyhow, as long as I stay below the 3Gb limit, all compatible memory works like a charm! Even the strangest combinations (for example, different brands, different speeds etc) work fine.

I think I tested over 50 strips of memory in these systems, so I think we can rule the memory out.

If there are any more suggestions, feel free. And thanks for all the answers, greatly appreciated!

hmm.. well that was basically the only thing i could come up with..

however after a quick search on google i found a topic somewhat related to yours..

http://www.mail-archive.com/linux-il.../msg48585.html

it won't really solve your problem, but it gives you more information on the subject..

Thanks for the link. However, it's not really related, as I don't need PAE (because of the 64 bit kernel).

Thanks anyhow :-)

It sounds like a DMA problem.

Up to slightly more than 3Gb (but significantly less than 4) DMA is fairly simple. Beyond that, things get complicated and I believe depend on details of the motherboard chipset.

An OS can (and may be forced to) do all DMA within the low 3GB and when necessary copy the data to or from the intended destination higher in memory.

I don't recall what build or boot time options Linux has for deciding how DMA is managed. Maybe you can find something to either correctly support the motherboard chipset or understand the limits of the support (to do extra copying rather than attempting certain DMA operations that wouldn't work).

That's not really a meaningful statement. Any PC needs over 4Gb of addressing to support beyond a little over 3Gb of ram. Any 32-bit OS needs PAE (or its obsolete predecessor) to support over 4Gb of addressing. A 64-bit OS doesn't need PAE because the default mapping in 64-bit mode includes all the features of PAE. In 64-bit mode, the OS has no choice of mapping modes there is just 64-bit mapping, which is a superset of PAE. Only in 32-bit mode can you choose PAE or non PAE and in 32-mode your hardware does not support much more than 3Gb without PAE.

Not related.
That thread was by someone who didn't understand that you need more that 4Gb of addressing to have a full 4Gb of ram, so didn't understand that a 32-bit OS needs PAE to have a full 4Gb of ram.

Unless I totally misunderstood, this thread is about a situation in which 4Gb or more of ram is recognized and used and randomly malfunctions (I think because DMAs that happen to land in the first 3Gb work and DMAs that happen to land later don't).

That sounds like interesting information. However, I don't quite get you. AFAIK, DMA is used for peripherals to access the memory directly.

Are there any "general" settings for DMA?

Primarily we are talking about your hard disk. Transfers to and from the hard disk use DMA. There might be other DMA in use for something, but most of the DMA is for your hard drive(s).

I don't recall what settings exist. As I have read through various (kernel) build and boot options, I only half noticed DMA options because I'm not having any DMA problems.

I think you can turn DMA off entirely, which may result in poor performance but would at least confirm the nature of the problem.

I think/hope you can fine tune the DMA control more to fix the problem without trashing DMA. But I don't know how.

What chipset do these motherboards have? A google search on the chipset will probably turn up previous discussion of whatever the problem might be.

Also there is likely to be useful information early in one of the kernel log files, such as dmesg. Have you looked at what it says abot the memory, the chipset and the DMA?

I just did a Google search +"bounce buffer" +linux hoping to find a good description of the issue to give you. But I just found a bunch of pages telling you how to turn off bounce buffers (to increase performance) when Linux is using them unnecessarily. If I'm guessing correctly, your problem is the opposite. Linux is not using bounce buffers but would need to use them to get correct results because the kernel does not understand the motherboard chipset well enough to correctly avoid the use of bounce buffers.

Hello and thanks for the information.

The systems I am testing use the Intel 975 and 3210 chipsets. Using Google I don't find any good information on these chipsets and Linux with more than 3Gb of memory.

Seems like I'm running into an odd compatibility issue...