LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Hardware (https://www.linuxquestions.org/questions/linux-hardware-18/)
-   -   Linux, 64 bit and >3Gb of memory (https://www.linuxquestions.org/questions/linux-hardware-18/linux-64-bit-and-3gb-of-memory-630709/)

Dee-ehn 03-26-2008 03:32 AM

Linux, 64 bit and >3Gb of memory
 
Hello all,

For the last few days I've been trying to install Linux on several PCs, without any success however.

All the systems I use are fine (seen from a hardware point of view). However, I cannot install any Linux version on them, as soon as they are equipped with more than 3Gb of internal memory.

I have tried several distributions: Ubuntu 7.10 AMD64, CentOS 5.1 AMD64, Debian 4.0 AMD64 etc. All the hardware I use does support >3Gb of memory, without using PAE / memory remapping (!). Used CPUs are Intel Core 2 series (E6420, E6600, Q6600). All the systems run fine with 64 bit XP or Vista, so I figure that if a Microsoft product can run fine, Linux should work like a charm :-).

All the Linux versions I tried work on a 8Gb dual Xeon system.

I guess there's some difference in the memory addressing between the desktop and the server systems.

Still, I need to get the desktop systems to work with a reasonable amount of memory (8Gb); how can this be done?

Best, Robin

key4ce 03-26-2008 03:52 AM

do you get any error messages? if so.. please list it.

without a bit more details about the problem it is hard to actually figure out what is actually going on.

syg00 03-26-2008 03:59 AM

Quote:

Originally Posted by Dee-ehn (Post 3100738)
All the systems I use are fine (seen from a hardware point of view). However, I cannot install any Linux version on them, as soon as they are equipped with more than 3Gb of internal memory.

Does this mean you are adding memory as part of this investigation ???.
Quote:

I guess there's some difference in the memory addressing between the desktop and the server systems.
Nope.
Different uses maybe, same architecture. This laptop is trundling along quite nicely on 4 Gig - either Ubuntu 7.10 or 8.04 Beta; both AMD64. I also have both of those plus Archlinux and Vista on a q6600 with 4 Gig - all run 64 bit happily.

Sounds like you have a memory mismatch - even if they say the same thing on the outside. Linux is much less tolerant of memory - and unfortunately even memtest doesn't always show the same issue.

Dee-ehn 03-26-2008 04:03 AM

Quote:

Originally Posted by key4ce (Post 3100748)
do you get any error messages? if so.. please list it.

without a bit more details about the problem it is hard to actually figure out what is actually going on.

Actually, the error messages are pretty random. For example, Ubuntu and Debian fail while installing packages with an error such as "Debootstrap failed...". CentOS also fails while installing packages, producing some random Python error.

Sadly enough, the error messages seem to be a bit random, at best.

Dee-ehn 03-26-2008 04:11 AM

Quote:

Originally Posted by syg00 (Post 3100754)
Does this mean you are adding memory as part of this investigation ???.Nope.

Well more or less. As the systems didn't function properly I started fiddling with (amongst others) memory. Then I noticed that the system could run quite fine with just 3Gb installed.

Quote:

Originally Posted by syg00 (Post 3100754)
Sounds like you have a memory mismatch - even if they say the same thing on the outside. Linux is much less tolerant of memory - and unfortunately even memtest doesn't always show the same issue.

I know... I use matching pairs of Kingston memory (kit of 2) which are actually identical. Memtest indeed doesn't reveal any issues.

How should I proceed from here?

syg00 03-26-2008 05:01 AM

Dunno - you seem to have done what you could test.

key4ce 03-26-2008 07:21 AM

well..
i have to say your problem is very confusing.

however.. a kit of 2 that gives you 3 gb memory in total?...
also.. you might try to switch to 1 gb or 2.. (just one memory) see what it does then...

as without a mismatching chip or a broken memory.. this problem seems.. even more odd.

Dee-ehn 03-26-2008 08:13 AM

Quote:

Originally Posted by key4ce (Post 3100896)
well..
i have to say your problem is very confusing.

however.. a kit of 2 that gives you 3 gb memory in total?...
also.. you might try to switch to 1 gb or 2.. (just one memory) see what it does then...

as without a mismatching chip or a broken memory.. this problem seems.. even more odd.

No I don't have a kit-of-2 3Gb set... as there's no such thing :)

I actually found out that 2Gb works fine, while 4Gb doesn't. Then I started testing with numerous configurations (we've got quite some memory in stock, so that makes life a bit easier ;)).

Anyhow, as long as I stay below the 3Gb limit, all compatible memory works like a charm! Even the strangest combinations (for example, different brands, different speeds etc) work fine.

I think I tested over 50 strips of memory in these systems, so I think we can rule the memory out.

If there are any more suggestions, feel free. And thanks for all the answers, greatly appreciated!

key4ce 03-26-2008 08:40 AM

hmm.. well that was basically the only thing i could come up with..

however after a quick search on google i found a topic somewhat related to yours..

http://www.mail-archive.com/linux-il.../msg48585.html

it won't really solve your problem, but it gives you more information on the subject..

Dee-ehn 03-26-2008 09:42 AM

Quote:

Originally Posted by key4ce (Post 3100978)
hmm.. well that was basically the only thing i could come up with..

however after a quick search on google i found a topic somewhat related to yours..

http://www.mail-archive.com/linux-il.../msg48585.html

it won't really solve your problem, but it gives you more information on the subject..

Thanks for the link. However, it's not really related, as I don't need PAE (because of the 64 bit kernel).

Thanks anyhow :-)

johnsfine 03-26-2008 09:46 AM

It sounds like a DMA problem.

Up to slightly more than 3Gb (but significantly less than 4) DMA is fairly simple. Beyond that, things get complicated and I believe depend on details of the motherboard chipset.

An OS can (and may be forced to) do all DMA within the low 3GB and when necessary copy the data to or from the intended destination higher in memory.

I don't recall what build or boot time options Linux has for deciding how DMA is managed. Maybe you can find something to either correctly support the motherboard chipset or understand the limits of the support (to do extra copying rather than attempting certain DMA operations that wouldn't work).

Quote:

Originally Posted by Dee-ehn (Post 3100738)
All the hardware I use does support >3Gb of memory, without using PAE / memory remapping (!).

That's not really a meaningful statement. Any PC needs over 4Gb of addressing to support beyond a little over 3Gb of ram. Any 32-bit OS needs PAE (or its obsolete predecessor) to support over 4Gb of addressing. A 64-bit OS doesn't need PAE because the default mapping in 64-bit mode includes all the features of PAE. In 64-bit mode, the OS has no choice of mapping modes there is just 64-bit mapping, which is a superset of PAE. Only in 32-bit mode can you choose PAE or non PAE and in 32-mode your hardware does not support much more than 3Gb without PAE.

Quote:

Originally Posted by key4ce (Post 3100978)
however after a quick search on google i found a topic somewhat related to yours.

Not related.
That thread was by someone who didn't understand that you need more that 4Gb of addressing to have a full 4Gb of ram, so didn't understand that a 32-bit OS needs PAE to have a full 4Gb of ram.

Unless I totally misunderstood, this thread is about a situation in which 4Gb or more of ram is recognized and used and randomly malfunctions (I think because DMAs that happen to land in the first 3Gb work and DMAs that happen to land later don't).

Dee-ehn 03-26-2008 09:56 AM

Quote:

Originally Posted by johnsfine (Post 3101045)
It sounds like a DMA problem.

Up to slightly more than 3Gb (but significantly less than 4) DMA is fairly simple. Beyond that, things get complicated and I believe depend on details of the motherboard chipset.

An OS can (and may be forced to) do all DMA within the low 3GB and when necessary copy the data to or from the intended destination higher in memory.

I don't recall what build or boot time options Linux has for deciding how DMA is managed. Maybe you can find something to either correctly support the motherboard chipset or understand the limits of the support (to do extra copying rather than attempting certain DMA operations that wouldn't work).

That sounds like interesting information. However, I don't quite get you. AFAIK, DMA is used for peripherals to access the memory directly.

Are there any "general" settings for DMA?

johnsfine 03-26-2008 10:10 AM

Quote:

Originally Posted by Dee-ehn (Post 3101052)
I don't quite get you. AFAIK, DMA is used for peripherals to access the memory directly.

Are there any "general" settings for DMA?

Primarily we are talking about your hard disk. Transfers to and from the hard disk use DMA. There might be other DMA in use for something, but most of the DMA is for your hard drive(s).

I don't recall what settings exist. As I have read through various (kernel) build and boot options, I only half noticed DMA options because I'm not having any DMA problems.

I think you can turn DMA off entirely, which may result in poor performance but would at least confirm the nature of the problem.

I think/hope you can fine tune the DMA control more to fix the problem without trashing DMA. But I don't know how.

What chipset do these motherboards have? A google search on the chipset will probably turn up previous discussion of whatever the problem might be.

Also there is likely to be useful information early in one of the kernel log files, such as dmesg. Have you looked at what it says abot the memory, the chipset and the DMA?

I just did a Google search +"bounce buffer" +linux hoping to find a good description of the issue to give you. But I just found a bunch of pages telling you how to turn off bounce buffers (to increase performance) when Linux is using them unnecessarily. If I'm guessing correctly, your problem is the opposite. Linux is not using bounce buffers but would need to use them to get correct results because the kernel does not understand the motherboard chipset well enough to correctly avoid the use of bounce buffers.

Dee-ehn 03-28-2008 04:56 AM

Quote:

Originally Posted by johnsfine (Post 3101066)
Primarily we are talking about your hard disk. Transfers to and from the hard disk use DMA. There might be other DMA in use for something, but most of the DMA is for your hard drive(s).

I don't recall what settings exist. As I have read through various (kernel) build and boot options, I only half noticed DMA options because I'm not having any DMA problems.

I think you can turn DMA off entirely, which may result in poor performance but would at least confirm the nature of the problem.

I think/hope you can fine tune the DMA control more to fix the problem without trashing DMA. But I don't know how.

What chipset do these motherboards have? A google search on the chipset will probably turn up previous discussion of whatever the problem might be.

Also there is likely to be useful information early in one of the kernel log files, such as dmesg. Have you looked at what it says abot the memory, the chipset and the DMA?

I just did a Google search +"bounce buffer" +linux hoping to find a good description of the issue to give you. But I just found a bunch of pages telling you how to turn off bounce buffers (to increase performance) when Linux is using them unnecessarily. If I'm guessing correctly, your problem is the opposite. Linux is not using bounce buffers but would need to use them to get correct results because the kernel does not understand the motherboard chipset well enough to correctly avoid the use of bounce buffers.

Hello and thanks for the information.

The systems I am testing use the Intel 975 and 3210 chipsets. Using Google I don't find any good information on these chipsets and Linux with more than 3Gb of memory.

Seems like I'm running into an odd compatibility issue...


All times are GMT -5. The time now is 07:16 AM.