I have an iwill server, with 8 dual core opteron cpu's. If I use 32bit live cd's, they boot fine (but of course, can't address more than 4gb of RAM, and running 32bit isn't an option).

If I use a 64bit live cd, and boot with more than 16gb of ram installed (ram has been tested for 48 hours in memtest), strange things occur, for instance, a soft lockup on a random cpu (could be 1, could be 3, could be 4). I have tried mem={theexactamount}, and mem={exactamountminus1}, however this does not work. I have 32gb of RAM installed, and it just simply won't boot. If I remove 16gb, and bring it to 16gb, the machine boots, and I am able to install, (and I can install the other 16gb once I'm done with the stage3 install, but it's unstable).

Is there some sort of limit on what linux really supports on installed RAM I'm not aware of? Is there a specific kernel argument I need to pass for that much RAM? I have tried gentoo 2006.0, and 2006.1 and 2007.0, to no avail. I have also tried fedora (to no avail).

Any help will be greatly appreciated.

Thanks,
Michael

Leto

..and I take crude about having 2gig installed. I think you are going to have a limited pool of people with real experience on this one. The only thing I could find applies to Redhat(maybe others?). "append the kernel command line with mem=32G"

As in:

title Red Hat Enterprise Linux Server (2.6.18-4.elxen)
root (hd0, 0)
kernel /xen.gz-2.6.18-4-el5 mem=32G
module /vmlinuz -2.6.18-4.el5xen ro root=LABEL=/
module /initrd-2.6.18-4.el5xen.img



Hope this helps.

Keep us posted.

Lazlow

Hi,

Perhaps if you boot with only the 16 gig memory, and compile a kernel with "CONFIG_HIGHMEM64G" set (in other words, give the kernel support for up to 64 GB mem), that might resolve your problem.

Sam

I have another of the exact machines, and that's how I did it, booted, and installed with 16, then installed the 32. However, as I said, it seems unstable.

Michael

Well what I meant was to boot with your 16 gig mem, and DON'T install the rest of the memory until you have compiled the new kernel and are ready to boot to it.

Sam

Woohoo, that'd be my idea of a Gentoo box. Bet you get X compiled faster than I did ...

Sounds like dodgy (most likely mismatched) memory. Memtest doesn't exercise things like the kernel does. Unfortunate but true - same with Windoze; it'll happily run on memory Linux gets all wobbly kneed about.

Mmmm - Opterons; is the memory "local" to the processors ???. I had a feeling the Opterons were (psuedo ???) NUMA architecture.
If so, maybe try using all the (current) memory on half the chips - use the remainder on the others. Can't harm to try.