"Failed to allocate mem resource" with Broadcom BCM5704 and 8GB of RAM
 

I'm building a new server to host virtual machines (Yea! VMware Server is free!). Here are the specs on the server:

Tyan Thunder K8SRE (S2891) (BIOS V2.01)
w/ Integrated Broadcom BCM5704 Dual GigE Ethernet Controller
8x1G DDR400 Registered ECC Corsair Memory

With all 8GB of RAM installed I get the following message using Centos 4.2 (kernel 2.6.9-22 and 2.6.9-22.0.2.106):

Feb 14 04:24:52 localhost kernel: PCI: Failed to allocate mem resource #2:10000@240000000 for 0000:0a:09.0
Feb 14 04:24:52 localhost kernel: PCI: Failed to allocate mem resource #0:10000@240000000 for 0000:0a:09.1

and later in the boot I get:

Feb 14 04:24:53 localhost kernel: tg3.c:v3.27-rh (May 5, 2005)
Feb 14 04:24:53 localhost kernel: ACPI: PCI interrupt 0000:0a:09.0[A] -> GSI 28 (level, low) -> IRQ 217
Feb 14 04:24:53 localhost kernel: PCI: Unable to reserve mem region #3:fffffffe9f400000@240000000 for device 0000:0a:09.0
Feb 14 04:24:53 localhost kernel: tg3: Cannot obtain PCI resources, aborting.
Feb 14 04:24:53 localhost kernel: tg3: probe of 0000:0a:09.0 failed with error -16
Feb 14 04:24:53 localhost kernel: ACPI: PCI interrupt 0000:0a:09.1[B] -> GSI 29 (level, low) -> IRQ 225
Feb 14 04:24:53 localhost kernel: PCI: Unable to reserve mem region #1:fffffffe9f400000@240000000 for device 0000:0a:09.1
Feb 14 04:24:53 localhost kernel: tg3: Cannot obtain PCI resources, aborting.
Feb 14 04:24:53 localhost kernel: tg3: probe of 0000:0a:09.1 failed with error -16
Feb 14 04:24:53 localhost kernel: ACPI: PCI interrupt 0000:00:02.1[B] -> GSI 21 (level, high) -> IRQ 193

The driver (tg3) seems to be loading, but not getting the memory mappings it wants.

If I remove half of the RAM (4GB installed) it starts to work, but I still get these error messages:

Feb 14 10:49:50 dhcp-host-196 kernel: PCI: Cannot allocate resource region 2 of device 0000:0a:09.0
Feb 14 10:49:51 dhcp-host-196 kernel: PCI: Cannot allocate resource region 0 of device 0000:0a:09.1
Feb 14 10:49:51 dhcp-host-196 kernel: PCI-DMA: Disabling IOMMU.

I've been searching Google all day and I can't find anything that seems to help. I'm going to try Fedora Core 4 tomorrow to see if anything changes, but I don't want to use that in production. Anybody else run into problems like this? Are there any BIOS settings or kernel parameters that I need to use to get the netwoking working when I have 8GB of RAM?

Thanks in advance!

Kernel 2.6.9 is an old version and vulnerable. I suggest at least upgrade to 2.6.12 or the next stable kernel version. I am not sure that VMware modules can be compiled well with gcc version 4 compiler which Fedora core 4 uses.

Try running lspci and find out what device points to 0000:0a:09.0 and 0000:0a:09.1. Start disabling SATA, USB, Communication ports or serial ports, parallel ports. If disabling the NICs fixed the problem, call Tyan about the problem.

FYI: Centos 4 = RHEL 4, the 2.6.9 kernel I'm using has all the security patches backported. I'm not sure what other stuff has been backported, but the idea is to have a well tested stable platform for production envroniments.

I did not disable the network card, the only thing I changed is how much RAM I had installed. If I use 2GB of RAM everything works fine, 4GB and I get errors, but the network adapter still works, 6 or 8GB and the network adapter stops working, but still shows up in lspci. The device it is complaining about is the network adapter which I already knew:

00:00.0 Memory controller: nVidia Corporation CK804 Memory Controller (rev a3)
00:01.0 ISA bridge: nVidia Corporation CK804 ISA Bridge (rev a3)
00:01.1 SMBus: nVidia Corporation CK804 SMBus (rev a2)
00:02.0 USB Controller: nVidia Corporation CK804 USB Controller (rev a2)
00:02.1 USB Controller: nVidia Corporation CK804 USB Controller (rev a3)
00:06.0 IDE interface: nVidia Corporation CK804 IDE (rev f2)
00:07.0 IDE interface: nVidia Corporation CK804 Serial ATA Controller (rev f3)
00:08.0 IDE interface: nVidia Corporation CK804 Serial ATA Controller (rev f3)
00:09.0 PCI bridge: nVidia Corporation CK804 PCI Bridge (rev a2)
00:0e.0 PCI bridge: nVidia Corporation CK804 PCIE Bridge (rev a3)
00:18.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
00:18.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
00:18.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
00:18.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
00:19.0 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] HyperTransport Technology Configuration
00:19.1 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Address Map
00:19.2 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] DRAM Controller
00:19.3 Host bridge: Advanced Micro Devices [AMD] K8 [Athlon64/Opteron] Miscellaneous Control
01:07.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
08:0a.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8131 PCI-X Bridge (rev 12)
08:0a.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC (rev 01)
08:0b.0 PCI bridge: Advanced Micro Devices [AMD] AMD-8131 PCI-X Bridge (rev 12)
08:0b.1 PIC: Advanced Micro Devices [AMD] AMD-8131 PCI-X IOAPIC (rev 01)
0a:09.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 03)
0a:09.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5704 Gigabit Ethernet (rev 03)

That is why I'm thinking a BIOS setting (IOMMU?) or kernel boot parameter will get the network adapter working.

Can't help, but I also pulled Centos 4.2 yesterday to play with VMWare.
4 x Xeon (PIII) with 4 Gig of memory.
Seems to run fine with Centos (both base and guest) and an Arch guest - everything gets recognised o.k.

Fedora Core 4 and the 2.6.15-1 kernel seem to work. I'm still getting some of the same errors about not being able to allocate memory, but the network adapter is working and I have 8GB of RAM installed. I'm going to assume at this point that this is a kernel bug, not a BIOS setting.

Thanks for everybodies help!

Try compile the kernel with 64 GB support.