[SOLVED] Kernel fails to assign memory to PCIe device

dbrazeau · 12-05-2013, 07:18 PM

I'm attempting to workaround an issue where a PCIe card does not show up on the PCIe bus after boot. I have discovered issuing a rescan of the PCIe bus via "echo 1 > /sys/bus/pci/rescan" results in the card showing up, but the kernel fails to assign memory to the device. I have tried loading the PCIe hotplug driver before doing the rescan but that doesn't help. I have also tried adding "pci=nobios" to the kernel commamnd line but that doesn't help either.

Here is a dump from the failure:

Code:

[   54.539874] pci 0000:0b:00.0: BAR 6: can't assign mem pref (size 0x200000)
[   54.546719] pci 0000:0b:00.0: BAR 3: can't assign mem (size 0x10000)
[   54.553042] pci 0000:0b:00.0: BAR 1: can't assign mem (size 0x4000)
[   54.559277] pci 0000:0b:00.0: BAR 0: assigned [io  0x5000-0x50ff]
[   54.565344] pci 0000:0b:00.0: BAR 0: set to [io  0x5000-0x50ff] (PCI address [0x5000-0x50ff])

I am running Fedora 15 kernel 3.0.3.

Any ideas what I can do to get the kernel to successful assign memory to a new pcie device post boot?

smallpond · 12-06-2013, 01:50 PM

The message is printed by _pci_assign_resource when it tries to allocate the space requested by the PCI config registers (BARs) on the card. The most likely reason for failure is a conflict - it thinks the resources are already assigned. It probably found the card at boot, but the driver failed to start and did not properly release the resources. Bad driver, no cookie. This is not a BIOS problem or kernel problem. Get a new driver.

dbrazeau · 12-06-2013, 03:21 PM

Quote:

Originally Posted by smallpond

The message is printed by _pci_assign_resource when it tries to allocate the space requested by the PCI config registers (BARs) on the card. The most likely reason for failure is a conflict - it thinks the resources are already assigned. It probably found the card at boot, but the driver failed to start and did not properly release the resources. Bad driver, no cookie. This is not a BIOS problem or kernel problem. Get a new driver.

I have added the driver to the blacklist (/etc/modprobe.d/blacklist.conf) so it is not being loaded at all. Also the pci device doesn't show up at boot, only after I issue the rescan.

dbrazeau · 12-06-2013, 05:06 PM

So I've found the solution. If I remove the Root Port from the PCI bus before doing the retrain the memory windows get successfully assigned to the PCI device.

Code:

# lspci
00:00.0 Host bridge: Intel Corporation Ivy Bridge DRAM Controller (rev 09)
00:01.0 PCI bridge: Intel Corporation Ivy Bridge PCI Express Root Port (rev 09)
00:01.1 PCI bridge: Intel Corporation Ivy Bridge PCI Express Root Port (rev 09)
...

#echo 1 > /sys/bus/pci/devices/0000\:00\:01.1/remove
#echo 1 > /sys/bus/pci/rescan

syg00 · 12-06-2013, 06:03 PM

Thanks for the post-back. Might help some-one on a search one day.

Clarkage · 03-27-2018, 11:32 AM

Thanks very much indeed, helped me out with some experiments I'm doing related to downstream port containment.
I was getting BAR issues depending on how the system booted up but removing the entire upstream bridge (that is DPC capable) and then re-scanning looks to have solved that aspect.