Your BIOS doesn't leave a aperture memory hole

torsionbar · 01-06-2011, 09:47 PM

the date on this thread is old, but it is still one of the top hits when googling for this error message.

I have an IOMMU bios option on my Tyan S2882 system board. The manual says this option is specifically for Linux, and allows the kernel to perform I/O mapping between 32 bit and 64 bit addresses. Also, this Tyan S2882 does not have AGP.

Some additional reading on AMD's implementation of IOMMU for Linux:

http://developer.amd.com/documentati...892006101.aspx

I turned on the IOMMU bios option, and this kernel error message went away.

H_TeXMeX_H · 01-07-2011, 12:38 PM

Thanks for the answer, I'm glad that changing the BIOS options works ... like I said in the third post and was later criticized for it.

JZL240I-U · 07-12-2011, 06:35 AM

Just to keep the subject up to date: On my Gigabyte GA-890GPA-UD3H (PCIe, no AGP) with kernel 2.6.29.2 (openSuSE) the error / warning still persists.

Google led me here, but there are newer posts in the net meanwhile. In short, the problem is not solved on the kernel side.

JZL240I-U · 07-13-2011, 07:41 AM

Further digging in /var/log/boot.msg revealed this:

Code:

...
<6>[    0.000000] Checking aperture...
<6>[    0.000000] No AGP bridge found
<6>[    0.000000] Node 0: aperture @ a0000000 size 32 MB
<6>[    0.000000] Aperture pointing to e820 RAM. Ignoring.
<6>[    0.000000] Your BIOS doesn't leave a aperture memory hole
<6>[    0.000000] Please enable the IOMMU option in the BIOS setup
<6>[    0.000000] This costs you 64 MB of RAM
<6>[    0.000000] Mapping aperture over 65536 KB of RAM @ a0000000
<6>[    0.000000] PM: Registered nosave memory: 00000000a0000000 - 00000000a4000000
...
<6>[    0.731050] PCI-DMA: Disabling AGP.
<6>[    0.731166] PCI-DMA: aperture base @ a0000000 size 65536 KB
<6>[    0.731168] PCI-DMA: using GART IOMMU.
<6>[    0.731170] PCI-DMA: Reserving 64MB of IOMMU area in the AGP aperture
...

If this is what I think it is, all is well...

P.S.: kernel boot parameter "iommu=noapg" switched off the integrated Radeon HD 4290, "iommu=soft" halted the integrated Radeon HD 4290, "iommu=memaper=2" doubled the reserved window size without other changes...

dbjh · 12-06-2011, 09:35 PM

I get the impression people are too focussed on getting rid of a warning message while not fully understanding the implications. I know *I* was too focussed on getting rid of that warning message :-) I am using a system (ASUS M5A87) with an AMD Phenom II X4 955, AMD Radeon HD 6570 (PCIe) and 4GB of memory. I am currently using Fedora 16, but the effect of iommu=noaperture is the same on Fedora 15.

While using iommu=noaperture ("ask the IOMMU not to touch the aperture for AGP") makes the following section of the kernel messages go from:

Code:

Dec  7 01:55:49 Odin kernel: [    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
Dec  7 01:55:49 Odin kernel: [    0.000000] Checking aperture...
Dec  7 01:55:49 Odin kernel: [    0.000000] No AGP bridge found
Dec  7 01:55:49 Odin kernel: [    0.000000] Node 0: aperture @ 0 size 32 MB
Dec  7 01:55:49 Odin kernel: [    0.000000] Your BIOS doesn't leave a aperture memory hole
Dec  7 01:55:49 Odin kernel: [    0.000000] Please enable the IOMMU option in the BIOS setup
Dec  7 01:55:49 Odin kernel: [    0.000000] This costs you 64 MB of RAM
Dec  7 01:55:49 Odin kernel: [    0.000000] Mapping aperture over 65536 KB of RAM @ c4000000
Dec  7 01:55:49 Odin kernel: [    0.000000] PM: Registered nosave memory: 00000000c4000000 - 00000000c8000000
Dec  7 01:55:49 Odin kernel: [    0.000000] Memory: 3964892k/4980736k available (4873k kernel code, 787416k absent, 228428k reserved, 6776k data, 944k init)

to:

Code:

Dec  7 02:06:32 Odin kernel: [    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
Dec  7 02:06:32 Odin kernel: [    0.000000] Memory: 4030428k/4980736k available (4873k kernel code, 787416k absent, 162892k reserved, 6776k data, 944k init)

It also makes the following section go from:

Code:

Dec  7 01:55:49 Odin kernel: [    1.423749] Freeing initrd memory: 16048k freed
Dec  7 01:55:49 Odin kernel: [    1.428945] PCI-DMA: Disabling AGP.
Dec  7 01:55:49 Odin kernel: [    1.429066] PCI-DMA: aperture base @ c4000000 size 65536 KB
Dec  7 01:55:49 Odin kernel: [    1.429068] PCI-DMA: using GART IOMMU.
Dec  7 01:55:49 Odin kernel: [    1.429071] PCI-DMA: Reserving 64MB of IOMMU area in the AGP aperture

to:

Code:

Dec  7 02:06:32 Odin kernel: [    1.413799] Freeing initrd memory: 16048k freed
Dec  7 02:06:32 Odin kernel: [    1.419009] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Dec  7 02:06:32 Odin kernel: [    1.419013] Placing 64MB software IO TLB between ffff8800cbf7e000 - ffff8800cff7e000
Dec  7 02:06:32 Odin kernel: [    1.419015] software IO TLB at phys 0xcbf7e000 - 0xcff7e000

In other words, where first on-chip hardware was used (GART IOMMU), after using the kernel option iommu=noaperture a software implementation will be used. I guess (but I'm not sure) this is not what I and with me, other people want. That is, compared to losing 64MB of memory. However, that appears to apply only to systems with an AMD 64-bits processor, because they contain an IOMMU. AFAIK systems with an Intel processor will always use a software implementation. My system will lose 64MB either way, because at address range 0xcbf7e000 - 0xcff7e000 there is memory according to the BIOS-provided physical RAM map.
Using the option iommu=noagp ("don't initialize the AGP driver and use full aperture") seems appropriate for a PCIe system, although I don't see any difference in the sections I referred to above (compared to not specifying the kernel argument iommu). So, this brings us back to "Please enable the IOMMU option in the BIOS setup". I don't know how to do that on my system...

To add to tredegar's explanation about where to specify kernel arguments: On Fedora 16 you would have to modify the file /boot/grub2/grub.cfg.

JZL240I-U · 12-07-2011, 01:10 AM

Quote:

Originally Posted by dbjh

I get the impression people are too focussed on getting rid of a warning message while not fully understanding the implications. ...

Exactly. But as you can see from this thread alone, there are no fast and easy explanations either, not to mention your

Quote:

Originally Posted by dbjh

So, this brings us back to "Please enable the IOMMU option in the BIOS setup". I don't know how to do that on my system...

Obviously nobody does. But I still hate "errors" with no chance of mitigating them.

Oh, and thanks for your post and explanations

.

dbjh · 12-07-2011, 06:35 AM

Quote:

Originally Posted by JZL240I-U

Exactly. But as you can see from this thread alone, there are no fast and easy explanations either, not to mention your

Excuse me, but my what?
It depends on the level of understanding and knowledge of the reader whether it is easy of course. As far as I understand the problem it is basically a shortcoming in BIOS implementations that do not offer the option to map part of the memory outside the 32-bit address space (to create a "hole" in the address space that can be used to communicate with the video card without eclipsing memory). To reach optimal performance (perhaps to make things work at all) all video memory should be accessible within the 32-bit address space of the processor (probably while also using a bank switching technique). A BIOS written for 64-bit AMD processors *should* offer that option. An explanation for not implementing it might be that the cost of doing so is thought of as being higher than the lost sales due to the lack of that feature. The problem does not show itself if you have at least 64MB less than 4GB in a system. If you have exactly 4GB in your system you lose 1.6% of your usable memory. That percentage gets lower as you install more memory.

Quote:

Originally Posted by JZL240I-U

Obviously nobody does. But I still hate "errors" with no chance of mitigating them.

I understand that :-) I meant it more like it is impossible to do that on my system. That people, including me even note and try to solve this problem is also because of the wording of the kernel log message. For example, I quoted this part of my /var/log/messages:

Code:

Dec  7 02:06:32 Odin kernel: [    1.419013] Placing 64MB software IO TLB between ffff8800cbf7e000 - ffff8800cff7e000
Dec  7 02:06:32 Odin kernel: [    1.419015] software IO TLB at phys 0xcbf7e000 - 0xcff7e000

I missed this before I read your post where you quoted part of your /var/log/boot.msg. Here 64MB is also being wasted without making it explicit to the casual reader. I guess we would have seen a thread on this site if a kernel message was added that said "This costs you 64 MB of RAM".
And yes, I also dislike warning or error messages without being able to do anything about it.

Quote:

Originally Posted by JZL240I-U

Oh, and thanks for your post and explanations

.

You are welcome. Thank you too for your posts :-) Thanks to you I realised there is a cost to using iommu=noaperture and that I cannot avoid losing 64MB on my system.

JZL240I-U · 12-07-2011, 06:47 AM

Quote:

Originally Posted by dbjh

Excuse me, but my what?...

Your next snippet of text I quoted immediately following my "..., not to mention your" -> So, this brings us back to "Please enable the IOMMU option in the BIOS setup". I don't know how to do that on my system....

Sorry, I found this self-explaining when I wrote it, I had no intention to be obscure.

Quote:

Originally Posted by dbjh

It depends on the level of understanding and knowledge of the reader whether it is easy of course...?

Woe unto me

.

Quote:

Originally Posted by dbjh

[more detailed explanations here]
...
...I cannot avoid losing 64MB on my system.

My first Atari MegaST computer had 1 (one!) MB, 360 kB more than the then biggest PC-memory available, and so it seemed huge at that time. I still have a bone-feeling of waste, when I read about the casual loss of 64 MB even though my system now has 4 GB ... must be getting old

.