Hello,

My background is in FPGA design, not kernel development. I'm pretty new to a lot of the things going on behind the scenes in the kernel. I've been trying to get my head wrapped around various details related to memory management in Linux. I appear to have hit a roadblock and could really use some help.

I'm trying to determine how to obtain the page structures of PCI-E virtual addresses I've mapped with ioremap. I have no problem actually reading or writing to memory so I'm sure that much as been done correctly. I get a virtual address back and was planning to to virt_to_page. However I noticed that virt_addr_valid returns false. Digging more into things two appear to be two kinds of virtual address within the kernel: logical and virtual (this seems like an unfortunate naming convention). Near as I can virt_addr_valid is checking for logical address. The virtual address space is often referred to as the vma_alloc area as well. This area is also where IO is mapped. Digging more I found the is_vmalloc_addr helper. Passing in the address return from ioremap returns true. Great, so now I can use vmalloc_to_page. Unfortunately This returns NULL for my PCI-Express virtual address. Passing in a pointer allocated with vmaalloc does return a page. At this point I've appeared to have exhausted all avenues for actually getting these pages. I may dig more into how ioremap in to get answers but I'm hoping a kind expert will be able to point me in the right direction.

Thanks for any assistance!

I'm not an expert on this, but I think ioremap returns a virtual address which is not in the page tables. Access to the PCIe memory is supported instead by request_mem_region which will prevent multiple processes from mapping the same addresses.

Why would you need a page structure for an I/O region? If you are trying to map to user space you just use the uio driver.

According to this random blog post, ioremap does create entries in the page table: http://learnlinuxconcepts.blogspot.c...s-ioremap.html.

Why do I need this? I'm trying to do some interesting work where I'm moving data between different hardware peripherals without going through system memory. Trying to paint into user space and then DMA from userspace isn't an oprion as I'm pushing a whole lot of data (gigabytes per second). Interestingly enough the 4.20 kernel has added support for this kind of functionality between PCI-Express endpoints (with certain limitations), but I also have a use case where I'm moving data where one of the endpoints isn't a PCI-Express device. I'm trying to keep Linux involved and orchestrating these transfers (especially in the case of when writing files to a disk). However I'm not super knowledgeable about anything outside of user space and basic driver development so I'm not 100% sure this is possible yet.

For those interested, playing around with the PCI Peer to Peer infrastructure bore some fruit. The pci_p2pdma_add_resource creates page structures. I'm not 100% sure if ioremap does or not. It creates something in the in the page tables, but I wasn't able to get a valid page structure. Fortunately the addresses returned from pci_alloc_p2pmem are kernel logical address. As a result virt_to_page works on them. Doing a page_to_phys returns the BAR address so all looks good. There are some limitations with the currently supported features, but I expect future versions of the kernel to expand upon details.