How to write a PCI Express device driver for Xilinx Virtex-5 LXT/SXT Dev Kit?

Originally Posted by nsaba I was in a similar situation as you are in, a few months ago; In general, in your driver you must enable PCI/PCI-Express device, initialize it and all its features need to be initialized for operation, provide access method/s to it and in from here comes special section that depends on your hardware internal architecture that you provide by programming FPGA. I started with ldd3, Linux essential device drivers, and some kernel development knowledge. Now that the device is getting ready to operate, I've developed driver as well in an incremental manner in parallel with device. I've provided access to device from user space through a character device node in File System (for testing device functionality) and, access from kernel space from kernel crypto api. Please give more description about your current status, errors you are involve and device architecture and functionality. I'll be pleased if I could help you.

First of all, thanks for your answer and sorry for my late reply.
I was finishing another part of my project this last week.

So, related to this topic..
I think my main problem is actually inexperience. :|
I'm a student and I've only had lessons for learning VHDL and working with FPGAs during one semester.. on top of that I've only worked with a Nexys-2 board so far.
So basically I've already a few projects developed and working on a Nexys-2. What I'm want to do for now is to adapt this projects to work on the Virtex-5 (which is all new to me) and use the PCI Express bus for data input.
From what I've read so far before I start developing the driver for Linux I think I should have the FPGA module working.. I've run into this "LogiCORE™ IP Endpoint Block Plus for PCI Express" at Xilinx website. Is this were I should start? Or do you have another suggestion?
I only have about 1 month to get this working so I would really appreciate any help from you.


Best regards.

Originally Posted by nsaba .. In general, in your driver you must enable PCI/PCI-Express device, initialize it and all its features need to be initialized for operation, provide access method/s to it and in from here comes special section that depends on your hardware internal architecture that you provide by programming FPGA. I started with ldd3, Linux essential device drivers, and some kernel development knowledge. Now that the device is getting ready to operate, I've developed driver as well in an incremental manner in parallel with device. .. I've provided access to device from user space through a character device node in File System (for testing device functionality) and, access from kernel space from kernel crypto api. Please give more description about your current status, errors you are involve and device architecture and functionality.

====
hello nsaba,
Great to hear you succeeded to make it working. Now I meet same situation where I want to develop one PCIe EP driver, connected to my board RC .. So your job is very interesting really ..

Also i'm interesting to your userspace testing method, could you share with us about your:
1. driver code if posisble ;
2. userspace testing method ;

Thanks veyr much!!

Du

Originally Posted by Restless86 First of all, thanks for your answer and sorry for my late reply. I was finishing another part of my project this last week. So, related to this topic.. I think my main problem is actually inexperience. :| I'm a student and I've only had lessons for learning VHDL and working with FPGAs during one semester.. on top of that I've only worked with a Nexys-2 board so far. So basically I've already a few projects developed and working on a Nexys-2. What I'm want to do for now is to adapt this projects to work on the Virtex-5 (which is all new to me) and use the PCI Express bus for data input. From what I've read so far before I start developing the driver for Linux I think I should have the FPGA module working.. I've run into this "LogiCORE™ IP Endpoint Block Plus for PCI Express" at Xilinx website. Is this were I should start? Or do you have another suggestion? I only have about 1 month to get this working so I would really appreciate any help from you. Best regards.

Hi,
Sorry for such a late reply;
I was not aware of this thread being updated .
But about your question "LogiCORE™ IP Endpoint Block Plus for PCI Express" is the right way to begin.

Sincerely.

Originally Posted by dudu1102 ==== hello nsaba, Great to hear you succeeded to make it working. Now I meet same situation where I want to develop one PCIe EP driver, connected to my board RC .. So your job is very interesting really .. Also i'm interesting to your userspace testing method, could you share with us about your: 1. driver code if posisble ; 2. userspace testing method ; Thanks veyr much!! Du

Hi Du,
I saw your reply to xilinx forums thread today. Sorry for late reply.

But now about your questions:
1. I will share part of my driver code as soon as possible. But for now as a quick reply, in driver I wrote data to some registers of device and read from those. For registers to be accessible you must first map some BAR of device that is created during generating of PCIe endpoint block plus using xilinx ISE.


2. In user space, as I said before, you must prepare some tool to make transfer of data between kernel and user space possible. I used char device mechanism for this goal. This provides you a node in file system e.g /dev/your_pci. Then you must open that node just like any ordinary file, read from and write to it. The format of data to be written to device might be dependent of the device architecture or simply raw data for testing purpose only.
I'll be glad if I can help you.

Good luck;

Hi,

I am supposed to send/receive data from xilinx spartan 6 to PC (this is atom processor running on Ubuntu embedded edition[UME]) through a PCIe port. I am currently struck as the basic hello world given in ldd3 doesnt compile in UME..It comes up with can't find file errors in /usr/block/Kconfig.iosched.. when i compare the source kernel files and the kernel files present in the UME, i find there are more files missing. But the UME loads the ubuntu interface properly. The UME has linux kernel version 2.6.22-menlow-default. I could find the missing files in my PC ubuntu .. I am booting the linux using an USB in the UME.. Since i can connect to internet from the UME device, can i download the latest kernel using apt-get directly..?? will the changes get installed in USB itself?

Basically i am new to linux..kindly help me..

Thanks,
Murali

Originally Posted by nsaba Hi Du, I saw your reply to xilinx forums thread today. Sorry for late reply. But now about your questions: 1. I will share part of my driver code as soon as possible. But for now as a quick reply, in driver I wrote data to some registers of device and read from those. For registers to be accessible you must first map some BAR of device that is created during generating of PCIe endpoint block plus using xilinx ISE. 2. In user space, as I said before, you must prepare some tool to make transfer of data between kernel and user space possible. I used char device mechanism for this goal. This provides you a node in file system e.g /dev/your_pci. Then you must open that node just like any ordinary file, read from and write to it. The format of data to be written to device might be dependent of the device architecture or simply raw data for testing purpose only. I'll be glad if I can help you. Good luck;

Hi there,
I'm hoping someone can help me.
I did step one and most of two- I wrote a driver based on pci_skel.c modeled as a char device. I've got a Virtex 5 with the
pci-express endpoint core on the other end.
I can access it via /dev/mytoy. I mapped BAR0 to be serviced by read() and write() and BAR1 to be done via ioctl()
I didn't do anything ambitious- writew/readw to/from the device. This appears to be working BUT...
I have one task that does nothing but ioctl()s to the device. Running cat bigfile >/dev/mytoy simultaneously works.
Running a task that does writes eventually hangs- in a manner that a reboot is required to get out. I can see the debug
messages that the /dev/mytoy was opened, but the subsequent writes are not there. The ioctl transactions have also stopped.
processes attached to /dev/mytoy are hung. Attempts to remove the driver return a refusal of being busy.

I've put in mutexes around all the write/reads to/from the hardware. Anything else I ought to be doing?
Thanks
Carl
Once I get past this- I got to look at the DMA stuff here. I never did figure out to send large pci-e transaction buffers.

Have anybody tried to do this with a network driver instead of a char driver?

Originally Posted by magda Hi, here you have links to pci skeletons that I've found useful, from the simplest to the most complicated: http://www.cs.fsu.edu/~baker/devices...pci/pci_skel.c http://true-random.com/homepage/proj...pci_template.c http://www.gelato.unsw.edu.au/lxr/so...pci-skeleton.c Linux Device Drivers is quite complete boot, just follow chapter 9 and edit pci_skel.c (first of the previous links) accordingly in order to get simple read and write operations. Also you need to decide if you want your driver to be a Character or a Block device, here you have some discussions: http://www.cab.u-szeged.hu/linux/doc/khg/node18.html http://www.linuxhq.com/guides/LKMPG/node14.html http://www.faqs.org/docs/kernel/x571.html In case you go for Character Device then you will need to create it from a linux terminal. I don´t have the info on how to do that yet. I hope this helps, Magda

Hi.
I have a problem in PCI driver.
I have a PCI Driver in 2.4 keneral work well.
But when I transplant it to 2.6 keneral.
The problem appear.
lseek() is not work.
I look PCI driver code in 2.4 keneral.
I don't see lseek().
same code .


static struct file_operations rubicon_fops = {
owner: THIS_MODULE,
read: rubicon_read,
write: rubicon_write,
poll: rubicon_poll,
ioctl: rubicon_ioctl,
open: rubicon_open,
release: rubicon_close,
};

My PCI is a Character Device.
Everyone know different in PCI driver with 2.4 keneral and 2.6 keneral.
Thanks.