LinuxQuestions.org
Go Job Hunting at the LQ Job Marketplace
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel
User Name
Password
Linux - Kernel This forum is for all discussion relating to the Linux kernel.

Notices

Reply
 
Search this Thread
Old 04-12-2010, 10:42 AM   #1
misterhw
LQ Newbie
 
Registered: Apr 2010
Posts: 1

Rep: Reputation: 0
Question Reloading Module stops MSIs


I'm running a 2.6.30.9 kernel on a SuperMicro/Intel chipset, and I'm writing a module for a PCIe device that uses MSI. The first time I load the module using insmod, interrupts work correctly, and the ISR is run as it should. When I do a rmmod and another insmod, interrupts are not caught by my module.

I've included the code from the init and exit functions below, but I have followed the directions in the MSI HOW-TO.txt file to call pci_enable_msi then request_irq in the init function. In the exit function, I call free_irq, then pci_disable_msi.

When interrupts are working (first insmod after reboot), /proc/interrupts shows the following after running a test program:

Code:
53:      33289     566723          0          0          0          0          0          0          0          0          0          0          0          0          0          0   PCI-MSI-edge      RMA Interrupt
When interrupts aren't working after reloading the module, /proc/interrupts show the following after running the same test program:

Code:
53:      0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0   PCI-MSI-edge      RMA Interrupt
I know that the problem isn't with the device because a reboot will fix the problem without restarting the device. Additionally, with a signal tap on the FPGA, the IRQ request signal remains high for the time when the first interrupt should come. The acknowledge signal isn't coming back to the device because the module isn't receiving the interrupts and going through the ISR.

Both when interrupts are working, and when they aren't, lspci -vv show the following:

Code:
05:00.0 Class ff00: Altera Corporation Unknown device 0004 (rev 01)
        Subsystem: Altera Corporation Unknown device 0004
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR-
        Latency: 0, Cache Line Size: 64 bytes
        Interrupt: pin A routed to IRQ 53
        Region 0: Memory at 680000000 (64-bit, prefetchable) [size=1024M]
        Capabilities: [50] Message Signalled Interrupts: 64bit+ Queue=0/2 Enable+
                Address: 00000000fee00000  Data: 40ba
        Capabilities: [68] MSI-X: Enable- Mask- TabSize=2
                Vector table: BAR=0 offset=00000400
                PBA: BAR=0 offset=00000500
        Capabilities: [78] Power Management version 3
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
                Status: D0 PME-Enable- DSel=0 DScale=0 PME-
        Capabilities: [80] Express Endpoint IRQ 0
                Device: Supported: MaxPayload 2048 bytes, PhantFunc 0, ExtTag-
                Device: Latency L0s <64ns, L1 <1us
                Device: AtnBtn- AtnInd- PwrInd-
                Device: Errors: Correctable- Non-Fatal- Fatal- Unsupported-
                Device: RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
                Device: MaxPayload 256 bytes, MaxReadReq 512 bytes
                Link: Supported Speed unknown, Width x4, ASPM L0s, Port 1
                Link: Latency L0s unlimited, L1 unlimited
                Link: ASPM Disabled RCB 64 bytes CommClk- ExtSynch-
                Link: Speed unknown, Width x4
Here is my source code for the init and exit functions for loading and unloading the module:

Code:
static int __init ssrma_init(void){
  int retval;
  ssrma_DEV_T *rma_dev;
  irq_handler_t irq_handler = &ssrma_pciIsr;
  rma_dev = &install_device;

  /* Register the driver with the PCI core */
  retval = pci_register_driver(&pci_driver);
  if(retval) return retval;

  /* Setup the cdev portion of the rma_device struct */
  retval = rma_setup_cdev(rma_dev);
  if(retval){
    goto registered;
  }

  /* Verify the device is found on a bus */
  rma_dev->pci_dev = pci_get_device(ssrma_VENDOR_ID, ssrma_DEVICE_ID, NULL);
  if(!rma_dev->pci_dev){
        retval = -ENODEV; // No such device
    goto cleanup;
  }
  /* Add the device to the device structure */
  pci_dev_put(rma_dev->pci_dev);

  /* Enable the MSI interrupts */
  if(pci_enable_msi(rma_dev->pci_dev)){
    retval = -1;
    goto cleanup;
  }
 /* Setup the interrupt ISR */
  if(request_irq(rma_dev->pci_dev->irq, irq_handler, IRQF_DISABLED, "RMA Interrupt", rma_dev)){
       retval = -1;
    goto msi_cleanup;
  }

  return 0;

  ioremap_cleanup:
    /* Unmap the mapped memory space */
    iounmap(rma_dev->virt_start_addr);
    /* Free the requested IRQ */
    free_irq(rma_dev->pci_dev->irq, rma_dev);
  
msi_cleanup:
    /* Disable the MSI interrupts */
    pci_disable_msi(rma_dev->pci_dev);
  
cleanup:
    /* Remove the device */
    cdev_del(&rma_dev->cdev);
 
 registered:
    /* Unregister the driver */
    pci_unregister_driver(&pci_driver);
  return retval;
} /* end rma_init */



static void __exit ssrma_exit(void){
  ssrma_DEV_T *rma_dev;
  rma_dev = &install_device;

  /* Unmap the mapped space */
  iounmap(rma_dev->virt_start_addr);

  /* Free the requested IRQ */
  free_irq(rma_dev->pci_dev->irq, rma_dev);

  /* Disable the MSI interrupts */
  pci_disable_msi(rma_dev->pci_dev);

  /* Remove the device */
  cdev_del(&rma_dev->cdev);

  /* Unregister the driver */
  pci_unregister_driver(&pci_driver);

} /* end rma_exit */
If you have had a similar problem, or know what I'm doing wrong, please let me know. If you need more information, I would be happy to provide it. It is frustrating to have to reboot everytime I change the module code, and it's something that needs to be fixed. Thank you.
 
Old 04-12-2010, 02:59 PM   #2
nini09
Senior Member
 
Registered: Apr 2009
Posts: 1,008

Rep: Reputation: 71
Use IRQF_SAMPLE_RANDOM instead of IRQF_DISABLED when you request interrupt handler.
 
  


Reply

Tags
interrupts, module, msi


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Reloading lp module bmcneely0 Suse/Novell 2 08-07-2006 04:50 PM
boot stops at "Updating module dependencies for 2.4..." everal Slackware 3 10-08-2005 01:55 PM
reloading eth0 module browser Slackware 2 07-31-2005 07:04 AM
VNC Works then stops! Reboot fixes but then it stops again! Leethal Linux - Software 1 02-26-2004 07:57 AM
Reloading fstab EnigmaX Linux - Newbie 1 09-23-2003 10:35 AM


All times are GMT -5. The time now is 01:29 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration