Modify APIC interrupts priorities

Hello,

My PC104 (PentiumM) is connected to a customized PCI hardware.
The hardware creates PCI interrupts every ~2ms.
When the e100 (NIC) and\or the ATA controller are working, sometimes
we are not getting an interrupt from the hardware.
Is it possible to modify the APIC driver in order to resolve this issue ?
The interrupt from the customized hardware is much more important than the e100 and other PCI devices.

Thanks.

Is your custom device using level triggering, or are you expecting to get edge triggering?

Originally Posted by neonsignal Is your custom device using level triggering, or are you expecting to get edge triggering?

This is the output of /proc/interrupts.
The line in bold (rdm) is my device.
Thanks.

CPU0
0: 4157 IO-APIC-edge timer
1: 2 IO-APIC-edge i8042
8: 0 IO-APIC-edge rtc0
9: 0 IO-APIC-fasteoi acpi
12: 4 IO-APIC-edge i8042
14: 0 IO-APIC-edge ata_piix
15: 37 IO-APIC-edge ata_piix
16: 190 IO-APIC-fasteoi uhci_hcd:usb2
20: 15331 IO-APIC-fasteoi eth0
21: 0 IO-APIC-fasteoi rcm
23: 2 IO-APIC-fasteoi ehci_hcd:usb1
NMI: 0 Non-maskable interrupts
LOC: 6011 Local timer interrupts
SPU: 0 Spurious interrupts
RES: 0 Rescheduling interrupts
CAL: 0 Function call interrupts
TLB: 0 TLB shootdowns
TRM: 0 Thermal event interrupts
ERR: 0
MIS: 0

Is the problem that you are missing the interrupts, or that they are not being serviced within the 2ms before the next interrupt comes in? It would be surprising if it was missing level-sensitive interrupts that are this slow.

You can 'grep intr /proc/stat' to look at interrupt counts, which may be different to the counts in /proc/interrupts.

If you have latency critical code in the interrupt handler, you may need to run at least part of the handler as a 'fast interrupt', ie, using the SA_INTERRUPT flag.

There are also real-time patches for the linux kernel if you require a more systematic approach.

Originally Posted by neonsignal Is the problem that you are missing the interrupts, or that they are not being serviced within the 2ms before the next interrupt comes in? It would be surprising if it was missing level-sensitive interrupts that are this slow. You can 'grep intr /proc/stat' to look at interrupt counts, which may be different to the counts in /proc/interrupts. If you have latency critical code in the interrupt handler, you may need to run at least part of the handler as a 'fast interrupt', ie, using the SA_INTERRUPT flag. There are also real-time patches for the linux kernel if you require a more systematic approach.

It seems the kernel does not miss interrupts.
In the ISR I call to:
flag = 1;
wake_up_interruptible(&wq);
The application checks for interrupt using an IOCTL and calls to the following in the kernel:
wait_event_interruptible(wq, flag != 0);
flag = 0;
The ioctl in the user space sleeps till an interrupt arrives.
Can you what is wronge with this mechanism?
Thank you for you time and effort.

There is nothing wrong with your mechanism; it is quite normal. However, it does not guarantee that the application will be woken the same number of times as the interrupt comes in (since the application may not even have reached the event queue when the next interrupt comes in).

This is not always a problem. Many devices are hardware buffered, so all that matters is that the read/write to the device is woken up when the device is ready.

But in cases where there is no buffering (eg a parallel port driver), the buffering must be done within the interrupt handler.

In general, it depends on the purpose of the interrupt.

case 1. All that is required is to update the application based on the interrupt count (such as a timer). Then either the count is available from the hardware, or the interrupt routine can increment the count. Later, the application reads this count.

case 2. The application must perform some action on every interrupt (but it is not time-critical). Then the interrupt routine will need to keep track of how many interrupts have occurred, so that the application can perform the appropriate number of actions (for a timer, this could be as simple as a counter, and the application keeps track of the previous value it saw).

case 3. Some action must be taken before the next interrupt comes in. Then the interrupt itself must undertake at least the critical portion of the action (such as buffering).

(the reason I am not directly answering the original question, about modifying the interrupt priority, is that I am assuming that the issue is that you are 'missing' interrupts in the application, rather than the microseconds latency of the handler)

Originally Posted by neonsignal There is nothing wrong with your mechanism; it is quite normal. However, it does not guarantee that the application will be woken the same number of times as the interrupt comes in (since the application may not even have reached the event queue when the next interrupt comes in). This is not always a problem. Many devices are hardware buffered, so all that matters is that the read/write to the device is woken up when the device is ready. But in cases where there is no buffering (eg a parallel port driver), the buffering must be done within the interrupt handler. In general, it depends on the purpose of the interrupt. case 1. All that is required is to update the application based on the interrupt count (such as a timer). Then either the count is available from the hardware, or the interrupt routine can increment the count. Later, the application reads this count. case 2. The application must perform some action on every interrupt (but it is not time-critical). Then the interrupt routine will need to keep track of how many interrupts have occurred, so that the application can perform the appropriate number of actions (for a timer, this could be as simple as a counter, and the application keeps track of the previous value it saw). case 3. Some action must be taken before the next interrupt comes in. Then the interrupt itself must undertake at least the critical portion of the action (such as buffering). (the reason I am not directly answering the original question, about modifying the interrupt priority, is that I am assuming that the issue is that you are 'missing' interrupts in the application, rather than the microseconds latency of the handler)

I would like to appreciate your help.

In my case the hardware has no buffering. The application can not afford to loose an interrupt. The application can read an interrupt counter incremented by the kernel but if it finds that it lost one, it is too late. So I think case 2 describes my system.

I wonder is the following code in the kerenl can help:
IOCTL:
wait_event_interruptible(wq, atomic_read(&flag)!=0);
atomic_dec (&flag)
ISR:
atomic_inc (flag);
wake_up_interruptible(&wq);

Do you think it can solve the problem ?

Best regards.

I wonder is the following code in the kerenl can help: IOCTL: wait_event_interruptible(wq, atomic_read(&flag)!=0); atomic_dec (&flag) ISR: atomic_inc (flag); wake_up_interruptible(&wq);

Yes, that's the kind of thing, as long as the IOCTL response is not time critical (because sometimes the flag will be greater than 1, and the task will iterate multiple times before suspending again).

In fact, if it is only the number of interrupts that matters, then you might find you don't even need to wake up the IOCTL task every time (depending on the application - you haven't explained the purpose of the interrupt).

Originally Posted by neonsignal Yes, that's the kind of thing, as long as the IOCTL response is not time critical (because sometimes the flag will be greater than 1, and the task will iterate multiple times before suspending again). In fact, if it is only the number of interrupts that matters, then you might find you don't even need to wake up the IOCTL task every time (depending on the application - you haven't explained the purpose of the interrupt).

Upon interrupt the application must write some 32bits registers to the PCI hardware. This data is used by the PCI core at the next hardware interrupt. It is OK that the user space will get the interrupt some time after the kernel gets it as long it will write all required data and do all processing before the next interrupt.

Thanks.

Upon interrupt the application must write some 32bits registers to the PCI hardware. This data is used by the PCI core at the next hardware interrupt. It is OK that the user space will get the interrupt some time after the kernel gets it as long it will write all required data and do all processing before the next interrupt.

To guarantee that the data is written before the next interrupt, then you will need to do the write inside the interrupt routine. You may be able to get the application to do most of the processing, and just buffer the values, but the time critical write should not be done in the application (the standard Linux kernel is not a real-time operating system).

I'm not saying it won't work most of the time if it is done in the application (and you can increase the priority level of the thread that does the writing), but the reliable way is to move time critical code to the interrupt.

If this is not workable (there is too much code that is time critical), then you need to consider real-time patches to the kernel.