Detecting when a process has been interrupted by CPU
 

Hello All,

Is it possible to detect when a particular process is interrupted? For example, let us say that we have three process running A, B, C. The scheduler is constantly switching tasks between A, B, and C. What I want to be able to do is detect when the CPU interrupts process B (here I want to clear an arbitrary flag) to run another process. Once the other process has completed, or the time quanta has expired, process B will resume. Once process B resumes, I want to set my arbritary flag that I had cleared earlier. Is this possible? Would I have to create my own kernel module, or would I have to modify the CPU scheduler (hopefully no)?

I am looking for any insight or help that could point me in the right direction. Thanks for your help and I look forward to your feedback.

Here is some info on my system:
2.6.23-gentoo-r9
i686 Intel(R) Pentium(R) 4 CPU 3.00GHz GenuineIntel GNU/Linux

Regards,
JQ

You would have to modify the CPU scheduler.

-----------------
Steve Stites

Detecting when a process's CPU utilization goes about 0.0 %
 

What about detecting when a process's cpu utilization goes above 0.0%? As you know, when a process is executing it is using the CPU. When the cpu is not executing the process, the process is idle and its cpu utilization should be zero (i.e. the process was preempted and the cpu is executing some other process). I know you can get good information on a particular process. For example, if you do something like:

top -b -n 1 | grep my_process

you would get the stats for my_process, i.e. PID, %CPU, %MEM, etc.

I wonder if there is a way to monitor the process I want, and when its CPU utilization rises above 0.0 %, I can set my flag. The problem is that I don't want to use a method that involves polling. Ideally I would like to use something that is interrupt driven. Is it possible to create such an interrupt (i.e. interrupt when process cpu > 0.0 and interrupt when process cpu == 0.0)? I'm hoping something like this is feasible. In either case, I look forward to any feedback you all may have. Thanks in advance.

Regards,
JQ

The interrupts are handled by the CPU scheduler. What you want to do means you would have to add extra logic to the CPU scheduler to record the data and/or notify your monitor code.

I wrote such code when I worked on developing IBM operating systems in the 1960s. You have to rewrite parts of the operating system to log events as they happen.

-------------------
Steve Stites

No I think the way it works is when the timer fires, it causes a hardware interrupt that makes it jump to a particular location, which is inside the OS. To do what you're suggesting, this location would have to have code to do the flag (which I think would be very dicey as the OS is not supposed to tamper with any user programs space), which, yes, I'd think means modifying linux.

Perhaps I am going about this the wrong way. Here is some background on exactly what I am doing and what I want to achieve. Maybe with this you guys can perhaps come up with a better plan of attack than the one I was trying to take.

Currently I have an OpenGL ES 1.1 demo. As you may know, graphical processes typically have lower priorities than other cpu processes. Thus, the cpu is constantly switching in and out of my graphical demo. What I want to know is how long my cpu is spending time doing graphical operations. When the cpu is executing a graphical operation, I want to be able to output a high signal. When the cpu is not executing a graphical operation, I want to output a low signal.

I don't know if this explanation helps, but I know it can't hurt. In either case, more details never hurt anything. Perhaps with this information one you all can suggest a solution that involves NOT modifying the scheduler. If not, well I appreciate your help anyways. Thanks again.

Regards,
JQ

I have done this several times, though a long time ago. I used specialized machinery (two different machines) made by IBM. These machines allowed you to put hardware probes into different memory locations of a computer. The machine was programmable and one of the things that you could do was measure the amount of CPU time spent within the memory of a particular program such as your graphical programs. The machine method had a huge advantage as there was no overhead associated with it on the subject machine and the timings and order of events were not distorted by the measurement.

I also wrote software called SIGNET that would come close to doing what you want. Like resetreset says I had to write code to handle all hardware interrupts and then pass the interrupt on to the regular kernel code. I also had to insert code into what Linux calls the scheduler. Then I had to write a logging function to accumulate the information from all the snippets of kernel code I wrote and write to a log file. My code did distort timings, sometimes a little, sometimes a lot depending on what I was trying to measure. The logging function took long enough that sometimes asynchronous events took place in a different order than they would have otherwise. The only advantage the software method had over the hardware method was that the software could pull details out of registers and such that just weren't available to the hardware monitoring machines.

I don't know how much time you have to spend on this but you are probably better off to use an existing sampling program like top. For a large sample size a sampling program will give answers reasonably close to the hardware monitor results and probably much better than the software monitor results.

--------------------
Steve Stites

First of all, thanks for your reply. Secondly, the work you did is impressive.

The problem with this method, although a good one, is that I don't have the resources for this.

Your right, this is similar, however, what I'm trying to do isn't really driven by hardware interrupts. Unless there is some interrupt, that I don't know about, that is flagged by the cpu doing graphical operations, then I feel like I am back to square one.

As a last note, as far as a time frame to get this done, I have about 2-3 weeks. Again, thanks for your reply. If you have any other suggestions or comments please continue to post your ideas.

Regards,
JQ

Yes what you want to measure is driven by hardware interrupts as you correctly stated in your first post.

And yes you have no way of knowing about all of these other interrupts except by placing your own hooks in the various upper half interrupt handler routines and by placing a hook in the CPU scheduler to keep track of what process the scheduler decides to schedule next after one of the interrupt handler routines has handled an interrupt.

---------------------
Steve Stites

Hmmm...I figured it was software interrupts rather than hardware interrupts since it is the scheduler that interrupts processes that have exceeded the defined time quanta. In either case (hardware or software interrupts), it seems that all roads lead to modifying the scheduler. Thanks for all your input.

Regards,
JQ

Maybe you could use markers, and just write the probe that needs to be hooked. You might even find the scheduler has them (for systemtap) - I saw somewhere Ingo say he had a patch that you could trace the scheduler, probably when he released the CFS.
There was some more tools being incorporated in the mainline as well. Been a while since I looked at lkml though - try having a search there.

Do you want to measure time spent in the graphical part of your program? You could do that by wrapping the calls to the graphic routines so that a high-resolution timer is updated as appropriate. You may be able to track time with times() (man 2 times). The scheduler updates that value, but keep in mind that the values may not be that accurate.

I dont follow - top will tell you what % your grphics program is using...?