LinuxQuestions.org
Did you know LQ has a Linux Hardware Compatibility List?
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel
User Name
Password
Linux - Kernel This forum is for all discussion relating to the Linux kernel.

Notices


Reply
 
Search this Thread
Old 01-11-2009, 10:15 PM   #1
AustinMarton
Member
 
Registered: May 2007
Location: New Zealand
Distribution: Fedora at home, Ubuntu 10.04 at work
Posts: 88

Rep: Reputation: 16
Userspace handling of driver interrupts


I am writing a driver to handle a network device on uClinux.

For reception of Ethernet frames, the interrupt handler receives the frame from the hardware, then it gets passed up and dealt with by the higher layers of the TCP/IP stack, so we do not have to do anything in user space.

The network device also sends and receives "control frames" which contain statistical information. When these interrupts are received, the same hardware reception procedure occurs (from a different buffer), but this time, we need to write a user space application that interprets the statistical information, and updates SNMP information.

How do we call a user space application from a kernel interrupt handler? Is that what I want to do or is there a better way? I think I am lacking an understanding of what the next step is after an interrupt handler executes?

Thanks,
Austin.
 
Old 01-11-2009, 10:21 PM   #2
jailbait
Guru
 
Registered: Feb 2003
Location: Blue Ridge Mountain
Distribution: Debian Wheezy, Debian Jessie
Posts: 7,636

Rep: Reputation: 219Reputation: 219Reputation: 219
Quote:
Originally Posted by AustinMarton View Post

How do we call a user space application from a kernel interrupt handler? Is that what I want to do or is there a better way? I think I am lacking an understanding of what the next step is after an interrupt handler executes?
Probably what you should do is have the application read or write to buffers and then wait for completion of the I/O. The driver handles the interrupt and posts the wait as complete.

-------------------------
Steve Stites
 
Old 01-12-2009, 02:56 PM   #3
AustinMarton
Member
 
Registered: May 2007
Location: New Zealand
Distribution: Fedora at home, Ubuntu 10.04 at work
Posts: 88

Original Poster
Rep: Reputation: 16
I have just been reading this article and I am wondering if it would be possible for me to append a custom packet header to my "control frames", and then call netif_rx to pass them to the TCP/IP protocol stack, where I would have a custom handler to run in user space "Another possible application of raw IP is, for example, realizing custom network protocols at the user level"

Would this be a feasible/efficient way of doing things?

Jailbait,
I don't quite understand how these buffers would be accessed in user space, or how to "post the wait". Would the user space app periodically poll a variable to see if there is a control frame waiting?
 
Old 01-12-2009, 03:46 PM   #4
jailbait
Guru
 
Registered: Feb 2003
Location: Blue Ridge Mountain
Distribution: Debian Wheezy, Debian Jessie
Posts: 7,636

Rep: Reputation: 219Reputation: 219Reputation: 219
Quote:
Originally Posted by AustinMarton View Post
I don't quite understand how these buffers would be accessed in user space, or how to "post the wait". Would the user space app periodically poll a variable to see if there is a control frame waiting?
No, the thread would simply wait. When a control frame is available in the buffer the device driver would tell the scheduler that the event that the thread has been completed ("post the wait" although that may not be the terminology used by the kernel documentation) and the scheduler would make the thread active again.

Quote:
Originally Posted by AustinMarton View Post
I don't quite understand how these buffers would be accessed in user space

The application would issue a read to read the control frame into a buffer and then wait for completion of the read.

------------------
Steve Stites
 
Old 01-12-2009, 05:56 PM   #5
sundialsvcs
Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 5,632

Rep: Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260
You need to re-think your strategy.

Kernel-space code can execute synchronously with the interrupts ... that is to say, the arrival of an interrupt will, well, interrupt whatever the userspace is doing and hand control over to your (kernel-space) handler.

Userland, on the other hand, is never synchronous with the hardware. In fact, "userland only gets to run whenever the kernel has nothing else better to do."

When a statistical-information packet arrives, the kernel-space code has to be the one to capture it. And, the kernel-space code has to be the one to "signal" "anyone in userland who might be interested" that this has just occurred. But, having done so, there must be an indeterminate delay between the time of this signal and the time that any userland process might get around to gathering-up the information.

Userland activities are always "loosely coupled" with whatever kernel-space is doing.

Let's toss out some kind of example. Let's say that you define a virtual-device that a userland process can read() to get statistical information. Simple enough... Let's say that it is going to be a well-behaved virtual character-device (a la /proc) ... "implementation details left as an exercise to the reader."

So, whenever a duly-authorized process reads from this device, he'll get a string representation of the latest statistics. These statistics are collected from the ring-buffer. "Nice and sweet and simple, just like /proc."

Let's fantasize and say that, each time a new entry is added to the ring buffer, a (kernel-maintained) counter is incremented, and this counter-value is stored as part of the data. Presto, we have a simple "dis-ambiguation" mechanism which will allow our user-land process to determine if he's seen a particular entry before ... and also to know how many times the buffer has been "overrun."

In my humble opinion, /proc stands out as one of the most elegant "hacks" ever to have been thought-of ... and you would do well to "go and do likewise." Now, I don't mean to imply that your code will be in that namespace, but "you get the idea."
 
Old 01-12-2009, 08:50 PM   #6
AustinMarton
Member
 
Registered: May 2007
Location: New Zealand
Distribution: Fedora at home, Ubuntu 10.04 at work
Posts: 88

Original Poster
Rep: Reputation: 16
Thanks for the reply sundailsvcs.

Quote:
Originally Posted by sundialsvcs View Post
Kernel-space code can execute synchronously with the interrupts ... that is to say, the arrival of an interrupt will, well, interrupt whatever the userspace is doing and hand control over to your (kernel-space) handler.

Userland, on the other hand, is never synchronous with the hardware. In fact, "userland only gets to run whenever the kernel has nothing else better to do."

When a statistical-information packet arrives, the kernel-space code has to be the one to capture it. And, the kernel-space code has to be the one to "signal" "anyone in userland who might be interested" that this has just occurred. But, having done so, there must be an indeterminate delay between the time of this signal and the time that any userland process might get around to gathering-up the information.

Userland activities are always "loosely coupled" with whatever kernel-space is doing.
I understand and agree with what you are saying above, and having an indeterminate delay is fine, we just don't know how to "signal" to the userland process, and how the process waits around to pick up the information?
 
Old 01-12-2009, 09:12 PM   #7
jiml8
Senior Member
 
Registered: Sep 2003
Posts: 3,171

Rep: Reputation: 115Reputation: 115
Here is a code fragment that does what you want to do:

In the kernel driver, in the interrupt handler:

Quote:
if(want to run statistical package) { //some flag to specify we want to talk to our userspace process
if(datacollector_pid){
kill_proc(datacollector_pid,SIGUSR1,1); //signal the userspace process
} else {
printk("<1>our userspace app not running\n");
}
}
In the initialization routine for the driver, we specify that it can read and write from /proc, like this:
Quote:
proc_file=create_proc_entry("mydrivernameinproc",0644, NULL);
if(!proc_file) {my_exit();}
proc_file->read_proc = read_procmem;
proc_file->write_proc = write_procmem;
Also in the kernel driver, we parse commands fed in through /proc, like this:

Quote:
int write_procmem(struct file *file, const char __user *buf,
unsigned long count, void *data)
{
char *loc_buf;
loc_buf = kmalloc(count,GFP_KERNEL);
if(copy_from_user(loc_buf,buf,count)) {return -EFAULT;}

and then process the command
And the userspace process, when it starts up, sends its pid to the kernel driver via a command entered by the entry in /proc. In the code I have provided here, the kernel driver stores that pid in the variable datacollector_pid. The userspace process also has a signal handler in place that captures SIGUSER1 when it is sent by the driver. In processing that signal, the userspace process obtains the info it needs from the driver, presumably through /proc, and proceeds to analyze the data.

Last edited by jiml8; 01-12-2009 at 09:18 PM.
 
Old 01-13-2009, 08:05 AM   #8
sundialsvcs
Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 5,632

Rep: Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260
I agree that the notion of using signals is a good one. The other choice is to implement the read() of your virtual device to be a "blocking read," which suspends the process if there is nothing to read, but I think that it's generally better that the read-request should finish without delay even if it returns zero bytes.

A signal is simply "hey you! wake up and go see if you've got any data!" The userland process might discover that it actually does, or it might discover that there's nothing to read. The signal will simply make it possible for the process to avoid "busy waiting."

You can implement virtual devices and pseudo-directories in any number of ways: /proc is a well-known example of the technique, but "that name-space is already taken." Still, the source code that implements these various useful gadgets within the Linux kernel is very readily-available, as are prototype dummies.
 
Old 01-13-2009, 01:58 PM   #9
AustinMarton
Member
 
Registered: May 2007
Location: New Zealand
Distribution: Fedora at home, Ubuntu 10.04 at work
Posts: 88

Original Poster
Rep: Reputation: 16
Re jiml8:
Many thanks for the code segment, that clarified things for me a lot! Two points...

Quote:
And the userspace process, when it starts up, sends its pid to the kernel driver via a command entered by the entry in /proc. In the code I have provided here, the kernel driver stores that pid in the variable datacollector_pid.
I now understand that while the app is running, it will have a PID and and I can access it's folder in the /proc directory, but what happens when the app finishes execution? Does it get assigned a new PID next time? Or will calling the old PID re-start the process?

Quote:
The userspace process also has a signal handler in place that captures SIGUSER1 when it is sent by the driver.
Can you explain a wee bit more how the user space app would capture the signal? Does it call an ioctl on the driver? Does it need to be sitting in a loop constantly checking for a signal? I don't get what code would make a process "sleep" or "wake".

Re sundialsvcs:
Quote:
A signal is simply "hey you! wake up and go see if you've got any data!" The userland process might discover that it actually does, or it might discover that there's nothing to read. The signal will simply make it possible for the process to avoid "busy waiting."
Exactly what I want to happen

Quote:
You can implement virtual devices and pseudo-directories in any number of ways: /proc is a well-known example of the technique, but "that name-space is already taken."
Are you saying that using /proc to communicate the signal between the kernel-user as described above is not going to work? To me it would seem silly to reinvent something of a different name when I think it sounds like it will be able to do all I need?
 
Old 01-13-2009, 06:26 PM   #10
jiml8
Senior Member
 
Registered: Sep 2003
Posts: 3,171

Rep: Reputation: 115Reputation: 115
Quote:
Originally Posted by AustinMarton View Post
Re jiml8:
Many thanks for the code segment, that clarified things for me a lot! Two points...


I now understand that while the app is running, it will have a PID and and I can access it's folder in the /proc directory, but what happens when the app finishes execution? Does it get assigned a new PID next time? Or will calling the old PID re-start the process?
Every time it runs, it gets a different pid assigned by the kernel.


Quote:
Can you explain a wee bit more how the user space app would capture the signal? Does it call an ioctl on the driver? Does it need to be sitting in a loop constantly checking for a signal? I don't get what code would make a process "sleep" or "wake".
Google on linux signals or linux signal handling or some such. Here is a nice reference:
http://www.cs.utah.edu/dept/old/texi...oc.html#SEC373
\
 
Old 01-14-2009, 07:44 AM   #11
sundialsvcs
Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 5,632

Rep: Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260Reputation: 1260
Your process needs to "open something," such as a virtual-device that is created by your kernel-module. You can then issue read() and ioctl() requests against it. And, "the kernel now knows who you are."

Study, and then implement, the "asynchronous I/O" protocols used already in Linux, so that your device implements the same sort of "well-behaved" methods that let you start a "non-blocking read()," as you can do with a "socket." You don't want to blaze new territory when what you're wanting to do resembles a well-understood way of doing things that already exists.

Once again, "use the Source, Luke!" You don't have to "puzzle out" how to do this stuff: you only have to "find" it. (And then, "cabbage it.") ("Even if it's 'road kill,' if it tastes good, eat it." Urgh... bad analogy. Really bad... )
 
Old 01-14-2009, 04:36 PM   #12
AustinMarton
Member
 
Registered: May 2007
Location: New Zealand
Distribution: Fedora at home, Ubuntu 10.04 at work
Posts: 88

Original Poster
Rep: Reputation: 16
Thanks for your replies. That last reference jiml8 provided was very good.

I have got a working (although I assume not optimum) solution now.
- My userspace app runs in the background (as a daemon?) sleeping, on startup it gave its PID (using an ioctl call) to the device driver who stores it for later.
- When a control frame hardware interrupt is received, the driver does the hardware reception, and copys the frame to a buffer.
- The driver then sends a signal (SIGUSR1) to the PID of the userspace app.
- This wakes up the userspace app, and calls it's handler, which reads from the /proc entry created by the device driver
- When read /proc entry returns the last control frame received to the userspace app.
- The userspace app then completes its processing on the frame.

I am not sure if this is exactly what either of you were suggesting. If you can see any fundamental flaws or issues that could arise from this method, or if I am not actually doing what I think I'm doing, please do let me know?

The code...

In my userspace app:
Code:
void synch_signal(int sig)
{
	int fd_proc, res, size;
	char *data;

	fd_proc = open("/proc/xx", O_RDWR);
	if (fd_proc < 0) {
		perror("xx_open:");
	}

	printf("This is the userspace signal handler\n");

	size = 8;
	res = read(fd_proc, data, size);

	printf("data: %s", data);

/* OTHER PROCESSING */

	interrupt_recvd = 1; /* global variable */

	return;
}

int get_sig(void)
{
	sigset_t mask, oldmask;
	struct sigaction usr_action;
	sigset_t block_mask;

	uint8_t data[16];
	int result;
	int fd = xx_open(CFG_IF);

	printf("PID: %d\n", getpid());
	ioctl(fd, XX_SEND_PID, getpid());

	/* Establish the signal handler.  */
	sigfillset (&block_mask);
	usr_action.sa_handler = synch_signal;
	usr_action.sa_mask = block_mask;
	usr_action.sa_flags = 0;
	sigaction (SIGUSR1, &usr_action, NULL);

	/* Set up the mask of signals to temporarily block. */ 
	sigemptyset(&mask); 
	sigaddset(&mask, SIGUSR1);

	/* Wait for a signal to arrive. */
	sigprocmask(SIG_BLOCK, &mask, &oldmask);
	printf("Waiting for signal to arrive\n");
	while (!interrupt_recvd) {
		sigsuspend(&oldmask);
	}
	sigprocmask(SIG_UNBLOCK, &mask, NULL);

	/* Now continue execution.  */
	printf("Finished!\n");

	return 0;
}
In my driver:
In the init
Code:
	struct proc_dir_entry *proc_file;
...
	/* initialize proc entry */
	proc_file = create_proc_entry("xx",0644, NULL);
	if(!proc_file) 
		goto fail;
	proc_file->read_proc = read_procmem;
	proc_file->write_proc = write_procmem;
At the end of the interrupt handler
Code:
	cframe_waiting = 1; /* global variable */

	if(datacollector_pid)	{
		kill_proc_info(SIGUSR1, info, datacollector_pid);
		printk("Sent signal to userspace app\n");
	} else {
		printk("<1>our userspace app not running\n");
	}
If someone reads the /proc entry
Code:
int read_procmem(char *buf, char **start, off_t offset, int count, int *eof, void *data)
{

	struct xx_cframe *cframe = &xx_device->cframe;
	struct cframe_buffer *wp = cframe->wp;
	int i, len = 0;

	/* if there has been a cframe recvd and not read by userspace, print it now */
	if (cframe_waiting)	{
		for (i = 0; i<10; i++)	{
			len += sprintf(buf, "%x", wp->frame[i]);
		}
	}
	len += sprintf(buf, "\n");

	cframe_waiting = 0;

	len += sprintf(buf,"The current pagesize is %i\n", (int)PAGE_SIZE); 

	return len;
}
In ioctl switch statement
Code:
	case XX_SEND_PID:
		printk("XX_SEND_PID\n");
		xx_recv_pid(arg);
To recv the pid
Code:
void xx_recv_pid(unsigned long arg)
{
	datacollector_pid = arg;
	printk("datacollector_pid: %d\n", datacollector_pid);

	return;
}
The only real problem I am having is in the way I am executing my userspace app. I have just been calling it from sash like "xxtest &", but once it has completed execution I get the message "sh 29: Child 32 died". I think I need to use "fork()", but I can't find the include file that defines it in uClinux...
 
Old 01-14-2009, 08:48 PM   #13
jailbait
Guru
 
Registered: Feb 2003
Location: Blue Ridge Mountain
Distribution: Debian Wheezy, Debian Jessie
Posts: 7,636

Rep: Reputation: 219Reputation: 219Reputation: 219
Quote:
Originally Posted by AustinMarton View Post

I have got a working (although I assume not optimum) solution now.
- My userspace app runs in the background (as a daemon?) sleeping, on startup it gave its PID (using an ioctl call) to the device driver who stores it for later.
- When a control frame hardware interrupt is received, the driver does the hardware reception, and copys the frame to a buffer.
- The driver then sends a signal (SIGUSR1) to the PID of the userspace app.
- This wakes up the userspace app, and calls it's handler, which reads from the /proc entry created by the device driver
- When read /proc entry returns the last control frame received to the userspace app.
- The userspace app then completes its processing on the frame.

I am not sure if this is exactly what either of you were suggesting.
I would have the application get the frame from the buffer and leave out all of the /proc processing by the application. Perhaps to do that the application would pass the buffer address to the driver as well as the PID.

------------------
Steve Stites
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Handling hardware interrupts zvivered Linux - Embedded & Single-board computer 2 12-11-2008 11:34 AM
Interrupts and precedence in an Ethernet driver TX/RX? Tasklets? 2 drivers 1 device? AustinMarton Linux - Kernel 2 12-09-2008 04:11 PM
getting interrupts form device driver bohemistanbul Linux - Hardware 1 07-30-2007 10:02 PM
Handling of Interrupts in thread context asurya Linux - Newbie 1 04-05-2006 09:15 AM
Help with converting a userspace PCI program to a Device Driver cstrask Linux - Hardware 0 10-31-2005 09:07 PM


All times are GMT -5. The time now is 10:34 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration