This is a study question

I'm interested in getting into building small applications that run in realtime on a PC or laptop. This is at the moment about realtime audio processing.
To give an example: Using a microphone input, grab sample by sample from an audio card, modify the signal, for example changing certain spectral characteristics, and send out a modified signal. The point of realtime processing hereby is to keep the delay between input and output as short as possible and most importantly constant.

I know enough about signal processing in general but have a hard time with getting into the programming aspect of it, namely the technical details of building kernel modules. In particular:

What free kernel should I use for real time. Is there one that's similar to the ones used for Fedora (5 or 6)?

How can I have a system that can support realtime processing but at the same time
behave pretty much like a normal PC, that is, windows system, internet, etc.

What's available that has been proven useful to you - tutorials, examples, books?

thanks for good pointers

You may start from standard kernel (newest possible) and configure it. Read help when choosing options, and it should be fine. But before anything else I'd check your time requirements. The delay you have, the processing power you need (and the time it takes) and the delay the system may cause. If it's not very critical, it may be fine to just deal with priorities. It's hard to say.

I'm actually mainly interested in what's called by some 'hard real time', where the an external interrupt (clock on AD converter) or an internal clock creates an interrupt event every, say 1/44000 seconds and an interrupt service routine promptly executes a small piece of code, during which it grabs some input, does some processing (updating filters, or whatever) and sends out some output data, for example, two samples of audio signal for a stereo output. Another application is a time critical control system, for example, controlling a robot arm, running every 1 ms, reading the sensors and generating motor control signals. Even though I understand the theory of this stuff to some extent, I usually felt at loss trying to understand kernel development in a hands-on way. That's why I'm looking for pointers to start from a fresh perspective.

Well, the first thing is to decide if you really need a hard real-time system. It has quite big consequences. Soft real-time can be done with standard kernel, with only minimal patching and configuration. For hard real-time you need complicated patches, like RTLinux (you may start reading from: http://www.linuxdevices.com/articles/AT3694406595.html).

Sound processing is not always hard real-time, as you need to get the data (typicaly a frame is 20ms), then process it. It means 50 frames/second, what can be done quite easily. If you can (need to) deal with 1/44000 second of data it's more complicated. Arm robot is another subject, usually a hard real-time, as it needs to be controlled carefully.

More links you can start from:
http://tldp.org/HOWTO/RTLinux-HOWTO.html
https://www.rtai.org/index.php?&MMN_position=1:1
http://linuxdevices.com/articles/AT8073314981.html

i'm not sure if you are building a system or just toying with kernel and app develpement.
if you really want a hard realtime process that process has to actually be the kernel itself and the linux kernel has to be run by your non preepmtable realtime core process at a lower priority in what we would call user space. thats what i would try anyway.
if you are just setting up a music system i can say what i do to the kernel and the rest of my configuration for audio processing but as others have pointed out i don't think this has anything to do with hard realtime. hardrealtime i think is not even be possible on the linux kernel.
i might be wrong on that one.
this works as a desktop machine just fine by the way.
first is the kernel configuration for soft realtime
apply this patch
http://people.redhat.com/mingo/realtime-preempt/
then in the config choose
Default I/O scheduler (CFQ)
and "Complete Preemption" as the preemption model
i use 1000 nanoseconds on the high res timer and i use a tickless system.
(tickless most likely doesn't effect realtime preemption) can't really remember why i did that.
make sure you choose Extended RTC operation
and of course Use RTC as default sequencer timer
make sure you have your hard disk set up in the fastest possible way.
OK you are also going to want PAM authentication that allows realtime access for normal users but for now you can do realtime as root till you figure that out.
next there is a utility "chrt"
http://www.die.net/doc/linux/man/man1/chrt.1.html
my soundcard is on IRQ 11 so i set at boot
chrt -f -p 82 `pidof "IRQ 11"`
for the life of me i can't remember the exact reasoning behind this but i had it well reasoned out when i did it ;-}
next adjust the reatime priorities of the RTC mine is on IRQ 8
chrt -f -p 98 `pidof "IRQ 8"`
next i set the latency on the soundcard a little higher than it was.
higher latency means larger bursts acrocc the bus
setpci -v -s '04:08.0' latency_timer=80
my soundcard is 04:08.0
this latency value you will have to experiment with to find the best settings for your machine.
ok and last but not least is to turn up the niceness of xorg
performance of xorg really sucks but it cant be so nice you don't have a fluid mouse curser so it's kind of something you have to play with as well. i use something like
renice +2 -p `ps -C Xorg -o pid=`
lastly you have to adjust your alsa and audio server buffers or for whatever app you are using and make them as small as possible for the lowest latency possible.
if you have a very fast soundcrd like m-audio or something you can go with very small buffers
without getting overruns.
if you have a slow audio card you will need to use larger ones but make them as small as possible.

Thanks, studioj and Mara, for these contributions. I appreciate your pointers. Looks like, I need to inform myself a bit more about sound cards and the drivers for them - I suppose the concept of one interrupt per sample (e.g., at 44000 Hz) isn't going to work with usual soundcards anyways. In the project (which is only in a state of brain storming so far) I'm interested in making the delay equal to the algorithmic delay, that is, the time necessary to have a causal system. For example, if I would do filtering with a low order filter that has an algorithmic delay of N samples, then I want the real delay not be a single sample longer than N/fs. Or if I had, for example, every 128 samples an overlap FFT of 512 samples in the algorithm, I would want the delay to be no more than those 128 samples. The purpose of this is to modify an audio input (speech or singing into a microphone) and feed back the modified microphone signal to the ears of the subject with minimal delay, in order to study the role of auditory feedback in speech or singing. It may be problematic with usual soundcards to operate them at very short buffer lengths, so that's what I have to look into.
Of course, the above discussion of hard versus not so hard is valid. I understand that a system that requires interrupt service every sample, like in a robot controller, is of a different nature than a filter system that may require an interrupt service routine only every 128 samples to process one fill of a ring buffer. A question is, where is the boundary between a "hard" realtime problem and a "soft" realtime problem. For example, if the sampling rate is 44kHz, and I have an algorithmic delay of only 16 samples, is an interrupt every 16 samples a "hard" realtime processing or a "soft" one?

It depends. Hard and soft real-time is not about the number of interrupts per second, but about the delay of reaction. For example, you have a robot. The robot may receive an interrupt from a sensor. The sensor interrupt means that there's something on the robot's way and it has to stop immediately. It doesn't matter if the interrupt comes (on average) once per minute or once per year - it's always hard real-time.

Sound processing is considered soft real-time, because it won't hurt anybody (except for some very special cases) if there's one frame delayed. It may break comfort (so it should happen as rarely as possible), but it's not critical.

there is no reason why you can't do this easily with Linux without alot of thought.
Linux has very complicated audio recording / multi-tracking programs that play in functional audible realtime.
i can play multiple pre recorded audio tracks and play on top of that either midi or audio from a mic input or keyboard or both without any perceptible delay at all including adding effects processing to either or both and manipulating the controlls for the effects. In my headphone its all realtime without any delays. "like it's real" real-time.
you cn also mix many different audio tracks together in realtime using effects and other signal processing all the while recording the output without any dropping or delay in the application of the GUI mixer program controlls pus manipulating controlls on the recording program. There is no sound latency at all ever even with many programs 4 or 5 hooked together.

realtime kernel is more about getting the kernel code to always yeild to the realtime interupts rather than have the interupts go into the soft IRQ processing for later. This includes the GUI controlls for what you are doing. So for instance i click a mouse click on a realtime recording program for a punch in.
It doesn't matter what the machine is doing at the time. i'm watching a movie and downoading a copy of another movie and snyping ebay and listening to music and playing a video game all at the same time. when i click that mouse click on the recording punch in recording begins imediately and proceeds uninterupted period.

thus we have full real-tme in the sense that there is ful logical corectness in audio playback and input just as if actual real speakers and musicians were "real" in the room.

hard realtime i think of as a simple very low level direct and unbreakable relationship between hard-ware. this can never be present in my mind in a complex operating system.

on the subject of interupts -- too many interupts and your system will be dead in the water handeling too many interupts so it's all about ballance and tuning and kernel code that is available and listening and not lost in a loop somewhere..

you can also start working with real time application interface(RTAI) which is point to start in the real time environments
you can downloads the source from
www.rtai.org

i thought of ths today in relation to this thread.

i couldn't figure out how t talk about interupts as you had said using very fast music vibrations type rtc settings was your approach.

the idea actually is since the computer can process music much faster that it happens in real time what you need to do to have apps seem in realtime in relation to everything else is optimize your use of the machines pipes.. that way you are more likely to get the results you want instead of trying to cram music throught those pipes in some realtime way but is inefficient and wil cause a pipe jam..