Hi,

On shared memory multiprocessors or multicores, is it possible to run all OS processes one core and application threads on the rest of the cores?
I'd like to have one core handle OS and control flow type of work and the others serve as workers(just like what IBM Cell does)
When worker cores have exceptions, I would like that exception to be passed to OS-running core and have it to take care of it.

I guess I can bound specific application threads to dedicated processors by using processor affinity. But was not sure if I could filter all other "non-application processes" or os-processes and bound them to one core.

I am thinking of modifying scheduler. There I expect to see all the processes/threads(regardless of it being OS or application). Before those processes are put into runqueue, I am hoping to set processor affinity to all the processes.

Do you think it makes sense? I will highly appreciate all your comments.

Thanks

Something like that ?

I think there is something else with about the same goals out there but I don't remember where...

[ deleted ]
I realized my posting wasn't helpful.

No.
The scheduler already attempts to re-dispatch on the same engine if possible to alleviate issues such as cache flush. I see no advantage in what you are attempting to do - but feel free to dive into the code; you may need some time and patience.

CKRM has effectively been supplanted by "containers" which I don't believe has reached the mainline yet. Has been to -mm, but updates caused it to drop out again. It may do some of what you want. Likewise cpusets, although this will suffer the same shortcoming you noted above for processor binding at the application level. Isolating your userspace to particular CPUs is likely to cause everything else to dispatch on the "other" CPU(s), but you can't guarantee it.

He's looking to make a trade-off that gives up some CPU cycles to ensure that system processes aren't at all affected by any CPU-intense user process (encoding video, for example).

For an example, assume four CPUs. Under a normal OS with decent SMP support, when enough processes exist to eat 100% of CPU on all 4 CPUs, the system processes will be fighting with the userland processes for CPU time. This solution allows the use of all 400% (yes, I know it should be 100, but this better illustrates the situation for me) of CPU cycles to be used, so it is effecient in that it doesn't waste any CPU cycles.

The scenario tryon16 is looking for would allow the userland processes to eat up 300% of CPU cycles, while reserving the last CPU solely for system processes. In this way, the system processes aren't fighting for CPU cycles with the userland processes. This comes at the cost of not using all of your CPU cycles. Say that system processes only used 2% of a given processor's available cycles, then the max usage would be 302% compared to the 400% of the first scenario.

Obviously, the trade-off is guaranteeing CPU availability to system processes at the cost of inefficient CPU usage.

With all of that said, I would tend to agree that I don't think this would be useful. Even if you guarantee CPU availability to your system processes, as a user, you'll still be waiting on the userland processes. You won't be speeding things up, just slowing them down by offering less CPU for use. Furthermore, it takes a hell of a lot to get four cores to max out all CPU usage. Finally, when a userland process does something that requires system process (reading from disk, for example), it won't be using any CPU until it gets the information from the disk, thus kind of forcing the necessary system process(es) into the spotlight.

Just my two cents. I could be completely wrong.

As far as running like Cell does, that's a bit different than what I just described, but since there are a few Linux distros that run on PS3, it seems like it would be worthwhile to look at the architecture-specific code and clone that.

-TDS-

Thank you so much to all of you for sharing your thoughts with me.

I think I should have told you the reason why I am doing this.

I am working on "Error resilient" architecture. It is multicore and only one core is "over-designed" to guarantee to be reliable. The rest of the cores-the worker cores- are normal or "less-designed" such that reliability is not guranteed. Why am I doing this? Because it is a cheaper way to fabricate a chip(than to make all cores reliable) and it would be ok for certain area of applications that are "robust" in themselves.

Since worker cores are unreliable, I don't want to run OS-processes on them. It would be much more difficult to recover OS failure than the application failure. I want OS processes to run on "reliable" core only. This is why I want to sequester OS processes from worker cores.

*****************************************************************************

Do you think using processor affinity would not work? As I mentioned in the original posting, if I set affinities to all the process/threads before they are put into runqueue, would I achieve what I want?

I heard some people saying no. Then could you tell me in more detail.

Thank you very very much.

Yes, maybe you should have.
My response was based on thinking you were planning on doing this on a "normal" multi-{core|cpu} machine. Not worth the effort for the average man in the street.
In your case, where you are probably already knee deep in kernel code, seems a reasonable option. Maybe setting "cpus_allowed" prior to dropping the task_struct on the runqueue might be one place to think about.