Question about threads scheduing behavior on kernel 2.6.26

hi,

Within my process, there are several threads (pthread). In certain scenarios, I want to make sure that only 1 thread of the process can be scheduled to run, while all the other threads of the same process should be blocked until this thread yeilds/blocks/finishes (it does not require to block other processes or threads belongs to other processes though).

I was thinking that i may be able to do this by changing the scheduling policy (to SCHED_RR, e.g.) and the priority (to maximum 99, e.g.) of the thread, but my test on 2.6.26 is not what i expected.

the following is the test code i wrote for the testing. basically in the process i created two threads, one is called "LOCKER" (which is supposed to be run without being preempted), the other is called "LOCKED" (which is supposed to be blocked whenever the LOCKER is running). in both thread, they update a global variable (adding a different DELTA to it) in a finite loop. If the LOCKER can not be preempted by the LOCKED thread (which is what i hoped), then in the loop of the LOCKER thread, it can check the increment of the global value is expected in each step. whenever it detects that the increament is not consistent, it means the global value has been updated by the LOCKED thread, which in turn means the LOCKED thread is still has chance to run while LOCKER thread is running--in this case, the process aborts.

the test result is that i got abort on my desktop:

[bruin@chn-bruin-xine boot]$ uname -a
Linux chn-bruin-xine 2.6.26.6-49.fc8 #1 SMP Fri Oct 17 15:59:36 EDT 2008 i686 i686 i386 GNU/Linux


i would like to know:
- is this expected?
- what should i do to fulfill my requirement? using what patches or libraries, or changing my program etc?


thank you very much,
/xiongyw



the test code -->


/*
* ============================================================================
* testing pthread scheduler behavior on Linux
*
*
* compile it with: "gcc sched.c -lpthread"
*
* created(bruin, 2009-03-30)
* ============================================================================
*/

#include <stdio.h>
#include <stdlib.h>
#include <assert.h>
#include <string.h>
#include <sys/time.h>
#include <limits.h>
#include <errno.h>
#include <pthread.h>



/*##############################################################
# definitions and globals
#############################################################*/

#define TEST_SCHED_POLICY SCHED_RR
#define LOCKER_PRIORITY 99
#define LOCKED_PRIORITY 1

#define LOCKER_DELTA 1
#define LOCKED_DELTA 100

/*
* every threads update this value (by adding a different DELTA to it) when they have chance.
*/
int volatile g_sum = 0;


const char g_locker[] = "LOCKER"; /* the thread has higher priority */
const char g_locked[] = "LOCKED"; /* the thread to be blocked */

/*##############################################################
# local function prototypes
#############################################################*/

int s_thread_create(void *(*entry)(void*), void* arg, int sched_policy, int prio);
void *s_locker_thread_entry(void* arg);
void *s_locked_thread_entry(void* arg);
void s_lengthy_work(void);


/*##############################################################
# main() function
#############################################################*/


int main(void)
{
struct sched_param param;

param.sched_priority = LOCKED_PRIORITY;
if(0 != sched_setscheduler(0, TEST_SCHED_POLICY, &param)){
fprintf(stderr, "sched_setscheduler() failed. you may need root privileges.\n");
abort();
}

printf("about to start the threads...\n");

s_thread_create(s_locker_thread_entry, (void*)g_locker, TEST_SCHED_POLICY, LOCKER_PRIORITY);
s_thread_create(s_locked_thread_entry, (void*)g_locked, TEST_SCHED_POLICY, LOCKED_PRIORITY);

sleep(20);
}


/*##############################################################
# local function implementations
#############################################################*/

/*
* create a thread with specified sched policy and priority
*/
int s_thread_create(void *(*entry)(void*), void* arg, int sched_policy, int prio)
{
pthread_t tid;
pthread_attr_t attr;
struct sched_param sched_param;
int rc;


/* init the thread attribute */
rc = pthread_attr_init(&attr);
assert(rc == 0);


/* sched inherit: explicit */
rc = pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED);
assert(rc == 0);

/* set sched policy */
rc = pthread_attr_setschedpolicy(&attr, sched_policy);
assert(rc == 0);

/* set priority */
sched_param.sched_priority = prio;
rc = pthread_attr_setschedparam(&attr, &sched_param);
assert(rc == 0);

/* create the thread */
rc = pthread_create(&tid, &attr, entry, (void *)arg);
if( rc != 0){
fprintf(stderr, "pthread_create()=%d failed.\n", rc);
if(rc == EPERM){
fprintf(stderr, "Tip: you may need the root privileges to run it!\n");
}
abort();
}
else{
/* printf("pthread_create() ok for '%s'\n", (const char*)arg); */
}

rc = pthread_attr_destroy(&attr);
assert(rc == 0);

return 0;
}


void s_lengthy_work(void)
{
/* do some work which takes time */
int i, j;
double d = 3.1415925;

for(i = 0; i < 2000; i ++)
for(j = 0; j < 1000; j ++)
d *= d;
}


void* s_locked_thread_entry(void* arg)
{
int i;

for(i = 0; i < 10; i ++){
s_lengthy_work();
g_sum += LOCKED_DELTA;
}

return arg;
}


void* s_locker_thread_entry(void* arg)
{
int i;
int pre;

pre = g_sum;

for(i = 0; i < 10; i ++){
g_sum += LOCKER_DELTA;
if((g_sum - pre) != (i + 1)){
abort();
}
s_lengthy_work();
}

return arg;
}

Today i disabled the SMP by building a UP kernel. by using the UP kernel, the process never aborts now. so it seems the test code only works on UP, not on SMP.

your further inputs or comments are still welcomed!

thanks,
/xiongyw