LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 07-18-2009, 09:37 AM   #1
pupillo
LQ Newbie
 
Registered: Jul 2009
Posts: 22

Rep: Reputation: 15
Question threads scheduling problem


Hi, I have a very simple C written program with 2 threads, each one just continuosly writing to ISA bus. I see the CPU time well shared between them (about some microseconds each slice). Whenever I add a 3rd thread (writing to ISA bus as well), I observe time holes up to 40milliSeconds where a thread (not one in particular) can't use the CPU. Time used by system tasks is very little and rare, it's not the problem.
I got it better by setting priority to 1 and SCHED_RR but the system hangs up....
I'm using ptherad library
What could I try?
Thanks
 
Old 07-18-2009, 01:43 PM   #2
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 453Reputation: 453Reputation: 453Reputation: 453Reputation: 453
Quote:
Originally Posted by pupillo View Post
Hi, I have a very simple C written program with 2 threads, each one just continuosly writing to ISA bus. I see the CPU time well shared between them (about some microseconds each slice). Whenever I add a 3rd thread (writing to ISA bus as well), I observe time holes up to 40milliSeconds where a thread (not one in particular) can't use the CPU. Time used by system tasks is very little and rare, it's not the problem.
I got it better by setting priority to 1 and SCHED_RR but the system hangs up....
I'm using ptherad library
What could I try?
Thanks
You write directly to HW from your program ?!
 
Old 07-19-2009, 03:53 AM   #3
pupillo
LQ Newbie
 
Registered: Jul 2009
Posts: 22

Original Poster
Rep: Reputation: 15
Yes, after issuing ioperm it can be done simply by outw(...)
 
Old 07-19-2009, 10:38 AM   #4
orgcandman
Member
 
Registered: May 2002
Location: dracut MA
Distribution: Ubuntu; PNE-LE; LFS (no book)
Posts: 594

Rep: Reputation: 102Reputation: 102
How many cores do you have? Especially when you say that going to sched_rr hung the system, I would suspect that you're trying to run 3 spinning threads on a system with fewer than 3 or 4 cores.
 
Old 07-19-2009, 01:42 PM   #5
pupillo
LQ Newbie
 
Registered: Jul 2009
Posts: 22

Original Poster
Rep: Reputation: 15
I have an Atom platform, parhaps one core I'm not sure but I don't pretend that the system doesn't hang up setting priority to 1, it's normal. My question is: is there any manner to have a simple round robin scheduling without setting the priority greater than the one of system tasks?
 
Old 07-19-2009, 02:33 PM   #6
simulacrum111
LQ Newbie
 
Registered: Jul 2009
Posts: 3

Rep: Reputation: 0
It doesn't make sense to me just yet.

What is running during these 40ms gaps when a thread can't get on the CPU? The scheduling order can't be disobeyed so it would have to be a system thread/process or thread from another application outside of your own. The CPU wouldn't be doing nothing during that time period so what is running?

Are you sure certain threads aren't sniffing the CPU at all during these gaps? It's not possible that they were simply pre-empted and placed at the back of the queue? Any thread/process with higher priority than 1 regardless of how long it was on the CPU (i.e. very fast but consistently executed) could cause this behaviour to repeatedly occur.
 
Old 07-19-2009, 07:44 PM   #7
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 453Reputation: 453Reputation: 453Reputation: 453Reputation: 453
Quote:
Originally Posted by pupillo View Post
Yes, after issuing ioperm it can be done simply by outw(...)
And do you use mutual exclusion if you are writing from different threads ?

And can you be 100% sure that your device access can't cause system freeze ?
 
Old 07-20-2009, 03:03 AM   #8
pupillo
LQ Newbie
 
Registered: Jul 2009
Posts: 22

Original Poster
Rep: Reputation: 15
The CPU is always assigned to any of those threads ( except when system tasks are doing theirs stuffs but it's very very rare). The problem is that I can often see up to 40milliSecs with the CPU assigned to the SAME thread. Whereas with only two threads it doesn't happen. In this case at most after 100 MICROseconds there is a context switch.

I don't need mutex as outw is atomic itself.
 
Old 07-20-2009, 05:37 PM   #9
orgcandman
Member
 
Registered: May 2002
Location: dracut MA
Distribution: Ubuntu; PNE-LE; LFS (no book)
Posts: 594

Rep: Reputation: 102Reputation: 102
Quote:
I got it better by setting priority to 1 and SCHED_RR but the system hangs up....
Quote:
have an Atom platform, parhaps one core I'm not sure but I don't pretend that the system doesn't hang up setting priority to 1, it's normal. My question is: is there any manner to have a simple round robin scheduling without setting the priority greater than the one of system tasks?
Just do a simple cat /proc/cpuinfo and see how many cores you have.

now for a little RT thread lesson =D

When you set a thread to execute at RT priority (basically, you call sched_setscheduler() and use a SCHED_RR or SCHED_FIFO as the scheduler type), you throw all the 'fairness' rules out the window. If the RT thread doesn't sleep, but endlessly spins, you will tie up whichever core it is running on.

As an example, here's some code:

This should go in task.h
Code:
/**
 * simple network bitrate test
 * Copyright (C) 2009, Aaron Conole <apconole@yahoo.com>
 * All Rights Reserved.
 */
#ifndef __TASK_H__
#define __TASK_H__

#include <stdint.h>
#include <sys/types.h>
#include <unistd.h>

typedef uint32_t (*entry)();

enum Priority
{
    Critical = 0,
    Critical1,
    Critical2,
    Critical3,
    Critical4,
    Critical5,
    High,
    High1,
    High2,
    High3,
    High4,
    High5,
    AboveNormal,
    AboveNormal1,
    AboveNormal2,
    AboveNormal3,
    AboveNormal4,
    AboveNormal5,
    Normal,
    Normal1,
    Normal2,
    Normal3,
    Normal4,
    Normal5,
    BelowNormal,
    BelowNormal1,
    BelowNormal2,
    BelowNormal3,
    BelowNormal4,
    BelowNormal5,
    BelowNormal6,
    AboveIdle,
    Idle
};

struct _tsk_desc
{
    /*parent, process, and "thread" or "task" id.*/
    pid_t ppid;
    pid_t pid;
    uint32_t tid;

    /*priority and stack information*/
    enum Priority prio;
    char *name;
    char state;
    int32_t cpuid;

    entry run;
    
    void *attributes;
};

typedef struct _tsk_desc task;

uint32_t create_task(const char *name, enum Priority pri, entry EntryPoint,
                     int32_t cpu, task *task_ptr);

#define TASK_RESULT_OK    0
#define TASK_RESULT_FAIL  1

#define TASK_STATE_STARTED 1
#define TASK_STATE_ENDED   2
#define TASK_STATE_RUNNING 3

#endif
this should go in task.c
Code:
/**
 * simple network bitrate test
 * Copyright (C) 2009, Aaron Conole <apconole@yahoo.com>
 * All Rights Reserved.
 */
#include "task.h"

/* the following is for certain plaforms not correctly using CPU_SET macros*/
#ifndef _GNU_SOURCE
#define _GNU_SOURCE
#endif

#ifndef __USE_GNU
#define __USE_GNU
#endif

#include <stdio.h>
#include <pthread.h>
#include <errno.h>
#include <string.h>
#include <malloc.h>
#include <unistd.h>
#include <sys/types.h>
#include <linux/unistd.h>
#include <sched.h>
#include <signal.h>

/*maps the enum to rt- priority */
static uint32_t linux_prio_mapping[] =
{77,76,75,74,73,72,
 71,70,69,68,67,66,
 55,54,53,52,51,50,
 35,34,33,32,31,30,
 25,24,23,22,21,20,
 15, 10, 1};

/*think of it as an internal task control block*/
struct linux_pthread_attributes
{
    pthread_t      threadId;
    pthread_attr_t threadAttributes;
    void *         taskInfo;
};

void * default_run(void *);

/*note: task.c is tied to pthreads. However, you cal always write a new
  task.c based on a different library */

uint32_t create_task(const char *name, enum Priority prio, entry EntryPoint,
                     int32_t cpu, task *task_ptr)
{
    int ret;
    struct sched_param sparam;
    struct linux_pthread_attributes *pTAttributes = 
        (struct linux_pthread_attributes *)malloc
        (sizeof(struct linux_pthread_attributes));

    if((task_ptr == NULL) || (name == NULL) || (EntryPoint == NULL))
    {
        fprintf
            (stderr,
             "<task:error> Creating task failure - t[%p] n[%p] e[%p] a[%p]\n",
             task_ptr, name, EntryPoint, pTAttributes);
        return TASK_RESULT_FAIL;
    }

    task_ptr->name = (char*) name;
    task_ptr->prio = prio;
    task_ptr->cpuid = cpu;
    task_ptr->run = EntryPoint;
    task_ptr->attributes = (void *)pTAttributes;
    pTAttributes->taskInfo = (void *)task_ptr;

    /* ok, now start a new thread */
    
    ret = pthread_attr_init(&(pTAttributes->threadAttributes));
    /* the below is to support realtime threads */
    ret = pthread_attr_setinheritsched(&(pTAttributes->threadAttributes),
                                       PTHREAD_EXPLICIT_SCHED);
    ret = pthread_attr_setdetachstate(&(pTAttributes->threadAttributes),
                                      PTHREAD_CREATE_JOINABLE);

    sparam.sched_priority = linux_prio_mapping[prio];
    if((getuid() == 0) || (geteuid() == 0))
    {
        /*if we're root, set our scheduling.*/
        ret = pthread_attr_setschedpolicy(&(pTAttributes->threadAttributes),
                                          SCHED_FIFO);
        ret = pthread_attr_setschedparam(&(pTAttributes->threadAttributes),
                                         &sparam);
    }

    task_ptr->state = TASK_STATE_STARTED;

    ret = pthread_create(&(pTAttributes->threadId),
                         &(pTAttributes->threadAttributes),
                         default_run,
                         task_ptr);
    
    switch(ret)
    {
    case 0:
        break;
    default:
        task_ptr->state = TASK_STATE_ENDED;
        printf("<task:error> pthread_create: %s\n", strerror(ret));
        return TASK_RESULT_FAIL;
        break;
    }

    return TASK_RESULT_OK;
}

/*this is where all tasks start.*/
void *default_run(void *arg)
{
    struct sched_param sparam;
    cpu_set_t cps;
    task *task_ptr = (task *)arg;

    if(arg != NULL)
    {
        printf("<task:debug> - starting: [%s]\n", task_ptr->name);
        task_ptr->pid  = getpid();
        task_ptr->ppid = getppid();
        task_ptr->tid  = syscall(__NR_gettid); /*yeah, this is not portable.*/

        if((geteuid() == 0) && (task_ptr->cpuid != -1))
        {
            CPU_ZERO(&cps);
            CPU_SET(task_ptr->cpuid, &cps);
            
            sched_setaffinity(task_ptr->tid, sizeof(cps), &cps);
        }

        if(geteuid() == 0)
        {
            sparam.sched_priority = linux_prio_mapping[task_ptr->prio];
            sched_setscheduler(task_ptr->tid, SCHED_FIFO, &sparam);
        }

        task_ptr->state = TASK_STATE_RUNNING;
        task_ptr->run(); /*call the entry point of the task.*/

        task_ptr->state = TASK_STATE_ENDED;
        pthread_exit(NULL);
    }else
    {
        /*I guess we could crash here, but it doesn't feel right.*/
        printf("<task:error> default_run called with invalid args\n");
    }
    
    return NULL;
}
and this should go in main.c
Code:
#include <stdio.h>
#include "task.h"

uint32_t a_task_entry()
{
    while(1); /*a cool test would be to put in a sleep and printf.*/
    return 0;
}


int main()
{
    int i;
    task aTasks[4];
    create_task("Task1", Critical1, a_task_entry, -1, &aTasks[0]);
    create_task("Task2", Critical1, a_task_entry, -1, &aTasks[1]);
    create_task("Task3", Critical1, a_task_entry, -1, &aTasks[2]);
    create_task("Task4", Critical1, a_task_entry, -1, &aTasks[3]);

    while(1)
    {
        sleep(1);
        for(i = 0; i < 4; ++i)
        {
            if(aTasks[i].state != TASK_STATE_RUNNING)
               printf("error - task [%s] not running!\n",
                      aTasks[i].name);
        }
    }
    return 0;
}
Compile like:
gcc -I. -o test main.c task.c -lpthread

and run. Add more tasks to fill up the CPUs you have, if 4 doesn't do
it. As long as you're running as root on a 2.6 kernel, your system
should halt.

This is just to show you why you would get a hang using RT threads.

For a system where you need a bounded worst-case latency, you should be using a realtime kernel. Even then, on newer x86 machines, you can't get true realtime bounded behavior (see the linux RT wiki for reasons why this is). You can find that at: http://rt.wiki.kernel.org/index.php/Main_Page

Hopefully this helps,
Aaron
 
Old 07-20-2009, 07:45 PM   #10
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 453Reputation: 453Reputation: 453Reputation: 453Reputation: 453
Quote:
Originally Posted by pupillo View Post
The CPU is always assigned to any of those threads ( except when system tasks are doing theirs stuffs but it's very very rare). The problem is that I can often see up to 40milliSecs with the CPU assigned to the SAME thread. Whereas with only two threads it doesn't happen. In this case at most after 100 MICROseconds there is a context switch.

I don't need mutex as outw is atomic itself.
??????????????????

Suppose one thread asserts, say, start bit. The bit should remain untouched until operation end.

Meanwhile another thread also asserts start bit - while your HW device operation is still in progress.
 
Old 07-21-2009, 04:19 AM   #11
pupillo
LQ Newbie
 
Registered: Jul 2009
Posts: 22

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by Sergei Steshenko View Post
??????????????????

Suppose one thread asserts, say, start bit. The bit should remain untouched until operation end.

Meanwhile another thread also asserts start bit - while your HW device operation is still in progress.

Outw is atomic for x86:
static inline void outw(u16 v, u16 port)
{
asm volatile("outw %0,%1" : : "a" (v), "dN" (port));
}
 
Old 07-21-2009, 04:21 AM   #12
pupillo
LQ Newbie
 
Registered: Jul 2009
Posts: 22

Original Poster
Rep: Reputation: 15
Aaron
I will try....
thanks
 
Old 07-21-2009, 04:33 PM   #13
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 453Reputation: 453Reputation: 453Reputation: 453Reputation: 453
Quote:
Originally Posted by pupillo View Post
Outw is atomic for x86:
static inline void outw(u16 v, u16 port)
{
asm volatile("outw %0,%1" : : "a" (v), "dN" (port));
}

The word "atomic" means the only one thread at a time can write to a HW port.

I am talking about a different scenario:
  1. thread 1 writes to port P; thread 2 doesn't even try to write to port P during thread 1 write;
  2. then thread 2 writes to port P; thread 1 doesn't even try to write to port P during thread 2 write.

So, it doesn't matter that write to port P operation is atomic - thread 2 shouldn't have written to port B at all - because start bit is already asserted.
 
Old 07-22-2009, 02:54 AM   #14
pupillo
LQ Newbie
 
Registered: Jul 2009
Posts: 22

Original Poster
Rep: Reputation: 15
Sorry, but I don't understand what is this start bit....
 
Old 07-22-2009, 01:16 PM   #15
Sergei Steshenko
Senior Member
 
Registered: May 2005
Posts: 4,481

Rep: Reputation: 453Reputation: 453Reputation: 453Reputation: 453Reputation: 453
Quote:
Originally Posted by pupillo View Post
Sorry, but I don't understand what is this start bit....

Generally speaking, if you have a HW device, it has control and status register(s).

Bits in control registers(s) define how the device operates (for example, clock rate, number of retransmissions, interrupts mask, etc.); bits in status register(s) upon operation completion indicate operation results (presence of errors, kind of errors, whether interrupts happened, etc.).

Often, but not necessarily, reading from status register resets bits in it.

The "start bit" is a generic (though sometimes exact) name of the bit which, being asserted, triggers the device operation, for example, DMA transfer of a block of data.

So, for a device to operate properly one should correctly set bits in control register(s) and to analyze results one should read status register(s).

If in the course of device operation the program asserts start bit again, the operation is typically screwed up.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Problem with scheduling shutdown edmondgyampoh Suse/Novell 1 05-18-2009 07:09 PM
process scheduling problem.. nick021 General 3 12-13-2007 03:41 PM
POSIX thread scheduling problem on Fedora Core 5 katlina Programming 3 11-16-2006 01:38 PM
scheduling problem using 'at' medmedia Linux - Software 7 12-05-2005 05:10 AM
linux scheduling problem lordofring Programming 2 08-30-2005 10:08 AM


All times are GMT -5. The time now is 11:01 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration