nightmare of big kernel lock

heroma · 06-16-2008, 08:22 AM

Quote:

Originally Posted by Randux

To answer your question: so who guarantees the precision (I think you mean what guarantees the serialization) of data? The answer is, good programming practice and a correct design. Everyone who accesses a resource that needs to be serialized has to do it the same way. One of the conditions required in this situation is that you must not sleep until you have committed any changes to data. Sleeping here is equivalent to releasing the lock.

You say with “good programming practice and a correct design” we can guarantee the datas inside critical section. OK,Thanks. But I have a question about security of these datas and scheduling of scheduler.In sched.c scheduler invokes two functions release_kernel_lock() and reaquire_kernel_lock(). If scheduler retrieve BKL with release_kernel_lock(), BKL is free now and another process can catchs it or can manipulates the datas that previous process was doing something on them. I can’t understand the manner of scheduler with BKL.how a good programming guarantees the security? Please help me in this problem.

heroma · 06-16-2008, 08:29 AM

Quote:

Originally Posted by jtshaw

Ok... so here we go.

When I say "process" I'm speaking strictly in terms of a user space process. When the process is executed it gets a heap and a stack in virtual memory with a task_struct in between (and the heap and stack grow towards each other). It cannot access another processes memory because the kernel does not allow the process to access memory outside of its virtual memory section. If its heap overruns its stack it will also overrun the task_struct (first) which virtually guarantees the program will crash before it can do anything nasty.

In the kernel all bets are off, because there is no line of defense to prohibit anyone from accessing anything (since the kernel IS the line of defense for user space).

So to start looking at the big kernel lock you should first consider it's history. Starting back in the 2.2 kernel, if I'm not mistaken, Linux started supporting SMP (Symmetric multiprocessing). Unfortunately there was a lot of kernel code that wasn't "thread-safe" meaning it wasn't safe to run both procedures simultaneously on different CPU's and insure data coherency. The big kernel lock was put in to protect those areas of code which were not "thread safe". Now during the development of the 2.4 kernel much of the system was completely redesigned to remove the dependance on the bkl and as a result it's use was decreased. In 2.6 they started working towards a preemptable kernel to, among other things, lower the latency on certain setups (like a Linux desktops or real-time linux systems for example). Part of making the system more preemptable was to make the bkl preemptable. I believe this happened around 2.6.8 but it could have been much sooner. This is likely what lead to the comments in the code your discussing... just a friendly reminder that if you've grabbed the lock you best not expect it to stay yours if you have to sleep for any reason, so code wisely.

Around 2.6.26rc2 somebody removed the preemtable part of the bkl again, which lead to another debate amongst the kernel folks. Linus seems to be most in favor of removing the bkl entirely going forward.

Really what it comes down to is you must be extra careful when using shared resources in the context of the kernel...

1. I know that 2.6.10’s BKL is non-preempted.
2. Do you mean that we haven’t any kernel mode process? but as I read the code of kernel, BKL gets to a process. in function lock_kernel() we have this: if lock_depth>=0 do something. Lock_depth is a field of task struct so we get the lock to a process. isn’t it?
3. In sched.c scheduler calls a function release_kernel_lock(prev). this function retrieves BKL from previous process. if scheduler retrieves BKL from previous process, other process that can enter the kernel can get the time from scheduler and manipulates the datas of kernel that previous process was doing something on them. we can say that a kernel mode process can threat the datas of other kernel mode process. can’t we?

Randux · 06-16-2008, 09:17 AM

Quote:

Originally Posted by heroma

You say with “good programming practice and a correct design” we can guarantee the datas inside critical section. OK,Thanks. But I have a question about security of these datas and scheduling of scheduler.In sched.c scheduler invokes two functions release_kernel_lock() and reaquire_kernel_lock(). If scheduler retrieve BKL with release_kernel_lock(), BKL is free now and another process can catchs it or can manipulates the datas that previous process was doing something on them. I can’t understand the manner of scheduler with BKL.how a good programming guarantees the security? Please help me in this problem.

You wrote "If scheduler retrieve BKL with release_kernel_lock(), BKL is free now and another process can catchs it or can manipulates the datas that previous process was doing something on them."

Did you mean "If scheduler *releases* BKL with release_kernel_lock(), BKL is free now and another process can catchs it or can manipulates the datas that previous process was doing something on them."

If that is what you meant to write, then my comments I made before (which you replied to now) are about as good of an explanation as I can provide.

The point is that good programming practice dictates that everybody agrees to update certain resources ONLY WHILE HOLDING THE BKL! If anybody will write code that updates those resources while NOT holding the BKL then we will have corruption, race conditions, etc. (Bad programming practice!)

You said "BKL is free now and another process can catchs it or can manipulates the datas that previous process was doing something on them"

That is correct, and that is exactly my point. Don't free the BKL until you have committed or backed-out your updates. If you do it correctly, anybody who follows the same rules can get the BKL and expect to be able to synchronously update the same resource. Only by this protocol (everybody agrees to use the BKL and agrees NOT TO UPDATE THIS DATA without it) can you guarantee consistent data.

I can't say any more than this, as I have no knowledge of the Linux kernel.

heroma · 06-18-2008, 10:18 AM

I need something like a book or ... about scheduling in 2.6 to understand BKL. Can anybody help me? (I describe this need in my previous post)

Randux · 06-18-2008, 10:25 AM

Some info here: http://www.kernel.org/pub/linux/docs/lkml/
and here http://lwn.net/Kernel/Index/
and possibly here http://rites.uic.edu/~solworth/linux-2.6.pdf

heroma · 06-18-2008, 01:22 PM

Thanks a lot.

heroma · 06-19-2008, 12:43 AM

I am here again! This is the code of sched.c that can confuse me! I can’t understand when previous process has the BKL how scheduler treat it?

These are in asmlinkage void __sched schedule(void)
//////////////////////////////////////
need_resched:
preempt_disable();
prev = current;
release_kernel_lock(prev);

need_resched_nonpreemptible:
rq = this_rq();
.
.
.
//////////////////////////////////////
if (likely(prev != next))
{
next->timestamp = now;
rq->nr_switches++;
rq->curr = next;
++*switch_count;
prepare_arch_switch(rq, next);
prev = context_switch(rq, prev, next);
barrier();
finish_task_switch(prev);
} else
spin_unlock_irq(&rq->lock);
prev = current;
if (unlikely(reacquire_kernel_lock(prev) < 0))
goto need_resched_nonpreemptible;
preempt_enable_no_resched();
if (unlikely(test_thread_flag(TIF_NEED_RESCHED)))
goto need_resched;
////////////////////////////////////////
I am very thankful.

heroma · 07-02-2008, 12:57 AM

Hi everybody!
1. As all you know, lock_depth is a variable that is defined in task_struct and shows the number of lock_kernel() that is invoken by a process and also each process has a personal task_struct . can I change the value of lock_depth of my user space process and grab the big kernel lock? is it possible in programming?

2. can anybody show me an example of sleeping process inside critical section of lock_kernel() and unlock_kernel()?
lock_kernel();
critical_section
unlock_kernel();

I ask my second question in post #6
thanks