LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel
User Name
Password
Linux - Kernel This forum is for all discussion relating to the Linux kernel.

Notices

Reply
 
Search this Thread
Old 04-10-2009, 03:48 AM   #1
merijnv
LQ Newbie
 
Registered: May 2007
Location: The Netherlands
Distribution: ubuntu-desktop 8.10 and ubuntu-server 8.10
Posts: 21

Rep: Reputation: 15
Cool Single cpu-core used on multicore system when processes communicate with pipe


For scientific experiments I am a user on an interesting computer. It has an enormous amount of ram (128G), and 16 cores (4 quad-core amd-64 processors). Uname -a says:
Linux cn51 2.6.27.19-78.2.30.fc9.x86_64 #1 SMP Tue Feb 24 19:44:45 EST 2009 x86_64 x86_64 x86_64 GNU/Linux


If I start three jobs which are cpubound, yet these processes communicate through a pipe, I see only one CPU in use; and the user-percentages count up to exactly 100% (one core). I must admit that an enormous amount of data flows through these pipes: i process megabyte a second or so.

In top, using the 1 line per processor, I see that the processes are always on the same CPU but on different cores. The user-times add up to almost exactly 100%, there is no significant sys-time measures; idle+user is also close to 100% per core.

Is there a kernel-level parameter that can be tweaked to ask the scheduler for another way of scheduling these?

Or, maybe even better, something in the environment I can set as user?

Last edited by merijnv; 04-10-2009 at 04:09 AM. Reason: more research done
 
Old 04-10-2009, 06:11 AM   #2
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 12,230

Rep: Reputation: 1019Reputation: 1019Reputation: 1019Reputation: 1019Reputation: 1019Reputation: 1019Reputation: 1019Reputation: 1019
Quote:
Originally Posted by merijnv View Post
Is there a kernel-level parameter that can be tweaked to ask the scheduler for another way of scheduling these?

Or, maybe even better, something in the environment I can set as user?
Nope.
If you are passing data via a pipe, you have a (effectively) synchronous write/read arrangement.
Hence you will never see more than what appears to be one processor in use.
The scheduler will attempt to enforce (hardware) locality totake advantage of cache coherency. Hence you see dispatch on just the one substrate.

If you can, redesign the work for better multi-tasking.
 
Old 04-10-2009, 07:47 AM   #3
johnsfine
Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,076

Rep: Reputation: 1110Reputation: 1110Reputation: 1110Reputation: 1110Reputation: 1110Reputation: 1110Reputation: 1110Reputation: 1110Reputation: 1110
Quote:
Originally Posted by merijnv View Post
Is there a kernel-level parameter that can be tweaked to ask the scheduler for another way of scheduling these?

Or, maybe even better, something in the environment I can set as user?
Quote:
Originally Posted by syg00 View Post
Nope.
That "Nope" is certainly correct.

Quote:
If you are passing data via a pipe, you have a (effectively) synchronous write/read arrangement.
But that is significantly overstating the problem.

You should be able to pass data via a pipe and still have overlapping processing.

Something is wrong with either your overall design, or the details of the way you write to each pipe.

You cannot get data out of a pipe before the other side puts the data in. So assuming the processing on the read side of the pipe requires the data, it must wait for it.

But when you write to a pipe, waiting for the other side to read the result is optional (I wish I understood the exact details myself. Some of the documentation is unclear, especially since I'm trying to use mostly the same source code between Windows and Linux. But I don't expect your situation is that complex).

Your write side almost certainly should not be waiting for the other side to read. It should be able to generate the next data as soon as it throws the last data into the pipe.

Quote:
If you can, redesign the work for better multi-tasking.
Of course. The first step is understanding why two out of three processes are waiting at any given moment.

If you explain a bit more about the data flow between the three processes, we might have some better ideas on how to identify and correct the undesired waits.
 
Old 04-10-2009, 09:17 PM   #4
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 12,230

Rep: Reputation: 1019Reputation: 1019Reputation: 1019Reputation: 1019Reputation: 1019Reputation: 1019Reputation: 1019Reputation: 1019
Quote:
Originally Posted by johnsfine View Post
But that is significantly overstating the problem.
I was trying to describe what I perceived the issue to be, not pipes per se.
Maybe I should have said something like
"If you are passing data via a pipe, it appears you have a (effectively) synchronous write/read arrangement."
 
  


Reply

Tags
multi, pipe, smp


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
SMP system - use a single CPU during boot bromanescu Programming 4 10-18-2008 04:29 PM
Run all OS processes on one core in multicore processor tryon16 Linux - Kernel 6 10-02-2007 05:13 PM
how to kill processes connected with pipe pelikan81 Programming 3 06-15-2007 03:50 AM
pipe to many processes? Thinking Programming 4 09-20-2005 11:22 AM
pipe commands into processes acren Linux - Newbie 1 07-04-2001 10:08 AM


All times are GMT -5. The time now is 01:23 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration