LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel
User Name
Password
Linux - Kernel This forum is for all discussion relating to the Linux kernel.

Notices


Reply
  Search this Thread
Old 11-03-2010, 10:00 PM   #1
cypkw
LQ Newbie
 
Registered: Nov 2010
Posts: 4

Rep: Reputation: 0
How to calculate the size of process cache footprint ?


Hi,everybody;

Recently i have learned something about process scheduling.I have some questions during my studying.Give me some advice on the following my questions if you know about it.
frist: process will take up some cache during executing.Each prcess may have different cache footprint.So could we have some methods to calculate the size of process cache footprint?

second: Each process have a life cycle. It will occupate cpu utilization when we bind a process to a CPU. Suppose it has only a process in the cpu, how to calculate cpu utilization during the whole process life cycle?

Hope that everybody could give me some methods on the above questions.
Best wishes for you.
 
Old 11-03-2010, 11:21 PM   #2
bsat
Member
 
Registered: Feb 2009
Posts: 347

Rep: Reputation: 72
i think the command "time" could help you in both the cases. please explore man pages for details.
 
Old 11-05-2010, 10:48 AM   #3
salasi
Senior Member
 
Registered: Jul 2007
Location: Directly above centre of the earth, UK
Distribution: SuSE, plus some hopping
Posts: 4,070

Rep: Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897
Quote:
Originally Posted by cypkw View Post
frist: process will take up some cache during executing.
And, if you run it several times you will probably find different amounts of cache used for each run, depending on what other demands there are on the cache memory...

Quote:
Each prcess may have different cache footprint.So could we have some methods to calculate the size of process cache footprint?
It follows from the previous, that there will be no exact calculation of the cache usage possible. It may be possible to approximate, but you may be better off measuring rather than calculating (and by measuring, I mean measuring and calculating eg, standard distribution, if that's the sort of thing that would be helpful to you).

Quote:
second: Each process have a life cycle. It will occupate cpu utilization when we bind a process to a CPU. Suppose it has only a process in the cpu, how to calculate cpu utilization during the whole process life cycle?
I only understand what you are asking to a limited extent;
  • I understand that each process has a life cycle; it is born, does stuff and dies
  • Your point that it will 'occupate' cpu utilisation is relevant how? Do you want the percentage of time that this process is active on this cpu or do you want the percentage of the cpu's cache it occupies? And if that, how do you want to handle the case of a multi-level cpu cache? Do you want total usage, or something else (if so, what?)?
  • And, you seem to assume that the only possibility is that a process is bound to a cpu. That is neither the only possibility nor is it the default case, for a device with more than one cpu.
  • And as, in the general case, the usage will fluctuate during the life cycle of the process, how do you want to handle that? Do you want the maximum cpu used (which will always be 100%, if the measurement timeframe is short enough) or are you trying to measure cpu time (as opposed to wall clock time) used?

I am afraid that I don't know how this helps you, and just maybe, the answer given by 'bsat' may be more helpful (but it wasn't really what you asked for). But then, you don't really explain what you would like to use the information for, when you get it.
 
Old 11-05-2010, 09:33 PM   #4
cypkw
LQ Newbie
 
Registered: Nov 2010
Posts: 4

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by salasi View Post
And, if you run it several times you will probably find different amounts of cache used for each run, depending on what other demands there are on the cache memory...



It follows from the previous, that there will be no exact calculation of the cache usage possible. It may be possible to approximate, but you may be better off measuring rather than calculating (and by measuring, I mean measuring and calculating eg, standard distribution, if that's the sort of thing that would be helpful to you).



I only understand what you are asking to a limited extent;
  • I understand that each process has a life cycle; it is born, does stuff and dies
  • Your point that it will 'occupate' cpu utilisation is relevant how? Do you want the percentage of time that this process is active on this cpu or do you want the percentage of the cpu's cache it occupies? And if that, how do you want to handle the case of a multi-level cpu cache? Do you want total usage, or something else (if so, what?)?
  • And, you seem to assume that the only possibility is that a process is bound to a cpu. That is neither the only possibility nor is it the default case, for a device with more than one cpu.
  • And as, in the general case, the usage will fluctuate during the life cycle of the process, how do you want to handle that? Do you want the maximum cpu used (which will always be 100%, if the measurement timeframe is short enough) or are you trying to measure cpu time (as opposed to wall clock time) used?

I am afraid that I don't know how this helps you, and just maybe, the answer given by 'bsat' may be more helpful (but it wasn't really what you asked for). But then, you don't really explain what you would like to use the information for, when you get it.

Thank you Salasi , bsat
Your reply are perfect.
I am sorry i don't exactly express what i really want to know.I will
describe my quesitions again.
In chip multi-core environment, suppose that there are the following system environment.
Two cores(core0, core1),independently L1\L2 cache,core1's load>core0's
load,and then core0 is a ilde core.It will trigger ilde load balance in linux.So how to select process to migrate to core0 from core1 to get better performance.I think you will obtain different performance if you migrate different type of processes.My thoughts are as follows:
1)Different process has different cache footprint.The process of larger cache footprint maybe pollute cache memory space,which reduce performance.
2)so when system happen ilde load balance,we should migrate the process of larger cache footprint to idle core to reduce cache pollution.
so my questions are as follows:
1)How should we measure process cache footprint? Are there good evaluation criterias?such as "time" or others?
2)Suppose that we migrate process in the light of their cache footprint when ilde core trigger load balance.The larger cache footprint process will be migrated to ilde core to execute.How to evaluate its superiority ? I agree with your point. Maybe process's executing time is a good evaluation criterias.

The above are my thoughs and questions.
I am looking forward for your reply.
Thank you for your reading.
 
Old 11-06-2010, 06:18 AM   #5
salasi
Senior Member
 
Registered: Jul 2007
Location: Directly above centre of the earth, UK
Distribution: SuSE, plus some hopping
Posts: 4,070

Rep: Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897Reputation: 897
Quote:
Originally Posted by cypkw View Post
I am sorry i don't exactly express what i really want to know.I will
describe my quesitions again.
1)How should we measure process cache footprint? Are there good evaluation criterias?such as "time" or others?
2)Suppose that we migrate process in the light of their cache footprint when ilde core trigger load balance.The larger cache footprint process will be migrated to ilde core to execute.How to evaluate its superiority ? I agree with your point. Maybe process's executing time is a good evaluation criterias.
My opinion, which you won't like, is that you are approaching this in a completely wrong way, and you haven't, at least in this thread, defined your aims satisfactorily in able to make progress.

I can't understand why you are so concerned with cache footprint. Do you think that a big cache footprint is a good thing or a bad thing? When you think about this question, bear in mind that using less cache than the amount that you have is to get less advantage out of the cache than you can; unused cache doesn't do any good. So, it doesn't seem to me that using cache size as a proxy for 'performance' for the total system will lead you to a good solution, whatever your problem is, however you measure cache footprint.

For an interesting view on this, you could consider the case of a disk access subsystem; if you 'improve' (however defined) the processor cache performance on this part of the system, doesn't that mean that the disk subsystem gets around to waiting for the disk more quickly? does this 'waiting more quickly' actually help system performance in any way?

Ok, when you say "we should migrate the process of larger cache footprint to idle core to reduce cache pollution" suggests that
  • You have a limited grip on the usage of the term cache pollution
  • You are continuing to try to use cache footprint, and now you use it to decide which CPU a process should run on. Given the other criteria which you could use (cpu usage, context switches run queue length, or almost anything else you could think of) it is unclear why you have selected this
  • Of course, if you do migrate processes, based on this, or other, criteria that is likely, in the short term, at least, to reduce the efficiency of usage of your cache and could lead to L/L2 cache pollution
.

You should also note that
  • In the 'default' case, moving processes from one core to another is handled by the kernel, when appropriate. What is it about your case that makes using the kernel's well-tested features the wrong thing to do? Why will some other approach be more suitable?
  • If you use the wrong metric(s), you will get a sub-optimal result. As far as I can see, you seem to be determined to use the wrong metric.
  • things like hit rate/miss rate seem to be far more useful as metrics, and, in the case that this is an embedded system execution time and/or latency for sub-systems. But, I have to repeat, there is more to optimising the system as a whole than just maximising (or minimising) the amount of cache that is used, so you need to explain why you are so fixed on that as an approach.
 
Old 11-11-2010, 02:16 AM   #6
cypkw
LQ Newbie
 
Registered: Nov 2010
Posts: 4

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by salasi View Post
My opinion, which you won't like, is that you are approaching this in a completely wrong way, and you haven't, at least in this thread, defined your aims satisfactorily in able to make progress.

I can't understand why you are so concerned with cache footprint. Do you think that a big cache footprint is a good thing or a bad thing? When you think about this question, bear in mind that using less cache than the amount that you have is to get less advantage out of the cache than you can; unused cache doesn't do any good. So, it doesn't seem to me that using cache size as a proxy for 'performance' for the total system will lead you to a good solution, whatever your problem is, however you measure cache footprint.

For an interesting view on this, you could consider the case of a disk access subsystem; if you 'improve' (however defined) the processor cache performance on this part of the system, doesn't that mean that the disk subsystem gets around to waiting for the disk more quickly? does this 'waiting more quickly' actually help system performance in any way?

Ok, when you say "we should migrate the process of larger cache footprint to idle core to reduce cache pollution" suggests that
  • You have a limited grip on the usage of the term cache pollution
  • You are continuing to try to use cache footprint, and now you use it to decide which CPU a process should run on. Given the other criteria which you could use (cpu usage, context switches run queue length, or almost anything else you could think of) it is unclear why you have selected this
  • Of course, if you do migrate processes, based on this, or other, criteria that is likely, in the short term, at least, to reduce the efficiency of usage of your cache and could lead to L/L2 cache pollution
.

You should also note that
  • In the 'default' case, moving processes from one core to another is handled by the kernel, when appropriate. What is it about your case that makes using the kernel's well-tested features the wrong thing to do? Why will some other approach be more suitable?
  • If you use the wrong metric(s), you will get a sub-optimal result. As far as I can see, you seem to be determined to use the wrong metric.
  • things like hit rate/miss rate seem to be far more useful as metrics, and, in the case that this is an embedded system execution time and/or latency for sub-systems. But, I have to repeat, there is more to optimising the system as a whole than just maximising (or minimising) the amount of cache that is used, so you need to explain why you are so fixed on that as an approach.
Thank you for your reply.
I am thinking for this issue that why i select cache footprint as metic of process migration when system happen idle load balance in these days.
Actually,I want to find another metric to migrate process.
Now,there have been some metrics to select process to migrate into idle core when idle load balance happen in recent Linux 2.6.30+,such as cache hot or cool,migration cost.
Generally, Linux scheduler don't migrate processes of busiest core that is cache hot or recently have executed into idle core when idle load balance happen.Suppose the followed scenario that there are two cores.One is busy core A, another is idle one,called B.Now B trigger idle load balance.According to linux scheduling function idle_load_balance(),B should migrate processes from busy core A to reach load balance. So how to select process to migrate idle core B?what metrics are used to migrate process in addtion to the above metrics(cache hot ,migration cost,cfs_rq) when idle load balance happen ?
the process's cache footprint could be used to migrate progress as a new metric?(If we suppose that there are some tools which measure cache footprint)
The above is my thinking.
Hope that you can give me some proposal.
Best wishes.
cypkw
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to calculate cache miss rate ? robertsong Linux - Software 0 03-24-2009 08:46 PM
Java Linux Process memory footprint with and without -Xmx rolandofghent Linux - General 5 09-27-2007 09:08 AM
how to calculate kernel structure size? cranium2004 Programming 1 04-15-2005 04:09 AM
how can i calculate an image size w/o creating one? rhxk Linux - Software 2 11-10-2004 04:52 PM
Help!!!: TCP code size/memory usage / footprint bichettelinux Linux - Networking 3 07-13-2003 05:31 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software > Linux - Kernel

All times are GMT -5. The time now is 01:59 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration