How to calculate the size of process cache footprint ?
Linux - KernelThis forum is for all discussion relating to the Linux kernel.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
How to calculate the size of process cache footprint ?
Hi,everybody;
Recently i have learned something about process scheduling.I have some questions during my studying.Give me some advice on the following my questions if you know about it.
frist: process will take up some cache during executing.Each prcess may have different cache footprint.So could we have some methods to calculate the size of process cache footprint?
second: Each process have a life cycle. It will occupate cpu utilization when we bind a process to a CPU. Suppose it has only a process in the cpu, how to calculate cpu utilization during the whole process life cycle?
Hope that everybody could give me some methods on the above questions.
Best wishes for you.
frist: process will take up some cache during executing.
And, if you run it several times you will probably find different amounts of cache used for each run, depending on what other demands there are on the cache memory...
Quote:
Each prcess may have different cache footprint.So could we have some methods to calculate the size of process cache footprint?
It follows from the previous, that there will be no exact calculation of the cache usage possible. It may be possible to approximate, but you may be better off measuring rather than calculating (and by measuring, I mean measuring and calculating eg, standard distribution, if that's the sort of thing that would be helpful to you).
Quote:
second: Each process have a life cycle. It will occupate cpu utilization when we bind a process to a CPU. Suppose it has only a process in the cpu, how to calculate cpu utilization during the whole process life cycle?
I only understand what you are asking to a limited extent;
I understand that each process has a life cycle; it is born, does stuff and dies
Your point that it will 'occupate' cpu utilisation is relevant how? Do you want the percentage of time that this process is active on this cpu or do you want the percentage of the cpu's cache it occupies? And if that, how do you want to handle the case of a multi-level cpu cache? Do you want total usage, or something else (if so, what?)?
And, you seem to assume that the only possibility is that a process is bound to a cpu. That is neither the only possibility nor is it the default case, for a device with more than one cpu.
And as, in the general case, the usage will fluctuate during the life cycle of the process, how do you want to handle that? Do you want the maximum cpu used (which will always be 100%, if the measurement timeframe is short enough) or are you trying to measure cpu time (as opposed to wall clock time) used?
I am afraid that I don't know how this helps you, and just maybe, the answer given by 'bsat' may be more helpful (but it wasn't really what you asked for). But then, you don't really explain what you would like to use the information for, when you get it.
And, if you run it several times you will probably find different amounts of cache used for each run, depending on what other demands there are on the cache memory...
It follows from the previous, that there will be no exact calculation of the cache usage possible. It may be possible to approximate, but you may be better off measuring rather than calculating (and by measuring, I mean measuring and calculating eg, standard distribution, if that's the sort of thing that would be helpful to you).
I only understand what you are asking to a limited extent;
I understand that each process has a life cycle; it is born, does stuff and dies
Your point that it will 'occupate' cpu utilisation is relevant how? Do you want the percentage of time that this process is active on this cpu or do you want the percentage of the cpu's cache it occupies? And if that, how do you want to handle the case of a multi-level cpu cache? Do you want total usage, or something else (if so, what?)?
And, you seem to assume that the only possibility is that a process is bound to a cpu. That is neither the only possibility nor is it the default case, for a device with more than one cpu.
And as, in the general case, the usage will fluctuate during the life cycle of the process, how do you want to handle that? Do you want the maximum cpu used (which will always be 100%, if the measurement timeframe is short enough) or are you trying to measure cpu time (as opposed to wall clock time) used?
I am afraid that I don't know how this helps you, and just maybe, the answer given by 'bsat' may be more helpful (but it wasn't really what you asked for). But then, you don't really explain what you would like to use the information for, when you get it.
Thank you Salasi , bsat
Your reply are perfect.
I am sorry i don't exactly express what i really want to know.I will
describe my quesitions again.
In chip multi-core environment, suppose that there are the following system environment.
Two cores(core0, core1),independently L1\L2 cache,core1's load>core0's
load,and then core0 is a ilde core.It will trigger ilde load balance in linux.So how to select process to migrate to core0 from core1 to get better performance.I think you will obtain different performance if you migrate different type of processes.My thoughts are as follows:
1)Different process has different cache footprint.The process of larger cache footprint maybe pollute cache memory space,which reduce performance.
2)so when system happen ilde load balance,we should migrate the process of larger cache footprint to idle core to reduce cache pollution.
so my questions are as follows:
1)How should we measure process cache footprint? Are there good evaluation criterias?such as "time" or others?
2)Suppose that we migrate process in the light of their cache footprint when ilde core trigger load balance.The larger cache footprint process will be migrated to ilde core to execute.How to evaluate its superiority ? I agree with your point. Maybe process's executing time is a good evaluation criterias.
The above are my thoughs and questions.
I am looking forward for your reply.
Thank you for your reading.
I am sorry i don't exactly express what i really want to know.I will
describe my quesitions again.
1)How should we measure process cache footprint? Are there good evaluation criterias?such as "time" or others?
2)Suppose that we migrate process in the light of their cache footprint when ilde core trigger load balance.The larger cache footprint process will be migrated to ilde core to execute.How to evaluate its superiority ? I agree with your point. Maybe process's executing time is a good evaluation criterias.
My opinion, which you won't like, is that you are approaching this in a completely wrong way, and you haven't, at least in this thread, defined your aims satisfactorily in able to make progress.
I can't understand why you are so concerned with cache footprint. Do you think that a big cache footprint is a good thing or a bad thing? When you think about this question, bear in mind that using less cache than the amount that you have is to get less advantage out of the cache than you can; unused cache doesn't do any good. So, it doesn't seem to me that using cache size as a proxy for 'performance' for the total system will lead you to a good solution, whatever your problem is, however you measure cache footprint.
For an interesting view on this, you could consider the case of a disk access subsystem; if you 'improve' (however defined) the processor cache performance on this part of the system, doesn't that mean that the disk subsystem gets around to waiting for the disk more quickly? does this 'waiting more quickly' actually help system performance in any way?
Ok, when you say "we should migrate the process of larger cache footprint to idle core to reduce cache pollution" suggests that
You have a limited grip on the usage of the term cache pollution
You are continuing to try to use cache footprint, and now you use it to decide which CPU a process should run on. Given the other criteria which you could use (cpu usage, context switches run queue length, or almost anything else you could think of) it is unclear why you have selected this
Of course, if you do migrate processes, based on this, or other, criteria that is likely, in the short term, at least, to reduce the efficiency of usage of your cache and could lead to L/L2 cache pollution
.
You should also note that
In the 'default' case, moving processes from one core to another is handled by the kernel, when appropriate. What is it about your case that makes using the kernel's well-tested features the wrong thing to do? Why will some other approach be more suitable?
If you use the wrong metric(s), you will get a sub-optimal result. As far as I can see, you seem to be determined to use the wrong metric.
things like hit rate/miss rate seem to be far more useful as metrics, and, in the case that this is an embedded system execution time and/or latency for sub-systems. But, I have to repeat, there is more to optimising the system as a whole than just maximising (or minimising) the amount of cache that is used, so you need to explain why you are so fixed on that as an approach.
My opinion, which you won't like, is that you are approaching this in a completely wrong way, and you haven't, at least in this thread, defined your aims satisfactorily in able to make progress.
I can't understand why you are so concerned with cache footprint. Do you think that a big cache footprint is a good thing or a bad thing? When you think about this question, bear in mind that using less cache than the amount that you have is to get less advantage out of the cache than you can; unused cache doesn't do any good. So, it doesn't seem to me that using cache size as a proxy for 'performance' for the total system will lead you to a good solution, whatever your problem is, however you measure cache footprint.
For an interesting view on this, you could consider the case of a disk access subsystem; if you 'improve' (however defined) the processor cache performance on this part of the system, doesn't that mean that the disk subsystem gets around to waiting for the disk more quickly? does this 'waiting more quickly' actually help system performance in any way?
Ok, when you say "we should migrate the process of larger cache footprint to idle core to reduce cache pollution" suggests that
You have a limited grip on the usage of the term cache pollution
You are continuing to try to use cache footprint, and now you use it to decide which CPU a process should run on. Given the other criteria which you could use (cpu usage, context switches run queue length, or almost anything else you could think of) it is unclear why you have selected this
Of course, if you do migrate processes, based on this, or other, criteria that is likely, in the short term, at least, to reduce the efficiency of usage of your cache and could lead to L/L2 cache pollution
.
You should also note that
In the 'default' case, moving processes from one core to another is handled by the kernel, when appropriate. What is it about your case that makes using the kernel's well-tested features the wrong thing to do? Why will some other approach be more suitable?
If you use the wrong metric(s), you will get a sub-optimal result. As far as I can see, you seem to be determined to use the wrong metric.
things like hit rate/miss rate seem to be far more useful as metrics, and, in the case that this is an embedded system execution time and/or latency for sub-systems. But, I have to repeat, there is more to optimising the system as a whole than just maximising (or minimising) the amount of cache that is used, so you need to explain why you are so fixed on that as an approach.
Thank you for your reply.
I am thinking for this issue that why i select cache footprint as metic of process migration when system happen idle load balance in these days.
Actually,I want to find another metric to migrate process.
Now,there have been some metrics to select process to migrate into idle core when idle load balance happen in recent Linux 2.6.30+,such as cache hot or cool,migration cost.
Generally, Linux scheduler don't migrate processes of busiest core that is cache hot or recently have executed into idle core when idle load balance happen.Suppose the followed scenario that there are two cores.One is busy core A, another is idle one,called B.Now B trigger idle load balance.According to linux scheduling function idle_load_balance(),B should migrate processes from busy core A to reach load balance. So how to select process to migrate idle core B?what metrics are used to migrate process in addtion to the above metrics(cache hot ,migration cost,cfs_rq) when idle load balance happen ?
the process's cache footprint could be used to migrate progress as a new metric?(If we suppose that there are some tools which measure cache footprint)
The above is my thinking.
Hope that you can give me some proposal.
Best wishes.
cypkw
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.