[SOLVED] why: the same program runs multiple times but get very different results
ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
why: the same program runs multiple times but get very different results
I used wait4 to time a child process. but the results differ each other dramatically. why?
my core source code lists as below.
to the point: in linux 2.6 kernel, how to time a process with a high precision. and, only its user-time rather than the elapsed time???
static struct rusage ruse;
static pid_t u_pid;
int main(int argc, char *argv[])
{
int i;
for (i = 0; i < 10; i++) {
u_pid = fork();
I used wait4 to time a child process. but the results differ each other dramatically. why?
my core source code lists as below.
to the point: in linux 2.6 kernel, how to time a process with a high precision. and, only its user-time rather than the elapsed time???
static struct rusage ruse;
static pid_t u_pid;
int main(int argc, char *argv[])
{
int i;
for (i = 0; i < 10; i++) {
u_pid = fork();
and my result:
1.08807
1.06407
1.08807
1.05607
1.06807
1.07607
1.06807
1.07607
1.07207
1.08407
Why do you call the differences dramatic, i.e. which differences would you consider to be non-dramatic ? Justify your calculations of non-dramatic differences and your expectation WRT to the results in general.
the differences is about 10ms. I define the precision 0.5ms. And in my opinion, time elapsed in user space can be attained according to usage. But it does work now. Why not? and How to measure the time a programe used for its execution in sys or usr space with the precision 0.5ms ??
Last edited by hillgreen; 12-09-2010 at 05:07 AM.
Reason: more info
the differences is about 10ms. I define the precision 0.5ms. And in my opinion, time elapsed in user space can be attained according to usage. But it does work now. Why not? and How to measure the time a programe used for its execution in sys or usr space with the precision 0.5ms ??
Do you understand how your computer, including the CPU, works ? Do you know which big parts the system consists of ? Have you ever heard the words "cache", "DRAM" ?
I used wait4 to time a child process. but the results differ each other dramatically. why?
They are all 1.0-something, or another. I agree with Sergei that this is not dramatic. It may not meet your requirements, but that is a rather different issue.
Did you actually look at your data? All of your data points (from this run) end ...07, in fact 207, 407, 607 and 807. Those will be completely random numbers, of course
Looking at your pattern, you could come to some conclusion about what could and could not be captured as a result of these runs.
To the OP: you apparently have an expectation of constant execution time, and this expectation is wrong - regardless of timer accuracy. That's why I wrote:
Quote:
Justify your calculations of non-dramatic differences and your expectation WRT to the results in general.
The execution time probably is constant, but what the OP in the program was the total wall clock time to execute the program, which of course includes context switches, and other things that the system does during that time.
What the OP apparently wants is to determine the total CPU time taken by the specific child process.
Take a look at getrusage(). Also look at /proc/stat. The answers you want are in those areas.
No, it is not. By construction of modern HW in the first place, by context switches, by other HW processes.
Please try to pay attention, and read carefully and correctly.
The execution time probably is constant. Every time the program "anti" is run...whatever anti is...it probably runs in the same number of CPU cycles and bus cycles. This is almost certainly true. Period.
Now, if you had taken the time to pay attention and actually read the rest of the sentence I wrote - which you clearly did not because you only clipped the first portion of it, suggesting that your attention span is only that long - you would have seen that I stated that OP was actually getting the wall clock time which included things like context switches.
I've seen you do this too many times. And on this particular thread, you have done it again. And following your usual pattern you took a patronizing tone with OP while NOT presenting a solution. You will note that I did tell the OP where to look for a solution.
Remember one thing, Junior. No matter how much you think you know, there's bound to be someone around who knows more than you do. And in this case, on this topic, I'm that someone.
Sergei is absolutely correct - timings WON'T be accurate to the microsecond. Or probably not even to the millisecond.
@hillgreen -
Here are a couple of other quick/easy ways to get "timings". It might be interesting to compare the results. And - equally interesting - to compare the variance between successive runs.
1. Use "time" in a shell script
Code:
time anti
time anti
time anti
time anti
time anti
time anti
time anti
3. Remember -
a) Linux is NOT a "real-time operating system" (and, of course, neither is Windows )
b) The whole point of "real time" is NOT "instantaneous" or "as fast as possible"...
... rather, the point is "deterministic"
c) And (at the risk of duplicate redundancy ) - Linux is NOT "deterministic"
Sergei is absolutely correct - timings WON'T be accurate to the microsecond. Or probably not even to the millisecond.
CPU time will be. Wall clock time won't be. The time spent executing is CPU time. The time from start to finish is wall clock time.
Quote:
@hillgreen -
Here are a couple of other quick/easy ways to get "timings". It might be interesting to compare the results. And - equally interesting - to compare the variance between successive runs.
1. Use "time" in a shell script
Code:
time anti
time anti
time anti
time anti
time anti
time anti
time anti
You apparently haven't answered these questions. So, here are my answers.
Modern computers typically use DRAM. "D" stands for "dynamic", meaning capacitive storage elements. Capacitive storage elements need refresh cycles. Refresh cycles are implemented as completely independent HW process, i.e. there is a piece of HW doing this. While a refresh cycle is in progress, the DRAM is inaccessible. I.e. a read or write operation will take more time if it hits a refresh cycle.
Modern CPUs have caches, and a CPU job is to execute instruction and to while doing this to load and store data. Instructions fetches and data load/store operations can occur with/from cache - in such a case they are fast, or with/from RAM - in such a case they are (IIRC) ten(s) of times slower. Whenever the OS switches tasks, cache is essentially repopulated, so task switching does cause variations of execution time.
Modern (and not only modern) computers use DMA extensively. DMA is yet another HW process. If a DMA transfer is in progress, RAM (at least, the affected by DMA bank) is inaccessible to CPU, so it looks to it like slower RAM.
As I have already written many times, taking all this into consideration, execution time is not constant. Because it simply can't be.
...
P.S. I took part in development of DMA block of a pretty complex communications chip; the chip also contained a CPU. It was my day to day job to perform HDL simulations of the chip carefully watching bus activity. DMA typically has higher than CPU bus priority - a network packet can't wait, that's why, for example.
You apparently haven't answered these questions. So, here are my answers.
Modern computers typically use DRAM. "D" stands for "dynamic", meaning capacitive storage elements. Capacitive storage elements need refresh cycles. Refresh cycles are implemented as completely independent HW process, i.e. there is a piece of HW doing this. While a refresh cycle is in progress, the DRAM is inaccessible. I.e. a read or write operation will take more time if it hits a refresh cycle.
Modern CPUs have caches, and a CPU job is to execute instruction and to while doing this to load and store data. Instructions fetches and data load/store operations can occur with/from cache - in such a case they are fast, or with/from RAM - in such a case they are (IIRC) ten(s) of times slower. Whenever the OS switches tasks, cache is essentially repopulated, so task switching does cause variations of execution time.
Modern (and not only modern) computers use DMA extensively. DMA is yet another HW process. If a DMA transfer is in progress, RAM (at least, the affected by DMA bank) is inaccessible to CPU, so it looks to it like slower RAM.
As I have already written many times, taking all this into consideration, execution time is not constant. Because it simply can't be.
...
P.S. I took part in development of DMA block of a pretty complex communications chip; the chip also contained a CPU. It was my day to day job to perform HDL simulations of the chip carefully watching bus activity. DMA typically has higher than CPU bus priority - a network packet can't wait, that's why, for example.
Do you know what? That's all true. I won't argue with any of it. But I will put it into context.
If a process has to wait on a memory refresh (which could happen) the process will be marked "not ready", the processor will do a context switch, and therefore the process is not executing. So the time charged against it is wall clock time, not processor time.
As for processor caching, the devil is in the details. I will agree that if process B is waiting and process A winds up using all the onboard cache, then when process B gets the processor back it will have to wait for fetches from RAM. And this will cause some variance in the CPU time associated with the execution, which will be charged against the process since the wait states associated with waiting on RAM are charged against the process. However, even in worst case, the time variance associated with doing this will be a very very small fraction of the total time variance associated with measuring wall clock time, which includes context switches and waits for I/O and so forth.
Similarly, extensive DMA can affect RAM access times, but often enough that will cause a CPU context switch if the data or instructions are not already cached, and again the time is not charged against the process.
So. What you say is true. But it is also a significant impact only at a scale that is ordinarily well below the scale at which the OP is working, and is at best an extremely trivial contributor to the variance OP was asking about, presuming the OP's computer is reasonably modern. How trivial? Very hard to say. But if I had to take a whack at it, I'd place it someplace on the order of a thousandth of a percent or less of OP's indicated variance in a modern PC-class computer. That's just a WAG, and if you can present real numbers, please do so.
If a process has to wait on a memory refresh (which could happen) the process will be marked "not ready", the processor will do a context switch, and therefore the process is not executing. So the time charged against it is wall clock time, not processor time.
...
Nonsense. The CPU has no notion of memory refresh. The CPU can't do anything externally while in wait cycles, for example, it can't push current task's registers on stack, it can't process an interrupt.
To the same extent the CPU has no knowledge about DMA.
...
Spend about $100 or something like that (well, more - you'll need an oscilloscope too), buy a cheap development board for a simple controller - preferably without pipeline, and do something on bare metalfrom scratch. I.e. write your own BIOS first.
For me it was quite revealing in the late eighties to develop an in-circuit emulator and to use it to debug HW and SW.
Last edited by Sergei Steshenko; 12-09-2010 at 03:45 PM.
As I have already written many times, taking all this into consideration, execution time is not constant. Because it simply can't be.
Quote:
Spend about $100 or something like that (well, more - you'll need an oscilloscope too), buy a cheap development board for a simple controller - preferably without pipeline, and do something on bare metal from scratch. I.e. write your own BIOS first.
Or, even better:
* Take your favorite (CPU- and memory- intensive) program, and time it any three or four or five ways you like.
* Run it ten or 100 or 1000 times for each timing method you choose.
* Compute the variation for each method.
I'll bet your timings might be a lot closer than hillgreen's (with his fork()'s and "wait()'s", which introduce a HUGE amount of latency). But I seriously doubt they'll consistently line up to the nearest millisecond, either
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.