How to reduce used memory and dedicate processor to a single process
Hello,
I am a new user with a few questions (for the time being though). First of all, let me try to explain what we will do with Linux: - We have a CFD (computational fluid dynamics) tool which requires really high computing power in order to calculate basic state equations of a fluid dynamics problem. We are going to use Linux for this (actually we already use it but we need some improvements). - The reliability of the result and the accuracy of the calculations depend on the total amount of RAM the system has. - The speed of the calculations is highly dependent on the speed of the processor. Now, as the hardware we have: Processor: AMD Phenom II X4 @ 3.0 GHz Memory: 4x2 GiB (the maximum amount the motherboard allows) As the Sotfware: GNU/Linux Ubuntu 64 bit Version 8.10 for AMD64 Our Issue: 1- We want to reduce the total amount of memory the system uses for other applications (it uses 337 MiB for the time being and every single byte is important for us). 2- We want to dedicate at least, if not all, 75% of the cpu resources to this CFD program to speed up the process. The cpu has 4 cores and 75% means that we want at least 3 cores to be dedicated to this program. For the time being, Linux uses only 1 processor in 100% capacity (which is equivalent to 25% overall). The first issue is relatively simple for most intermediate or advanced Linux user yet the second issue is, I guess, rather complicated and needs the opinion of a developer. Any opinion is wellcome and thanks to any helper in advance. Bora |
Quote:
If you have the opportunity to make it multi-threaded, you can use cgroups to create a container for that process with 3 CPUs/cores - all other work can be isolated to the other core. I use cgroups for benchmarks that I expect to "burn" the system, and I can reserve a CPU so I can monitor what's happening and not be "locked out". See ../Documentation/cpusets.txt (the old name for this) and ../Documentation/cgroups/cgroups.txt |
Quote:
However, note that 'memory usage, but not particularly active' shouldn't be hurting you much. Assuming that you have swap, it should be swapped if there is pressure on memory usage and it should largely stay there. What evidence do you have that the memory is an issue? Commonly people look at memory in use and conlude that they have a problem when linux is using its 'haven't anything better to do with this ram' algorithm and would swap, if helpful. Quote:
Quote:
In the interim, note that this may mean that there is not a dramatic slowdown in running several simulations simultaneously on the same processor (assuming that you have the memory, of course, which it sounds as if you don't currently, and also assuming that you aren't I/O limited) as might be advantageous if you were doing gentic algorithms and/or Monte Carlo / Tagucchi for optimisation over many runs. |
To me it looks like there may be a misunderstanding or several.
Post the output of free and you'll get something like: Code:
bash-3.1$ free -m syg00 is right about the multithreading, if your program is not multithreaded then it will always only be able to use 1 core or processor. You may need to rewrite the program. If you have not already done so, compile a kernel specifically for the processor, choose 'No Forced Preemption (Server)', Disable 'Preempt The Big Kernel Lock', 'Timer frequency = 100 Hz', which will give you maximum processing power and lowest latency. You may have already done this. |
Here's still another take. Given your application, computational fluid dynamics, I'm surprised it's not already multi-thread capable. If it is, and if there is a way to tell it to use more threads, I think you will be surprised at just how well Linux's CPU usage works without your getting involved.
|
Quote:
That is a hard problem to redesign as multi-threaded. Is the work of changing the application to multi-threaded being done by the engineer who wrote the original code? Or by a more skilled engineer with access to all the original source code, with enough ability to understand and improve on the original design? If "no" to both of those, then forget multi-threaded. The application will remain single threaded. Quote:
Quote:
Quote:
Quote:
Quote:
Quote:
Quote:
If you don't have the source code to the application, you're not going to improve anything by tweaking your Linux system. If you have the source code, I would start by using Oprofile to see where the code spends its time. Other profilers give you more information but they distort the information, often quite a lot. Plus they are harder to use (not that Oprofile is all that easy). You may have a lot of room for improvement just by improving the code in the hot spots. You might also think of a way to locally multithread just in some hot spots. As long as your application has one to three active threads on a four core system, you don't need anything complicated to make sure it gets the maximum share of CPU time it can use. Just give it a higher priority than anything else using significant CPU time. If you manage to multi-thread to more than three useful threads, then you can revisit the then complicated question of how to give it most of the CPU power without locking up the system so you can't even look around. |
All times are GMT -5. The time now is 06:22 AM. |