[Log in to get rid of this advertisement]

Hi:

I have been coding an elaborate PHP site using Fedora Linux, mySQL, and apache for a year now. We're about to launch and, in a fit of optimism, we are discussing a server upgrade and wondering how to determine where our system's bottlenecks are going to be.

I'm familiar with these linux commands to measure resource usage but was wondering if people could recommend others so i can get more detail:

free - reports total memory usage. is there one that does so by application?
netstat - reports internet connections and bandwidth consumed by each.
w - reports instantaneous cpu usage by user? do i have that right?

can anyone recommend others?

i'm basically wanting to see if can determine whether our bottleneck is more likely to be RAM, CPU, disk drives, or bandwidth. any help would be much appreciated.

You might want to try gkrellm and top.

If you dont find it here you wont find it anywhere.
http://ltp.sourceforge.net/tooltable.php

you guys rock. thanks so much. EXACTLY what i was looking for.

that tools page looks pretty extensive but i'm a noob to benchmarking/testing. i will read up there and see if i can figure it out.


by the way, it seems a bit disconcerting to me that all my memory is used up. "free" produces the following:

[root@mydomain root]# free
total used free shared buffers cached
Mem: 247416 215416 32000 0 61152 60640
-/+ buffers/cache: 93624 153792
Swap: 1052248 70872 981376

Thats because you are used to windows.Just remember its not windows,it handles memory different and is obviously more efficiant.
It will distributed memory to wherever needed when needed and on time too!

Just a note (that is really handy to know), RAM in Linux is cached once used, instead of being freed up entirely (ala Windows). This allows the cached RAM to be quickly accessed, instead of having to add the extra step of taking the free RAM and recaching it first. So when you run free, make sure to add the actual free column to the buffers and cached row. For example:
If I just went by the free column, I would think I only had a measly 448mb left. But when I add cached and buffers to free, I end up with a much friendlier number of ~850.

I think the real thing you should be worried about is that your swap is actually being used. It normally takes a lot before things start swapping out (I don't even have a swap partition, and I've never run out of RAM).
As was said above, I'll reinforce the idea of getting gkrellm, it's a solid tool for all your monitoring needs (and displaying correct free RAM values).

ok...how did you get that nice lovely formatting? can one type html in here?

total used free shared buffers cached
Mem: 247416 215416 32000 0 61152 60640
-/+ buffers/cache: 93624 153792
Swap: 1052248 70872 981376


ALSO, i tried TOP with all threads showing. it looks like apache is consuming all the memory, but every single process appears to be sleeping. i *think* i understand what you mean. memory gets allocated to a process and stays with it until somebody else needs the memory? my sleeping process are gobbling up the memory but seem more than happy to relinquish it to a new process that needs it.

Vgui, I'm familiar with the concept of caching but i don't really understand what you wrote. Do you mean to say that once RAM is allocated, it remains allocated? and that the RAM remains 'allocated' so that it can be used again quickly without reallocating? it is my understanding that 'caching' memory means saving it to either the CPU cache or to the hard drive when you page out memory.

by the way, i *really* appreciate your help on this.

The formatting is just using code tags (enclose what you want formatted in code]/code], but with the leading [).
As I said before, I recommend gkrellm, unless you are running a pure non-X server (which is probably fairly likely). What apache is probably doing is requested X amount of RAM, but it isn't actually using the RAM, and will hand it off as needed. I had read in a few places that the kernel doesn't bother to free up RAM unless something else needs it (to stop needless overhead). Perhaps there is some apache configuration you can perform (such as setting initial RAM usage, etc.).
As for how the caching works. First, the Windows way. Say you open up Eclipse, and it takes up 60mb of RAM. That RAM is allocated to Eclipse, and isn't free or cached. When you close Eclipse, Windows will move the RAM back to "free", which basically means cleaning out any data left in the RAM. Say you reopen Eclipse. The RAM will again have to be allocated. Basically, Windows adds a step either way.
With Linux, any RAM you use (from running programs, etc.), once the program is closed, the RAM is moved to the "cached" stage, instead of freed up entirely. So, you still have old data kicking around in the cached RAM, and it is a lot (no hard numbers, sorry) speedier for the kernel to reallocate the cached RAM than having to move from free to used RAM. So, in this case Linux removes the 2 steps, and doesn't create unnecessary overhead.
Hopefully that clears things up a bit more, it's getting a bit late after a night of coding, so this post is probably just garbled ramblings

thanks vgui, it's pretty clear and pretty much what i thought you were saying.

i have tried to get gkrellm...downloaded the source files and tried to make on my fedora linux machine. no X windows here. the make command gave me the error:

"Package glib-2.0 was not found in the pkg-config search path"

i noticed a couple of tutorials to get it going on windows and stuff but i can't even get the server part done.

That's ok...if you are not using x windows you wouldn't be able to use gkrellm. Gkrellm is a useful tool that monitors (among other things) cpu usage, memory usage, interface (e.g. eth0) traffic and places the information it finds on your desktop in a nice bar-graph style.

As another recommendation, I use the 'df' command a lot for monitoring filesystem usage. It is helpful to use it in a cron-like job to monitor memory thresholds. For example, I get emails when my filesystem reaches 80% capacity.

Actually, there appear to be 2 components to gkrellm....or so the tutorials suggest anyway. one part is the 'server' part which gathers the information. the other part is the client. Some guy has written a client for windows machines. They say that debian linux has all the necessary libraries to compile the server in the linux distribution.

i'm hip to DF. i also like top quite a bit. i'm more concerned about RAM issues than disk issues.