Hi friends

I have a server running RHEL AS 4.7 which is being used for high end visualization applications. It has 128 GB of RAM but the swap space currently set on this server is 30GB.


free -g
total used free shared buffers cached
Mem: 125 60 65 0 0 35
-/+ buffers/cache: 25 100
Swap: 30 0 30


Can anybody tell how much swap space can be set on this server? Should I set it to 256GB !!! going by the thumb rule or should it be equal to size of memory?

The users who have tested their applications on this server say that it takes the same time to process as it would on dell 690 workstation which has 8GB RAM in it.

FYI this server has 2 NVIDIA GRAPHIC CARDS

->% more /proc/driver/nvidia/cards/0
Model: Quadro FX 5600
IRQ: 217
Video BIOS: 60.80.1e.00.05
Card Type: PCI-E
DMA Size: 40 bits
DMA Mask: 0xffffffffff
Bus Location: 05.00.0
->% more /proc/driver/nvidia/cards/1
Model: Quadro FX 5600
IRQ: 217
Video BIOS: 60.80.1e.00.05
Card Type: PCI-E
DMA Size: 40 bits
DMA Mask: 0xffffffffff
Bus Location: 90.00.0

While the 690 Dell workstation has 1 NVIDIA GRAPHIC CARD

Model: Quadro FX 4500
IRQ: 169
Video BIOS: 05.70.02.41.03
Card Type: PCI-E
DMA Size: 39 bits
DMA Mask: 0x7fffffffff
Bus Location: 07.00.0



Any suggestions as to how the server performance can be improved?

Considering your application (high end visualization) I wouldn't think that any swap usage would be tolerable. So, I'd say even less than what you have.

With this size of ram you should rarely need swap. Usually swap is twice the size of ram but if you have 128gb of ram then you might not need swap. It should depend on how heavy usage will be and what applications are going to run. My desktop machine does not even have the amount of hard drive as big as your ram.

Usually in the sizes of 16 GB + Ram you consider the SWAP disc as big as your OS.

If you use RHEL i would consider it about 5GB.
Everything else doesnt make sense.

Remember Swap discs are just for unused programms which reside in memory which can be swaped out for performance/space issues.

Your Application will use the Libraries in Shared memory anyways so the only data which resides in the memory is the user specific app data. Which will be (hopefully) destroyed on disconnect.

As you can apply the rule of thumb here. ( and i dont know your whole setup/environment/influences)

I would recommend you to have a look overall if you use this system activly log the ram usage and the swap usage.

Maybe even copy a filelist down from the swap disc. Maybe do this over 1 week or 2 (depending if you have a full workload of people on the server) and figure out by yourself how much is really needed.

Recommendation from me is 5 GB Swap. +- your adjustment.
I would monitor it anyway after making changes on this setting. For example you dont want to have a full memory after while your apps stop to swap out and stay in your ram.

If you'd used code tags for that output, you'd have noticed that its saying,
SWAP: total 30 used 0 free 30
IOW, its not swapping ....

on my box (RHEL 5.3)
If you run top, you'll see it more like

[QUOTE=shaikjaveed1;3477882]


If this is normal operation with a representative workload, normal operation should never cause any swapping.


No
No

What you want, in addition to having enough memory for normal operations, which it seems that you have, is, when something goes wrong, for the system to slow down gracefully rather than to crash. 2G would do this as would 5 or 10, but I cannot see what having more than that would accomplish. I guess, given your spec, that you have enough disk space...

Which is a bit disappointing, but then it doesn't seem that you are anywhere needing the amount of ram that you have installed.

Right now, no. The algorithm is to find the bottlenecks and cure them and I am afraid that nothing about this post gives me any idea what the bottlenecks are (apart from that its not the amount of memory, if the memory usage data above is recorded in a reasonably heavy working scenario).

Try to set in /etc/sysctl.conf this:

vm.swappiness = 0
(tell the kernel to use only RAM memory, only in critical situations to use swap, this should help a lot)

Add the line and then run: sysctl -p

Good luck!

Until you place a much heavier load on the system than shown in that free output, the amount of swap space won't matter at all.

The "rule of thumb" for swap space is usually nonsense and in this case extreme nonsense.

Assuming you plan to put a much heavier load on this system's memory, you may want to think through what the swap space would be for in order to estimate how much you want.

1) Stale pages in background tasks. Swap space lets the system kick those out in favor of more file caching (but so far you're not doing enough file I/O for the system to even want more file caching). When memory is almost full including file caching (if it ever gets there) you could look at the total of the RES sizes of several background tasks selected by lowest accumulated CPU time in long sleeping tasks. That would give a very crude estimate of how much swap you might want for this reason. Most likely it will be a very small number.

2) Your "high end visualization applications" might be very excessive in their mapping of COW and Demand-Zero memory areas, causing issues with the limits on memory commit level, such that correct operation requires allocating a big swap area that the system would never actually use, but whose existence keeps the kernel from thinking it has over committed memory. It is an unlikely situation but possible. Look for absurdly high values in the VIRT column in top to see how possible it might be.

You seem to be using a lot more than 8GB. But I guess the important task is cpu bound and uses less than 8GB.

The rule of thumb was true back when computers had 16MB of RAM. Those days are long gone.

That's 60GB used, 25 for applications and 35 for buffers and cache. Then 65GB completely unused. Then 30GB swap space, but 0GB of it used. You do not have to worry about swap space, and you should not increase its size.

Read here for one perspective: http://www.linux.com/feature/121916

It doesn't speak precisely to your situation, but it is useful nonetheless.

IMO, you've sized your swap space just about right. With 128GB RAM I probably also would have set up a swap space between 24 and 32GB (nothing especially scientific there, BTW). What you want to do is build yourself a comfortable buffer in case your server does begin to use swap space. This will help you avoid a system crash.