Hi,

How do I check what amount of cache + buffer is actively being used.
Was just googling this and found that Active and Inactive parameters in /proc/meminfo gives this info. But if that was that case (Active + Inactive)kB should be equal to (cache + buffer)kB right ? But this doesn't seem to be true. Active + Inactive is a much bigger number than Cache + Buffer. Please help me, I am trying to troubleshoot an issue where the swap space usage is almost 100%, even though cache + buffer ~ 1.5G on 6G RAM system.

Thanks in advance.
Joe

These figures might never be "exact sums."

Very high swap-space usage implies that your applications have a memory-leak somewhere. The amount of virtual memory that they think they need, and therefore that they individually have access to, is much larger than the amount of physical memory that the system can afford to give them. The culprits will have very high "swap-I/O" counts and are probably spending most of their times in "swap wait." You might smell funny hot-smoke smells from those poor disk drives.

The system will always reserve some amount of buffer memory, for the actively-open files, and this buffer allocation is probably, in this case, just being used for open files.

Thanks for the reply.
But we can almost 3 processes which are using swap heavily, which is why I didn't suspect a memory leak.
I am going to check the swap in/out, but the highly suspect that this is not going to be high as the IO on the system is very low.
If swap in/out was was high, it should have resulted in high IO.
Also where can I see "swap wait" ?

Then concentrate on the real problem, not some irrelevant symptom.That makes no sense. If they are heavy swap users, they are almost certainly the problem - especially if they are increasing their usage. Swap only contains anonymous memory - that means applications.High swap occupancy doesn't necessarily mean high swap I/O - poorly designed code can allocate memory and rarely access it. Then (continually) go do the same. Eventually the "old" allocation goes to swap and stays there.
That's a classic memory leak.

1 members found this post helpful.

An excellent point.

Many programmers these days cut their teeth on languages that provide a "garbage collector," which will periodically sweep-up memory (unless(!) confounded by "circular references"). Other languages, like C, do not. Either situation can result in a subtle bug: a memory leak. The "leaked" memory winds up in the swap file, and stays there.

Active and Inactive values in meminfo are not relevant swap activity. You'll find that linux memory reporting is fairly convoluted. I'd recommend you start with by looking at rates of swap and identify when and under what trend the swap partition reached 100%. i.e. was this a sudden spike in swapping or a constant slow swap rate, most importantly, do you see thrashing. Each system activity report (sar) file will contain a segment headed by pswpin/s and pswpout/s. These are your swap rates. View the current day's with sar -W or search the text archives under /var/log/sa/sarXX where XX is the day of the month.

After identifying the time and trend you can examine other stats in sar during that time frame as well as the kernel logs (messages) and any specific application logs which may shed light on application activity during the issue. Unfortunately the default historical record keeping is limited in linux. In the case of swapping it's likely that the sum total of userspace demand is exceeding physical RAM. Linux doesn't keep a record of per process usage so you likely won't be able to identify a root cause without additional logging.

Ultimately, you'll need to adress the following:
[1] Break down of memory utilization: kernel/userspace/per-process (ps aux and meminfo during the swap activity will suffice)
[2] If a specific application is utilizing a large amount of memory (usually the case) is this expected? i.e. potential bug
[3] If not, is the application and os tuned to the application workload (refer app tuning guide from the app vendor)
[4] Is the application foot print expected to reside within the allocated amount of RAM (i.e. should RAM be increased)