hi
i have a linux server which is running squid proxy server, the configuration is 160GB HDD ,6GB ram, od is centos 5.5 64bit

the problem i am facing is that the swap memeory consumption is showing me 0, screen shot is attached below with this mail.
i tried swapon /dev/sda5 it says device busy

my fstab file is below
[root@proxyserver ~]# cat /etc/fstab
LABEL=/ / ext3 defaults 1 1
LABEL=/var /var ext3 defaults 1 2
LABEL=/cache /cache ext3 defaults 1 2
LABEL=/boot /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
LABEL=SWAP-sda5 swap swap defaults 0 0
LABEL=SWAP-sda6 swap swap defaults 0 0


vmstat output
[root@proxyserver ~]# vmstat
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 0 40992 415656 2345000 0 0 0 2 1 6 0 0 100 0 0

Attached Thumbnails

 

And, what is the problem?

The less your system use the swap the better.
The SWAP is only here to take over the RAM if you consume it all.

I do have the same on my system:
Does that answer your issue?

Angel.

Hi

To add on to the previous poster, your swap usage should be zero for a normally running system. SWAP is a reserved partition on disk (you would choose the size at installation) that only will get used if your real memory is exhausted. If you have sufficient RAM to cope with the load on your server; the server should never dip into the swap memory.

You don't want your machine using swap since its slower than RAM and will thus definitely slow the system down. In practice if your system is ever dipping into swap for an extended period of time; this implies that either you have too little RAM on the box OR you have a poorly written application that is consuming all of your RAM.

Either way you should be looking at rectifying the fault and getting the machine to have zero swap utilization.

k thanx a lot i was thinking that may be swap must be in use always.
anyways one of my application on this server is eating a lot of memory that y i needed to check this . is there a way to check memory leak and how do i fix this i am not a developer

this application was written by previous employee , i think it has memory leak as my squid server hangs every week.

this application he has written reads the database value of the IP and the access that IP has and provides the info to squid which does the necessary stuff

is there any other application like this one.


advice

I suggest you setup something like Cacti or Nagios and monitor your CPU and memory utilization for a few days so you can identify when exactly the memory use spikes.

If there is a memory leak in the application; the only way to fix would be to go through the code and figure out where the memory leak is happening - so thus you'd need to get a developer involved.

To monitor squid logs, you can make your own choice here:
http://www.squid-cache.org/Scripts/

If feasible create a cronjob to restart the application prior to the normal hang period. That will at least get you by until you can fix it and at least then you can control when/how it is being brought down.

Regarding the swap usage, I disagree with the previous posts. I think something is configured wrong. Your system should be using some swap space.

Unless there is an unreasonable setting in swapiness, Linux should give only moderate preference to anonymous memory vs. non anonymous. Looking at the very high res memory amounts, low free memory and moderate caching, it is likely there is a very large amount of resident anonymous memory. Some of that memory must be much more stale than non anonymous memory that is being dropped.

I think the system would run slightly better if it were using some swap space. You ought to check the settings that are stopping it from using swap space. Many people believe swapping is bad and caching is a strictly less important use of memory. That idea is based on a misunderstanding of memory management.

Regarding the hang, I don't see that you have any basis for blaming that on a memory leak. Maybe it is caused by some memory leak. Maybe you have a basis for that partial diagnosis. But you haven't given enough information in this thread for any kind of guess at the diagnosis.

Hi johnsfine,

I'm quite confused by this answer...
And after some search on internet I still didn't find what would be the advantage to use some SWAP if you still have enough RAM available?

Please stop me if I'm wrong, but, storing informations in RAM have the following advantage compare to SWAP:
- Written faster.
- Retrieved faster.
- Consume less system resources to do both operations mention above.

If you are agree (or not) with the 3 statement above can you please explain me, why, and in which case, using some SWAP would be an advantage?

PS: I'm not looking for trouble here, but I just want to understand and correct my thought if I'm wrong.

Best regards,
Angel.

That information is correct but the application of that information to the situation is not correct.

Notice how little free ram there is in the system we are discussing (in this thread). There is nothing inherently wrong with that. But it does indicate that questions about swap usage are not questions about whether pages will be in ram vs. disk; They are questions about which pages will be in ram vs. disk.

Most pages that could be resident in a typical process are either read/write/dirty anonymous or read-only non-anonymous (there are other possibilities, but for performance strategy those are the kinds to focus on).

I am intentionally ignoring some levels in between being resident for a process and being really non resident in physical memory: Either of the above kinds of pages can be bumped out of memory and when it is needed again will be read from disk.

A read/write/dirty anonymous page must be written to the swap area when it is bumped out and must be read back from swap when it is needed again. A read-only non-anonymous page is identical to an existing page at a known location on disk. So to bump it out, the kernel can just discard the copy in ram. To bring it back, the original disk location is reread.

So it costs half as much to bump out (and later read back) a non-anonymous page (which doesn't go to swap) compared to an anonymous page. So the system is biased in favor of bumping out non-anonymous pages (and thus not using swap).

But the whole theory of memory management is based on locality of access, meaning there is a strong correlation between how recently a page has been accessed and the probability it will be accessed again soon.

If you bump out a page that is accessed again soon, you are "wasting" the disk access to bring it back, because you need to bump out some other page to bring it back and bumping out a different page in the first place would have avoided that.

If a non-anonymous page were twice as likely to be accessed soon than a non-anonymous page, it would be better to bump the anonymous page even though doing so is ultimately twice the work per page bumped. It halves the probable number of pages bumped and the associated latency.

The system can only infer the probability of a page being reused soon from the length of time since that page was accessed. But in doing so the system should sometimes infer an anonymous page is less than half as likely to be reused soon as some non-anonymous page, so it should bump out that anonymous page to swap even though it doesn't strictly need to.

In concrete terms, large service processes have a moderate amount of data that is written only during startup, but kept around. Maybe it simply was too tricky to discard it and reuse the space. Maybe the data will be used again for cleanup operations when the service shuts down. But as that service sits around for many days waiting for work to do intermixed with doing work, that data never gets touched. That data is wasting ram that would be better used for cache. The system should be able to recognize that stale data (where it fills whole pages) and bump it to swap space. In a modern system with plenty of ram, it may take many hours of real system use before such data is stale enough to overcome a reasonable bias in favor anonymous pages. But the system whose stats we saw at the start of this thread clearly qualifies. The service processes are big enough and have been running long enough with free ram low enough that it is almost certain there stale anonymous pages in those processes that should have been recognized by the memory management system.

If you set swapiness to zero on such a system, the stale anonymous pages will stay in ram forever (we are assuming there is enough ram to never absolutely need swap space). Meanwhile some non-anonymous page gets bumped out (which is zero work) then needed again later and read back in (disk delay) causing a different non anonymous page to get bumped, then that one is later read back, and so on continually using more disk accesses than would be needed if the stale anonymous pages had been bumped out to swap and left there.

If I do a vmstat on one of my servers while it is fairly busy, I see swpd is 280MB but si and so are zero. That means 280MB of stale anonymous pages were long ago written to swap and zero of those pages are read back in a typical second. This system has plenty of ram, so I expect zero of those pages will ever be read back until those services are shut down.

1 members found this post helpful.

Hi johnsfine,

Thank you for your reply. I rely appreciate that you took time to answer my question.

Although it took me some times to understand what you were saying, I think I do get it now.

But if I understand it well, the one which will benefit the most of this, will be servers that never shutdown/reboot.
Anyway, I still think that this kind of tuning have to be done by perfectionist as the performance benefit will be minimum.

Best regards,
Angel.

hi guys

As i was saying earlier i have this application is was customised for our enviroment.

the problem i am facing is that, this application uses a lot of memory it starts with 2GB and then slowly as days pass the memory utilization increases and at one point of time the system hangs where in i have to reboot the server and the application is back to normal , the reason is that the memory is not being freed since we dont have a developer to assist me in debugging the code is there a way i can free the memory on regular basis without stopping the application.

the operating system is centos 5.5

Rechecking some basic facts:

You have 6GB of ram and a 64 bit kernel.

Is the problem application a 64 bit application or 32 bit?

Earlier you showed 12GB of swap space available and none used. Did that change? Especially when the problem application was near hanging?

Most application memory leaks in 64 bit applications provide significant opportunity for the kernel to swap out pages of the application without harming that application's performance. So if you have 12GB of swap space and a memory leaking 64 bit application and still no swap space is used, something strange is happening (maybe swapiness is set very low).

That is very strange for a memory leak in one application. The system should slow way down once that application is using almost all of the 6GB of physical ram, but very slow should be noticeably different from hung.

NO! You probably can stop/restart just the application (not the whole computer) to release the memory. But you CANNOT release the leaked memory without stopping the application and you almost certainly cannot fix the application to not leak memory without a competent developer.

johnsfine, your two replies are more than just*fine*, they are very helpful.

Hi Johnsfine

thanx a lot for your response

its a 64 bit application, yes i have 12gb of ram available

one the application reaches 5.3 gb of physical RAM then the application makes use of the swap memory
i have 2 swap partition each of 6gb, so when 6 gb of one swap is utilised then the system hangs

i cant login to the system so, the only option is to reboot the system then everything is back to normal

the application is written in c , it was written by one developer in the organisation who is not longer available with us.

if i use the command swapoff -a wait for a while and then swapon -a
will it affect the running of the application, this is a critical application where users are affected if there is a downtime