LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-09-2010, 01:53 AM   #1
fernfrancis
Member
 
Registered: Feb 2009
Location: Goa(India)-Sharjah(UAE)
Distribution: RHEL,centos,fedora,ubuntu
Posts: 229

Rep: Reputation: 18
swap memory


hi
i have a linux server which is running squid proxy server, the configuration is 160GB HDD ,6GB ram, od is centos 5.5 64bit

the problem i am facing is that the swap memeory consumption is showing me 0, screen shot is attached below with this mail.
i tried swapon /dev/sda5 it says device busy

my fstab file is below
[root@proxyserver ~]# cat /etc/fstab
LABEL=/ / ext3 defaults 1 1
LABEL=/var /var ext3 defaults 1 2
LABEL=/cache /cache ext3 defaults 1 2
LABEL=/boot /boot ext3 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
LABEL=SWAP-sda5 swap swap defaults 0 0
LABEL=SWAP-sda6 swap swap defaults 0 0


vmstat output
[root@proxyserver ~]# vmstat
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 0 40992 415656 2345000 0 0 0 2 1 6 0 0 100 0 0
Attached Thumbnails
Click image for larger version

Name:	Screenshot.png
Views:	34
Size:	210.8 KB
ID:	5144  
 
Old 11-09-2010, 02:11 AM   #2
angel115
Member
 
Registered: Jul 2005
Location: France / Ireland
Distribution: Debian mainly, and Ubuntu
Posts: 528

Rep: Reputation: 78
And, what is the problem?

The less your system use the swap the better.
The SWAP is only here to take over the RAM if you consume it all.

I do have the same on my system:
Code:
~$ vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  0      0  28656  63608 104760    0    0     3    64   20   38  0  0 99  0
Does that answer your issue?

Angel.

Last edited by angel115; 11-09-2010 at 02:13 AM.
 
Old 11-09-2010, 02:39 AM   #3
alli_yas
Member
 
Registered: Apr 2010
Location: Johannesburg
Distribution: Fedora 14, RHEL 5.5, CentOS 5.5, Ubuntu 10.04
Posts: 559

Rep: Reputation: 92
Hi

To add on to the previous poster, your swap usage should be zero for a normally running system. SWAP is a reserved partition on disk (you would choose the size at installation) that only will get used if your real memory is exhausted. If you have sufficient RAM to cope with the load on your server; the server should never dip into the swap memory.

You don't want your machine using swap since its slower than RAM and will thus definitely slow the system down. In practice if your system is ever dipping into swap for an extended period of time; this implies that either you have too little RAM on the box OR you have a poorly written application that is consuming all of your RAM.

Either way you should be looking at rectifying the fault and getting the machine to have zero swap utilization.
 
Old 11-09-2010, 02:47 AM   #4
fernfrancis
Member
 
Registered: Feb 2009
Location: Goa(India)-Sharjah(UAE)
Distribution: RHEL,centos,fedora,ubuntu
Posts: 229

Original Poster
Rep: Reputation: 18
k thanx a lot i was thinking that may be swap must be in use always.
anyways one of my application on this server is eating a lot of memory that y i needed to check this . is there a way to check memory leak and how do i fix this i am not a developer

this application was written by previous employee , i think it has memory leak as my squid server hangs every week.

this application he has written reads the database value of the IP and the access that IP has and provides the info to squid which does the necessary stuff

is there any other application like this one.


advice
 
Old 11-09-2010, 02:59 AM   #5
alli_yas
Member
 
Registered: Apr 2010
Location: Johannesburg
Distribution: Fedora 14, RHEL 5.5, CentOS 5.5, Ubuntu 10.04
Posts: 559

Rep: Reputation: 92
I suggest you setup something like Cacti or Nagios and monitor your CPU and memory utilization for a few days so you can identify when exactly the memory use spikes.

If there is a memory leak in the application; the only way to fix would be to go through the code and figure out where the memory leak is happening - so thus you'd need to get a developer involved.
 
Old 11-09-2010, 03:05 AM   #6
angel115
Member
 
Registered: Jul 2005
Location: France / Ireland
Distribution: Debian mainly, and Ubuntu
Posts: 528

Rep: Reputation: 78
To monitor squid logs, you can make your own choice here:
http://www.squid-cache.org/Scripts/
 
Old 11-09-2010, 09:39 AM   #7
jwl17330536
Member
 
Registered: Feb 2010
Location: Raleigh, NC
Posts: 83

Rep: Reputation: 22
If feasible create a cronjob to restart the application prior to the normal hang period. That will at least get you by until you can fix it and at least then you can control when/how it is being brought down.
 
Old 11-09-2010, 10:15 AM   #8
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181
Regarding the swap usage, I disagree with the previous posts. I think something is configured wrong. Your system should be using some swap space.

Unless there is an unreasonable setting in swapiness, Linux should give only moderate preference to anonymous memory vs. non anonymous. Looking at the very high res memory amounts, low free memory and moderate caching, it is likely there is a very large amount of resident anonymous memory. Some of that memory must be much more stale than non anonymous memory that is being dropped.

I think the system would run slightly better if it were using some swap space. You ought to check the settings that are stopping it from using swap space. Many people believe swapping is bad and caching is a strictly less important use of memory. That idea is based on a misunderstanding of memory management.

Regarding the hang, I don't see that you have any basis for blaming that on a memory leak. Maybe it is caused by some memory leak. Maybe you have a basis for that partial diagnosis. But you haven't given enough information in this thread for any kind of guess at the diagnosis.
 
Old 11-10-2010, 04:32 AM   #9
angel115
Member
 
Registered: Jul 2005
Location: France / Ireland
Distribution: Debian mainly, and Ubuntu
Posts: 528

Rep: Reputation: 78
Quote:
Originally Posted by johnsfine View Post
Regarding the swap usage, I disagree with the previous posts. I think something is configured wrong. Your system should be using some swap space.

Unless there is an unreasonable setting in swapiness, Linux should give only moderate preference to anonymous memory vs. non anonymous. Looking at the very high res memory amounts, low free memory and moderate caching, it is likely there is a very large amount of resident anonymous memory. Some of that memory must be much more stale than non anonymous memory that is being dropped.

I think the system would run slightly better if it were using some swap space. You ought to check the settings that are stopping it from using swap space. Many people believe swapping is bad and caching is a strictly less important use of memory. That idea is based on a misunderstanding of memory management.
Hi johnsfine,

I'm quite confused by this answer...
And after some search on internet I still didn't find what would be the advantage to use some SWAP if you still have enough RAM available?

Please stop me if I'm wrong, but, storing informations in RAM have the following advantage compare to SWAP:
- Written faster.
- Retrieved faster.
- Consume less system resources to do both operations mention above.

If you are agree (or not) with the 3 statement above can you please explain me, why, and in which case, using some SWAP would be an advantage?

PS: I'm not looking for trouble here, but I just want to understand and correct my thought if I'm wrong.

Best regards,
Angel.
 
Old 11-10-2010, 08:45 AM   #10
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181
Quote:
Originally Posted by angel115 View Post
Please stop me if I'm wrong, but, storing informations in RAM have the following advantage compare to SWAP:
- Written faster.
- Retrieved faster.
- Consume less system resources to do both operations mention above.

If you are agree (or not) with the 3 statement above can you please explain me, why, and in which case, using some SWAP would be an advantage?
That information is correct but the application of that information to the situation is not correct.

Notice how little free ram there is in the system we are discussing (in this thread). There is nothing inherently wrong with that. But it does indicate that questions about swap usage are not questions about whether pages will be in ram vs. disk; They are questions about which pages will be in ram vs. disk.

Most pages that could be resident in a typical process are either read/write/dirty anonymous or read-only non-anonymous (there are other possibilities, but for performance strategy those are the kinds to focus on).

I am intentionally ignoring some levels in between being resident for a process and being really non resident in physical memory: Either of the above kinds of pages can be bumped out of memory and when it is needed again will be read from disk.

A read/write/dirty anonymous page must be written to the swap area when it is bumped out and must be read back from swap when it is needed again. A read-only non-anonymous page is identical to an existing page at a known location on disk. So to bump it out, the kernel can just discard the copy in ram. To bring it back, the original disk location is reread.

So it costs half as much to bump out (and later read back) a non-anonymous page (which doesn't go to swap) compared to an anonymous page. So the system is biased in favor of bumping out non-anonymous pages (and thus not using swap).

But the whole theory of memory management is based on locality of access, meaning there is a strong correlation between how recently a page has been accessed and the probability it will be accessed again soon.

If you bump out a page that is accessed again soon, you are "wasting" the disk access to bring it back, because you need to bump out some other page to bring it back and bumping out a different page in the first place would have avoided that.

If a non-anonymous page were twice as likely to be accessed soon than a non-anonymous page, it would be better to bump the anonymous page even though doing so is ultimately twice the work per page bumped. It halves the probable number of pages bumped and the associated latency.

The system can only infer the probability of a page being reused soon from the length of time since that page was accessed. But in doing so the system should sometimes infer an anonymous page is less than half as likely to be reused soon as some non-anonymous page, so it should bump out that anonymous page to swap even though it doesn't strictly need to.

In concrete terms, large service processes have a moderate amount of data that is written only during startup, but kept around. Maybe it simply was too tricky to discard it and reuse the space. Maybe the data will be used again for cleanup operations when the service shuts down. But as that service sits around for many days waiting for work to do intermixed with doing work, that data never gets touched. That data is wasting ram that would be better used for cache. The system should be able to recognize that stale data (where it fills whole pages) and bump it to swap space. In a modern system with plenty of ram, it may take many hours of real system use before such data is stale enough to overcome a reasonable bias in favor anonymous pages. But the system whose stats we saw at the start of this thread clearly qualifies. The service processes are big enough and have been running long enough with free ram low enough that it is almost certain there stale anonymous pages in those processes that should have been recognized by the memory management system.

If you set swapiness to zero on such a system, the stale anonymous pages will stay in ram forever (we are assuming there is enough ram to never absolutely need swap space). Meanwhile some non-anonymous page gets bumped out (which is zero work) then needed again later and read back in (disk delay) causing a different non anonymous page to get bumped, then that one is later read back, and so on continually using more disk accesses than would be needed if the stale anonymous pages had been bumped out to swap and left there.

If I do a vmstat on one of my servers while it is fairly busy, I see swpd is 280MB but si and so are zero. That means 280MB of stale anonymous pages were long ago written to swap and zero of those pages are read back in a typical second. This system has plenty of ram, so I expect zero of those pages will ever be read back until those services are shut down.

Last edited by johnsfine; 11-10-2010 at 09:03 AM.
 
1 members found this post helpful.
Old 11-15-2010, 02:26 AM   #11
angel115
Member
 
Registered: Jul 2005
Location: France / Ireland
Distribution: Debian mainly, and Ubuntu
Posts: 528

Rep: Reputation: 78
Hi johnsfine,

Thank you for your reply. I rely appreciate that you took time to answer my question.

Although it took me some times to understand what you were saying, I think I do get it now.

But if I understand it well, the one which will benefit the most of this, will be servers that never shutdown/reboot.
Anyway, I still think that this kind of tuning have to be done by perfectionist as the performance benefit will be minimum.

Best regards,
Angel.
 
Old 12-01-2010, 03:33 AM   #12
fernfrancis
Member
 
Registered: Feb 2009
Location: Goa(India)-Sharjah(UAE)
Distribution: RHEL,centos,fedora,ubuntu
Posts: 229

Original Poster
Rep: Reputation: 18
hi guys

As i was saying earlier i have this application is was customised for our enviroment.

the problem i am facing is that, this application uses a lot of memory it starts with 2GB and then slowly as days pass the memory utilization increases and at one point of time the system hangs where in i have to reboot the server and the application is back to normal , the reason is that the memory is not being freed since we dont have a developer to assist me in debugging the code is there a way i can free the memory on regular basis without stopping the application.

the operating system is centos 5.5
 
Old 12-01-2010, 08:22 AM   #13
johnsfine
LQ Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,286

Rep: Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181Reputation: 1181
Rechecking some basic facts:

Quote:
Originally Posted by fernfrancis View Post
6GB ram, od is centos 5.5 64bit
You have 6GB of ram and a 64 bit kernel.

Is the problem application a 64 bit application or 32 bit?

Earlier you showed 12GB of swap space available and none used. Did that change? Especially when the problem application was near hanging?

Quote:
Originally Posted by fernfrancis View Post
one of my application on this server is eating a lot of memory that y i needed to check this . is there a way to check memory leak
Most application memory leaks in 64 bit applications provide significant opportunity for the kernel to swap out pages of the application without harming that application's performance. So if you have 12GB of swap space and a memory leaking 64 bit application and still no swap space is used, something strange is happening (maybe swapiness is set very low).

Quote:
Originally Posted by fernfrancis View Post
at one point of time the system hangs where in i have to reboot the server
That is very strange for a memory leak in one application. The system should slow way down once that application is using almost all of the 6GB of physical ram, but very slow should be noticeably different from hung.

Quote:
Originally Posted by fernfrancis View Post
since we dont have a developer to assist me in debugging the code is there a way i can free the memory on regular basis without stopping the application.
NO! You probably can stop/restart just the application (not the whole computer) to release the memory. But you CANNOT release the leaked memory without stopping the application and you almost certainly cannot fix the application to not leak memory without a competent developer.
 
Old 12-01-2010, 10:34 AM   #14
malekmustaq
Senior Member
 
Registered: Dec 2008
Location: root
Distribution: Slackware & BSD
Posts: 1,613

Rep: Reputation: 433Reputation: 433Reputation: 433Reputation: 433Reputation: 433
johnsfine, your two replies are more than just*fine*, they are very helpful.
 
Old 12-06-2010, 01:29 AM   #15
fernfrancis
Member
 
Registered: Feb 2009
Location: Goa(India)-Sharjah(UAE)
Distribution: RHEL,centos,fedora,ubuntu
Posts: 229

Original Poster
Rep: Reputation: 18
Hi Johnsfine

thanx a lot for your response

its a 64 bit application, yes i have 12gb of ram available

one the application reaches 5.3 gb of physical RAM then the application makes use of the swap memory
i have 2 swap partition each of 6gb, so when 6 gb of one swap is utilised then the system hangs

i cant login to the system so, the only option is to reboot the system then everything is back to normal

the application is written in c , it was written by one developer in the organisation who is not longer available with us.

if i use the command swapoff -a wait for a while and then swapon -a
will it affect the running of the application, this is a critical application where users are affected if there is a downtime
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
process, memory, swap, virtual memory wakatana Linux - Hardware 1 08-31-2009 08:55 AM
swap memory rehan999 Linux - Software 5 04-09-2008 08:45 AM
Why is 1 GB of my memory Swap? ThaHabbis Linux - Hardware 10 03-18-2008 08:28 PM
Swap Memory / Virtual Memory in Fedora Core 6 Jojo_CFT Linux - Newbie 2 10-15-2007 05:23 AM
Difference between Swap Virtrual memory and Swap Parition Nappa Slackware 4 11-27-2003 08:58 PM


All times are GMT -5. The time now is 05:49 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration