LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 11-25-2007, 10:02 AM   #1
dralexpe
LQ Newbie
 
Registered: Jun 2007
Posts: 27

Rep: Reputation: 15
System uses large amounts of memory and does not release it


I am encountering a situation where large amounts of memory are used and not released.

The system is Centos 5 x86_64 with 8Gb RAM, two hard drives of 500 Gb each, using a single volume with LVM, filesystem is ext3, swap space is 2Gb (for some reason when I installed Centos it wouldn't let me allocate more than that, although I had plenty of disk space). I am using VMware to run XP within it and I reserved about 4Gb RAM for the virtual machine, yet when I am monitoring the system resources with ksysguard I see the used memory goes up to almost the whole 8Gb, with only about 40kb left. I thought it was a VMware server 1.0.4 problem, but the same thing appeared when I copied a virtual machine with a disk of 80 Gb. The command I used was cp -ax source destination, and it took more than an hour. The system was almost brought to its knees, it was barely responding during that time, which I can understand, but what puzzled me was that after the operation has been successfully completed the memory appeared still in use, as if it wasn't released. Logging off and then back on did not release that used memory.

Since I am new to Linux, I have installed kind of everything from the installation CD's, but still the processes that are running do not seem to use that much memory.

Any hints would be appreciated.
Thank you.
 
Old 11-25-2007, 10:23 AM   #2
pusrob
Member
 
Registered: Jan 2006
Distribution: Debian
Posts: 463

Rep: Reputation: 35
Hi.
You mean, that there is no much free, absolutely unused memory. Right? Well, it is because linux puts almost the whole amount of memory into cache, so it can be accessed faster. A GNU/Linux OS is well-configured, when there is no much free memory left.
To be sure that the memory is cached, and not used by something else, type in a terminal this: free -h This will show the memory usage in a table. Post the results here.

Last edited by pusrob; 11-25-2007 at 10:24 AM.
 
Old 11-25-2007, 02:00 PM   #3
reddazz
Guru
 
Registered: Nov 2003
Location: N. E. England
Distribution: Fedora, CentOS, Debian
Posts: 16,298

Rep: Reputation: 73
This article explains more about how Linux memory management works, so you may find it useful.
 
Old 11-25-2007, 02:33 PM   #4
jay73
Guru
 
Registered: Nov 2006
Location: Belgium
Distribution: Ubuntu 11.04, Debian testing
Posts: 5,019

Rep: Reputation: 130Reputation: 130
Quote:
swap space is 2Gb (for some reason when I installed Centos it wouldn't let me allocate more than that, although I had plenty of disk space).
swap partitions normally can't be greater than 2GB. If you need more, you have to make multiple swap partitions.

Quote:
The system was almost brought to its knees,
Nothing to do with memory. Copying data involves far more hardware components than just memory. The usual bottlenecks in such a case include disk controllers and buses. It also depends on the type of file system. Some tend to monopolize your hardware during intensive reads and writes, other ones are more modest.

Last edited by jay73; 11-25-2007 at 02:38 PM.
 
Old 11-25-2007, 03:30 PM   #5
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 12,130

Rep: Reputation: 986Reputation: 986Reputation: 986Reputation: 986Reputation: 986Reputation: 986Reputation: 986Reputation: 986
In addition to that lot, how many (actual) CPUs do you have ??? (don't say just one ...).

@jay73, I thought at some point Redhat had released a patch to enable the (x86) swap extents to be bigger. Don't know about _64, and I don't use RH/Centos myself so I haven't checked.
 
Old 11-25-2007, 05:29 PM   #6
jay73
Guru
 
Registered: Nov 2006
Location: Belgium
Distribution: Ubuntu 11.04, Debian testing
Posts: 5,019

Rep: Reputation: 130Reputation: 130
http://kbase.redhat.com/faq/FAQ_80_7794.shtm

Yes, you appear to be right. But obviously it would still be better (in terms of performance ) to create multiple swap partitions and spread them out out over multiple disks.
 
Old 11-26-2007, 09:31 PM   #7
dralexpe
LQ Newbie
 
Registered: Jun 2007
Posts: 27

Original Poster
Rep: Reputation: 15
Thank you all for all the information.

I have tried to see the output of free -ml the way pusrob has suggested and here is what I've got:


total used free shared buffers cached <--- this is right after boot
Mem: 7982 691 7291 0 37 378
Low: 7982 691 7291
High: 0 0 0
-/+ buffers/cache: 274 7707
Swap: 1919 0 1919

[dralexpe@new-screamer ~]$ free -ml <---this is with VMware running, and watching a movie in Windows XP
total used free shared buffers cached
Mem: 7982 5453 2529 0 44 5036
Low: 7982 5453 2529
High: 0 0 0
-/+ buffers/cache: 372 7610
Swap: 1919 0 1919

[dralexpe@new-screamer ~]$ free -ml <---this is after I shutdown XP and closed VMware
total used free shared buffers cached
Mem: 7982 1900 6082 0 45 1560
Low: 7982 1900 6082
High: 0 0 0
-/+ buffers/cache: 294 7688
Swap: 1919 0 1919

[dralexpe@new-screamer ~]$ free -ml <-- just a Firefox instance open
total used free shared buffers cached
Mem: 7982 1951 6031 0 46 1581
Low: 7982 1951 6031
High: 0 0 0
-/+ buffers/cache: 323 7659
Swap: 1919 0 1919

[dralexpe@new-screamer ~]$ free -ml<-- just a Firefox instance open
total used free shared buffers cached
Mem: 7982 1964 6017 0 46 1581
Low: 7982 1964 6017
High: 0 0 0
-/+ buffers/cache: 336 7646
Swap: 1919 0 1919
[dralexpe@new-screamer ~]$

From what I understand this should be pretty much expected behavior.

To answer syg00 I have 2 CPU's, Intel Xeon X5355 Quad Core @ 2.66GHz on a Tyan S2696 mobo.

The filesystem is ext3.

Thanks again.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Difference between resident memory,shared memory and virtual memory in system monitor mathimca05 Linux - Newbie 1 11-11-2007 04:05 AM
Kernel panics when trying to transfer large amounts of data from or to my hardrive CuriouserAndCur Debian 3 01-10-2007 11:53 AM
how to limit swapping - prevent processes allocating huge amounts of memory david@linuxquestions Linux - General 10 12-21-2006 07:26 AM
Using wget to copy large amounts of data via ftp. AndrewCAtWayofthebit Linux - General 1 05-11-2006 11:55 AM
rm command is choking on large amounts of data? Jello Linux - General 18 02-28-2003 07:11 PM


All times are GMT -5. The time now is 04:56 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration