LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 03-02-2009, 12:21 PM   #1
Jinouchi
Member
 
Registered: Jul 2007
Location: WI, USA
Distribution: Debian, Lenny
Posts: 111

Rep: Reputation: 19
Excessive RAM/swap usage


For some reason, if I leave my Linux box running for several days, the swap space and RAM slowly fill up until my system is so slow that it takes around 15 seconds just to open a new tab if Firefox (Iceweasel, specifically). I have 512GB RAM and almost a gig of swap; how on earth does it fill up so much? Even if I close all my programs, there's still over 600MB swap used and all RAM is full. I've included a screenshot of 'top' running just about two minutes after I closed all my running programs. (Before I closed it, I had only 71MB swap free.)

I know that Linux is supposed to make good usage of RAM, but isn't this over the top? Is there a way to force it to use only required memory with no or little extras kept in RAM?

EDIT: Just thought I'd add in the fact that I'm running Xfce as opposed to KDE or GNOME in an attempt to have a smoother running system on my old hardware. Also, what's the "VIRT" column?
Attached Images
File Type: png res.png (14.9 KB, 20 views)

Last edited by Jinouchi; 03-02-2009 at 12:23 PM.
 
Old 03-02-2009, 01:06 PM   #2
johnsfine
Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,051

Rep: Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100
Quote:
Originally Posted by Jinouchi View Post
EDIT: Just thought I'd add in the fact that I'm running Xfce
I think the subject line of your thread should mention xfdesktop, so you will attract the attention of someone who might know why it is acting that way.

Quote:
Also, what's the "VIRT" column?
Virtual memory size. I think it includes "demand zero" and other forms of totally insignificant virtual memory use. It also includes memory mapped files, so it can be quite misleading. But it also is the place top reports the value that matters for your problem. So processes with a high value there should be investigated.

You can investigate details of a process's memory use with a command such as
Code:
less /proc/4691/maps
(the 4691 is the pid from the image you posted)
The results might help you understand the memory use or they might be too verbose and need some post processing. I expect there are tools to post process that, but I don't know them.

Ordinary memory use (the kind that fills up your swap file) shows up in that output with a 0 on the end of the line (instead of a big number followed by a file name), but I think "demand zero" memory (which your should ignore if you can) also has just a 0 on the end, so I'm not sure how you tell the difference.

The first two hex numbers on the line are beginning and end of that mapping. The size is the difference between those two numbers, which is one of the reasons some post processing would really help.

Last edited by johnsfine; 03-02-2009 at 01:26 PM.
 
Old 03-02-2009, 01:23 PM   #3
TBC Cosmo
Member
 
Registered: Feb 2004
Location: NY
Distribution: Fedora 10, CentOS 5.4, Debian 5 Sparc64
Posts: 355

Rep: Reputation: 43
I see you have a relatively long running sort using some ram. I would like to see a top without ntfs-3g, without sort, and with no iowait. And give it more that a couple minutes.
 
Old 03-02-2009, 01:52 PM   #4
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,965
Blog Entries: 11

Rep: Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865
I'm with cosmo on the NTFS thing ... what have you got mounted there,
why is it munching CPU like cookies?


Btw, you could have easily copy & pasted the text from the terminal
and posted it in code-tags rather than making a screen shot ;} ...



Cheers,
Tink
 
Old 03-02-2009, 02:37 PM   #5
johnsfine
Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,051

Rep: Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100
Quote:
Originally Posted by Tinkster View Post
I'm with cosmo on the NTFS thing ... what have you got mounted there,
why is it munching CPU like cookies?
That certainly looks strange in the top output. If you asked why the system is so slow, that might be a place I'd suggest looking.

But the question was about ram and swap use. Only the top three lines in the output of top have high enough values in VIRT to be significant parts of the problem.

In general, VIRT might be high for a process without that process consuming a lot of memory resource. VIRT might even be low and the process could be consuming a lot of ram indirectly by doing I/O that drives up file caching.

But VIRT cannot be low for a process consuming a lot of swap space. The amount of swap space a process is using is less than VIRT minus RES (how much less is hard to discover, but definitely less). Something there is consuming a lot of swap space. The only candidates are the three processes with high values for VIRT.

As for the system speed, I'm going to apply Occam's razor: The use of swap space is unreasonably high. That use of swap space most likely involves a major drain on system performance. It is the first place to look for the performance problem.

It is certainly possible for processes to use a lot of swap space without a significant performance cost. It is certainly possible the performance problems are in the ntfs process and entirely unrelated to the three processes that are filling the swap file. I just wouldn't look at ntfs first.

Last edited by johnsfine; 03-02-2009 at 02:42 PM.
 
Old 03-02-2009, 02:51 PM   #6
Tinkster
Moderator
 
Registered: Apr 2002
Location: in a fallen world
Distribution: slackware by choice, others too :} ... android.
Posts: 22,965
Blog Entries: 11

Rep: Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865Reputation: 865
Quote:
I just wouldn't look at ntfs first.
I would ;}

I'm not certain how ntfs-3g works, but I'd be concerned that, since no
processes consume anywhere near enough RAM to exhaust the resources
available on the system, ntgs-3g (maybe via way of a copy process?)
not only munches up CPU but also RAM in a non-transparent way.

And that's not occams razor, that's just my gut.


Cheers,
Tink
 
Old 03-02-2009, 03:06 PM   #7
johnsfine
Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,051

Rep: Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100
Quote:
Originally Posted by Tinkster View Post
no
processes consume anywhere near enough RAM to exhaust the resources
available on the system
Where do you get that claim?

686412KB of swap is in use. If a significant fraction of that is active areas, the swap file thrashing would overwhelm a 512MB system.

xfdesktop VIRT-RES = 330MB
xgce4-menu-plug VIRT-RES = 293MB
Xorg VIRT-RES = 110MB

Quote:
Originally Posted by Tinkster View Post
no
processes consume anywhere near enough RAM
The fact that no process has a high value in RES meaning they don't consume a lot of physical RAM, might just be a symptom of the RAM pressure itself. It does not mean those three processes aren't trying very hard to use a lot more RAM and just failing to get it.

In any case, it is unsound to run so close to swap area full. If there is a good reason for those three processes to use so much swap area and they aren't the performance problem (in other words, they have a lot of stale data they don't use much) then the swap area should be made larger so there will be room for all that stale data.

More likely, something is wrong with the memory use of those three processes.

Quote:
why is it munching CPU like cookies?
The system was up 3998 minutes, probably idle for most of them, but Xorg used 604 minutes of CPU time, and you want to know why ntfs is using so much CPU with its total of 10 minutes !??

Last edited by johnsfine; 03-02-2009 at 03:23 PM.
 
Old 03-02-2009, 05:55 PM   #8
lazlow
Senior Member
 
Registered: Jan 2006
Posts: 4,362

Rep: Reputation: 171Reputation: 171
Xorg uses a lot of cpu time just ideling (a little bit at a time), which is one of the reasons a lot of servers do not run X.

When you are using a lot of swap there are two basic possibilities. First that you have a rogue process of some sort, this (at least on the surface) does not appear to be the case. Second that you just need more ram. It looks like to me you have several processes constantly fighting over the ram. The fight over the ram is probably taking up more system resources than the work those processes are doing is. Ram is dirt cheap right now. Bump your ram up to at least 1GB. If you can you MAY be better off just maxing out the ram the motherboard supports. You could also "borrow" some ram to check out the possibilities.
 
Old 03-02-2009, 07:21 PM   #9
johnsfine
Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,051

Rep: Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100
Quote:
Originally Posted by lazlow View Post
Xorg uses a lot of cpu time just ideling
But that was 6 CPU seconds for every 40 seconds of up time! That can't be normal.

I expect my CPU is a bit faster than the OP's which should reduce that ratio for me. But my desktop is 1920x1200, which is a bit larger than typical and I think increases the Xorg CPU time, and my display has been pretty actively changing since boot up (which I think would use more CPU than idling).

Xorg used 5 seconds of CPU time just to start up. Since then it used another 20 seconds in 40 minutes of up time. That is 0.33 per 40, not 6 per 40. That 18 to one difference in rate of Xorg CPU use, is probably more than minor things like CPU speed.

I'm not nearly as sure something is excessive in the OP's Xorg CPU use as I am sure something is excessive in his xfce desktop memory use. But it does seem like his Xorg is working too hard and I suspect something excessive being done by that desktop is as fault there as well.
 
Old 03-09-2009, 09:39 AM   #10
Jinouchi
Member
 
Registered: Jul 2007
Location: WI, USA
Distribution: Debian, Lenny
Posts: 111

Original Poster
Rep: Reputation: 19
Wow, thanks for all the discussion!

MOST RAM IS dirt cheap, but unfortunately I run an older mobo that uses SDR-SDRAM which is still rather expensive. Besides, my mobo only supports 512 RAM anyway.

Well, I tried removing that ntfs line from /etc/fstab since I rarely use it anyway and can mount it by hand just fine. Since I removed that, I haven't noticed 'sort' running much anymore either. Unfortunately, it hasn't helped. In fact, yesterday my system actually froze up when it reached 0kb swap and 5Mb RAM, and I had to hard reboot.

I included another screenshot - it's from yesterday; I only today bothered thinking about pasting the text as Tinkster suggested, otherwise I would have done that instead. This one was taken before I closed anything - my system crashed right after I attempted to close everything. I still don't have any more ideas, but in the meantime I'll try using something other than Xfce in case that is causing a problem.
Attached Images
File Type: png res2.png (15.1 KB, 7 views)
 
Old 03-09-2009, 09:59 AM   #11
johnsfine
Guru
 
Registered: Dec 2007
Distribution: Centos
Posts: 5,051

Rep: Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100Reputation: 1100
Are you short of disk space as well? Why didn't you increase the amount of swap space?

Until you solve the software problem (if you ever can), it would be better to have the system slow down rather than lock up. If you had more swap area, it could do that.

You also don't seem to have done any investigation, such as looking at the /proc/pid/maps info I suggested. If you don't provide any more info, we can't give you any better advice.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Excessive RAM usage on boot - unnecessary modules? Jinouchi Linux - Newbie 17 01-15-2009 03:30 PM
ram and swap usage? FredericV Linux - General 6 03-25-2007 04:56 PM
After upgrade, excessive RAM usage kuratkull Debian 2 11-05-2006 04:26 AM
Odd swap usage even though there is free RAM Artanicus Linux - Hardware 5 09-28-2005 02:25 AM
RAM/SWAP usage ganja_guru Linux - Software 13 12-29-2004 03:17 AM


All times are GMT -5. The time now is 12:37 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration