For some reason, if I leave my Linux box running for several days, the swap space and RAM slowly fill up until my system is so slow that it takes around 15 seconds just to open a new tab if Firefox (Iceweasel, specifically). I have 512GB RAM and almost a gig of swap; how on earth does it fill up so much? Even if I close all my programs, there's still over 600MB swap used and all RAM is full. I've included a screenshot of 'top' running just about two minutes after I closed all my running programs. (Before I closed it, I had only 71MB swap free.)

I know that Linux is supposed to make good usage of RAM, but isn't this over the top? Is there a way to force it to use only required memory with no or little extras kept in RAM?

EDIT: Just thought I'd add in the fact that I'm running Xfce as opposed to KDE or GNOME in an attempt to have a smoother running system on my old hardware. Also, what's the "VIRT" column?

Attached Thumbnails

 

I think the subject line of your thread should mention xfdesktop, so you will attract the attention of someone who might know why it is acting that way.

Virtual memory size. I think it includes "demand zero" and other forms of totally insignificant virtual memory use. It also includes memory mapped files, so it can be quite misleading. But it also is the place top reports the value that matters for your problem. So processes with a high value there should be investigated.

You can investigate details of a process's memory use with a command such as
(the 4691 is the pid from the image you posted)
The results might help you understand the memory use or they might be too verbose and need some post processing. I expect there are tools to post process that, but I don't know them.

Ordinary memory use (the kind that fills up your swap file) shows up in that output with a 0 on the end of the line (instead of a big number followed by a file name), but I think "demand zero" memory (which your should ignore if you can) also has just a 0 on the end, so I'm not sure how you tell the difference.

The first two hex numbers on the line are beginning and end of that mapping. The size is the difference between those two numbers, which is one of the reasons some post processing would really help.

I see you have a relatively long running sort using some ram. I would like to see a top without ntfs-3g, without sort, and with no iowait. And give it more that a couple minutes.

I'm with cosmo on the NTFS thing ... what have you got mounted there,
why is it munching CPU like cookies?


Btw, you could have easily copy & pasted the text from the terminal
and posted it in code-tags rather than making a screen shot ;} ...



Cheers,
Tink

That certainly looks strange in the top output. If you asked why the system is so slow, that might be a place I'd suggest looking.

But the question was about ram and swap use. Only the top three lines in the output of top have high enough values in VIRT to be significant parts of the problem.

In general, VIRT might be high for a process without that process consuming a lot of memory resource. VIRT might even be low and the process could be consuming a lot of ram indirectly by doing I/O that drives up file caching.

But VIRT cannot be low for a process consuming a lot of swap space. The amount of swap space a process is using is less than VIRT minus RES (how much less is hard to discover, but definitely less). Something there is consuming a lot of swap space. The only candidates are the three processes with high values for VIRT.

As for the system speed, I'm going to apply Occam's razor: The use of swap space is unreasonably high. That use of swap space most likely involves a major drain on system performance. It is the first place to look for the performance problem.

It is certainly possible for processes to use a lot of swap space without a significant performance cost. It is certainly possible the performance problems are in the ntfs process and entirely unrelated to the three processes that are filling the swap file. I just wouldn't look at ntfs first.

I would ;}

I'm not certain how ntfs-3g works, but I'd be concerned that, since no
processes consume anywhere near enough RAM to exhaust the resources
available on the system, ntgs-3g (maybe via way of a copy process?)
not only munches up CPU but also RAM in a non-transparent way.

And that's not occams razor, that's just my gut.


Cheers,
Tink

Where do you get that claim?

686412KB of swap is in use. If a significant fraction of that is active areas, the swap file thrashing would overwhelm a 512MB system.

xfdesktop VIRT-RES = 330MB
xgce4-menu-plug VIRT-RES = 293MB
Xorg VIRT-RES = 110MB

The fact that no process has a high value in RES meaning they don't consume a lot of physical RAM, might just be a symptom of the RAM pressure itself. It does not mean those three processes aren't trying very hard to use a lot more RAM and just failing to get it.

In any case, it is unsound to run so close to swap area full. If there is a good reason for those three processes to use so much swap area and they aren't the performance problem (in other words, they have a lot of stale data they don't use much) then the swap area should be made larger so there will be room for all that stale data.

More likely, something is wrong with the memory use of those three processes.

The system was up 3998 minutes, probably idle for most of them, but Xorg used 604 minutes of CPU time, and you want to know why ntfs is using so much CPU with its total of 10 minutes !??

Xorg uses a lot of cpu time just ideling (a little bit at a time), which is one of the reasons a lot of servers do not run X.

When you are using a lot of swap there are two basic possibilities. First that you have a rogue process of some sort, this (at least on the surface) does not appear to be the case. Second that you just need more ram. It looks like to me you have several processes constantly fighting over the ram. The fight over the ram is probably taking up more system resources than the work those processes are doing is. Ram is dirt cheap right now. Bump your ram up to at least 1GB. If you can you MAY be better off just maxing out the ram the motherboard supports. You could also "borrow" some ram to check out the possibilities.

But that was 6 CPU seconds for every 40 seconds of up time! That can't be normal.

I expect my CPU is a bit faster than the OP's which should reduce that ratio for me. But my desktop is 1920x1200, which is a bit larger than typical and I think increases the Xorg CPU time, and my display has been pretty actively changing since boot up (which I think would use more CPU than idling).

Xorg used 5 seconds of CPU time just to start up. Since then it used another 20 seconds in 40 minutes of up time. That is 0.33 per 40, not 6 per 40. That 18 to one difference in rate of Xorg CPU use, is probably more than minor things like CPU speed.

I'm not nearly as sure something is excessive in the OP's Xorg CPU use as I am sure something is excessive in his xfce desktop memory use. But it does seem like his Xorg is working too hard and I suspect something excessive being done by that desktop is as fault there as well.

Wow, thanks for all the discussion!

MOST RAM IS dirt cheap, but unfortunately I run an older mobo that uses SDR-SDRAM which is still rather expensive. Besides, my mobo only supports 512 RAM anyway.

Well, I tried removing that ntfs line from /etc/fstab since I rarely use it anyway and can mount it by hand just fine. Since I removed that, I haven't noticed 'sort' running much anymore either. Unfortunately, it hasn't helped. In fact, yesterday my system actually froze up when it reached 0kb swap and 5Mb RAM, and I had to hard reboot.

I included another screenshot - it's from yesterday; I only today bothered thinking about pasting the text as Tinkster suggested, otherwise I would have done that instead. This one was taken before I closed anything - my system crashed right after I attempted to close everything. I still don't have any more ideas, but in the meantime I'll try using something other than Xfce in case that is causing a problem.

Attached Thumbnails

 

Are you short of disk space as well? Why didn't you increase the amount of swap space?

Until you solve the software problem (if you ever can), it would be better to have the system slow down rather than lock up. If you had more swap area, it could do that.

You also don't seem to have done any investigation, such as looking at the /proc/pid/maps info I suggested. If you don't provide any more info, we can't give you any better advice.