Swap Partition Ruining PC Performance?
 

I used my machine normally for 2 days and then I see this:

Then I launched a VBox machine (restore not fresh bootup) and my Xfce desktop becomes virtually unusable for 3 solid minutes (the mouse barely responds a window take 20 seconds to come to forefront if I click on one) while the VBox machine comes up and the disk is audibly grinding away that entire time. Once the machine was up:

To get the machine to restore and then close it (save state) straight away took nearly 5 minutes total. I also noticed that once it was up and ready to use (i.e. the 99% thing went away) the disk continued to grind for another 20-30 seconds. Even after I saved it, the noise carried on 10 seconds after it appeared to be finished.

Then I open two news sites in Chromium, each in a fresh tab, and the disk again grinds for 20 seconds straight until both are loaded. Granted, they're heavy pages, but they're just Internet pages. Once they're loaded, it's quiet and the CPU is totally quiet.

So then I hard rebooted my machine, got back into my Xfce desktop, and launched no apps whatsoever except one terminal to get this:

Tons of free RAM. I launched that same VBox machine (restore) and it came up in 14 seconds and with very little noise. Shut down (save state) in 15 seconds. I launched it again (14 seconds) and then got this:

I closed it and launched Chromium and opened those 2 news sites. They opened in 10 seconds and there was a bit of noise.

My partitions are all mostly empty:

My fstab is OK I think:

I wonder if all this points to a problem with my swap partition maybe? The VBox performance with and without a lot of unused RAM is strikingly different...

I'm not quite sure what you're asking.

You have 3 GiB of RAM.
You are using VirtualBox to virtualise a guest Operating System.
VirtualBox assigns a large section of RAM to the virtual machine by reserving the configured amount all at once.
Your host OS runs out of RAM and starts using swap.
Your hard disk read/write performance is awful when compared to RAM read/write performance.
All Operating System tasks and applications that are loaded into swap, or are accessing data in swap take a lot longer to process.
Also since your swap space and your root partition are both on the same disk, the same read/write heads are coping with not just the excess RAM load, but also with reading and writing to the root filesystem.

This is all normal. Get more RAM, or assign less RAM to VirtualBox.

Or to state it another way ...
Swap partition(s) don't affect performance, using the swap partition(s) is what hurts. If you don't swap, it don't matter.

Every performance/tuning problem I've ever seen can be fixed by more memory - somewhere; central, {L1/2/3/4} cache, disk controller cache, GPU, whatever.

Also see:
http://linuxpoison.blogspot.com/2009...linux-for.html

What was running at that point? That looks like quite a lot of memory use with very little buffer and cache use.

You should use top (or similar tool) and sort on the RES column to get a first estimate of which processes are using a lot of memory (ignore VIRT and SWAP columns in top, they are less meaningful than you might think).

Many application program problems can cause X to leak, and some versions of X leak due to their own bugs (rather than due to flaws in applications using them). You haven't given enough info to support a real diagnosis, so I am just making a guess based on experience, but everything you have reported seems to fit the behavior one would expect from a memory leak in X.

The big memory stress of launching VBOX is OK when everything else is using no significant amount of memory, but when your memory is already heavily used, launching VBOX is a performance disaster.

Thank you everyone for your comments. To explain, I have sort of jumped into the middle of a saga here. I posted my VBox test to clearly illustrate my problem, which is that my disk makes crazy noise a LOT and this PC does not perform as well as I think it should. It runs Arch Linux and Xfce and has "Intel(R) Core(TM) i5 CPU 650 @ 3.20GHz" and "Hitachi 1 TB sata 7200 RPM" and should be blazing fast, I would think.

I have a tool called ps_mem from http://www.pixelbeat.org/scripts/ which shows which processes use the most RAM. Two weeks ago, I open a 100MB file in mousepad and got this:

Which is crazy I think. 800 M for a 100 M file?

I tried opening the same file in gedit and I got a message at the top of gedit (after 20 or 30 seconds of it being blank) that it's "LOADING file.sql..." with a progress bar. I let it go 3.5 minutes, it got to less than half finished, and then I closed it.
Before I closed it, I got this:

To be 100% clear, this is the file I am talking about, as viewed by "ls -l" :

It's a 100 M file!

Before I closed gedit, my PC was almost unusable--it was responding very slowly to clicks and apps were very slow to render.

I'm a bit confused by all this. Something is wrong here and so I came up with my VBox test to demonstrate the problem.

zordak, your point

makes good sense. However, I installed zramswap and so now I see 9 G Swap. Anyhow right now I see:

no swap used and tons of free RAM. I open those two news sites in new tabs in Chromium and I hear the disk grinding.

I have an Antec Three Hundred Mini-Tower which is supposed to be a silent case. That means, I think, that my disk is super loud. I even replaced it right after I bought this machine a few months ago b/c I thought it was so loud. Same make/model. This one, the replacement, is better--it's quieter and the performance is better, but it still has these problems.

I realize that Firefox is a RAM hog and I do often restart it just to get more RAM back, but still, it shouldn't make my system SO slow.

Overall, anyhow, I have no explanation for why 800 M are needed for a 100 M file, nor why the disk should grind for a news site in Chromium, when there is plenty of free RAM for such.

However, the idea from zordak is the only one I have yet heard (I have posted about this issue on another forum) that really makes sense. I suppose I could put in a spare drive I have and use that for swap and see. But the issues in my previous paragraph I don't think will be fixed by that.

If the disks heads must go back and forth between the swap partition and the regular partition, the PC will make a lot of noise and run slow.

You could get more memory, or use less memory, or get a different disk for swap.

So where would I get any hint what those numbers mean? I know what the various memory numbers in top mean (numbers most Linux users don't actually understand). But that just makes me more aware of the wide range of possibilities for memory use measurements.
Does it have a lot of short lines?
I never used mousepad, but a typical text editor locates all the line boundaries in a text file. If the lines are short, the data needed to remember where all the line boundaries are might be much larger than the total data in the file.

Something else is seriously wrong with this system. Looking at everything you have posted so far gives me a new guess at that. You made some earlier attempt to adjust Linux settings to improve performance, misunderstood what you were changing, and wrecked the performance.

My more specific guess is swappiness. Your system seems to act as if swappiness is set way too low. I'm guessing you set it to zero. That does decrease the use of swap space, but by greatly increasing the number of reads needed from executable files. Even while it managed to suppress use of swap space, it was already slowing the system. But once it isn't enough to suppress use of swap space, it may be a disaster as every read or write with swap must be alternated with reading executable files at the other end of the same disk.

BTW, you might be able to get more ram simply by enabling it. You seem to have more than 3GB but far less than 4GB, which almost always means you have 4GB of ram physically installed but only have 32bit physical addressing, which only supports 3 and a fraction GB of ram.
If the BIOS has enabled more ram, then you can switch to a PAE kernel to use that additional ram in 32 bit Linux. If the BIOS hasn't enabled the extra ram, that may be harder to solve, but still worth the effort. If the motherboard is limited to 32bit physical addressing then the problem can't be fixed. I don't know how to get a PAE kernel in Arch Linux. If that is a problem for you, search that question or start a new thread to ask it.

All I know is what it shows as the last line:

That is also the first line of the output of this Python script. The URL I posted above has the source code.

Finally, someone commiserates and agrees with me that something is wrong here! :)

I was suggested to play with this setting but I never did. I see now:

which I don't think could be too low. Let me know if you disagree.

You are a genius. I noticed that I posted elsewhere once that I have 4 GB of RAM. I forgot about that--I think I do! I am working now on getting PAE running--I do the hardware for it. Hopefully this will help.

The only other suggestion that makes sense thus far, as far as I can tell, is to use a separate disk for swap. I will also work on that.

Thank you!

I am assuming this is a desktop and not a laptop or netbook.

Besides "swappiness" possible causing this, I would also check to make sure the drive is not failing or it isn't some type of firmware bug where the drive spins itself down.

So, post the output of:

I have checked the Smart tests many times. They always come back good:

It's also not an inherently slow disk:

Yeah, the drive seems normal in term of performance, and it is not failing.

Yes, this is a desktop machine, with excellent specs.

So the only two suggestions which seem usable thus far are:

1. Get PAE kernel. This, however, should only provide another .5 G RAM and so will only help the problem, not fix it.

2. Put swap partition on a separate physical drive. I am going to try this as soon as my new USB disk arrives. I realize USB 2.0 is slower than a SATA disk, but it's an easy way to test this hypothesis--the hard drive shouldn't grind anymore anyhow. If it's better, I will put in a real drive from another machine and put swap there and see if it gets even better.

Are there are any other ideas? I feel that something is still not right here, because many if not most desktop users (and probably virtually all laptop users) indeed only have one physical drive and so their swap is on that drive.

I think I have clearly configured something wrong on this machine. But I have no idea what.

I don't have a swap partition, because when it kicks in, it does ruin performance.

However, there is this experimental option:
http://www.webupd8.org/2011/10/incre...inux-with.html

Oh I have zswapram. My physical swap space is 6 G. Yet free shows 9 G:

I haven't noticed that it helps yet.

Perhaps the bottom line is that my work habits require more than 3.5 G RAM and that's it.

My guess about swappiness was wrong. You didn't (and shouldn't) mess with it.

In the info you posted, the amount in cache is very low, relative to fully free memory, and even more so relative to really used memory. I have been guessing that is a connective detail (between cause and effect) of some serious performance problem. But other than swappiness, I can't think of a plausible cause.

When cache use is low and there is significant memory pressure, one would expect the system to constantly page in code from the original main executables and .so files. That seems to fit the overall performance symptoms. But I don't see any explanation for cache being so low. If almost all the memory use were anonymous (programs using very little code but a lot of heap and/or stack) the memory stats could look as they do without the low cache being a major performance factor. But that isn't as good a fit for the described performance symptoms.

That seems to be a big part of the problem, maybe a very big part. So why is Firefox a RAM hog and why does restarting it make a big difference? Two theorys seem obvious: 1) Firefox might have a memory leak, so you really need to restart it often to keep it from dominating your ram. 2) Firefox has user settable parameters for how much ram and how much hard drive space is used for its caching of web content and you have the ram parameter set way too high. Your hard drive should be much faster than your internet connection and generally fast enough for your display, so web content should be mostly cached on disk and not in ram.