Originally Posted by Simon Bridge
Swap may still be used a little bit, sometimes. swappiness=0 guarantees it is not used at all, ever.
False. swappiness controls the tradeoff between writing modified memory pages to swap and discarding (probably to reread later) unmodified pages. A swappiness of 0 just means the maximum possible number of unmodified pages will be discarded before any modified pages are written to swap.
A higher value lets the OS make a more effective tradeoff, so it can minimize the number of disk accesses needed to deal with any memory pressure and maximize the throughput of the system.
If you leave an interactive program idle for a while during which you run some background task that does a lot of file I/O, the system's normal behavior (to maximize throughput) would swap out the idle task to use more memory for file caching to speed up the task that is still running. When you switch back to that interactive task it will be sluggish for a short time as it recovers its data from disk.
If you do things like that a lot (run file I/O intensive background jobs while you leave interactive tasks idle) and you are bothered by the sluggishness when switching back to the interactive task, you might want to set swappiness way down to sacrifice the throughput of that background task in favor of faster restarts of the interactive task.
But most people should leave swappiness alone rather than lower it based on some misunderstanding of memory management.