Wipe previously used memory from a shutdown VM without shutting down the host

henly · 05-05-2023, 02:06 PM

I have a server that hosts a number of kvm/qemu VMs. I'd like to assure that after a VM is shutdown, sensitive information from the memory used by the VM is overwritten under the assumption that a memory dump or something could extract the information even after the VM is destroyed. This is sort of like defending against cold boot attacks but I don't want to shut down the whole server to assure things are cleared so the traditional defenses don't really apply.

I haven't been able to figure out how reuse of memory space when stopping and restarting a virtual machine would play out so to be safe I would have to assume that no memory would be reused so that potentially any available memory on the host could still contain the previous VMs state. Cold boot defenses like rebooting to memtest or running shmem or other wipers on shutdown of the VM couldn't really be relied upon based on the same assumption. Or maybe that isn't true? So overwriting all available memory after the VM is shutdown seems the only logical path.

In researching I've seen users suggest filling up 95% of memory and having the OMM kill the task to prevent the system from crashing. My system by default has no swap at all. I thought of enhancing this idea somewhat by temporarily enabling a swap file to prevent the crash and so that all available memory can be hit.

I guess I'm just looking for any blindspots in my thinking or better ideas

sundialsvcs · 05-05-2023, 03:01 PM

Others here may disagree with me, @henly, but I am quite confident that your concerns are unfounded.

When a virtual machine ceases to exist, all of its resources are released. Sensitive data might be here-or-there in physical memory "for the time being," but none of it is accessible. I do not consider the scenario that you are worrying about to be a credible threat.

henly · 05-05-2023, 03:44 PM

I'll be happy to be wrong on this topic! But by what measure is none of the previously used memory accessible? I'm aware of AMD supporting a method of memory encryption for virtual machines but my processor does not support it.

dugan · 05-05-2023, 06:26 PM

Get real. If you're hosting material that's actually sensitive (illegal?) enough that you're thinking about this, then you would smash that power button so fast that no-one would even see you move.

dugan · 05-05-2023, 06:43 PM

Some search hits I got that might interest you:

https://security.stackexchange.com/q...ory-space-afte

https://superuser.com/questions/8944...y-applications

The first link has a link to what you're asking for. Direct link:

https://en.wikibooks.org/wiki/Grsecu...l_freed_memory

henly · 05-05-2023, 09:43 PM

Thanks for the links @dugan The idea seems similar to a bash script I saw here:

https://stackoverflow.com/questions/...-snapshot-size

Quote:

#!/bin/bash

echo 3 > /proc/sys/vm/drop_caches
memfree=$(free -m | awk '/^Mem/ {print $4-32}')
if [ $memfree -gt 0 ]; then
dd if=/dev/zero of=/dev/null bs=${memfree}M count=1
fi

I guess my real question is now, how does swap behave in an OOM situation like this? Does linux start swapping out previously used blocks in an effort to maintain some available real memory (for performance) or does it exhaust the real and only then start using swap?

-----------------

This is adjacent to my question somewhat but this was an interesting discussion to me:
https://security.stackexchange.com/q...boot-clear-ram

This suggested that if a PC was running or simply suspended to ram that getting a dump of (almost) everything in ram was pretty trivial as you could just reboot into a custom OS that was designed for that purpose. I'd never thought about that before, my PC suspends to ram when not in use or I leave it on. I hardly ever reboot so all my bank passwords are probably just sitting there!

teckk · 05-06-2023, 02:44 PM

Quote:

This suggested that if a PC was running or simply suspended to ram that getting a dump of (almost) everything in ram was pretty trivial as you could just reboot into a custom OS that was designed for that purpose.

Code:

sudo head /dev/mem | hexdump -C
sudo cat /dev/mem | hexdump -C | less
dd if=/dev/mem of=ramfile

Edit:

Quote:

Does linux start swapping out previously used blocks in an effort to maintain some available real memory

Depends on where you have swappiness set to.

sundialsvcs · 05-08-2023, 12:55 PM

Quote:

... by what measure is none of the previously used memory accessible?

Here's a thirty-second primer on virtual memory. Every process runs in an environment where it appears to have "memory" exclusive to itself. But the reality is that hardware features are used to translate the addresses issued by the program to the (unknowable ...) actual location in physical-RAM where the information resides. If the information is not now present in physical RAM, an interrupt is generated which causes Linux to retrieve ("swap in") the requested information and make it available ... suspending the process until this has been done, at which time the process resumes execution, "none the wiser."

The information in question may have been "swapped out" to disk since it was "not recently used." In which case the data will be read ("swapped") back in. Or it may be that the location had never been referenced before, in which case Linux will allocate a physical memory page, set it entirely to zero, and then make it available.

Either way, the "memory" that is available to any process is always its purely-virtual picture. Data "left over" from any other process is never available. As far as any process is concerned, "it is the only game in town."

When actual "pressure" exists for the physical-RAM resource, Linux is constantly scanning for "not recently used" memory-pages that can be "swapped out" to disk to make room. In a crisis situation that should never happen, the "OOM killer = Out-of-Memory killer" can start killing-off processes to free up room.

A virtual-machine (VM) monitor uses the same mechanisms to manage the VMs.