Upgrading to VMWare Virtual Machine Hardware version 7.0 steals available memory
Linux - VirtualizationThis forum is for the discussion of all topics relating to Linux Virtualization. Xen, KVM, OpenVZ, VirtualBox, VMware, Linux-VServer and all other Linux Virtualization platforms are welcome. Note that questions relating solely to non-Linux OS's should be asked in the General forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
Upgrading to VMWare Virtual Machine Hardware version 7.0 steals available memory
Our team recently updated each of our esx hosts to 4.0. However, we *did not* immediately upgrade the Virtual Machine Hardware version. We ran 4.0 for quite some time with no issues. After powering down the VM, upgrading to Virtual Machine Hardware v7.0, and powering back on we discovered that our VM had mysteriously "lost" ~ 800MB of memory. I've pasted data from two VMs below to illustrate this point.
This first VM is running Virtual Machine Hardware v4.0 and is reporting correct memory numbers all around.
This second VM is missing memory. It is running Virtual Machine Hardware v7.0 (all else being equal). The memory that is reported is inconsistent and dependent on the source.
What is interesting is that it looks like the BIOS is providing a memory map that includes address space which goes above and beyond the 4GB mark (0000000100000000). From what I understand, this is typical for instances wherein memory is being "remapped" by the BIOS. I also understand that the OS should be able to use this newly addressed space without having to go to a PAE kernel. What's confusing is that the the amount of memory that has been remapped is *not* equal to the amount of missing memory. There looks to be several factors involved
Distribution: Debian PPC/i386/AMD64 5.0(Lenny), Vista, XP , WIN7, Server 03/08
Posts: 1,164
Rep:
I am making an assumption here, but I have seen some newer Virtualization products start to use dynamic memory allocation, maybe it is somehting like that?
We investigated ESX memory management to correct an issue we had a while back. As far as I know, ESX will dynamically allocate and reclaim memory using some pretty cool algorithms that depend on several variables. The balloon driver is particularly impressive. However, it's my understanding that all of this is managed at the hypervisor level. As far as the guest OS is concerned, it's convinced that its always had whatever memory it was originally configured with (ie. VM settings). That is, it has no visibility into resource management at the hypervisor level. I also think the dmseg output occurs long before esx has had a chance to invoke its memory management process. That is, the memory goes missing right out of the gate.
----clip----
Solution
The PC architecture reserves a portion of the address space that is below 4GB for PCI devices. This space cannot be used for system memory. Also, note that this is one of the main reasons for guests not recognizing all allocated physical memory.
This table lists the address space size available for system memory under 4GB for each hardware version.
Hardware Version Reserved Memory Supported Memory
7 1024MB 3GB (3072MB)
6 512MB 3.5GB (3584MB)
4 256MB 3.75GB (3840MB)
3 496MB* 3.52GB (3600MB)
----clip----
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
Upgrading to VMWare Virtual Machine Hardware version 7.0 steals available memory
