Understanding the Kernel through virtualization

gilius · 02-15-2019, 03:38 PM

I am still considered a newbie at Linux and trying to make sense of this world. One glaring thing is that most distributions have virtualization turned off, and apparently, it's to do with security.

For virtualization you need to have a "KVM-enabled kernel", i.e. KVM appears under /dev/kvm. But if this is missing then how exactly do we "enable" this on any Linux distro? On Windows we simply go to the bios and turn it on there, but for Linux it's different kettle of fish entirely.

One guy who maintains 2 different Linux distributions of his own shockingly did not seem to know anything about it when I pointed out the lack of virtualization/KVM, and his answer was:
"KVM: I don't know about that, but the kernel component of KVM is included in mainline Linux, as of 2.6.20."

Not much help as you can see, but I did manage to get this answer:
"Unfortunately, KVM is not enabled in the shipped gentoo-on-rpi3-64bit kernel, as it uses the "vanilla" upstream bcmrpi3_defconfig (which omits this feature). But, it isn't hard to create a custom binary kernel package supporting this. What exactly do you need turned on for your use case, CONFIG_KVM, CONFIG_VHOST_NET and CONFIG_VHOST_CROSS_ENDIAN_LEGACY?"

How exactly do we create such a kernel package and how does that fit in with the rest of the distro? Specifically, can you find the appropriate section for the above inside this book:
http://www.linuxfromscratch.org/
? That would really be a big help for me if you can.

Or is there another academic source I can consult to learn more about this and how it fits into the big picture? I feel that if I cannot understand how virtualization can be enabled in any Linux distro then something is terribly amiss.

berndbausch · 02-17-2019, 12:33 AM

I don't know about Gentoo, but the distros that I have worked with (OpenSuse/SLES, Ubuntu, Centos/RHEL) only require installing the software, that is the KVM kernel modules as well as management software like libvirt. You can do this by checking a box during the installation process, or by a one-line command on the running system.

The kernels that ship with those distros don't require any special configuration. /dev/kvm appears if the CPU has virtualization extensions.

syg00 · 02-17-2019, 01:05 AM

I've been holding off commenting so this could stay on the zero-reply list and keep getting bumped for others to read.
There are several aspects that are of concern. I too have never had an issue with KVM not being available, although I use "techo" distros - for instance here are the applicable options on Fedora 26 - 3 releases old.

Code:

CONFIG_KVM_GUEST=y
# CONFIG_KVM_DEBUG_FS is not set
CONFIG_VHOST_RING=m
CONFIG_PTP_1588_CLOCK_KVM=m
CONFIG_DRM_I915_GVT_KVMGT=m
CONFIG_HAVE_KVM=y
CONFIG_HAVE_KVM_IRQCHIP=y
CONFIG_HAVE_KVM_IRQFD=y
CONFIG_HAVE_KVM_IRQ_ROUTING=y
CONFIG_HAVE_KVM_EVENTFD=y
CONFIG_KVM_MMIO=y
CONFIG_KVM_ASYNC_PF=y
CONFIG_HAVE_KVM_MSI=y
CONFIG_HAVE_KVM_CPU_RELAX_INTERCEPT=y
CONFIG_KVM_VFIO=y
CONFIG_KVM_GENERIC_DIRTYLOG_READ_PROTECT=y
CONFIG_KVM_COMPAT=y
CONFIG_HAVE_KVM_IRQ_BYPASS=y
CONFIG_KVM=m
CONFIG_KVM_INTEL=m
CONFIG_KVM_AMD=m
CONFIG_KVM_MMU_AUDIT=y
CONFIG_VHOST_NET=m
CONFIG_VHOST_SCSI=m
CONFIG_VHOST_VSOCK=m
CONFIG_VHOST=m
# CONFIG_VHOST_CROSS_ENDIAN_LEGACY is not set

Those that are "not set" are generally not required - unless you are are a KVM dev.

The linuxfromscratch "book" is not an academic introduction to Linux for new users - it is a technical how-to for building Linux from the very basic source code. As the link states in the first sentence.
Probably not the place to start your Linux journey, although I know folks that did.

gilius · 02-17-2019, 06:01 AM

Thanks for your replies!

I can confirm that 99% of distros of the ARM64 architecture DO NOT have a KVM-enabled kernel; this includes 99% of Android-based Linux kernels - though most x86/x64 Intel-based distros did not pose this problem, such as Ubuntu. However, the moment you switch to Ubuntu on the Raspberry Pi or some such then KVM/virtualization is disabled!

So where might I start on this journey? I am lacking in a lot of fundamentals here, as you can probably tell:
1) Where/how would we change those parameters?
2) Do we have to recompile the kernel before those features can be used in a distro or can we just text-edit a file and then restart the distro?
3) In other words: how can we manage those changes to the kernel - and can it be dealt with separately from the distro - or does the whole distro have to be "redone"?
4) Which academic source(s) would you suggest I consult - even if it means learning a new programming language?

I want to have an understanding of:
*How the kernel and distro work together
*How I can make changes to the kernel based on this example of virtualization

Any help would be much appreciated!

syg00 · 02-17-2019, 07:23 PM

OK, in future if you are asking about ARM, say so - in the subject header or early in the initial post.
To quote the KVM home page, ARM is a "work in progress". Virtualisation needs hardware support - even in the x86 world it took a while for both Intel and AMD to add it. ARM has been the wild west of computing for years - each manufacturer does its own thing trying to scrape out its own piece of turf. Has taken quite a while to even get enough consensus to have a decent accepted branch in the kernel tree - it all used to be separately "supported" (for want of a better word) by each manufacturer.
After that, somebody has to be interested enough to port all the necessary changes across to the ARM architecture and test/support it. This takes time - for any feature, not just KVM.
Looks like it's getting there - Redhat appear to have it as a technology preview (use it at your own risk) on their latest enterprise releases. Probably means Fedora will have it too, but I haven't looked.

As for recommendation for a good book on where to start, I'll leave that to others - I picked it up through osmosis over the years.

In general you shouldn't need to (re-)configure the kernel; distros usually provide one that is generic enough for most day-to-day uses. The kernel provides a mechanism that allows functionality to be added after boot, as modules that can be loaded and are then treated as part of the kernel. This can also be dynamic, so if you plug in some USB thingy, the system will recognise it needs to load the module to support it, and will do so, and your thingy will then become usable. All without touching the kernel.

gilius · 02-18-2019, 02:08 PM

syg00, I don't think you have it quite right re: ARM. All Cortex ARM64 CPU chips have virtulization capabilities built into them, so the only thing lacking is the "KVM-Enabled Kernel". If the ARM devs were adapting a kernel from x86/x64 to ARM presumably they would copy and paste the kernel and modify where necessary, so I would guess that they have deliberately disabled the KVM or removed the feature entirely from the Kernel flags? Note: I don't really know much about how the kernel works yet...

Quote:

Originally Posted by syg00

The kernel provides a mechanism that allows functionality to be added after boot, as modules that can be loaded and are then treated as part of the kernel. This can also be dynamic, so if you plug in some USB thingy, the system will recognise it needs to load the module to support it, and will do so, and your thingy will then become usable. All without touching the kernel.

I'm not so sure this is possible either, without recompiling the kernel each time? It's a nice thought though - I wish it could be true! There were some commands I had to run to finalise the KVM in one particular scenario, but I still think having a "KVM-enabled kernel" was a prerequisite?
sudo insmod kvm.ko
sudo insmod kvm-intel.ko