VirtualBox on sc64's Kernel 5.10.x

rahrah · 01-04-2021, 07:23 PM

Hi,

Virtualbox 6.1.16 kernel modules did not compile against the latest 5.10.x kernel in slackware64-current. The attached patch worked for me.

More details can be found upstream, here:

https://www.virtualbox.org/ticket/20055

with the patch here:

https://www.virtualbox.org/raw-attac...j-fix-r0.patch

sha1: ee313af087310b4d2f7126fb74710360c9f1fddf

The 6.1.17 testbuild (as others have pointed out) is available here:

https://www.virtualbox.org/wiki/Testbuilds

This is a patch for 5.10.x as a host.

Guests running 5.10.x kernels with VB Guest Additions, may need further tweaking.
Look at Sérgio Basto's comment #12 (and following) at the link above.
Sergio (sergiomb2 on github) has a nice git clone of the VB repo, which seems to be up to date.

The patch at:

https://github.com/sergiomb2/Virtual...5fb495e2d04e97

(the patch now in the vb svn trunk) is substantially similar to the one attached here, but for a comment:

Code:

/** @def IPRT_USE_APPLY_TO_PAGE_RANGE_FOR_EXEC
 * alloc_vm_area was removed with 5.10 so we have to resort to a different way
 * to allocate executable memory.
 * It would be possible to remove IPRT_USE_ALLOC_VM_AREA_FOR_EXEC and use
 * this path execlusively for 3.2+ but no time to test it really works on every
 * supported kernel, so better play safe for now.
 */

and this:

Code:

- uint32_t idxPg = (uAddr - (unsigned long)pMemLnx->Core.pv) >> PAGE_SHIFT;
+ size_t   idxPg = (uAddr - (unsigned long)pMemLnx->Core.pv) >> PAGE_SHIFT;

Guest Additions

I haven't tested any vb guest running 5.10.x with vb guest additions.
I have tested a Slackware-current guest with the standard, vboxsf, vboxguest, vboxvideo, in tree kernel based modules: most things substantially work.
Just load the modules (you may need to kick them into action with an xradr -s trickery).
For full functionality, you might have to mess with the guest addition scripts, a little so as not to attempt to build the already built in-tree modules.

Cheers,

===Rich

File: linux-5.10-r0drv-memobj-fix-r0.patch

Code:

Index: src/VBox/Runtime/r0drv/linux/memobj-r0drv-linux.c
===================================================================
--- src/VBox/Runtime/r0drv/linux/memobj-r0drv-linux.c	(Revision 141658)
+++ src/VBox/Runtime/r0drv/linux/memobj-r0drv-linux.c	(Arbeitskopie)
@@ -56,9 +56,12 @@
  * Whether we use alloc_vm_area (3.2+) for executable memory.
  * This is a must for 5.8+, but we enable it all the way back to 3.2.x for
  * better W^R compliance (fExecutable flag). */
-#if RTLNX_VER_MIN(3,2,0) || defined(DOXYGEN_RUNNING)
+#if RTLNX_VER_RANGE(3,2,0, 5,10,0) || defined(DOXYGEN_RUNNING)
 # define IPRT_USE_ALLOC_VM_AREA_FOR_EXEC
 #endif
+#if RTLNX_VER_MIN(5,10,0) || defined(DOXYGEN_RUNNING)
+# define IPRT_USE_APPLY_TO_PAGE_RANGE_FOR_EXEC
+#endif
 
 /*
  * 2.6.29+ kernels don't work with remap_pfn_range() anymore because
@@ -502,7 +505,43 @@
 }
 
 
+#ifdef IPRT_USE_APPLY_TO_PAGE_RANGE_FOR_EXEC
 /**
+ * User data passed to the apply_to_page_range() callback.
+ */
+typedef struct LNXAPPLYPGRANGE
+{
+    /** Pointer to the memory object. */
+    PRTR0MEMOBJLNX pMemLnx;
+    /** The page protection flags to apply. */
+    pgprot_t       fPg;
+} LNXAPPLYPGRANGE;
+/** Pointer to the user data. */
+typedef LNXAPPLYPGRANGE *PLNXAPPLYPGRANGE;
+/** Pointer to the const user data. */
+typedef const LNXAPPLYPGRANGE *PCLNXAPPLYPGRANGE;
+
+/**
+ * Callback called in apply_to_page_range().
+ *
+ * @returns Linux status code.
+ * @param   pPte                Pointer to the page table entry for the given address.
+ * @param   uAddr               The address to apply the new protection to.
+ * @param   pvUser              The opaque user data.
+ */
+static DECLCALLBACK(int) rtR0MemObjLinuxApplyPageRange(pte_t *pPte, unsigned long uAddr, void *pvUser)
+{
+    PCLNXAPPLYPGRANGE pArgs = (PCLNXAPPLYPGRANGE)pvUser;
+    PRTR0MEMOBJLNX pMemLnx = pArgs->pMemLnx;
+    uint32_t idxPg = (uAddr - (unsigned long)pMemLnx->Core.pv) >> PAGE_SHIFT;
+
+    set_pte(pPte, mk_pte(pMemLnx->apPages[idxPg], pArgs->fPg));
+    return 0;
+}
+#endif
+
+
+/**
  * Maps the allocation into ring-0.
  *
  * This will update the RTR0MEMOBJLNX::Core.pv and RTR0MEMOBJ::fMappedToRing0 members.
@@ -584,6 +623,11 @@
         else
 # endif
         {
+#  if defined(IPRT_USE_APPLY_TO_PAGE_RANGE_FOR_EXEC)
+            if (fExecutable)
+                pgprot_val(fPg) |= _PAGE_NX; /* Uses RTR0MemObjProtect to clear NX when memory ready, W^X fashion. */
+#  endif
+
 # ifdef VM_MAP
             pMemLnx->Core.pv = vmap(&pMemLnx->apPages[0], pMemLnx->cPages, VM_MAP, fPg);
 # else
@@ -1851,6 +1895,21 @@
         preempt_enable();
         return VINF_SUCCESS;
     }
+# elif defined(IPRT_USE_APPLY_TO_PAGE_RANGE_FOR_EXEC)
+    PRTR0MEMOBJLNX pMemLnx = (PRTR0MEMOBJLNX)pMem;
+    if (   pMemLnx->fExecutable
+        && pMemLnx->fMappedToRing0)
+    {
+        LNXAPPLYPGRANGE Args;
+        Args.pMemLnx = pMemLnx;
+        Args.fPg = rtR0MemObjLinuxConvertProt(fProt, true /*fKernel*/);
+        int rcLnx = apply_to_page_range(current->active_mm, (unsigned long)pMemLnx->Core.pv + offSub, cbSub,
+                                        rtR0MemObjLinuxApplyPageRange, (void *)&Args);
+        if (rcLnx)
+            return VERR_NOT_SUPPORTED;
+
+        return VINF_SUCCESS;
+    }
 # endif
 
     NOREF(pMem);

Aeterna · 01-04-2021, 09:20 PM

the problem with VB development and testbuilds is that all this randomly crashes VB client kernel (at boot time) 5.10.x (no problems with VB host)

Is you patchset better in the terms of VB client stability?

rahrah · 01-05-2021, 11:46 AM

Hello,

Firstly, 'my patchset' is *not* mine! I just lifted it from the VB ticket site. I take no credit for the code.

I've run vb for a couple of days with this one patch. I have had no stability issues with a Slackware64-current host running Pat's 5.10.4 generic kernel with any of these guest OSs:

Linux Mint 20.0 running: 5.8.0-33-generic kernel and VB 6.1.16 guest additions.

Ubuntu 20.04 running 5.4.0-59-generic kernel and VB 6.1.16 guest additions.

Slackware64-current running: 5.10.4 generic kernel with NO guest additions installed.**

Windows 10 with VB 6.1.16 guest additions.

** I have experimented with with VB 6.1.16 guest additions under a slackware64-current snapshot, but kept the kernel modules from Pat's kernel (ie Linus' release kernel with no 3rd party code, and no VB module code from the VB guest additions package). I had no crashes, but didn't test extensively. I could provoke no crashes; I had VB file sharing, screen resizing working, under xfce and kde.

If you are having stability problems, I'd be interested to know:

What graphics chipset your host is running?

Have you enabled 3D Acceleration (generally not good)?

What VB video driver you are using?

Can you reproduce the 'random' crashes with any sort of consistency (is there any way you can exacerbate the crashes)?

===R

aikempshall · 01-05-2021, 05:01 PM

Quote:

Originally Posted by rahrah

Hello,

Slackware64-current running: 5.10.4 generic kernel with NO guest additions installed.**

I'm running the above at the moment in a test environment as a guest. Shared folders work. Bi-directional copy and paste doesn't work, probably need Guest Additions for this to work.

If I get a free moment or two I will try applying the patch.

Alex

Aeterna · 01-05-2021, 06:24 PM

ok, so instead of patching, just download dev/test from VirtualBox site.

my VB host (Slackware-current) runs 5.10.x
VB clients (e.g. Slackware-current, Gentoo, antiX Artix, Venom) run without issues 5.10.x as long as VirtualBox additions are not installed. Adding VB additions makes kernel unstalble.

This situation is long known.

aikempshall · 01-13-2021, 05:56 AM

Quote:

Originally Posted by aikempshall

I'm running the above at the moment in a test environment as a guest. Shared folders work. Bi-directional copy and paste doesn't work, probably need Guest Additions for this to work.

If I get a free moment or two I will try applying the patch

Alex

Got that free time.

With 5.10.6 and VBoxGuestAdditions_6.1.16 installed I was getting this error recorded in file /var/log/vboxadd-setup.log

Quote:

In file included from /tmp/vbox.0/combined-os-specific.c:33:
/tmp/vbox.0/r0drv/linux/memobj-r0drv-linux.c: In function 'rtR0MemObjLinuxVMap':
/tmp/vbox.0/r0drv/linux/memobj-r0drv-linux.c:560:34: error: implicit declaration of function 'alloc_vm_area' [-Werror=implicit-function-declaration]
560 | pMemLnx->pArea = alloc_vm_area(pMemLnx->Core.cb, papPtes); /* Note! pArea->nr_pages is not set. */
| ^~~~~~~~~~~~~
/tmp/vbox.0/r0drv/linux/memobj-r0drv-linux.c:560:32: warning: assignment to 'struct vm_struct *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
560 | pMemLnx->pArea = alloc_vm_area(pMemLnx->Core.cb, papPtes); /* Note! pArea->nr_pages is not set. */
| ^
cc1: some warnings being treated as errors
make[2]: *** [scripts/Makefile.build:279: /tmp/vbox.0/combined-os-specific.o] Error 1
make[2]: *** Waiting for unfinished jobs....
./tools/objtool/objtool orc generate --module --no-fp --retpoline --uaccess /tmp/vbox.0/combined-agnostic.o
make[1]: *** [Makefile:1805: /tmp/vbox.0] Error 2
make: *** [/tmp/vbox.0/Makefile-footer.gmk:117: vboxguest] Error 2

as root

successfully ran the patch
restarted the guest
guest started successfully
no shared clipboard

Now get this error recorded in /var/log/vboxadd-setup.log

Quote:

UILD_BASENAME='"regops"' -DKBUILD_MODNAME='"vboxsf"' -c -o /tmp/vbox.0/regops.o /tmp/vbox.0/regops.c
/tmp/vbox.0/regops.c: In function 'vbsf_lock_user_pages_failed_check_kernel':
/tmp/vbox.0/regops.c:1406:24: error: 'USER_DS' undeclared (first use in this function); did you mean 'USER_HZ'?
1406 | && uPtrFrom >= USER_DS.seg)
| ^~~~~~~
| USER_HZ
/tmp/vbox.0/regops.c:1406:24: note: each undeclared identifier is reported only once for each function it appears in
./tools/objtool/objtool orc generate --module --no-fp --retpoline --uaccess /tmp/vbox.0/lnkops.o
make[2]: *** [scripts/Makefile.build:279: /tmp/vbox.0/regops.o] Error 1
make[2]: *** Waiting for unfinished jobs....
./tools/objtool/objtool orc generate --module --no-fp --retpoline --uaccess /tmp/vbox.0/vfsmod.o
./tools/objtool/objtool orc generate --module --no-fp --retpoline --uaccess /tmp/vbox.0/dirops.o
make[1]: *** [Makefile:1805: /tmp/vbox.0] Error 2
make: *** [/tmp/vbox.0/Makefile-footer.gmk:117: vboxsf] Error 2

Was there anything else I was supposed to do?

Alex

bassmadrigal · 01-14-2021, 12:18 PM

I don't know if the stable release supports the 5.10.x kernel yet, but the testbuilds should. Here's the link for Guest additions v6.1.17-141968

rahrah · 01-14-2021, 03:45 PM

Hi,

== Intro

The patch above is for the *host*, it is *not* for the guests. I've included more instructions, below, on how to use the patch for the host.

The rationale for the patch is to compile virtualbox 6.1.16 host drivers. This obviates use of test build which has *many* patches and may be unstable. It *only* patches one file. That's all you need to get vb 6.1.16 going as a host on 5.10.x kernels.

That said, here's a bit on Slackware guests. I've also included full instructions on patching.

== VB Guests

Linux, the kernel at least, runs fine under vb without any other additions. However, for added functionality, VB provides guest
additions. The file, VBoxLinuxAdditions.run contains and installs, amongst other stuff, i) user programs to allow certain
functions (like screen resizing under X, file sharing between guest and host, and shared clipboard), and ii) source for kernel
modules for the *guest* kernel. These guest kernel modules provide the kernel support for the functions in i). The kernel modules are:
vboxguest.ko vboxvideo.ko and vboxsf.ko.

Before some kernel revision (5.6 perhaps???) vboxsf.ko and vboxvideo.ko were not included in the mainline kernel. With kernel 5.10.x, (and maybe 5.9.x??) all three modules are in the kernel. The code bases between what's in the VBoxLinuxAdditions.run and the mainline kernel might be different, but some versions of those modules are now in the kernel. So, my assumption is don't bother with with the installation of the kernel modules included in the VBoxLinuxAdditions.run script, just install the other stuff, and most things will work.

On installation of VBoxLinuxAdditions.run, all the other stuff will be installed, you'll get the error message when the installer tries to
install the kernel modules, but it doesn't matter. Guest additions should still work. To stop the errors, just do this at the command prompt:

Code:

export INSTALL_NO_MODULE_BUILDS=true

Then install the guest additions like this:

Code:

./VBoxLinuxAdditions.run

Then add the line:

Code:

export INSTALL_NO_MODULE_BUILDS=true

before the line:

Code:

export LC_ALL=C

towards the top of the file:

Code:

/etc/rc.d/init.d/vboxadd

All this assumes a slackware-current guest running a 5.10.x kernel. Most of it can be adapted to other distros (but few are running 5.10.x kernels).

Then restart the guest.

Everything seems to work bar drag and drop (which, I never use).

== VB Hosts

Getting hosts going with vb 6.1.16 whilst running 5.10.x kernels was the main purpose of my original post. Here's it is again in
more detail.

This has only been tested for 5.10.x and vb 6.1.16 only.

Check the integrity of the patch

Code:

sha1sum linux-5.10-r0drv-memobj-fix-r0.patch
e313af087310b4d2f7126fb74710360c9f1fddf  linux-5.10-r0drv-memobj-fix-r0.patch

Run these commands

Code:

cd /opt/VirtualBox/src/vboxhost/vboxdrv

patch -p3 --dry-run --verbose  < path_to_patch/linux-5.10-r0drv-memobj-fix-r0.patch

That command should show:

Code:

Hmm...  Looks like a unified diff to me...
The text leading up to this was:
--------------------------
|Index: src/VBox/Runtime/r0drv/linux/memobj-r0drv-linux.c
|===================================================================
|--- src/VBox/Runtime/r0drv/linux/memobj-r0drv-linux.c	(Revision 141658)
|+++ src/VBox/Runtime/r0drv/linux/memobj-r0drv-linux.c	(Arbeitskopie)
--------------------------
checking file r0drv/linux/memobj-r0drv-linux.c
Using Plan A...
Hunk #1 succeeded at 56.
Hunk #2 succeeded at 505.
Hunk #3 succeeded at 623.
Hunk #4 succeeded at 1895.
done

###
If that doesn't happen then the something is wrong with the patch or command. That command was a dry run, it didn't actually patch the files.

###
Once you have checked that the patch works, patch for real with this command:

Code:

patch -p3 --verbose --backup < path_to_patch/linux-5.10-r0drv-memobj-fix-r0.patch

You should see the same messages.

Now run:

Code:

cd /opt/VirtualBox/src/vboxhost
make
make install

Then, if you are running the 5.10.6 kernel, do this:

Code:

ls -la /lib/modules/5.10.6/misc

You should see three modules:

Code:

-rw-r--r-- 1 root root 732928 Jan 10 15:10 vboxdrv.ko
-rw-r--r-- 1 root root  56792 Jan 10 15:11 vboxnetflt.ko
-rw-r--r-- 1 root root  18776 Jan 10 15:11 vboxnetadp.ko

These are the host modules.

You can then just do

Code:

sh /opt/VirtualBox/vboxdrv.sh start

(This may attempt to build the drivers again, but it should succeed).

Then start vb and you are away. You will have to run:

Code:

sh /opt/VirtualBox/vboxdrv.sh start

as root before every invocation of vb (or shove it in /etc/rc.d/rc.local).

Cheers,

===Rich

aikempshall · 01-17-2021, 04:54 AM

On my guest using the 5.10.7 kernel without VBoxGuestAdditions I

can share between guest and host
can't copy and paste between guest and host

Would like to get copy/paste working. This worked well before the 5.10.x kernels but I had to install VBoxGuestAdditions.

So followed rahrah's instructions for VB Guests

Quote:

Originally Posted by rahrah

Hi,

== VB Guests

Linux, the kernel at least, runs fine under vb without any other additions. However, for added functionality, VB provides guest
additions. The file, VBoxLinuxAdditions.run contains and installs, amongst other stuff, i) user programs to allow certain
functions (like screen resizing under X, file sharing between guest and host, and shared clipboard), and ii) source for kernel
modules for the *guest* kernel. These guest kernel modules provide the kernel support for the functions in i). The kernel modules are:
vboxguest.ko vboxvideo.ko and vboxsf.ko.

Before some kernel revision (5.6 perhaps???) vboxsf.ko and vboxvideo.ko were not included in the mainline kernel. With kernel 5.10.x, (and maybe 5.9.x??) all three modules are in the kernel. The code bases between what's in the VBoxLinuxAdditions.run and the mainline kernel might be different, but some versions of those modules are now in the kernel. So, my assumption is don't bother with with the installation of the kernel modules included in the VBoxLinuxAdditions.run script, just install the other stuff, and most things will work.

On installation of VBoxLinuxAdditions.run, all the other stuff will be installed, you'll get the error message when the installer tries to
install the kernel modules, but it doesn't matter. Guest additions should still work. To stop the errors, just do this at the command prompt:

Code:

export INSTALL_NO_MODULE_BUILDS=true

Then install the guest additions like this:

Code:

./VBoxLinuxAdditions.run

Then add the line:

Code:

export INSTALL_NO_MODULE_BUILDS=true

before the line:

Code:

export LC_ALL=C

towards the top of the file:

Code:

/etc/rc.d/init.d/vboxadd

All this assumes a slackware-current guest running a 5.10.x kernel. Most of it can be adapted to other distros (but few are running 5.10.x kernels).

Then restart the guest.

Everything seems to work bar drag and drop (which, I never use).

Used VBoxGuestAdditions_6.1.17-141968.iso. Didn't get any error messages.

My guest is running 5.10.7 kernel with VBoxGuestAdditions-6.1.17. I get the shares as expected. I get this

Unfortunately the directories I have shared between host and guest are all owned by root
copy and paste between host and guest doesn't work.

Alex

rahrah · 01-19-2021, 09:23 AM

Hi,

It was not my intention to post regarding test builds. It was only my intention to post regarding getting vb 6.1.16 going on 5.10.x kernels.

I hadn't tested any of the test builds. There are far too many patches for my liking. VB is a part of my daily workflow and stability is important. It was my intent to just show the patch for getting 6.1.16 going on later host kernels on sc. All the stuff I posted re VB Hosts was for 6.1.16. When the next release comes out, I'll dump the patch and go with mainline.

Yesterday, I successfully replicated your issue regarding shared files. Consequently, I would strongly advise you not to mix the stuff I posted here with any test build stuff, unless you are on a snapshot and/or only want to test. Use the test builds in their entirety, or use 6.1.16 as per this post. I could not replicate your clipboard problem: the bidirectional cut and pasting worked fine for me using the test build method you describe.

I would agree with the point, should it be made, that the in kernel modules might not be fully compatible with later guest addition software. But while the test of test build may corroborate that, it doesn't prove it. The in kernel guest modules do seem to be compatible with 6.1.16 guest additions.

For me, there's no need or wish to use test builds. All I want is a working system. I get that with sc and vb 6.1.16 with the patch in the original post. The guest addition chicanery was really just a kludge just to demonstrate that 6.1.16 guest additions work with *existing* in kernel modules.

Cheers,

===Rich

willysr · 01-19-2021, 07:55 PM

6.1.18 is now out, which should be working out of the box with 5.10.x

aikempshall · 01-20-2021, 11:39 AM

Quote:

Originally Posted by willysr

6.1.18 is now out, which should be working out of the box with 5.10.x

Thanks for the update. I shall raise a separate thread about my not good experience with the 6.1.18 GuestAdditions.

Alex