Greetings!
I've recently dusted my old GTX550Ti in order to try and prepare a virtual machine (x64 Windows 10) with GPU pass-through but I've hit a few issues on the way.
I've followed the guides in this topic (as well as the linked ones) and I managed to go as far as binding the GTX550Ti to vfio-pci but I cannot manage to have a working VM with qemu.
I'm using an up-to-date Slackware64-current installation: I had to recompile the kernel (I used the default kernel's config file and enabled vfio), blacklisted both snd_hda_intel and the nvidia drivers (later to be loaded with modprobe using /etc/rc.d/rc.local) and bound the graphic card to vfio, both the GPU and the audio portion.
lspci -nnk outputs the following (relevant entries only):
Code:
06:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF116 [GeForce GTX 550 Ti] [10de:1244] (rev a1)
Subsystem: ASUSTeK Computer Inc. GF116 [GeForce GTX 550 Ti] [1043:83be]
Kernel driver in use: vfio-pci
Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia
06:00.1 Audio device [0403]: NVIDIA Corporation GF116 High Definition Audio Controller [10de:0bee] (rev a1)
Subsystem: ASUSTeK Computer Inc. GF116 High Definition Audio Controller [1043:83be]
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel
iommu groups are ok, as seen here:
Code:
IOMMU group 0
00:00.0 Host bridge [0600]: Intel Corporation 4th Gen Core Processor DRAM Controller [8086:0c00] (rev 06)
IOMMU group 1
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller [8086:0c01] (rev 06)
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM206 [GeForce GTX 950] [10de:1402] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:0fba] (rev a1)
IOMMU group 2
00:14.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB xHCI [8086:8c31] (rev 05)
IOMMU group 3
00:16.0 Communication controller [0780]: Intel Corporation 8 Series/C220 Series Chipset Family MEI Controller #1 [8086:8c3a] (rev 04)
IOMMU group 4
00:1a.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #2 [8086:8c2d] (rev 05)
IOMMU group 5
00:1b.0 Audio device [0403]: Intel Corporation 8 Series/C220 Series Chipset High Definition Audio Controller [8086:8c20] (rev 05)
IOMMU group 6
00:1c.0 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #1 [8086:8c10] (rev d5)
IOMMU group 7
00:1c.2 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #3 [8086:8c14] (rev d5)
IOMMU group 8
00:1c.3 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev d5)
04:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge [1b21:1080] (rev 04)
IOMMU group 9
00:1c.4 PCI bridge [0604]: Intel Corporation 8 Series/C220 Series Chipset Family PCI Express Root Port #5 [8086:8c18] (rev d5)
IOMMU group 10
00:1d.0 USB controller [0c03]: Intel Corporation 8 Series/C220 Series Chipset Family USB EHCI #1 [8086:8c26] (rev 05)
IOMMU group 11
00:1f.0 ISA bridge [0601]: Intel Corporation H87 Express LPC Controller [8086:8c4a] (rev 05)
00:1f.2 SATA controller [0106]: Intel Corporation 8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode] [8086:8c02] (rev 05)
00:1f.3 SMBus [0c05]: Intel Corporation 8 Series/C220 Series Chipset Family SMBus Controller [8086:8c22] (rev 05)
IOMMU group 12
03:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
IOMMU group 13
06:00.0 VGA compatible controller [0300]: NVIDIA Corporation GF116 [GeForce GTX 550 Ti] [10de:1244] (rev a1)
06:00.1 Audio device [0403]: NVIDIA Corporation GF116 High Definition Audio Controller [10de:0bee] (rev a1)
The other graphic card (a GTX 950) works as usual with the nVidia proprietary drivers.
I'm using alien's qemu and vde packages if this can be relevant. I also checked if the GTX550Ti works properly and I verified both a Windows installation (dual boot) and my Slackware system (prior to binding the GTX550Ti to vfio) could use the card without issues and output to the monitor I connected.
Once I reached this point, I started to run into several issues.
- Problem #1: no output on the GTX550Ti
I cannot output anything on the GTX550Ti using qemu (the monitor I hooked up keeps giving the NO SIGNAL warning).
I'm using this script to launch qemu:
Code:
#!/bin/sh
INSTALLFILE=/home/kthxbye/vm/win10-uefi-x64.qcow2
FILESIZE=200G
INSTALLCD=/home/kthxbye/archive/windows/win10/Win10_1511_1_English_x64.iso
DRIVERCD=/home/kthxbye/archive/windows/virtio-win-0.1.118.iso
# PCI address of the passtrough devices
DEVICE1="06:00.0"
DEVICE2="06:00.1"
if [ ! -e $INSTALLFILE ]; then
qemu-img create -f qcow2 -o preallocation=metadata,compat=1.1,lazy_refcounts=on $INSTALLFILE $FILESIZE
fi
cp /usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd /tmp/my_vars.fd
QEMU_AUDIO_DRV=alsa
qemu-system-x86_64 \
-enable-kvm \
-m 8192 \
-smp cores=4,threads=2 \
-cpu host,kvm=off \
-vga none \
-soundhw hda \
-boot menu=on \
-usb \
-device vfio-pci,host=$DEVICE1,multifunction=on \
-device vfio-pci,host=$DEVICE2 \
-drive if=pflash,format=raw,readonly,file=/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd \
-drive if=pflash,format=raw,file=/tmp/my_vars.fd \
-device virtio-scsi-pci,id=scsi \
-drive file=$INSTALLCD,id=isocd,format=raw,if=none -device scsi-cd,drive=isocd \
-drive file=$INSTALLFILE,id=disk,format=qcow2,if=none,cache=writeback -device scsi-hd,drive=disk \
-drive file=$DRIVERCD,id=virtiocd,if=none,format=raw -device ide-cd,bus=ide.1,drive=virtiocd
- Problem #2: BSOD on the VM if using -vga std
If I start the vm with the
-vga std option I can install Windows 10; doing so allowed me to verify that the VM does actually see the GTX550Ti with no issues. Windows proceeded to download the latest updates and the drivers for the nVidia card but at the next reboot it started going into a BSOD loop giving one of the two following errors:
Code:
system_service_exception
or
Code:
system_service_exception nvlddmkm.sys
This happens both if I let Windows download the updates and the drivers by itself or if I install the latest drivers myself.
- Problem #3: freezes and garbled audio.
Prior to installing the drivers, the VM works reasonably well, although it's a bit slow at times and I had better success by using
Code:
-cdrom $INSTALLCD \
-hda $INSTALLFILE
instead of
Code:
-device virtio-scsi-pci,id=scsi \
-drive file=$INSTALLCD,id=isocd,format=raw,if=none -device scsi-cd,drive=isocd \
-drive file=$INSTALLFILE,id=disk,format=qcow2,if=none,cache=writeback -device scsi-hd,drive=disk \
-drive file=$DRIVERCD,id=virtiocd,if=none,format=raw -device ide-cd,bus=ide.1,drive=virtiocd
in the qemu script I use.
It does, however, freeze every time it needs to output sound: this happens both with alsa and pulseaudio. The terminal shows this warning:
Code:
main-loop: WARNING: I/O thread spun for 1000 iterations
On top of that, the sound is also heavily garbled. The sound coming from the host is not affected.
Does anyone have any idea about how to solve any of those issues, or where to look for a possible solution?
If I'll manage to have a working VM with proper pass-through and virtualisation I'll also write a step by step guide taken to solve the aforementioned issues.
Thanks in advance!
Cheers,
~kthxbye
