[SOLVED] Arch VM bridged networking

InventiousTech · 10-09-2016, 01:20 PM

Hi,

I'm having a bit of an issue with networking two VMs in an Arch linux host running kernel 4.7.2-1-vfio (for ACS IOMMU ungrouping), and I haven't been able to get it resolved. The machine itself is Z87 based, with an i7 6700k, 16Gb of 3200MHz DDR4, a GTX 1060, and a GTX 770. I've got two Qemu/KVM virtual machines running, with one graphics card passed to each. Each guest is running Windows 7 x64. After several straight days of fighting with drivers, I got everything working except the networking.

As long as only one of the VMs is running, everything is fine and the adapter bridging works great at full speed. However, when both are running, only one network connection can be active at a time, on either or both machines. So between the two of them, only one application can talk to the internet at a time. If I'm running an online game on one machine, then start a Youtube video on the second, the game connection gets reset. Even if the two connections are from the same machine, it doesn't work.

I'm stumped, and I haven't been able to find anyone else with similar issues. It may be a simple configuration parameter issue, but I'm not sure. I've tried switching the KVM virtual network adapter line from '-netdev' to '-net' but no dice. Here is my configuration:

netctl bridge service:

Code:

Description="Bridge for VMs"
Interface=br0
Connection=bridge
BindsToInterfaces=(enp0s31f6 tap0 tap1)
IP=dhcp
## Ignore (R)STP and immediately activate the bridge
#SkipForwardingDelay=yes

One of the VM Qemu startup scripts (they are effectively identical)

Code:

#!/bin/bash


qemu-system-x86_64 \
    -nodefaults \
    -nodefconfig \
    -serial none \
    -parallel none \
    -enable-kvm \
    -vga std \
    -nographic \
    -name Windows \
    -cpu host,kvm=off,check \
    -smp sockets=1,cores=2,threads=2 \
    -m 8192 \
    -rtc base=localtime \
    -device virtio-scsi-pci,id=scsi \
    -device vfio-pci,host=01:00.0,multifunction=on,x-vga=on \
    -device vfio-pci,host=01:00.1 \
    -device vfio-pci,host=00:14.0 \
    -device vfio-pci,host=00:14.2 \
    -net nic,model=virtio -net tap,ifname=tap0,script=no,downscript=no,vhost=on \
    -drive if=pflash,format=raw,readonly,file=/usr/share/ovmf/x64/ovmf_code_x64.bin \
    -drive if=pflash,format=raw,file=/mnt/blah/ovmf_vars.bin \
    -drive file=/mnt/blah/blah.qcow2,id=disk,format=qcow2,if=none,bus=1,cache=writeback -device ide-hd,drive=disk 


ip link set tap0 down

Script for starting the VMs, each with its own screen window

Code:

ip tuntap add dev tap0 mode tap
ip tuntap add dev tap1 mode tap
ip link set tap0 up
ip link set tap1 up

netctl start bridge


if ! screen -list | grep -q "VM1"; then
    echo "Starting VM1."
    screen -dmS VM1 /home/blah/VM1Startup.sh
    sleep 2
    if ! screen -list | grep -q "VM1"; then
	echo "VM1 start FAILED!"
    else
	echo "VM1 successfully started."
    fi
else
    echo "VM1 already running, doing nothing"
fi

if ! screen -list | grep -q "VM2"; then
    echo "Starting VM2."
    screen -dmS VM2/home/blah/VM2Startup.sh
    sleep 2
    if ! screen -list | grep -q "VM2"; then
	echo "VM2 start FAILED!"
    else
	echo "VM2 successfully started."
    fi
else
    echo "VM2 already running, doing nothing"
fi

Thanks in advance for any help or tips you can provide.

business_kid · 10-10-2016, 11:59 AM

If I get the picture right, you have 2 copies of windows-7 x64 running on the same machine at the same time. That, I would conjecture, is the problem. You can rely on M$ to make a dog's dinner of a situation like that. Does any M$ based machine run 2 independent copies of the same version of windows simultaneously without puking?

You could probably test that by adding a linux vm. All you need is a basic system, a browser, and plugins to get YouTube. Play the game on windows, try youtube on the linux vm and see how it goes.

InventiousTech · 10-10-2016, 01:22 PM

Thanks for the reply. Just for clarification, one of the guests is Win7 Pro, the other is Home Premium, and they have different product keys, so it shouldn't be a MS licencing issue. I can try setting up a linux VM when I get some time to play with it, but setting up guest video drivers is a pain in the neck since I don't have a monitor connected to the host video output. I could get a PCI network card to pass through to one of the VMs, or a USB ethernet adapter, which I'm sure would work, but that shouldn't really be necessary.

I may also try wiresharking some traffic to see if I see anything obvious.

business_kid · 10-11-2016, 04:01 PM

The sort of error I envisaged was interrupts, I/O addresses or identically named but different roles, that sort of thing. Linux has good separation and can run many instances of itself. I doubt that windows can, as they didn't design it as a guest OS. Who knows?

Product keys are a software licencing thing, and don't affect software operation. Try another guest OS.

InventiousTech · 10-12-2016, 12:20 PM

Yeah, that was it. As usual, it was a simple problem that I didn't even think of. Even though the two machines received different IP addresses from the DHCP server, the MAC addresses were the same, since they both used the RedHat VirtIO network adapter. Incrementing the least significant bit of one of the MAC addresses solved the problem. Thanks for the ideas, Business_Kid, I appreciate it.

business_kid · 10-13-2016, 04:44 PM

Glad to spare someone else the grey hairs I picked up along the way.

There's a thing you can click on to approve of posts that help you. Check the bottom right.