[SOLVED] QEMU (4.1.0) Connect a usb-sata-hdd

Luridis · 11-24-2019, 02:43 PM

(Apologies for the original post, it was supposed to be a new thread and somehow ended up a repy, I removed that one since it was off-topic to that thread.)

I've been trying to do this for hours and can't figure out why it is not working. I've dug through dozens of guides, webpages and manuals and none of the examples seem to work completely. So first, here is my ls{pci,usb,scsi} (trimmed):

Code:

lspci
00:14.0 USB controller: Intel Corporation 7 Series/C210 Series Chipset Family USB xHCI Host Controller (rev 04)
00:1c.0 PCI bridge: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 1 (rev c4)
00:1c.1 PCI bridge: Intel Corporation 7 Series/C210 Series Chipset Family PCI Express Root Port 2 (rev c4)
00:1c.3 PCI bridge: Intel Corporation 7 Series/C216 Chipset Family PCI Express Root Port 4 (rev c4)
00:1d.0 USB controller: Intel Corporation 7 Series/C216 Chipset Family USB Enhanced Host Controller #1 (rev 04)
00:1f.0 ISA bridge: Intel Corporation HM76 Express Chipset LPC Controller (rev 04)
00:1f.2 SATA controller: Intel Corporation 7 Series Chipset Family 6-port SATA Controller [AHCI mode] (rev 04)
lsusb
Bus 004 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 004 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 004: ID 13d3:5719 IMC Networks 
Bus 001 Device 003: ID 8087:07da Intel Corp. 
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 004: ID 13fd:1e40 Initio Corporation INIC-1610P SATA bridge
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
lsscsi
[0:0:0:0]    disk    ATA      ST750LX003-1AC15 SM12  /dev/sda 
[2:0:0:0]    cd/dvd  MATSHITA DVD-RAM UJ8B2    1.00  /dev/sr0 
[6:0:0:0]    disk    SWI      SD Card          2.31  /dev/sdb 
[7:0:0:0]    disk    ST       ST9500420AS      3.00  /dev/sdc

Here are the exmples I've found:

qemu-system-x86_64 <my other options> -usb -usbdevice disk:/dev/sdc
Doesn't work, warns depreciated, can't find updated version of command, assumed a dead end.

qemu-system-x86_64 <my other options> -hda /dev/sdc
This works for the most part, but things like cgdisk won't work because it prevents writing to block 0. Note the warning:

Code:

WARNING: Image format was not specified for '/dev/sdc' and probing guessed raw.
Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
Specify the 'raw' format explicitly to remove the restrictions.

Yet there is no way to secify things like cache=none and format=raw using this method. Though if I boot an archlinux iso, everything appears to be fine, except the sector zero block.

What about connecting via usb: qemu-system-x86_64 <my other options>-usb -device usb-host,hostbus=2,hostaddr=4
NOPE! If I boot from the arch iso the kerenel completely throws a tantrum and keeps disconnecting and reconnecting the drive.

I tried several other methods, virtio-scsi, etc. (complained of no pcie bus)

I'm really lost on this one. Because the -hda method does work with limitations, I just need to know how to do that while specifying a raw format, etc. But nothing I've found explains how to do that. Has anyone managed to connect a usb-sata enclosure drive to qemu and get it properly recognized?

Luridis · 11-24-2019, 10:38 PM

Okay, I figured this out and I'll post what happened here in case anyone else runs into the issue. Basically, what I had was a perfect storm of little issues that made the system unbootable.

First, what was actually preventing QEMU from booting from the usb hdd was a scrambled writable copy of the TianoCore efi variables nvram file. (/usr/share/ovmf/x64/OVMF_VARS.fd copied to my user directory.) This is what stores your non-volitile efi entries. I read on one website that if you've been fiddling with things and then booting different configurations, this can get scrambled and you need to copy over a fresh one. If you don't need persistent efivars, you can simply omit it from the QEMU command line and the bios: OVMF_CODE.fd (always read-only) will regenerate a fresh efi table during post, but any entries you create won't be saved on poweroff. Removing the nvram line from my QEMU command allowed it to pick up the EFI-GRUB on my usb hdd.

Second, -hda /dev/sdb will work (and make sure it's not mounted via gvfs), though you'll still get an auto-detect raw format warning. A more correct way is using -drive file=/dev/sdb,format=raw,media=disk which will eliminate the warnings. An even better way is to specify the disk by-id so that if the usb/ahci enumeration changes your QEMU launcher script will still work, i.e. -drive /dev/disk/by-id/ata-ST9500420AS_XXXXXXXX,format=raw,media=disk

Finally, I mentioned an error in c{f,g}disk about not being able to write to sector 0. This happened when booting with -hda /dev/sdb specifed, and the guest running the livecd could see this as /dev/sda. At first, I thought this was related to the auto-detect raw warning that QEMU put in stderr upon launching. A closer look at the udev error "print_req_error: I/O error, dev fd0, sector 0" reveals that it doesn't say anything about /dev/sda. When I fixed the efivar problem and got QEMU to book from the HDD, I didn't see this error at all when using cf/cgdisk. I think what is actually causing the error is the arch-linux.iso. This is a mixed-mode bootable cd, meaning it supports both UEFI and old El Torito MBR emulation for bootable cdroms. If you're unfamiliar, El Torito works by including a floppy disk image to act as the boot sector for the cdrom. Some UEFI bioses will pick up both modes during post, my ASUS laptop has two boot options when I use this arch-linux ISO, one is named "Boot from XXXXX CDROM" and the other "UEFI: Boot from XXXX CDROM". So, I think that the TianoCore was somehow picking up this read-only floppy disk image and mounting it as fd0, then the partitioning utilities complained that it was not writable. As I said, when I finally managed to boot from the HDD, cf/cgdisk didn't creat udev complaints.

So, hopefully that will help if anyone finds it in a future search for issues with booting physical hard disks from QEMU.

Luridis · 12-10-2019, 12:47 PM

Came back to add this...

Code:

-usb -device usb-host,hostbus=XXX,hostaddr=XXX

Will work for connecting a physical usb disk to QEMU, but there are two caveats. First, the user running QEMU must own that device in /dev/bus/usb/XXX/XXX. Second, if the port involved is xHCI (USB3), the emulated PC platform must be changed to ICH9 by adding -M q35 to the QEMU command line, since the other PIIX chipsets don't speak newer USB. Last, if QEMU's CPU utilization is high after switching to ICH9, the guest kernel is likely missing some QEMU guest kernel modules (balloon, virtio, etc.).