Re: QEMU booting of non-NetBSD CDs and virtual HDs

[Robert Nestor <rnestor%mac.com@localhost>]

On Aug 23, 2020, at 1:57 PM, Chavdar Ivanov <ci4ic4%gmail.com@localhost> wrote:

> On Sun, 23 Aug 2020 at 15:41, Robert Nestor <rnestor%mac.com@localhost> wrote:
>> 
>> I received a couple of messages off list that suggested a few things and it prompted me to try investigating further with just components found in NetBSD.
>> 
>> This test was run on a fairly recent NetBSD build of 9.99.70.  I downloaded the amd64 images for 9.99.71 (the ISO and IMG files), and tried booting them with qemu using -nvmm and the OVMF binaries currently in pkgsrc with the following:
>> 
>> qemu-system-x86_64 -m 4096 -machine q35 -accel=nvmm -boot menu=on \
> 
> -accel nvmm
> 
>>    -device qemu-xhci -device usb-tablet -device usb-mouse -smbios type=2 \
>>    -drive if=pflash,format=raw,readonly,file=/usr/pkg/share/OVMFX64.fd \
> 
> the OVMFX64.fd file is actually in /usrpkg/share/ovmf directory, but
> perhaps this is a typo. Anyway. I have no idea about this particular
> way of specifying the bios; anyway, with
> 
> 
> -bios /usr/pkg/share/ovmf/OVMFX64.fd \
> 
> it boots just fine. Otherwise I get the same crash as you.
> 
> 
>>    -device ich9-ahci,id=sata \
>>    -device ide-cd,bus=sata.0,drive=disk \
>>    -drive id=disk,if=none,media=cdrom,format=raw,file=NetBSD-9.99.71-amd64.iso
>> 
>> This produces an immediate “failed to start VCPU” and results in a core dump. Also tried the NetNSD-9.99.71-amd64-install.img file with:
> 
>> 
>> qemu-system-x86_64 -m 4096 -machine q35 -accel=nvmm -boot menu=on \
>>    -device qemu-xhci -device usb-tablet -device usb-mouse -smbios type=2 \
>>    -drive if=pflash,format=raw,readonly,file=/usr/pkg/share/OVMFX64.fd \
>>    -device ich9-ahci,id=sata \
>>    -device ide-hd,bus=sata.0,drive=disk \
>>    -drive id=disk,if=none,media=disk,format=raw,file=NetBSD-9.99.71-amd64-install.img
>> 
>> And it provides the same results - “failed to start VCPU” and a core dump.
>> 
>> Removing the “-accel=nvmm” from both of the scripts allows the boot to proceed, but the OVMF code fails to find the CD or HD image and boot falls back to attempting to boot over the network.  This appears to be a bug in the version of OVMF found in pkgsrc which is based on stable2018.  Replacing the OVMF with binaries obtained from a build of stable202005 fixes the disk access issue and the boot then succeeds brining up the NetBSD installer.
>> 
>> I then proceeded to do two installations of NetBSD under qem; one using the defaults for an MBR setup and one for a GPT setup.  The resulting MBR disk doesn’t boot under qemu; the GPT disk does boot however.  In the case of the MBR disk it appears the problem is that OVMF can’t find the disk or anything bootable on it.
>> 
>> I’ve opened two PRs for these issues.  PR-55582 for the NVMM issues and PR-55582 for the OVMF issue.
>> 

Indeed, using the OVMFX64.fd file as a -bios parameter does eliminate the NVMM crash and it does boot up the NetBSD CD ISO file.  From there I was able to do two installations of NetBSD, one specifying an MBR partition setup and one with GPT taking the defaults for both.  However, neither of the created disk image files will boot with the OVMFX64.fd file.  Using a newer version from a stable202005 OVMF build and one from a build done in the last week or so does boot up the GPT created disk image but not the newly created MBR disk image.

So it appears there is a bug in NVMM that causes an abort and dump when any OVMF file is configured as a pflash device rather than a BIOS.  From what I can see the primary difference in the setup of the use of OVMF is how it stores any EFI variables it might need or use. Using it as a BIOS option is the most restrictive for storing and saving UEFI variables. (This may no longer be true and current versions of OVMF may actually store EFI variables in a NuVars file in the boot path though.)  Using the OVMF file that contains the read-only code and the read-write variable space is less restrictive, and using a separate OVMF code and variable files provides the most flexibility but only the BIOS usage seems compatible withe NVMM at this point.

And it seems there’s a bug in all versions of OVMF at this time that prevents some disk images from booting even though they will boot on real HW.  Or there’s something strange about the way the NetBSD installer creates these disks that is incompatible with OVMF at present - or both.

-bob