NetBSD DomU (PV) hangs on Xen 4.13.1 / NetBSD 9.1

[Matthias Petermann <mp%petermann-it.de@localhost>]

Hello together,

re-building my Xen host is proving to be more of a challenge than I 
initially thought. Now that the machine boots from RAID, there is a 
problem running the virtual machines. Is there any known problem in the 
combination mentioned in the subject?

This is my setup:

Host
----

- Xen Kernel 4.13.1 from pkgsrc-2020Q3 with NetBSD 9.1 DOM0
- domain config

name = "builder91"
type="pv"
kernel = "/netbsd-XEN3_DOMU.gz"
memory = 16384
vcpus = 8
vif = [ 'mac=00:16:3E:74:32:42,bridge=bridge0' ]
disk = [
   'phy:/dev/zvol/rdsk/tank0/volumes/builder91,hda,rw',
   'file:/tank1/archive/NetBSD-9.1-amd64.iso,ioemu:hdc:cdrom,r'
]
boot = 'cd'
vnc=1
vnclisten = "0.0.0.0"
vncpasswd = "in.secure"
vncunused=1
vga='stdvga'
usbdevice = "tablet"

Guest
-----

- NetBSD 9.1 installed to a ZVOL using HVM mode --> ok
- switched to PV mode --> hangs on boot

And this are the only lines "xl console builder91" shows after creation 
of the domain:

[   1.0000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 
2003, 2004, 2005,
[   1.0000000]     2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 
2015, 2016, 2017,
[   1.0000000]     2018, 2019, 2020 The NetBSD Foundation, Inc.  All 
rights reserved.
[   1.0000000] Copyright (c) 1982, 1986, 1989, 1991, 1993
[   1.0000000]     The Regents of the University of California.  All 
rights reserved.

[   1.0000000] NetBSD 9.1 (XEN3_DOMU) #0: Sun Oct 18 19:24:30 UTC 2020
[   1.0000000] 
mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/xen/compile/XEN3_DOMU
[   1.0000000] total memory = 16384 MB
[   1.0000000] avail memory = 15854 MB
[   1.0000000] running cgd selftest aes-xts-256 aes-xts-512 done
[   1.0000000] mainbus0 (root)
[   1.0000000] hypervisor0 at mainbus0: Xen version 4.13.1nb1
[   1.0000000] vcpu0 at hypervisor0
[   1.0000000] vcpu0: AMD FX(tm)-8350 Eight-Core Processor           , 
id 0x600f20
[   1.0000000] vcpu0: package 0, core 3, smt 0
[   1.0000000] vcpu1 at hypervisor0
[   1.0000000] vcpu1: AMD FX(tm)-8350 Eight-Core Processor           , 
id 0x600f20
[   1.0000000] vcpu1: package 0, core 6, smt 0
[   1.0000000] vcpu2 at hypervisor0
[   1.0000000] vcpu2: AMD FX(tm)-8350 Eight-Core Processor           , 
id 0x600f20
[   1.0000000] vcpu2: package 0, core 5, smt 0
[   1.0000000] vcpu3 at hypervisor0
[   1.0000000] vcpu3: AMD FX(tm)-8350 Eight-Core Processor           , 
id 0x600f20
[   1.0000000] vcpu3: package 0, core 1, smt 0
[   1.0000000] vcpu4 at hypervisor0
[   1.0000000] vcpu4: AMD FX(tm)-8350 Eight-Core Processor           , 
id 0x600f20
[   1.0000000] vcpu4: package 0, core 4, smt 0
[   1.0000000] vcpu5 at hypervisor0
[   1.0000000] vcpu5: AMD FX(tm)-8350 Eight-Core Processor           , 
id 0x600f20
[   1.0000000] vcpu5: package 0, core 2, smt 0
[   1.0000000] vcpu6 at hypervisor0
[   1.0000000] vcpu6: AMD FX(tm)-8350 Eight-Core Processor           , 
id 0x600f20
[   1.0000000] vcpu6: package 0, core 0, smt 0
[   1.0000000] vcpu7 at hypervisor0
[   1.0000000] vcpu7: AMD FX(tm)-8350 Eight-Core Processor           , 
id 0x600f20
[   1.0000000] vcpu7: package 0, core 7, smt 0
[   1.0000000] xenbus0 at hypervisor0: Xen Virtual Bus Interface
[   1.0000000] xencons0 at hypervisor0: Xen Virtual Console Driver
[   1.9900279] xenbus: can't get state for device/suspend/event-channel (2)
[   1.9900279] xbd0 at xenbus0 id 768: Xen Virtual Block Device Interface
[   1.9900279] xbd1 at xenbus0 id 5632: Xen Virtual Block Device Interface
[   1.9900279] unknown type vfb at xenbus0 id 0 not configured
[   1.9900279] unknown type vkbd at xenbus0 id 0 not configured
[   1.9900279] xennet0 at xenbus0 id 0: Xen Virtual Network Interface
[   1.9900279] xennet0: MAC address 00:16:3e:74:32:42
[   1.9900279] balloon0 at xenbus0 id 0: Xen Balloon driver
[   1.9900279] balloon0: current reservation: 16777216 KiB
[   1.9900279] xennet0: using RX copy mode
[   2.0000302] xenbus: can't get state for device/suspend/event-channel (2)
[   2.0000302] unknown type vfb at xenbus0 id 0 not configured
[   2.0000302] unknown type vkbd at xenbus0 id 0 not configured
[   2.0000302] balloon0: current reservation: 4194304 pages => target: 
4194304 pages
[   2.0000302] ignore shutdown request:
[   2.0000302] xenbus: can't get state for device/suspend/event-channel (2)
[   2.0000302] unknown type vfb at xenbus0 id 0 not configured
[   2.0000302] unknown type vkbd at xenbus0 id 0 not configured

The identical configuration had recently worked on another system with 
NetBSD 9.0 host and Xen 4.11. Unfortunately, I can't do anything with 
the log transcript. My research also showed that the "unknown type 
vfb/vkbd" messages are not critical, because there are no emulated 
devices available in PV mode. Is there a way to inspect the activity of 
the DomU kernel more closely in this state (like with ktrace for 
processes...)? I would like to know what the kernel is doing or waiting 
for, but don't know how to get this information. Of course, if I have a 
thinking error or an obvious configuration problem, that would be 
satisfactory for now. Actually I just wanted to put my builder back into 
operation this morning ;-)

Kind regards
Matthias