kern/56969: Kernel panic on host when qemu-nvmm virtual machine exits

[bernd%niob.bersie.home@localhost]

>Number:         56969
>Category:       kern
>Synopsis:       Kernel panic on host when qemu-nvmm virtual machine exits
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Aug 17 15:50:00 +0000 2022
>Originator:     bernd.sieker%posteo.net@localhost
>Release:        NetBSD 9.3
>Organization:
>Environment:
System: NetBSD niob.bersie.home 9.3 NetBSD 9.3 (NIOB_DEBUG) #1: Wed Aug 17 12:07:51 CEST 2022 bernd%niob.bersie.home@localhost:/usr/src/sys/arch/amd64/compile/NIOB_DEBUG amd64
SunFire X2270 M2, Dual Xeon X5675, 56 GB ECC RAM
Architecture: x86_64
Machine: amd64
>Description:
I have recently upgraded a 9.2_STABLE system to 9.3 RELEASE, built and installed from local source copy using build.sh.
I also reinstalled all packages from pkgsrc-2022Q2, including qemu 7.0.0.
The virtual machine also runs NetBSD 9.3 RELEASE, also completely upgraded and reinstalled.
Whenever a virtual machine quits (either shutting down the NetBSD guest with "shutdown -p" or killing the qemu process with TERM signal, the host machine kernel panics and the machine reboots.

I have built a kernel with DEBUG and LOCKDEBUG enabled, here are the last lines extracted from the crashdump using dmesg:

[  1260.922078] panic: kernel diagnostic assertion "semcnt >= 0" failed: file "../../../../kern/kern_uidinfo.c", line 241
[  1260.922078] cWpuA9R:N IBNeGg:i nS PtLr aNcOeTb aLcOkW.E.R.E
[  1260.922078] D ON SYSCALL 2 675736328 EXIT ff844ed0 7
[  1260.922078] WARNING: SPL NOT LOWERED ON SYSCALL 2 675736328 EXIT ff844ed0 7
[   126.000000] 0 7v8p4a]n iWcA(R)N IaNtG : SPL NOT LOWERED ON SYSCALL 2 675736328 EXIT ff844ed0 7
[  1260.922078] + 0WxA1R6N0I
[  1260.922078] NG: SPL NOT LOWERED ON SYSCALL 0 675736328 EXIT ff844ed0 7
[  1260.922078] ugen_get_alt_index() at netbsd:ugen_get_alt_index
[  1260.922078] chgsemcnt() at netbsd:chgsemcnt+0x56
[  1260.922078] ksem_release() at netbsd:ksem_release+0x6a
[  1260.932083] ksem_close_fop() at netbsd:ksem_close_fop+0x49
[  1260.932083] closef() at netbsd:closef+0x6d
[  1260.932083] fd_close() at netbsd:fd_close+0x2b1
[  1260.932083] sys__ksem_destroy() at netbsd:sys__ksem_destroy+0x9c
[  1260.932083] syscall() at netbsd:syscall+0x196
[  1260.932083] --- syscall (number 255) ---
[  1260.932083] 7be3fd84384a:
[  1260.932083] cpu9: End traceback...

Possibly relevant kernel options include:

options         SVS             # Separate Virtual Space
makeoptions     SPECTRE_V2_GCC_MITIGATION=1     # GCC Spectre variant 2
                                                # migitation
options         SPECTRE_V2_GCC_MITIGATION
[...]
# Diagnostic/debugging support options
options         DIAGNOSTIC      # inexpensive kernel consistency checks
                                # XXX to be commented out on release branch
options         DEBUG           # expensive debugging checks/support
options         LOCKDEBUG       # expensive locking checks/support
[...]
makeoptions     COPTS="-O2 -fno-omit-frame-pointer"
options         DDB             # in-kernel debugger
options         DDB_COMMANDONENTER="bt" # execute command when ddb is entered
options         DDB_ONPANIC=1   # see also sysctl(7): `ddb.onpanic'
options         DDB_HISTORY_SIZE=512    # enable history editing in DDB
#options        KGDB            # remote debugger
#options        KGDB_DEVNAME="\"com\"",KGDB_DEVADDR=0x3f8,KGDB_DEVRATE=9600
makeoptions     DEBUG="-g"      # compile full symbol table for CTF
[...]

KUBSAN, KASAN, KLEAK and KCOV are not enabled.

>How-To-Repeat:
Start a virtual machine using qemu with nvmm acceleration, stop the VM. A kernel panic ensues.
>Fix:
Unknown