kern/59371: Xen domU uvm_fault since FPU state allocation patch

[imil%home.imil.net@localhost]

>Number:         59371
>Category:       kern
>Synopsis:       Xen domU uvm_fault since FPU state allocation patch
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Apr 28 05:40:01 +0000 2025
>Originator:     Emile `iMil' Heitor
>Release:        NetBSD 10.99.14
>Organization:
	NetBSD
>Environment:
System: NetBSD outcast 10.99.14 NetBSD 10.99.14 (XEN3_DOM0) #1: Wed Apr 23 12:03:35 CEST 2025  imil@tatooine:/home/imil/src/github.com/NetBSD-src/sys/arch/amd64/compile/obj/XEN3_DOM0 amd64
Architecture: x86_64
Machine: amd64
>Description:
	Starting April 25, NetBSD domU kernel would crash with an uvm_fault:

[   1.0000000] cpu_rng: rdrand/rdseed
[   1.0000000] entropy: ready
[   1.0000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003,
[   1.0000000]     2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013,
[   1.0000000]     2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023,
[   1.0000000]     2024, 2025
[   1.0000000]     The NetBSD Foundation, Inc.  All rights reserved.
[   1.0000000] Copyright (c) 1982, 1986, 1989, 1991, 1993
[   1.0000000]     The Regents of the University of California.  All rights reserved.

[   1.0000000] NetBSD 10.99.14 (XEN3_DOMU) #7: Mon Apr 28 06:06:42 CEST 2025
[   1.0000000] 	imil@tatooine:/home/imil/src/NetBSD/src/sys/arch/amd64/compile/obj/XEN3_DOMU
[   1.0000000] total memory = 256 MB
[   1.0000000] avail memory = 235 MB
[   1.0000000] mainbus0 (root)
[   1.0000000] hypervisor0 at mainbus0: Xen version 4.18.4_20241221nb0
[   1.0000000] vcpu0 at hypervisor0
[   1.0000000] vcpu0: Intel(R) Core(TM) i5-7300U CPU @ 2.60GHz, id 0x806e9
[   1.0000000] vcpu0: node 0, package 0, core 0, smt 0
[   1.0000000] xenbus0 at hypervisor0: Xen Virtual Bus Interface
[   1.0000000] xencons0 at hypervisor0: Xen Virtual Console Driver
[   1.0000030] xenbus0: can't get state for device/suspend/event-channel (2)
[   1.0000030] uvm_fault(0xffffffff8094a300, 0x0, 2) -> e
[   1.0000030] fatal page fault in supervisor mode
[   1.0000030] trap type 6 code 0x2 rip 0xffffffff8062795c cs 0xe030 rflags 0x10202 cr2 0 ilevel 0 rsp 0xffffffff80adad38
[   1.0000030] curlwp 0xffffffff8078f880 pid 0.0 lowest kstack 0xffffffff80ad62c0
kernel: page fault trap, code=0
Stopped in pid 0.0 (system) at  netbsd:memset+0x2c:     repe stosq      %es:(%rdi)
memset() at netbsd:memset+0x2c
lwp_create() at netbsd:lwp_create+0x2f1
fork1() at netbsd:fork1+0x42c
main() at netbsd:main+0x44f
ds          40
es          100
fs          1
gs          107
rdi         0
rsi         200
rbp         ffffffff80adad90
rbx         ffff930042b2e000
rdx         200
rcx         40
rax         0
r8          ffffffff8047a38c    start_init
r9          0
r10         fffffe00
r11         ffffffff80adabcc
r12         0
r13         ffff93000092e800
r14         ffffffff8078f880    lwp0
r15         0
rip         ffffffff8062795c    memset+0x2c
cs          e030
rflags      10202
rsp         ffffffff80adad38
ss          e02b
netbsd:memset+0x2c:     repe stosq      %es:(%rdi)
db{0}>

This behavior seems linked to this commit:
https://mail-index.netbsd.org/source-changes/2025/04/24/msg156552.html

Riastradh@ suggested that I try this workaround:

--- sys/arch/x86/x86/fpu.c      2025-04-24 16:57:38.905367169 +0200
+++ sys/arch/x86/x86/fpu.c.patch        2025-04-24 16:58:39.368608934 +0200
@@ -475,6 +475,9 @@
                return;
        }
 
+#ifdef XENPV
+       pcb2->pcb_savefpu = &pcb2->pcb_savefpusmall;
+#endif
        /* For init(8). */
        if (__predict_false(l1->l_flag & LW_SYSTEM)) {
                memset(pcb2->pcb_savefpu, 0, x86_fpu_save_size);

which indeed permitted the domU to proceed with boot.

>How-To-Repeat:
	Boot a NetBSD Xen/domU kernel from April 25 2025
>Fix:
	Please.