Re: Xen MP panics in cpu_switchto()

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

On Mon, Jan 13, 2020 at 11:42:17AM +0000, Andrew Doran wrote:
> Hi Manuel,
> 
> On Mon, Jan 13, 2020 at 10:56:23AM +0100, Manuel Bouyer wrote:
> > Hello,
> > A current Xen domU kernel fails to boot with:
> > [   1.0000000] hypervisor0 at mainbus0: Xen version 4.11.3nb1
> > [   1.0000000] vcpu0 at hypervisor0
> > [   1.0000000] vcpu0: Intel(R) Xeon(TM) CPU 3.00GHz, id 0xf64
> > [   1.0000000] vcpu0: node 0, package 0, core 1, smt 1
> > [   1.0000000] vcpu1 at hypervisor0
> > [   1.0000000] vcpu1: Intel(R) Xeon(TM) CPU 3.00GHz, id 0xf64
> > [   1.0000000] vcpu1: node 0, package 1, core 0, smt 0
> > [   1.0000000] xenbus0 at hypervisor0: Xen Virtual Bus Interface
> > [   1.0000000] xencons0 at hypervisor0: Xen Virtual Console Driver
> > [   1.9901295] uvm_fault(0xffffffff80d5c120, 0x0, 1) -> e
> > [   1.9901295] fatal page fault in supervisor mode
> > [   1.9901295] trap type 6 code 0 rip 0xffffffff8020209f cs 0x8 rflags 0x10246 cr2 0x28 ilevel 0 rsp 0xffffb7802b19de88
> > [   1.9901295] curlwp 0xffffb7800083b500 pid 0.15 lowest kstack 0xffffb7802b1992c0
> > kernel: page fault trap, code=0
> > Stopped in pid 0.15 (system) at netbsd:cpu_switchto+0xf:        movq    28(%r13),%rax
> > cpu_switchto() at netbsd:cpu_switchto+0xf
> > 
> > both amd64 and i386. A boot with vcpus=1 succeeds, so I guess something is
> > missing in initialisations of secondary CPUs.
> > This happens with the 202001101800Z but the problem is probably older than
> > that (the testbed used vcpus=1 until today)
> > 
> > Any idea ?
> 
> It should work now with revision 1.199 of src/sys/arch/amd64/amd64/locore.S. 

The same problem happens with i386.

> Nothing else in tree calls cpu_switchto() with prevlwp=NULL any more.  Can
> Xen's cpu_hatch() call idle_loop() directly?

Maybe it could, but cpu_switchto() does some extra work (switch the stack,
set curlwp at last). Maybe this is already done but I'll have to double check.

-- 
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
     NetBSD: 26 ans d'experience feront toujours la difference
--