NetBSD and Linux KVM

[makoto%hauN.ORG@localhost (Minoura Makoto)]

I tracked down the problem that the recent Linux KVM
hypervisor cannot boot NetBSD (5/6/current, at least).

 1 It hangs after acpicpus are probed
 2 Disabling ACPI (qemu -no-acpi/no options ACPI/boot -2/...)
  works
 3 Disabling IOAPIC (boot -c and disable ioapic/no ioapic*
  at mainbus?) works
 4 Avoiding KVM in-kernel APIC implementation (qemu -no-kvm-irqchip)
  works
 5 Somehow pressing the virtual power button (qemu monitor
  system_powerdown) continues the boot process

1. Backtraces show it hangs at spl0(); at the last of
cpu_configure(), meaning that an interrupt condition is not
cleared forever, even after the interrupt handlers are
called.

There's a hint in the mp_verbose dmesg:

# dmesg | grep 'ioapic0: pin 9'
ioapic0: pin 9 attached to isa0 irq 9 (type 0x0<type=0x0> flags 0xd<pol=0x1=Act 
Hi,trig=0x3=Level>)
ioapic0: pin 9 attached to pci0 device 1 INT_A (type 0x0<type=0x0> flags 
0xf<pol=0x3=Act Lo,trig=0x3=Level>)

Look at the polarity settings.  Same interrupt line shows
both active-high and -low.

The former line comes from ACPI MADT, describing it is active-high:

        Type=INT Override
        BUS=0
        IRQ=9
        INTR=9
        Flags={Polarity=active-hi, Trigger=level}

The latter is from PCI0 _PRT, saying nothing about its polarity:

                Package (0x04)
                {
                    0x0001FFFF, // Address: dev 0 func any
                    Zero,       // Pin: INT_A
                    Zero,       // Source: global interrupt
                    0x09        // SourceIndex: 9
                }, 

NetBSD reads _PRT at mpacpi_pciroute() in mpacpi.c.

                if (ptrp->Source[0] != 0) {
                        /* snip */
                } else {
                        /* snip */
                        /* Defaults for PCI (active low, level triggered) */
                        mpi->redir =
                            (IOAPIC_REDLO_DEL_FIXED<<IOAPIC_REDLO_DEL_SHIFT) |
                            IOAPIC_REDLO_LEVEL | IOAPIC_REDLO_ACTLO;
                        /* snip */
                }

I think we have to honor the MADT interrupt source override
here (it's a `global' interrupt).  Attached patch looks through
all the legacy interrupt settings to do it.

2. The problem was found in our ACPI code.  Good.

3. And in IOAPIC initialization. Ok.

4. Qemu IOAPIC implementation omits polarity settings.  Right?

5. This actually asserts ioapic0 pin 9, but looks it's
 cleared to NetBSD...

Comments?

Index: mpacpi.c
===================================================================
RCS file: /cvsroot/src/sys/arch/x86/x86/mpacpi.c,v
retrieving revision 1.92
diff -u -p -r1.92 mpacpi.c
--- mpacpi.c    1 Jul 2011 18:21:31 -0000       1.92
+++ mpacpi.c    17 Feb 2013 11:02:50 -0000
@@ -851,6 +851,7 @@ mpacpi_pciroute(struct mpacpi_pcibus *mp
                        if (mp_verbose > 1)
                                printf("pciroute: done adding entry\n");
                } else {
+                       int i;
                        if (mp_verbose > 1)
                                printf("pciroute: dev %d INT%c on globint %d\n",
                                    dev, 'A' + (ptrp->Pin & 3),
@@ -880,6 +881,15 @@ mpacpi_pciroute(struct mpacpi_pcibus *mp
                                mpi->ioapic_ih = pin;
                        mpi->linkdev = NULL;
                        mpi->flags = MPS_INTPO_ACTLO | (MPS_INTTR_LEVEL << 2);
+                       for (i = 0; i < NUM_LEGACY_IRQS-1; i++) {
+                               if (mp_intrs[i].ioapic == pic &&
+                                   mp_intrs[i].ioapic_pin == pin) {
+                                       mpi->flags = mp_intrs[i].flags;
+                                       mpi->redir = mp_intrs[i].redir;
+                                       mpi->sflags = mp_intrs[i].sflags;
+                                       break;
+                               }
+                       }
                        if (mp_verbose > 1)
                                printf("pciroute: done adding entry\n");
                }