Issues with NetBSD-5.1, xen and amd64 machines.

[buhrow%lothlorien.nfbcal.org@localhost (Brian Buhrow)]

        Hello.  I've been trying to get Xen running on a dual quad-core amd64
box with 12GB of RAM. NetBSD-5.1 runs fine, but when I try to boot dom0
with the xen kernel, I get a hang, as shown below.  Interestingly enough,
even though I've built the xen3kernel package from pkgsrc on this box, the
resultant installed binary, once gunzipped, shows as a 32-bit ELF
executable.  Shouldn't it be a 64-bit executable?  Although I've built a
custom dom0 kernel from netbsd-5 sources newer than the 5.1 release, I'm
actually trying to do a test boot using the netbsd-xen3-dom0 kernel as
provided in the binary distribution for NetBSD-5.1.  I am booting using the
NetBSD boot loader, as shown in boot.cfg(5)'s man page.
        this is a Supermicro 1-U server with a lights-out BMC board in it
which provides network console access on com1.
        I'm sure I'm doing something obviously wrong, but I'm not sure what it
is.  Does anyone have any thoughts?
Oh, yes, I'm trying to boot the xen-debug.gz file from
/usr/pkg/xen3-kernel, but I copied and gunzipped it into /xen/xen on the
root filesystem.

(I'm showing the boot sequence, the /boot.cfg file, and dmesg for the
machine when it boots native NetBSD.)

-thanks
-Brian


>> NetBSD/x86 BIOS Boot, Revision 5.2 (from NetBSD 5.1_STABLE)
>> Memory: 607/3136000 k

     1. Boot Xen and Dom0
     2. Boot normally
     3. Boot single user
     4. Disable ACPI
     5. Disable ACPI and SMP
     6. Drop to boot prompt

Choose an option; RETURN for default; SPACE to stop countdown.
Option 1 will be chosen in 5 seconds.
951808+371200=0x1430ac
Loading /netbsd console=com1 

<hang>


Script started on Wed Sep 21 22:26:35 2011
xen2# file /xen/xen
/xen/xen: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically 
linked, stripped
xen2# cat /boot.cfg
menu=Boot Xen and Dom0:load /netbsd console=com1;multiboot /xen/xen 
dom0_mem=1024M console=com1 com1=115200,8n1
menu=Boot normally:boot netbsd.hw
menu=Boot single user:boot netbsd -s
menu=Disable ACPI:boot netbsd -2
menu=Disable ACPI and SMP:boot netbsd -12
menu=Drop to boot prompt:prompt
default=1
timeout=5

(dmesg from native hardware boot)

Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
    2006, 2007, 2008, 2009, 2010
    The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
    The Regents of the University of California.  All rights reserved.

NetBSD 5.1_STABLE (RBL) #0: Wed May 18 13:21:03 PDT 2011
        
buhrow%lothlorien.nfbcal.org@localhost:/usr/local/netbsd/obj/sys/arch/amd64/compile/RBL
total memory = 12279 MB
avail memory = 11890 MB
timecounter: Timecounters tick every 10.000 msec
timecounter: Timecounter "i8254" frequency 1193182 Hz quality 100
SMBIOS rev. 2.6 @ 0x99c00 (81 entries)
iXsystems iX1204-563TUB (1234567890)
mainbus0 (root)
WARNING: can't reserve area for I/O APIC.
cpu0 at mainbus0 apid 0/usr/local/netbsd/src/sys/arch/x86/x86/mtrr_i686.c: 
FIXME: more than 8 MTRRs
: Intel 686-class, 2400MHz, id 0x206c2
cpu1 at mainbus0 apid 2: Intel 686-class, 2400MHz, id 0x206c2
cpu2 at mainbus0 apid 18: Intel 686-class, 2400MHz, id 0x206c2
cpu3 at mainbus0 apid 20: Intel 686-class, 2400MHz, id 0x206c2
cpu4 at mainbus0 apid 32: Intel 686-class, 2400MHz, id 0x206c2
cpu5 at mainbus0 apid 34: Intel 686-class, 2400MHz, id 0x206c2
cpu6 at mainbus0 apid 50: Intel 686-class, 2400MHz, id 0x206c2
cpu7 at mainbus0 apid 52: Intel 686-class, 2400MHz, id 0x206c2
cpu8 at mainbus0 apid 1: Intel 686-class, 2400MHz, id 0x206c2
cpu9 at mainbus0 apid 3: Intel 686-class, 2400MHz, id 0x206c2
cpu10 at mainbus0 apid 19: Intel 686-class, 2400MHz, id 0x206c2
cpu11 at mainbus0 apid 21: Intel 686-class, 2400MHz, id 0x206c2
cpu12 at mainbus0 apid 33: Intel 686-class, 2400MHz, id 0x206c2
cpu13 at mainbus0 apid 35: Intel 686-class, 2400MHz, id 0x206c2
cpu14 at mainbus0 apid 51: Intel 686-class, 2400MHz, id 0x206c2
cpu15 at mainbus0 apid 53: Intel 686-class, 2400MHz, id 0x206c2
ioapic0 at mainbus0 apid 6: pa 0xfec00000, version 20, 24 pins
ioapic1 at mainbus0 apid 7: pa 0xfec8a000, version 20, 24 pins
acpi0 at mainbus0: Intel ACPICA 20080321
acpi0: X/RSDT: OemId <SMCI  ,        ,20110105>, AslId <MSFT,00000097>
acpi0: SCI interrupting at int 9
acpi0: fixed-feature power button present
timecounter: Timecounter "ACPI-Fast" frequency 3579545 Hz quality 1000
ACPI-Fast 24-bit timer
acpi: activated PNP0C0F
acpi: activated PNP0C0F
attimer1 at acpi0 (TMR, PNP0100): io 0x40-0x43 irq 0
pcppi1 at acpi0 (SPKR, PNP0800): io 0x61
midi0 at pcppi1: PC speaker (CPU-intensive output)
spkr0 at pcppi1
sysbeep0 at pcppi1
UAR1 (PNP0501) [16550A-compatible COM port] at acpi0 not configured
UAR2 (PNP0501) [16550A-compatible COM port] at acpi0 not configured
hpet0 at acpi0 (HPET, PNP0103): mem 0xfed00000-0xfed003ff
timecounter: Timecounter "hpet0" frequency 14318179 Hz quality 2000
acpibut0 at acpi0 (PWRB, PNP0C0C-170): ACPI Power Button
attimer1: attached to pcppi1
ipmi0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1
pci0: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
pchb0 at pci0 dev 0 function 0
pchb0: Intel product 0x3406 (rev. 0x22)
ppb0 at pci0 dev 1 function 0: Intel product 0x3408 (rev. 0x22)
ppb0: unsupported PCI Express version
pci1 at ppb0 bus 1
pci1: i/o space, memory space enabled, rd/line, wr/inv ok
wm0 at pci1 dev 0 function 0: 82576 1000BaseT Ethernet, rev. 1
wm0: interrupting at ioapic1 pin 4
wm0: PCI-Express bus
wm0: 65536 word (16 address bits) SPI EEPROM
wm0: Ethernet address 00:25:90:35:xx:xx
igphy0 at wm0 phy 1: i82566 10/100/1000 media interface, rev. 1
igphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 
1000baseT-FDX, auto
wm1 at pci1 dev 0 function 1: 82576 1000BaseT Ethernet, rev. 1
wm1: interrupting at ioapic1 pin 16
wm1: PCI-Express bus
wm1: 65536 word (16 address bits) SPI EEPROM
wm1: Ethernet address 00:25:90:35:xx:xx
igphy1 at wm1 phy 1: i82566 10/100/1000 media interface, rev. 1
igphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 
1000baseT-FDX, auto
ukphy0 at wm1 phy 16: Generic IEEE 802.3u media interface
ukphy0: OUI 0x000000, model 0x0000, rev. 0
ukphy0: 10baseT-FDX, 100baseTX-FDX, 100baseT4, auto
ppb1 at pci0 dev 3 function 0: Intel product 0x340a (rev. 0x22)
ppb1: unsupported PCI Express version
pci2 at ppb1 bus 2
pci2: i/o space, memory space enabled, rd/line, wr/inv ok
ppb2 at pci0 dev 5 function 0: Intel product 0x340c (rev. 0x22)
ppb2: unsupported PCI Express version
pci3 at ppb2 bus 3
pci3: i/o space, memory space enabled, rd/line, wr/inv ok
ppb3 at pci0 dev 7 function 0: Intel product 0x340e (rev. 0x22)
ppb3: unsupported PCI Express version
pci4 at ppb3 bus 4
pci4: i/o space, memory space enabled, rd/line, wr/inv ok
ppb4 at pci0 dev 9 function 0: Intel product 0x3410 (rev. 0x22)
ppb4: unsupported PCI Express version
pci5 at ppb4 bus 5
pci5: i/o space, memory space enabled, rd/line, wr/inv ok
Intel product 0x342d (interrupt system, interface 0x20, revision 0x22) at pci0 
dev 19 function 0 not configured
Intel product 0x342e (interrupt system, revision 0x22) at pci0 dev 20 function 
0 not configured
Intel product 0x3422 (interrupt system, revision 0x22) at pci0 dev 20 function 
1 not configured
Intel product 0x3423 (interrupt system, revision 0x22) at pci0 dev 20 function 
2 not configured
Intel product 0x3438 (interrupt system, revision 0x22) at pci0 dev 20 function 
3 not configured
Intel product 0x3430 (miscellaneous system, revision 0x22) at pci0 dev 22 
function 0 not configured
Intel product 0x3431 (miscellaneous system, revision 0x22) at pci0 dev 22 
function 1 not configured
Intel product 0x3432 (miscellaneous system, revision 0x22) at pci0 dev 22 
function 2 not configured
Intel product 0x3433 (miscellaneous system, revision 0x22) at pci0 dev 22 
function 3 not configured
Intel product 0x3429 (miscellaneous system, revision 0x22) at pci0 dev 22 
function 4 not configured
Intel product 0x342a (miscellaneous system, revision 0x22) at pci0 dev 22 
function 5 not configured
Intel product 0x342b (miscellaneous system, revision 0x22) at pci0 dev 22 
function 6 not configured
Intel product 0x342c (miscellaneous system, revision 0x22) at pci0 dev 22 
function 7 not configured
uhci0 at pci0 dev 26 function 0: Intel product 0x3a37 (rev. 0x00)
uhci0: interrupting at ioapic0 pin 16
usb0 at uhci0: USB revision 1.0
uhci1 at pci0 dev 26 function 1: Intel product 0x3a38 (rev. 0x00)
uhci1: interrupting at ioapic0 pin 21
usb1 at uhci1: USB revision 1.0
uhci2 at pci0 dev 26 function 2: Intel product 0x3a39 (rev. 0x00)
uhci2: interrupting at ioapic0 pin 19
usb2 at uhci2: USB revision 1.0
ehci0 at pci0 dev 26 function 7: Intel product 0x3a3c (rev. 0x00)
ehci0: interrupting at ioapic0 pin 18
ehci0: EHCI version 1.0
ehci0: companion controllers, 2 ports each: uhci0 uhci1 uhci2
usb3 at ehci0: USB revision 2.0
uhci3 at pci0 dev 29 function 0: Intel product 0x3a34 (rev. 0x00)
uhci3: interrupting at ioapic0 pin 23
usb4 at uhci3: USB revision 1.0
uhci4 at pci0 dev 29 function 1: Intel product 0x3a35 (rev. 0x00)
uhci4: interrupting at ioapic0 pin 19
usb5 at uhci4: USB revision 1.0
uhci5 at pci0 dev 29 function 2: Intel product 0x3a36 (rev. 0x00)
uhci5: interrupting at ioapic0 pin 18
usb6 at uhci5: USB revision 1.0
ehci1 at pci0 dev 29 function 7: Intel product 0x3a3a (rev. 0x00)
ehci1: interrupting at ioapic0 pin 23
ehci1: EHCI version 1.0
ehci1: companion controllers, 2 ports each: uhci3 uhci4 uhci5
usb7 at ehci1: USB revision 2.0
ppb5 at pci0 dev 30 function 0: Intel 82801BA Hub-PCI Bridge (rev. 0x90)
pci6 at ppb5 bus 6
pci6: i/o space, memory space enabled
vga0 at pci6 dev 1 function 0: Matrox product 0x0532 (rev. 0x0a)
wsdisplay0 at vga0 kbdmux 1
wsmux1: connecting to wsdisplay0
drm at vga0 not configured
pcib0 at pci0 dev 31 function 0
pcib0: Intel ICH10R Watchdog timer (rev. 0x00)
ahcisata0 at pci0 dev 31 function 2: Intel ICH10R AHCI Controller
ahcisata0: interrupting at ioapic0 pin 19
ahcisata0: AHCI revision 1.2, 6 ports, 32 command slots, features 0xef20e0e0
atabus0 at ahcisata0 channel 0
atabus1 at ahcisata0 channel 1
atabus2 at ahcisata0 channel 2
atabus3 at ahcisata0 channel 3
atabus4 at ahcisata0 channel 4
atabus5 at ahcisata0 channel 5
ichsmb0 at pci0 dev 31 function 3: Intel ICH10 SMBus Serial Bus 1 (rev. 0x00)
ichsmb0: interrupting at ioapic0 pin 18
iic0 at ichsmb0: I2C bus
spdmem0 at iic0 addr 0x52
spdmem0: DDR3 SDRAM memory, no ECC, 1333MHz (PC3--132644)
spdmem0: 11 rows, 13 cols, 8 internal banks, 1 physical banks, 1.500ns cycle 
time
spdmem0: tAA-tRCD-tRP-tRAS: 8-8-8-24
isa0 at pcib0
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
com1: console
pckbc0 at isa0 port 0x60-0x64
timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
uhub0 at usb0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhub1 at usb2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
uhub2 at usb6: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub2: 2 ports with 2 removable, self powered
uhub3 at usb5: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub3: 2 ports with 2 removable, self powered
uhub4 at usb1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub4: 2 ports with 2 removable, self powered
uhub5 at usb3: Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub5: 6 ports with 6 removable, self powered
uhub6 at usb4: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub6: 2 ports with 2 removable, self powered
ahcisata0 port 0: device present, speed: 3.0Gb/s
ahcisata0 port 1: device present, speed: 3.0Gb/s
uhub7 at usb7: Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub7: 6 ports with 6 removable, self powered
wd0 at atabus0 drive 0: <WDC WD5003ABYX-01WERA0>
wd0: drive supports 16-sector PIO transfers, LBA48 addressing
wd0: 465 GB, 969021 cyl, 16 head, 63 sec, 512 bytes/sect x 976773168 sectors
wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 6 (Ultra/133)
wd0(ahcisata0:0:0): using PIO mode 4, DMA mode 2, Ultra-DMA mode 6 (Ultra/133) 
(using DMA)
wd1 at atabus1 drive 0: <WDC WD5003ABYX-01WERA0>
wd1: drive supports 16-sector PIO transfers, LBA48 addressing
wd1: 465 GB, 969021 cyl, 16 head, 63 sec, 512 bytes/sect x 976773168 sectors
wd1: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 6 (Ultra/133)
wd1(ahcisata0:1:0): using PIO mode 4, DMA mode 2, Ultra-DMA mode 6 (Ultra/133) 
(using DMA)
uhidev0 at uhub4 port 1 configuration 1 interface 0
uhidev0: American Megatrends Inc. Virtual Keyboard and Mouse, rev 1.10/1.00, 
addr 2, iclass 3/1
ukbd0 at uhidev0
wskbd0 at ukbd0 mux 1
wskbd0: connecting to wsdisplay0
uhidev1 at uhub4 port 1 configuration 1 interface 1
uhidev1: American Megatrends Inc. Virtual Keyboard and Mouse, rev 1.10/1.00, 
addr 2, iclass 3/1
ums0 at uhidev1: 3 buttons
wsmouse0 at ums0 mux 0
ipmi0: version 2.0 interface KCS iobase 0xca2/2 spacing 1
Kernelized RAIDframe activated
pad0: outputs: 44100Hz, 16-bit, stereo
audio0 at pad0: half duplex, playback, capture
raid0: RAID Level 1
raid0: Components: /dev/wd0a /dev/wd1a
raid0: Total Sectors: 976772992 (476939 MB)
boot device: raid0
root on raid0a dumps on raid0b
root file system type: ffs
raid0: Device already configured!
ipmi0: critical under limit on 'P2-DIMM3A Temp'
ipmi0: critical under limit on 'P2-DIMM2A Temp'
Accounting started