Attempt at using Xen 4.1 on -current

To: port-xen <port-xen%netbsd.org@localhost>
Subject: Attempt at using Xen 4.1 on -current
From: Dustin Marquess <dmarquess%gmail.com@localhost>
Date: Thu, 19 May 2011 08:35:29 -0700
I attempted to move back from XCP 1.0.0 to NetBSD/xen this last week.
I ran into a few issues along the way.

- A 32-bit XEN3PAE_DOM0 -current kernel and a 64-bit Xen kernel don't
seem to get along anymore.  (Errors below).

No problem, gave up and went back to a 64-bit XEN3_DOM0.

- Attempted to use LVM as a storage backend (with a ported version of
haad's patch)

Accidentally created the LV with the wrong size.  Went to go remove it
and ran into PR bin/44947.
Okay, no problem, I'll use the old reliable file: backend!

Setup an OpenIndiana domU.  Works fine, go and add a phy:/dev/rsd0d
disk to point to my 1TB ZFS drive.  VM boots fine, sees the correct
drive size.  Go to zpool import it and:

- zpool import completely hangs.  phy: seems broken. I've seen some
mention of this problem w/ FreeBSD domUs here.  I don't believe
there's a PR for it.

This is with dom0_mem set and autoballon disabled in both the xm
config file & the xl config file.  The domU was started using xm, as
PV is known to be broken with xl.

The LVM patch is basically identical to haad's xen 3.3 patch, except
it patches tools/ioemu-qemu-xen/block-raw-posix.c now.  The one issue
I ran into was qemu seems to use a modified version of NetBSD's
sys/queue.h and even defines _SYS_QUEUE_H_, so that the NetBSD version
never gets included.  This means SLIST_ENTRY never gets defined, which
breaks the sys/disk.h include.  Not sure of a clean way to solve this,
so I just defiend SLIST_ENTRY in block-raw-posix.c, which is a bad
kludge.  Not sure if this even works, as I ended up not even using
LVM, but maybe this is used for phy: too?

All in all, which does work seems to work really well, and to me
domU's seem 'faster' than they do under a Linux dom0.  I really
appreciate all of the hard work that went into this!

-Dustin

Xen kernel output under a XEN3PAE_DOM0:

WARNING: couldn't open ffs (//ffs/ffs.kmod)
Loading /netbsd-XEN3PAE_DOM0
WARNING: 1 module failed to load
 __  __            _  _    _   ___
 \ \/ /___ _ __   | || |  / | / _ \
  \  // _ \ '_ \  | || |_ | || | | |
  /  \  __/ | | | |__   _|| || |_| |
 /_/\_\___|_| |_|    |_|(_)_(_)___/

(XEN) Xen version 4.1.0 (root@) (gcc version 4.1.3 20080704 prerelease (NetBSD n
b2 20081120)) Tue May 10 22:18:47 UTC 2011
(XEN) Latest ChangeSet: unavailable
(XEN) Bootloader: unknown
(XEN) Command line: dom0_mem=752M console=com1 com1=115200,8n1
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: none; EDID transfer time: 0 seconds
(XEN)  EDID info not retrieved because no DDC retrieval method detected
(XEN) Disc information:
(XEN)  Found 2 MBR signatures
(XEN)  Found 2 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009e000 (usable)
(XEN)  000000000009e000 - 00000000000a0000 (reserved)
(XEN)  00000000000e4000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000e7ee0000 (usable)
(XEN)  00000000e7ee0000 - 00000000e7ee9000 (ACPI data)
(XEN)  00000000e7ee9000 - 00000000e7f00000 (ACPI NVS)
(XEN)  00000000e7f00000 - 00000000e8000000 (reserved)
(XEN)  00000000f0000000 - 00000000f4000000 (reserved)
(XEN)  00000000fec00000 - 00000000fec10000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 0000000118000000 (usable)
(XEN) ACPI: RSDP 000F5F90, 0014 (r0 PTLTD )
(XEN) ACPI: RSDT E7EE2BEC, 003C (r1 PTLTD    RSDT    6040000  LTP        0)
(XEN) ACPI: FACP E7EE8E60, 0074 (r1 INTEL            6040000 PTL         3)
(XEN) ACPI: DSDT E7EE4014, 4E4C (r1  INTEL GLENWOOD  6040000 MSFT  100000E)
(XEN) ACPI: FACS E7EE9FC0, 0040
(XEN) ACPI: MCFG E7EE8ED4, 003C (r1 PTLTD    MCFG    6040000  LTP        0)
(XEN) ACPI: HPET E7EE8F10, 0038 (r1 PTLTD  HPETTBL   6040000  LTP        1)
(XEN) ACPI: APIC E7EE8F48, 0090 (r1 PTLTD        APIC    6040000  LTP        0)
(XEN) ACPI: BOOT E7EE8FD8, 0028 (r1 PTLTD  $SBFTBL$  6040000  LTP        1)
(XEN) ACPI: SSDT E7EE2C28, 13EC (r1  PmRef    CpuPm     3000 INTL 20050228)
(XEN) System RAM: 4094MB (4192760kB)
(XEN) Domain heap initialised
(XEN) Processor #0 6:15 APIC version 20
(XEN) Processor #1 6:15 APIC version 20
(XEN) Processor #2 6:15 APIC version 20
(XEN) Processor #3 6:15 APIC version 20
(XEN) IOAPIC[0]: apic_id 4, version 32, address 0xfec00000, GSI 0-23
(XEN) IOAPIC[1]: apic_id 5, version 32, address 0xfecc0000, GSI 24-47
(XEN) Enabling APIC mode:  Flat.  Using 2 I/O APICs
(XEN) Table is not found!
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2394.069 MHz processor.
(XEN) Initing memory sharing.
(XEN) I/O virtualisation disabled
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN) HVM: ASIDs disabled.
(XEN) HVM: VMX enabled
(XEN) Brought up 4 CPUs
(XEN) *** LOADING DOMAIN 0 ***
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 32-bit, PAE, lsb, paddr 0xc0100000 -> 0xc0b2451c
(XEN)  Dom0 symbol map 0xc0b2451c -> 0xc0c06160
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000113000000->0000000114000000 (188416 pages to be all
ocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: 00000000c0100000->00000000c0c06160
(XEN)  Init. ramdisk: 00000000c0c07000->00000000c0c07000
(XEN)  Phys-Mach map: 00000000c0c07000->00000000c0cc3000
(XEN)  Start info:    00000000c0cc3000->00000000c0cc34b4
(XEN)  Page tables:   00000000c0cc4000->00000000c0cd2000
(XEN)  Boot stack:    00000000c0cd2000->00000000c0cd3000
(XEN)  TOTAL:         00000000c0000000->00000000c1000000
(XEN)  ENTRY ADDRESS: 00000000c0100000
(XEN) Dom0 has maximum 4 VCPUs
(XEN) Scrubbing Free RAM: ................................done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: Errors and warnings
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch
input to Xen)
(XEN) Freed 204kB init memory.
(XEN) mm.c:2424:d0 Bad type (saw 1400000000000001 != exp 7000000000000000) for m
fn 113cc4 (pfn cc4)
(XEN) mm.c:907:d0 Error getting mfn 113cc4 (pfn cc4) from L1 entry 0000000113cc4
003 for l1e_owner=0, pg_owner=0
(XEN) mm.c:1222:d0 Failure in alloc_l1_table: entry 196
(XEN) mm.c:2177:d0 Error while validating mfn 113ce0 (pfn ce0) for type 10000000
00000000: caf=8000000000000003 taf=1000000000000001
(XEN) mm.c:915:d0 Attempt to create linear p.t. with write perms
(XEN) mm.c:1366:d0 Failure in alloc_l2_table: entry 6
(XEN) mm.c:2177:d0 Error while validating mfn 113cd9 (pfn cd9) for type 22000000
00000000: caf=8000000000000003 taf=2200000000000001
(XEN) mm.c:1476:d0 Failure in alloc_l3_table: entry 3
(XEN) mm.c:2177:d0 Error while validating mfn 113cd4 (pfn cd4) for type 30000000
00000000: caf=8000000000000003 taf=3000000000000001
(XEN) mm.c:2985:d0 Error while pinning mfn 113cd4
(XEN) domain_crash_sync called from entry.S
(XEN) Domain 0 (vcpu#0) crashed on cpu#0:
(XEN) ----[ Xen-4.1.0  x86_64  debug=n  Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e019:[<00000000c022a094>]
(XEN) RFLAGS: 0000000000000246   EM: 1   CONTEXT: pv guest
(XEN) rax: 0000000000000001   rbx: 0000000000000104   rcx: 0000000000000000
(XEN) rdx: 00000000c0a37d54   rsi: 00000000c0a37d54   rdi: 0000000000000002
(XEN) rbp: 00000000c0a3db14   rsp: 00000000c0a3db08   r8:  0000000000000000
(XEN) r9:  0000000000000000   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: 0000000000000000   r13: 0000000000000000   r14: 0000000000000000
(XEN) r15: 0000000000000000   cr0: 000000008005003b   cr4: 00000000000026f0
(XEN) cr3: 0000000114383000   cr2: 0000000000000000
(XEN) ds: e021   es: e021   fs: 0000   gs: 0000   ss: e021   cs: e019
(XEN) Guest stack trace from esp=c0a3db08:
(XEN)   c022a094 0001e019 00000046 c0a3db54 c0658533 c0a31209 c0a3db60 00000000
(XEN)   00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)   c0a3db60 c0a3db70 00007ff0 c0a3db84 c0787da5 c0a37d54 00000000 00000000
(XEN)   00000000 00000000 00000002 00113cd4 00000003 00003350 00cd4000 c0a3dc24
(XEN)   c07882cc 13cd4000 00000001 00000002 00000000 00000000 00000000 00cd8000
(XEN)   00000000 00000000 00000000 00000000 00000000 00000008 0000000e c0cd4000
(XEN)   c0cc6000 c0cd5000 00cd5000 c0ce0708 c0cd4000 c0cd5000 00000000 c0cda000
(XEN)   c0a3d000 c0ce2000 00000008 00000607 c0ce2000 00000000 c0ce2000 00cd8000
(XEN)   00000000 00000000 c0cda000 00000000 c0cd4000 00000007 00000008 c0a3dc54
(XEN)   c0788641 00000008 00000000 00000000 0000000e c0cd4000 00000008 c0cc6000
(XEN)   756e6547 c0cc3200 c0b22200 00000000 c0100041 00000000 c09959d0 c0992960
(XEN)   00000000 00000000 c0989060 c098aa28 c07fdf64 00000000 00000000 00000000
(XEN)   00000000 00000000 00000000 00000000 c0995180 00000000 c07fdb80 c095bac0
(XEN)   c0982100 c0958ec0 00000000 c07fe7a0 00000000 00000000 00000000 00000000
(XEN)   00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)   00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)   00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)   00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)   00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN)   00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
Follow-Ups:
- Re: Attempt at using Xen 4.1 on -current
  - From: Hugo Silva
Prev by Date: Re: Dom0 ballooning: crash on guest start
Next by Date: Hypervisor Crash, xen4.1
Previous by Thread: broken link on xen article
Next by Thread: Re: Attempt at using Xen 4.1 on -current
Indexes:
Home | Main Index | Thread Index | Old Index