problems exporting PCI devices to domU

[Karl Janmar <karl%utopiafoundation.org@localhost>]

Hi,

I have got xen working on my machine and I'm able to run both Linux and netbsd
as a domU.
For Linux I use a physical disk and for netbsd I use a file together with vnd,
that seems working pretty well.

But when I try to give my netbsd domU access to a PCI device it fail. I have
read the xen/howto instructions on www.netbsd.org and also some post about
exporting PCI devices, but I still doesn't get it working.

I have hidden the PCI device from my dom0 and now I won't see the device when I
list the PCI bus with pcictl.
Prepared my domU kernel with the extra lines in the kernel config to enable the
PCI bus (from www.netbsd.org xen/howto).
Then I add a "pci = ['3,13,0']" line to the xen config.
After that when I do "xm create netbsd" I get this:
Using config file "/etc/xen/netbsd".
Error: Error creating domain: (19, 'Operation not supported by device')

A "xm log" says (sorry for the bad layout):
----------------- output cut -------------------------
[2005-10-09 00:20:34 xend] DEBUG (XendDomainInfo:720) init_domain> Created
domain=2 name=netbsd memory=256
[2005-10-09 00:20:34 xend] INFO (console:94) Created console id=11 domain=2
port=9602
[2005-10-09 00:20:34 xend] DEBUG (XendDomainInfo:1130) Creating vbd dom=2
uname=phy:/dev/vnd0d
[2005-10-09 00:20:34 xend] DEBUG (blkif:155) Connecting blkif
<BlkifBackendInterface 2 0>
[2005-10-09 00:20:34 xend] DEBUG (XendDomainInfo:1178) Creating pci device dom=2
bus=3 dev=13 func=0
[2005-10-09 00:20:34 xend] ERROR (SrvBase:162) op=create: Error creating domain:
(19, 'Operation not supported by device')
Traceback (most recent call last):
File "/usr/pkg/lib/python2.4/site-packages/xen/xend/server/SrvBase.py", line
107, in _perform
val = op_method(op, req)
File "/usr/pkg/lib/python2.4/site-packages/xen/xend/server/SrvDomainDir.py",
line 71, in op_create
raise XendError("Error creating domain: " + str(ex))
XendError: Error creating domain: (19, 'Operation not supported by device')
[2005-10-09 00:20:34 xend] INFO (XendRoot:113) EVENT> xend.console.create [11,
2, 9602]
-------------------------- end ---------------------------------

My dom0 system is a 3.0_BETA from the anoncvs about the same time the kernel is
compiled, this is the beginning of the dmesg for the dom0 system:
---------------- dmesg cut ------------------------------------
NetBSD 3.0_BETA (NGONG_XEN) #1: Sun Oct  2 10:40:23 CEST 2005
        
karl%ngong.utopiafoundation.org@localhost:/usr/src/sys/arch/i386/compile/NGONG_XEN
total memory = 1457 MB
avail memory = 1425 MB
mainbus0 (root)
cpu0 at mainbus0: (uniprocessor)
cpu0: Intel Pentium 4 (686-class), 2806.43 MHz, id 0xf29
cpu0: features bfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR>
cpu0: features bfebfbff<PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX>
cpu0: features bfebfbff<FXSR,SSE,SSE2,SS,HTT,TM,SBF>
cpu0: I-cache 12K uOp cache 8-way, D-cache 8 KB 64B/line 4-way
cpu0: L2 cache 512 KB 64B/line 8-way
cpu0: ITLB 4K/4M: 64 entries
cpu0: DTLB 4K/4M: 64 entries
cpu0: 16 page colors
hypervisor0 at mainbus0
debug virtual interrupt using event channel 1
misdirect virtual interrupt using event channel 0
Domain controller: using event channel 2
npx0 at hypervisor0: using exception 16
pci0 at hypervisor0 bus 0
pci0: i/o space, memory space enabled
pchb0 at pci0 dev 0 function 0
pchb0: Intel 82875P Host (rev. 0x02)
pchb0: random number generator enabled
agp0 at pchb0: aperture at 0xf0000000, size 0x8000000
ppb0 at pci0 dev 1 function 0: Intel 82875P AGP (rev. 0x02)
pci at ppb0 bus 1 not configured
ppb1 at pci0 dev 3 function 0: Intel 82875P PCI-CSA Bridge (rev. 0x02)
pci at ppb1 bus 2 not configured
------------------ end cut --------------------------------

If I remove my "pci = ['3,13,0']" line from the xen config I'm able to boot the
domU again. Here is a snippit of the dmesg from that domU, this might help
understand what's going on:
--------------- dmesg cut --------------------------
hypervisor0 at mainbus0
debug virtual interrupt using event channel 2
misdirect virtual interrupt using event channel 0
Domain controller: using event channel 1
xencons0 at hypervisor0: Xen Virtual Console Driver
xencons0: console major 143, unit 0
Initialising Xen virtual ethernet frontend driver.
npx0 at hypervisor0: using exception 16
PCI_PROBE_ROOT_BUSES: 0x0 0x0 0x0 0x0 0x0 0x0 0x0 0x0
Xen clock: using event channel 3
-------------- end cut -----------------------

Note that I use netbsd-current as a domU and 3.0_BETA as dom0,
could this be a problem?
I personally don't think so as the error seems to occur before the kernel is
loaded, but I don't know enough of xen to say something for sure.

regards,
Karl Janmar
karl%utopiafoundation.org@localhost