Re: PR/55822 CVS commit: src/sys/dev/pci

[Bert Kiers <kiers%original.xs4all.nl@localhost>]

The following reply was made to PR kern/55822; it has been noted by GNATS.

From: Bert Kiers <kiers%original.xs4all.nl@localhost>
To: s ymgch <s.ymgch228%gmail.com@localhost>
Cc: gnats-bugs%netbsd.org@localhost, kiers%original.xs4all.nl@localhost,
	Shoichi Yamaguchi <yamaguchi%netbsd.org@localhost>, gnats-admin%netbsd.org@localhost,
	netbsd-bugs%netbsd.org@localhost
Subject: Re: PR/55822 CVS commit: src/sys/dev/pci
Date: Wed, 2 Dec 2020 13:10:11 +0100

 On Wed, Dec 02, 2020 at 10:45:40AM +0900, s ymgch wrote:
 > Hi,
 
 Hi,
 
 > The reason why iavf(4) did not work on bhyve may be the combination between
 > bhyve and NetBSD, because it didn't appear on ESXi and Linux as far as I know.
 > 
 > Can you let me know about the environment of the VM?
 > 
 > 1. Which version of bhyve did you use?
 
 The one that comes with FreeBSD 12.2-RELEASE r366954.
 bhyve nor bhyvectl have an option that shows the version, afaik.
 
 > 2. Which driver version of ixl did the bhyve use?
 >    - This is not the FW version shown in a probe message.
 
 I see
 
 ixl0: <Intel(R) Ethernet Connection X722 for 10GBASE-T - 2.3.0-k> mem 0xc4000000-0xc4ffffff,0xc5008000-0xc500ffff irq 36 at device 0.0 numa-domain 0 on pci8
 
 so 2.3.0-k
 
 On
 
 https://downloadcenter.intel.com/download/25160/Intel-Network-Adapter-Driver-for-PCIe-40-Gigabit-Ethernet-Network-Connection-under-FreeBSD-
 
 I see version 1.12.3.  I'll go try that.  But I do not understand how
 2.3.0-k relates to 1.12.3 yet.
 
 > 3. Can other network devices using PCI passthrough, for example ixl, ixg and ixv
 >    work well on the same VM (not NetBSD-9 host)
 
 I just have ixl NICs on this machine.  So I don't know.
 
 (In an other server I have a X540 NIC, that is ix on the FreeBSD side
 and ixv in the NetBSD guest.  That works stable and fast, but just for
 *one* guest, I documented that story on
 https://netbsd.itsx.net/freebsd+bhyve+sriov.html, which is work in 
 progress).
 
 On this machine with ixl, multiple FreeBSD and Linux guests work.
 
 
 > And can you tell me the message when iavf(4) is loaded with -i
 > debug_level=3 option?
 > You may already know, but just in case, the usage of the debug option
 > is following:
 > 
 > 1. build and install kernel not containing iavf(4)
 > 2. build and install modules
 > 3. run the kernel and do "modload -i debug_level=3 if_iavf" command
 
 Boot messages:
 
 [     1.033888] Intel XL710 Ethernet Virtual Function (ethernet network, revision 0x01) at pci0 dev 2 function 0 not configured
 [     1.033888] Intel X722 10GbE VF (ethernet network, revision 0x09) at pci0 dev 3 function 0 not configured
 
 and then
 
 # modload -i debug_level=3 if_iavf
 [  61.5983006] iavf: debug level=3
 [  61.6083181] iavf0 at pci0 dev 2 function 0iavf0: post
 [  61.6283146] iavf0: flags 0x3400<SI,BUF,DB> opcode 0801
 [  61.6283146] iavf0: datalen 8 retval 0
 [  61.6283146] iavf0: vc-opcode 1 (GET_VERSION)
 [  61.6381634] iavf0: vc-retval 0
 [  61.6381634] iavf0: cookie 0000000000000001
 [  61.6381634] iavf0: 00000000 00000000 00000001 07fd0000
 [  61.9183239] iavf0: atq timedout
 :unable to get VF interface version
 [  61.9379397] iavf1 at pci0 dev 3 function 0iavf1: post
 [  61.9483230] iavf1: flags 0x3400<SI,BUF,DB> opcode 0801
 [  61.9483230] iavf1: datalen 8 retval 0
 [  61.9483230] iavf1: vc-opcode 1 (GET_VERSION)
 [  61.9483230] iavf1: vc-retval 0
 [  61.9582973] iavf1: cookie 0000000000000001
 [  61.9582973] iavf1: 00000000 00000000 00000001 080d5000
 [  62.2182973] iavf1: atq timedout
 :unable to get VF interface version
 
 
 I'll put complete dmesg and things on
 https://netbsd.itsx.net/PR55822/ soon.
 
 Grtnx,
 --
 Bert