kern/59174: ixg(4) pci_machdep.c setting mismatch

[uchiyama.yasushi%gmail.com@localhost]

>Number:         59174
>Category:       kern
>Synopsis:       ixg(4) pci_machdep.c setting mismatch
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    kern-bug-people
>State:          open
>Class:          support
>Submitter-Id:   net
>Arrival-Date:   Thu Mar 13 13:05:00 +0000 2025
>Originator:     UCHIYAMA Yasushi
>Release:        NetBSD 10.99.12
>Organization:
>Environment:
NetBSD diamond 10.99.12 NetBSD 10.99.12 (SICKLEMOON) #16: Wed Mar 12 12:32:46 UTC 2025  uch@diamond:/usr/src/sys/arch/amd64/compile/SICKLEMOON amd64
>Description:
INSPUR Intel X540-T2(X540-AT2)
HP ProLiant N54L

this machine have MSI but disabled @sys/arch/x86/pci_machdep.c
	/*
	 * Don't enable MSI on a HyperTransport bus.  In order to
	 * determine that bus 0 is a HyperTransport bus, we look at
	 * device 24 function 0, which is the HyperTransport
	 * host/primary interface integrated on most 64-bit AMD CPUs.
	 * If that device has a HyperTransport capability, bus 0 must
	 * be a HyperTransport bus and we disable MSI.
	 */
	if (24 < pci_bus_maxdevs(pc, 0)) {
		tag = pci_make_tag(pc, 0, 24, 0);
		if (pci_get_capability(pc, tag, PCI_CAP_LDT, NULL, NULL)) {
			pba->pba_flags &= ~PCI_FLAGS_MSI_OKAY;
			pba->pba_flags &= ~PCI_FLAGS_MSIX_OKAY;
		}
	}

In ixgbe_configure_interrupts()@ixbge.c, directly check the MSI-X.
	/* First try MSI-X */
	msgs = pci_msix_count(sc->osdep.pc, sc->osdep.tag);
pci_msix_count returns 64. so driver use MSI.
but pba->pba_flags PCI_FLAGS_MSI_OKAY|PCI_FLAGS_MSIX_OKAY were disabled.
then pci_msix_alloc_exact() returns ENODEV.

However, on this machine, enabling MSI on the HyperTransport bus @pci_machdep.c performs better (using MSI-X).

>How-To-Repeat:

>Fix:
diff -u -r1.354 ixgbe.c
--- ixgbe.c	10 Jul 2024 03:26:30 -0000	1.354
+++ ixgbe.c	12 Mar 2025 12:27:07 -0000
@@ -924,6 +924,17 @@
 	hw->mac.ops.set_lan_id(hw);
 	ixgbe_init_device_features(sc);
 
+	/* Check the host bridge functionality */
+	if (!(pa->pa_flags & PCI_FLAGS_MSI_OKAY)) {
+		sc->feat_cap &= ~(IXGBE_FEATURE_MSI | IXGBE_FEATURE_MSIX);
+		aprint_normal_dev (dev, "PCI host bridge does not "
+		    "support MSI. Use legacy interrupts.\n");
+	} else if (!(pa->pa_flags & PCI_FLAGS_MSIX_OKAY)) {
+		sc->feat_cap &= ~IXGBE_FEATURE_MSIX;
+		aprint_normal_dev (dev, "PCI host bridge does not "
+		    "support MSI-X. Drop MSI-X feature of card.\n");
+	}
+
 	if (ixgbe_configure_interrupts(sc)) {
 		error = ENXIO;
 		goto err_out;