[src/trunk]: src/sys/dev/pci/ixgbe Fix ixg(4) Rx interrupt stall when Rx buff...

[knakahara <knakahara%NetBSD.org@localhost>]

details:   https://anonhg.NetBSD.org/src/rev/ab4e879925d2
branches:  trunk
changeset: 958746:ab4e879925d2
user:      knakahara <knakahara%NetBSD.org@localhost>
date:      Mon Jan 18 09:09:04 2021 +0000

description:
Fix ixg(4) Rx interrupt stall when Rx buffers are exhausted.

Current ixgbe_rxeof() implementation calls ixgbe_refresh_mbufs()(and
ixgbe_getjcl()) after ixgbe_rx_input()(and if_percpuq_enqueue()).
And ixg(4) uses its own m_ext structure.  That causes Rx interrupt
stall when Rx buffers are exhausted.  e.g. TCP iperf3 with large
windows size.  Furthermore, Rx descriptor problem has occurred
after stoppping the load.

To fix that problem, ixgbe_rxeof() must call ixgbe_getjcl() before
ixgbe_rx_input(), and must discard the new packet when ixgbe_getjcl()
fails.

By the way, ixg(4) should use MCLGET() instead of own m_ext structure.

ok'ed by msaitoh@n.o, thanks.

diffstat:

 sys/dev/pci/ixgbe/ix_txrx.c |  20 +++++++++++++++++---
 1 files changed, 17 insertions(+), 3 deletions(-)

diffs (58 lines):

diff -r 460e7d74e49f -r ab4e879925d2 sys/dev/pci/ixgbe/ix_txrx.c
--- a/sys/dev/pci/ixgbe/ix_txrx.c       Mon Jan 18 08:29:32 2021 +0000
+++ b/sys/dev/pci/ixgbe/ix_txrx.c       Mon Jan 18 09:09:04 2021 +0000
@@ -1,4 +1,4 @@
-/* $NetBSD: ix_txrx.c,v 1.63 2020/04/17 02:21:25 msaitoh Exp $ */
+/* $NetBSD: ix_txrx.c,v 1.64 2021/01/18 09:09:04 knakahara Exp $ */
 
 /******************************************************************************
 
@@ -1818,6 +1818,7 @@
 
        for (i = rxr->next_to_check; count != 0;) {
                struct mbuf *sendmp, *mp;
+               struct mbuf *newmp;
                u32         rsc, ptype;
                u16         len;
                u16         vtag = 0;
@@ -1860,6 +1861,15 @@
                        goto next_desc;
                }
 
+               /* pre-alloc new mbuf */
+               newmp = ixgbe_getjcl(&rxr->jcl_head, M_NOWAIT, MT_DATA, M_PKTHDR,
+                   rxr->mbuf_sz);
+               if (newmp == NULL) {
+                       rxr->rx_discarded.ev_count++;
+                       ixgbe_rx_discard(rxr, i);
+                       goto next_desc;
+               }
+
                bus_dmamap_sync(rxr->ptag->dt_dmat, rbuf->pmap, 0,
                    rbuf->buf->m_pkthdr.len, BUS_DMASYNC_POSTREAD);
 
@@ -1908,7 +1918,8 @@
                 */
                sendmp = rbuf->fmp;
                if (sendmp != NULL) {  /* secondary frag */
-                       rbuf->buf = rbuf->fmp = NULL;
+                       rbuf->buf = newmp;
+                       rbuf->fmp = NULL;
                        mp->m_flags &= ~M_PKTHDR;
                        sendmp->m_pkthdr.len += mp->m_len;
                } else {
@@ -1927,10 +1938,13 @@
                                        sendmp->m_len = len;
                                        rxr->rx_copies.ev_count++;
                                        rbuf->flags |= IXGBE_RX_COPY;
+
+                                       m_freem(newmp);
                                }
                        }
                        if (sendmp == NULL) {
-                               rbuf->buf = rbuf->fmp = NULL;
+                               rbuf->buf = newmp;
+                               rbuf->fmp = NULL;
                                sendmp = mp;
                        }