[src/trunk]: src/sys/dev/pci/ixgbe Fix ixg(4) Rx interrupt stall when Rx buff...

[knakahara <knakahara%NetBSD.org@localhost>]

details:   https://anonhg.NetBSD.org/src/rev/3e1052f686b2
branches:  trunk
changeset: 979970:3e1052f686b2
user:      knakahara <knakahara%NetBSD.org@localhost>
date:      Mon Jan 18 09:09:04 2021 +0000

description:
Fix ixg(4) Rx interrupt stall when Rx buffers are exhausted.

Current ixgbe_rxeof() implementation calls ixgbe_refresh_mbufs()(and
ixgbe_getjcl()) after ixgbe_rx_input()(and if_percpuq_enqueue()).
And ixg(4) uses its own m_ext structure.  That causes Rx interrupt
stall when Rx buffers are exhausted.  e.g. TCP iperf3 with large
windows size.  Furthermore, Rx descriptor problem has occurred
after stoppping the load.

To fix that problem, ixgbe_rxeof() must call ixgbe_getjcl() before
ixgbe_rx_input(), and must discard the new packet when ixgbe_getjcl()
fails.

By the way, ixg(4) should use MCLGET() instead of own m_ext structure.

ok'ed by msaitoh@n.o, thanks.

diffstat:

 sys/dev/pci/ixgbe/ix_txrx.c |  20 +++++++++++++++++---
 1 files changed, 17 insertions(+), 3 deletions(-)

diffs (58 lines):

diff -r 8694af9dc7b1 -r 3e1052f686b2 sys/dev/pci/ixgbe/ix_txrx.c
--- a/sys/dev/pci/ixgbe/ix_txrx.c       Mon Jan 18 08:29:32 2021 +0000
+++ b/sys/dev/pci/ixgbe/ix_txrx.c       Mon Jan 18 09:09:04 2021 +0000
@@ -1,4 +1,4 @@
-/* $NetBSD: ix_txrx.c,v 1.63 2020/04/17 02:21:25 msaitoh Exp $ */
+/* $NetBSD: ix_txrx.c,v 1.64 2021/01/18 09:09:04 knakahara Exp $ */
 
 /******************************************************************************
 
@@ -1818,6 +1818,7 @@
 
        for (i = rxr->next_to_check; count != 0;) {
                struct mbuf *sendmp, *mp;
+               struct mbuf *newmp;
                u32         rsc, ptype;
                u16         len;
                u16         vtag = 0;
@@ -1860,6 +1861,15 @@
                        goto next_desc;
                }
 
+               /* pre-alloc new mbuf */
+               newmp = ixgbe_getjcl(&rxr->jcl_head, M_NOWAIT, MT_DATA, M_PKTHDR,
+                   rxr->mbuf_sz);
+               if (newmp == NULL) {
+                       rxr->rx_discarded.ev_count++;
+                       ixgbe_rx_discard(rxr, i);
+                       goto next_desc;
+               }
+
                bus_dmamap_sync(rxr->ptag->dt_dmat, rbuf->pmap, 0,
                    rbuf->buf->m_pkthdr.len, BUS_DMASYNC_POSTREAD);
 
@@ -1908,7 +1918,8 @@
                 */
                sendmp = rbuf->fmp;
                if (sendmp != NULL) {  /* secondary frag */
-                       rbuf->buf = rbuf->fmp = NULL;
+                       rbuf->buf = newmp;
+                       rbuf->fmp = NULL;
                        mp->m_flags &= ~M_PKTHDR;
                        sendmp->m_pkthdr.len += mp->m_len;
                } else {
@@ -1927,10 +1938,13 @@
                                        sendmp->m_len = len;
                                        rxr->rx_copies.ev_count++;
                                        rbuf->flags |= IXGBE_RX_COPY;
+
+                                       m_freem(newmp);
                                }
                        }
                        if (sendmp == NULL) {
-                               rbuf->buf = rbuf->fmp = NULL;
+                               rbuf->buf = newmp;
+                               rbuf->fmp = NULL;
                                sendmp = mp;
                        }