Re: RFC: softint-based if_input

[Taylor R Campbell <campbell+netbsd-tech-kern%mumble.net@localhost>]

   Date: Thu, 28 Jan 2016 11:58:05 +0900
   From: Ryota Ozaki <ozaki-r%netbsd.org@localhost>

   On Thu, Jan 28, 2016 at 12:17 AM, Taylor R Campbell
   <campbell+netbsd-tech-kern%mumble.net@localhost> wrote:
   >             So vanilla might make less efficient use of the CPU cache,
   > and vanilla might leave the rxq full for longer so that the device
   > cannot fill it as quickly with incoming packets.

   That might be true. If so, the real question may be why the old
   implementation isn't efficient compared to the new one.

By `the old implementation', do you mean the one with pcq instead of
ifq, softint-rx?  It's only a little bit slower than softint-rx-ifq.
My wild guess is that all the atomics and memory barriers in pcq slow
it down.

   > Another experiment that might be worthwhile is to bind the interrupt
   > to a specific CPU, and then use splnet instead of WM_RX_LOCK to avoid
   > acquiring and releasing a lock for each packet.

   In the measurements, all interrupts are already delivered to CPU#0.
   Removing the lock doesn't change the results. I guess acquiring and
   releasing a lock (w/o contentions) are low overhead. Note that
   wm has a RX lock per HW queue, so RX processing can be done with no
   lock contention basically.

OK, makes sense.

   >  (On Intel >=Haswell,
   > we should use transactional memory to avoid bus traffic for that
   > anyway (and maybe invent an MD pcq(9) that does the same).  But the
   > experiment with wm(4) is easier, and not everyone has transactional
   > memory.)

   How does transactional memory help?

It doesn't.  I was just blathering aimlessly like I often do about
things I don't understand well enough but am idly thinking about.  (It
also turns out -- I hadn't heard until today -- that transactional
memory is broken on Haswell!)