CVS commit: [netbsd-10] src/sys/arch/xen

["Martin Husemann" <martin%netbsd.org@localhost>]

Module Name:    src
Committed By:   martin
Date:           Mon Jul 31 15:23:03 UTC 2023

Modified Files:
        src/sys/arch/xen/include [netbsd-10]: hypervisor.h xenring.h
        src/sys/arch/xen/x86 [netbsd-10]: cpu.c
        src/sys/arch/xen/xen [netbsd-10]: if_xennet_xenbus.c xbd_xenbus.c
            xbdback_xenbus.c xencons.c xengnt.c xennetback_xenbus.c
        src/sys/arch/xen/xenbus [netbsd-10]: xenbus_comms.c

Log Message:
Pull up following revision(s) (requested by riastradh in ticket #268):

        sys/arch/xen/xenbus/xenbus_comms.c: revision 1.25
        sys/arch/xen/xenbus/xenbus_comms.c: revision 1.26
        sys/arch/xen/xen/xennetback_xenbus.c: revision 1.110
        sys/arch/xen/xen/xennetback_xenbus.c: revision 1.111
        sys/arch/xen/xen/xennetback_xenbus.c: revision 1.112
        sys/arch/xen/x86/cpu.c: revision 1.144
        sys/arch/xen/x86/cpu.c: revision 1.145
        sys/arch/xen/include/hypervisor.h: revision 1.56
        sys/arch/xen/include/hypervisor.h: revision 1.57
        sys/arch/xen/xen/xbdback_xenbus.c: revision 1.102
        sys/arch/xen/xen/xbdback_xenbus.c: revision 1.103
        sys/arch/xen/include/xenring.h: revision 1.7
        sys/arch/xen/xen/xennetback_xenbus.c: revision 1.109
        sys/arch/xen/xen/xengnt.c: revision 1.40
        sys/arch/xen/xen/xengnt.c: revision 1.41
        sys/arch/xen/xen/if_xennet_xenbus.c: revision 1.129
        sys/arch/xen/xen/xencons.c: revision 1.51
        sys/arch/xen/xen/xencons.c: revision 1.52
        sys/arch/xen/xen/xencons.c: revision 1.53
        sys/arch/xen/xen/xbd_xenbus.c: revision 1.130 (patch)
        sys/arch/xen/xen/xbd_xenbus.c: revision 1.131 (patch)

xen: Fix sense of xen_rmb/wmb to make sense.

Use membar_acquire and membar_release, not membar_consumer and
membar_producer, out of paranoia -- that better matches Linux's
rmb/wmb (at least for non-I/O loads and stores).

Proposed on port-xen:
https://mail-index.netbsd.org/port-xen/2022/07/13/msg010248.html

xen/x86/cpu.c: Membar audit.

I see no reason for store-before-load ordering here; as far as I'm
aware, evtchn_upcall_mask is only shared between a (v)CPU and its
(hypervisor) interrupts, not other (v)CPUs.

xennet(4): Membar audit.
- xennet_tx_complete: Other side owns rsp_prod, giving us responses
  to tx commands.  We own rsp_cons, recording which responess we've
  processed already.
  1. Other side initializes responses before advancing rsp_prod, so
     we must observe rsp_prod before trying to examine the responses.
     Hence load from rsp_prod must be followed by xen_rmb.
     (Can this just use atomic_load_acquire?)
  2. As soon as other side observes rsp_event, it may start to
     overwrite now-unused response slots, so we must finish using the
     response before advancing rsp_cons.  Hence we must issue xen_wmb
     before store to rsp_event.
     (Can this just use atomic_store_release?)
     (Should this use RING_FINAL_CHECK_FOR_RESPONSES?)
  3. When loop is done and we set rsp_event, we must ensure the other
     side has had a chance to see that we want more before we check
     whether there is more to consume; otherwise the other side might
     not bother to send us an interrupt.  Hence after setting
     rsp_event, we must issue xen_mb (store-before-load) before
     re-checking rsp_prod.
- xennet_handler (rx): Same deal, except the xen_mb is buried in
  RING_FINAL_CHECK_FOR_RESPONSES.  Unclear why xennet_tx_complete has
  this open-coded while xennet_handler (rx) uses the macro.

xbd(4): Membar audit.
After consuming slots, must issue xen_wmb before notifying the other
side that we've consumed them in RING_FINAL_CHECK_FOR_RESPONSES.
xbdback(4): Membar audit.

After consuming request slots, must issue xen_wmb notifying the other
side that we've consumed them in RING_FINAL_CHECK_FOR_REQUESTS.

xencons(4): Membar audit.
- xenconscn_getc: Once we have consumed an input slot, it is clearer
  to issue xen_wmb (release, i.e., load/store-before-store) before
  advancing in_cons so that the update becomes a store-release
  freeing the input slot for the other side to reuse.
- xenconscn_putc: After filling an output slot, must issue xen_wmb
  (release, i.e., load/store-before-store) before advancing out_prod,
  and another one before notifying the other side of the advance.

xencons(4): Reduce unnecessary membars.
- xencons_handler: After advancing in_cons, only need one xen_wmb
  before notifying the hypervisor that we're ready for more.
  (XXX Should this do xen_mb and re-check in_prod at that point, or
  does hypervisor_notify_via_evtchn obviate the need for this?)
- xenvonscn_getc: After reading in_prod, only need one xen_rmb before
  using the slots it is telling us are now ready.

xengnt(4): Membar audit.
This had the sense of membars reversed, presumably because xen_rmb
and xen_wmb had gotten reversed at some point.
xenbus_comms.c: Membar audit.

This had the sense of membars reversed, presumably because xen_rmb
and xen_wmb had gotten reversed at some point.

xennetback(4): Fix xennetback_evthandler loop.
- After observing the other side has produced pending tx requests by
  reading sring->req_prod, must issue xen_rmb before touching them.
  Despite all the effort to use the heavy-weight
  RING_FINAL_CHECK_FOR_REQUESTS on each request in the loop, this
  barrier was missing.
- No need to update req_cons at each iteration in the loop.  It's
  private.  Just update it once at the end.
- After consuming requests, must issue xen_wmb before releasing the
  slots with RING_FINAL_CHECK_FOR_REQUEST for the other side to
  reuse.

xennetback(4): Fix membars in xennetback_rx_copy_process.
- No need for barrier around touching req_cons and rsp_prod_pvt,
  which are private.
- RING_PUSH_RESPONSES_AND_CHECK_NOTIFY already issues xen_wmb, no
  need to add one explicitly.
- After pushing responses, must issue xen_wmb (not xen_rmb) before
  hypervisor_notify_via_evtchn.

xennetback(4): Omit needless membars in xennetback_connect.
xneti is a private data structure to which we have exclusive access
here; ordering the stores doesn't make sense.

xen/hypervisor.h: Nix trailing whitespace.
No functional change intended.

xen/x86/cpu.c: Nix trailing whitespace.
No functional change intended.

xbd(4): Nix trailing whitespace.

xbdback(4): Nix trailing whitespace.
No functional change intended.

xencons(4): Nix trailing whitespace.
No functional change intended.

xengnt(4): Nix trailing whitespace.
No functional change intended.

xenbus_comms.c: Nix trailing whitespace.
No functional change intended.

xennetback(4): Nix trailing whitespace.
No functional change intended.


To generate a diff of this commit:
cvs rdiff -u -r1.55 -r1.55.4.1 src/sys/arch/xen/include/hypervisor.h
cvs rdiff -u -r1.6 -r1.6.20.1 src/sys/arch/xen/include/xenring.h
cvs rdiff -u -r1.142 -r1.142.4.1 src/sys/arch/xen/x86/cpu.c
cvs rdiff -u -r1.128 -r1.128.20.1 src/sys/arch/xen/xen/if_xennet_xenbus.c
cvs rdiff -u -r1.129.20.1 -r1.129.20.2 src/sys/arch/xen/xen/xbd_xenbus.c
cvs rdiff -u -r1.101 -r1.101.4.1 src/sys/arch/xen/xen/xbdback_xenbus.c
cvs rdiff -u -r1.50 -r1.50.20.1 src/sys/arch/xen/xen/xencons.c
cvs rdiff -u -r1.39 -r1.39.4.1 src/sys/arch/xen/xen/xengnt.c
cvs rdiff -u -r1.108 -r1.108.4.1 src/sys/arch/xen/xen/xennetback_xenbus.c
cvs rdiff -u -r1.24 -r1.24.20.1 src/sys/arch/xen/xenbus/xenbus_comms.c

Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.