[src/netbsd-9]: src/sys/arch/x86/x86 Pull up following revision(s) (requested...

[martin <martin%NetBSD.org@localhost>]

details:   https://anonhg.NetBSD.org/src/rev/ea2393efcc35
branches:  netbsd-9
changeset: 963442:ea2393efcc35
user:      martin <martin%NetBSD.org@localhost>
date:      Tue Dec 17 12:56:45 2019 +0000

description:
Pull up following revision(s) (requested by riastradh in ticket #566):

        sys/arch/x86/x86/bus_space.c: revision 1.42
        sys/arch/x86/x86/bus_space.c: revision 1.43

Use LFENCE/SFENCE/MFENCE in x86 bus_space_barrier.

These are needed for BUS_SPACE_MAP_PREFETCHABLE mappings.  On x86,
these are WC-type memory regions, which means -- unlike normal
WB-type memory regions -- loads can be reordered with loads,
requiring LFENCE, and stores can be reordered with stores, requiring
SFENCE.

Reference: AMD64 Architecture Programmer's Manual, Volume 2: System
Programming, Sec. 7.4.1 `Memory Barrier Interaction with Memory
Types', Table 7-3 `Memory Access Ordering Rules'.

Skip fences in bus_space_barrier on I/O space.

I/O operations are issued in program order.  Not that I/O operations
are usually a performance bottleneck anyway, but maybe it is slightly
cheaper to avoid stalling on store buffers or pending loads, and
there's very little cost to the skipping criterion here.

diffstat:

 sys/arch/x86/x86/bus_space.c |  42 +++++++++++++++++++++++++++++++++++++++---
 1 files changed, 39 insertions(+), 3 deletions(-)

diffs (63 lines):

diff -r b42783d381cf -r ea2393efcc35 sys/arch/x86/x86/bus_space.c
--- a/sys/arch/x86/x86/bus_space.c      Tue Dec 17 12:55:10 2019 +0000
+++ b/sys/arch/x86/x86/bus_space.c      Tue Dec 17 12:56:45 2019 +0000
@@ -1,4 +1,4 @@
-/*     $NetBSD: bus_space.c,v 1.41 2019/02/11 14:59:33 cherry Exp $    */
+/*     $NetBSD: bus_space.c,v 1.41.4.1 2019/12/17 12:56:45 martin Exp $        */
 
 /*-
  * Copyright (c) 1996, 1997, 1998 The NetBSD Foundation, Inc.
@@ -31,7 +31,7 @@
  */
 
 #include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: bus_space.c,v 1.41 2019/02/11 14:59:33 cherry Exp $");
+__KERNEL_RCSID(0, "$NetBSD: bus_space.c,v 1.41.4.1 2019/12/17 12:56:45 martin Exp $");
 
 #include <sys/param.h>
 #include <sys/systm.h>
@@ -878,7 +878,43 @@
                  bus_size_t offset, bus_size_t len, int flags)
 {
 
-       /* Function call is enough to prevent reordering of loads. */
+       /* I/O instructions always happen in program order.  */
+       if (x86_bus_space_is_io(tag))
+               return;
+
+       /*
+        * For default mappings, which are mapped with UC-type memory
+        * regions, all loads and stores are issued in program order.
+        *
+        * For BUS_SPACE_MAP_PREFETCHABLE mappings, which are mapped
+        * with WC-type memory regions, loads and stores may be issued
+        * out of order, potentially requiring any of the three x86
+        * fences -- LFENCE, SFENCE, MFENCE.
+        *
+        * For BUS_SPACE_MAP_CACHEABLE mappings, which are mapped with
+        * WB-type memory regions (like normal memory), store/load may
+        * be reordered to load/store, potentially requiring MFENCE.
+        *
+        * We can't easily tell here how the region was mapped (without
+        * consulting the page tables), so just issue the fence
+        * unconditionally.  Chances are either it's necessary or the
+        * cost is small in comparison to device register I/O.
+        */
+       switch (flags) {
+       case 0:
+               break;
+       case BUS_SPACE_BARRIER_READ:
+               x86_lfence();
+               break;
+       case BUS_SPACE_BARRIER_WRITE:
+               x86_sfence();
+               break;
+       case BUS_SPACE_BARRIER_READ|BUS_SPACE_BARRIER_WRITE:
+               x86_mfence();
+               break;
+       default:
+               panic("unknown bus space barrier: 0x%x", (unsigned)flags);
+       }
 }
 
 void *