[src/trunk]: src/sys/arch/x86/x86 Use LFENCE/SFENCE/MFENCE in x86 bus_space_b...

[riastradh <riastradh%NetBSD.org@localhost>]

details:   https://anonhg.NetBSD.org/src/rev/0e8dca5dd797
branches:  trunk
changeset: 1005220:0e8dca5dd797
user:      riastradh <riastradh%NetBSD.org@localhost>
date:      Mon Dec 02 08:33:52 2019 +0000

description:
Use LFENCE/SFENCE/MFENCE in x86 bus_space_barrier.

These are needed for BUS_SPACE_MAP_PREFETCHABLE mappings.  On x86,
these are WC-type memory regions, which means -- unlike normal
WB-type memory regions -- loads can be reordered with loads,
requiring LFENCE, and stores can be reordered with stores, requiring
SFENCE.

Reference: AMD64 Architecture Programmer's Manual, Volume 2: System
Programming, Sec. 7.4.1 `Memory Barrier Interaction with Memory
Types', Table 7-3 `Memory Access Ordering Rules'.

diffstat:

 sys/arch/x86/x86/bus_space.c |  38 +++++++++++++++++++++++++++++++++++---
 1 files changed, 35 insertions(+), 3 deletions(-)

diffs (59 lines):

diff -r feef70de47a5 -r 0e8dca5dd797 sys/arch/x86/x86/bus_space.c
--- a/sys/arch/x86/x86/bus_space.c      Mon Dec 02 08:33:42 2019 +0000
+++ b/sys/arch/x86/x86/bus_space.c      Mon Dec 02 08:33:52 2019 +0000
@@ -1,4 +1,4 @@
-/*     $NetBSD: bus_space.c,v 1.41 2019/02/11 14:59:33 cherry Exp $    */
+/*     $NetBSD: bus_space.c,v 1.42 2019/12/02 08:33:52 riastradh Exp $ */
 
 /*-
  * Copyright (c) 1996, 1997, 1998 The NetBSD Foundation, Inc.
@@ -31,7 +31,7 @@
  */
 
 #include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: bus_space.c,v 1.41 2019/02/11 14:59:33 cherry Exp $");
+__KERNEL_RCSID(0, "$NetBSD: bus_space.c,v 1.42 2019/12/02 08:33:52 riastradh Exp $");
 
 #include <sys/param.h>
 #include <sys/systm.h>
@@ -878,7 +878,39 @@
                  bus_size_t offset, bus_size_t len, int flags)
 {
 
-       /* Function call is enough to prevent reordering of loads. */
+       /*
+        * For default mappings, which are mapped with UC-type memory
+        * regions, all loads and stores are issued in program order.
+        *
+        * For BUS_SPACE_MAP_PREFETCHABLE mappings, which are mapped
+        * with WC-type memory regions, loads and stores may be issued
+        * out of order, potentially requiring any of the three x86
+        * fences -- LFENCE, SFENCE, MFENCE.
+        *
+        * For BUS_SPACE_MAP_CACHEABLE mappings, which are mapped with
+        * WB-type memory regions (like normal memory), store/load may
+        * be reordered to load/store, potentially requiring MFENCE.
+        *
+        * We can't easily tell here how the region was mapped (without
+        * consulting the page tables), so just issue the fence
+        * unconditionally.  Chances are either it's necessary or the
+        * cost is small in comparison to device register I/O.
+        */
+       switch (flags) {
+       case 0:
+               break;
+       case BUS_SPACE_BARRIER_READ:
+               x86_lfence();
+               break;
+       case BUS_SPACE_BARRIER_WRITE:
+               x86_sfence();
+               break;
+       case BUS_SPACE_BARRIER_READ|BUS_SPACE_BARRIER_WRITE:
+               x86_mfence();
+               break;
+       default:
+               panic("unknown bus space barrier: 0x%x", (unsigned)flags);
+       }
 }
 
 void *