Re: MP locking?

[der Mouse <mouse%Rodents-Montreal.ORG@localhost>]

>> [...roll forward pre-MP code to become MP-ready...]

>> lock(9) outlines locking facilities which I believe I can use to do
>> [the locking] I want - but there are other issues [...]

> You need memory barriers, modern CPUs can do speculative reads, and
> some can reorder writes.  AFAIK, lock operations act as memory
> barriers.  atomic ops do not.

But are lock calls enough?  As I understand a memory barrier (which
understanding is mostly based on CPU documentation of barrier
instructions), it is not enough, since it does not imply any cache
synchronization with other CPUs.  If CPU A writes a shared data
structure and then issues a memory barrier, either (i) this has no
particular effect on CPU B's cache, leading B to possibly use stale
data, or (ii) this forces CPU B to discard its whole read cache,
incurring a mostly (possibly entirely) unnecessary performance hit.

(ii) can be avoided if memory barriers apply to only certain addresses,
but, since the locking calls do not take any addresses except those of
the lock, the implicit memory barrier you refer to cannot have that
kind of information associated with it.  (This actually leads me to
wonder: are those implicit memory barriers good for anything besides
device access?)  I find it hard to believe that releasing a lock causes
all other CPUs to discard their entire read caches....

This leaves me with (i).  I know some cache hardware does snooping of
writes by other CPUs (or, semi-equivalently, DMA engines) to avoid the
issue entirely, but I also know some hardware doesn't.  How does NetBSD
address the issue?  The only thing "man -k barrier" turned up, besides
a handful of pthread calls, is bus_space(9), which AIUI is not relevant
here because these are data structures in ordinary kernel memory, not
bus space - am I missing something?  Does this just mean that 4.0.1
(which is what I was checking the manpages on) does not support MP on
hardware which doesn't do cache snooping, or is there some facility
I've missed for arranging for other-CPU cache flushes, or what?

/~\ The ASCII                             Mouse
\ / Ribbon Campaign
 X  Against HTML                mouse%rodents-montreal.org@localhost
/ \ Email!           7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B