Re: pserialize(9) vs. TAILQ

[Taylor R Campbell <campbell+netbsd-tech-kern%mumble.net@localhost>]

   Date: Sat, 22 Nov 2014 01:03:58 +0900
   From: Masao Uebayashi <uebayasi%gmail.com@localhost>

   The problem of TAILQ_INSERT_*() macros with respect to pserialize(9)
   is that, as you are pointing out, they update new elm's next pointer
   and prev elm's next pointer almost simultaneously.  So you have to
   ensure those ordering in the reader's code path.

Dennis's point was that on most CPUs this ordering is guaranteed
already.  My point was that that is not true on all CPUs, hence my
proposal for a membar_datadep_consumer which is a no-op on most CPUs.

	pserialize_read_enter();
	TAILQ_FOREACH(...) {
		if (...) {
			/* Ensure that E_p.next and generation are sync'ed. */
			mutex_spin_enter();

It matters what's in the ...'s.  Are you doing this?

	TAILQ_FOREACH(e, head, next) {
		if (e->key == key) {
			mutex_spin_enter(&e->lock);
			...

If so, the reader may see a stale e->key -- the membar_enter implied
by mutex_spin_enter is too late.  The reader needs a memory barrier
after loading e, and before loading anything e points to:

	TAILQ_FOREACH(e, head, next) {
		membar_datadep_consumer();
		if (e->key == key) {
			mutex_spin_enter(&e->lock);
			...

I'm not clear on what you're trying to solve by versioning the tailqs,
but I don't think it will be necessary in the end for the purposes we
have in mind.

I expanded the example code in pserialize(9).  Perhaps it will be a
little more instructive now.