Re: What's an "MPSAFE" driver need to do?

[Brian Buhrow <buhrow%nfbcal.org@localhost>]

        Hello.  If you look in src/sys/net/bpf.c, you'll see calls to
something called kernel_lock().  I believe these are  calls to acquire and
release the kernel's giant lock.  If  a driver or subsystem needs these,
then it can call these primitives directly.  If you start a kernel thread,
setup an interupt or initialize a callout handle without explicitly saying
you're mpsafe, then the subsystems assume you're not and the kernel_lock is
taken on your behalf when these calls are executed for you later.
        I don't know enough about the memory barrier stuff to speak about it,
so I leave that to those smarter than I.
        As an example, I've seen comments from Manuel on these lists that
indicate the wd(4) driver isn't mpsafe.  When I look at wd.c, I see calls
to callout_init and the like without the MPSAFE flag, which means that
these callouts are executed with the giant lock enabled.  I have no idea
what in particular makes the wd(4) driver mpunsafe, but I suspect it is
because it may use scratch areas for transient bits of data that may be
accessed if multiple instances of the wd(4) driver were running
simultaneously on a system.
        I think if you use mutex(9) and rwlock(9), as well as  the MPSAFE
flags to  establish interrupts, create threads or initialize callouts,
you'll have an mpsafe driver.
        The goal is to get rid of the need for calls to kernel_lock(), but I
gather we're still a good ways from that ideal -- though I think we're a
lot closer than the FreeBSD guys were at a similar stage of development.
My understanding is that the networking stack is the the largest remaining
piece of kernel code that requires the kernel_lock() calls.  However, I'm
straying into territory I know very little about, so I'll be quiet now.
Lol.
-Brian

On Feb 28,  2:29am, Mouse wrote:
} Subject: Re: What's an "MPSAFE" driver need to do?
} > The way to write a driver to be mpsafe is to use the mutex(9) calls
} > for locking exclusively.
} 
} Well, assuming rwlock(9) is considered a subset of mutex(9) for the
} purposes of that sentence, I then have to ask, what else is there?
} spl(9), the traditional way, specifically calls out that those routines
} work on only the CPU they're executed on (which is what I'd expect,
} given what they have traditionally done - but, I gather from the
} manpage, no longer do).
} 
} This then leads me to wonder how a driver can _not_ be MPSAFE, since
} the old way doesn't/can't actually work and the new way is MPSAFE.
} 
} dholland says that a driver is MPSAFE "if it does not require
} kernel_lock to work correctly".  What's kernel_lock?  man -k turns up
} nothing; some grepping leads me to think it's what I've seen called on
} the lists a `giantlock', which is what I thought all this new locking
} stuff was supposed to be getting away from.
} 
} So now I'm puzzled.  I've clearly either missed something or
} misunderstood something, possibly both....
} 
} Also a bit disturbing is that I see memory barriers in various places,
} in particular some parts of the mutex implementation, but the
} documentation is completely silent on whether clients of the mutex
} interface may assume anything memory-barrier-ish.
} 
} While not directly related, I also note some issues in the memory
} barrier descriptions:
} 
} - membar_enter and membar_exit speak of loads "reach[ing] global
}    visibility", which makes no sense to me.  What does it mean for a
}    load to reach global visibility?
} 
} - membar_consumer is described with "All loads preceding the memory
}    barrier will complete before any loads after the memory barrier
}    complete".  That last "complete" needs to be "start" for this to be
}    a useful guarantee.
} 
} - membar_sync has each of the above issues (mutatis mutandis).
} 
} /~\ The ASCII                           Mouse
} \ / Ribbon Campaign
}  X  Against HTML              mouse%rodents-montreal.org@localhost
} / \ Email!         7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B
>-- End of excerpt from Mouse