--u0lqphHC8S+ob1K0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Wed, Jun 08, 2005 at 08:24:05AM -0700, Jason Thorpe wrote:
> You still need to fix everything at higher IPLs first, because as =20
> long as you could be interrupted by something that needs to take the =20
> kernel_lock, you could end up in a deadlock situation.

I understand how that can arise, if something in the kernel_lock path
then wants to aquire a lock you took before being interrupted. =20

What I don't really see is why there *must* be such a case, or rather
that there can't be (potentially substantial) amounts of code that
don't incur such contention. =20

Thus I can't see a reason to forbid this out of hand, though clearly
there's reason to be very careful. Perhaps I'm just reading too much
into generalised statements.

If we declare everything that needs to run inside the kernel lock as
'legacy' code (pick any other term if you prefer), then they won't be
trying to access the new finer-grained locks in an SMP-safe driver -
if they know about such locks, they're by definition no longer
'legacy' code and would need to be updated themselves to be SMP-safe.
The more of this code that can be moved out of the high-IPL path and
into lower-IPL continuations/workerthreads/softintrs/etc/whatever, the
less legacy non-SMP code needs to run at higher IPL and take the
kernel_lock, and the more we can progressively shrink that perimeter.

Thus it seems there can be SMP-safe drivers, that have fine-grained
locks on all their internal data structures and don't use any
(contested) external data structures.  They're probably rather
boring drivers, but there must be some at this extreme.

Then there are other (most) drivers that need to access other shared
data structures in the kernel. For these drivers to be SMP-safe, we
need SMP-safe data structures; ones that can be accessed outside of
the kernel lock. By the definition posited above, these data
structures can't also be accessed from inside the kernel lock by
'legacy' code at a higher IPL. My point is that this is not *quite*
the same thing as saying no 'legacy' code can run at the higher IPL
*at all*.

To me, this seems to be the factor that determines the size of a
'commit unit of work' for conversion of code to SMP-safe. One
quick-and-obvious view of this granularity comes from IPL ordering
(and especially memory allocation), but I'm sure that smaller
granularities can be found within an IPL based on the data structures
that need to be shared.  It also seems that gaining and enforcing
clear understanding and separation between these modules that don't
actually share data would be no bad thing for code structure overall.

Finally, making smp-safe datastructures doesn't necessarily involve
locks; there is much research on other primitives and constructions,
especially from those trying to scale to many-way SMP and other more
diverse architectures where lock-taking latency kills.

--
Dan.
--u0lqphHC8S+ob1K0
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (NetBSD)

iD8DBQFCqR4HEAVxvV4N66cRAk/QAJ9ZA3FlPeiQtwkYDEeHnxlEIbIewwCdEJ12
uHoHIT2hASBIAKi6FtVU8yc=
=4mlT
-----END PGP SIGNATURE-----

--u0lqphHC8S+ob1K0--