In message <ECF1DBA0-3105-11D8-91BB-000A957650EC@wasabisystems.com>,
Jason Thorpe writes:


>The hierarchy of simplelocks is simply not an acceptable solution:

s/strict hierarcy/whatever we have now/, then. That was all I really
meant. I hadnt noticed the splclock() changes. 

(Forgive me if I dont pay attention to Events that at 100Hz when I'm
focusing on event sthat hapen at several tens of KHz per device, with
mulpitle such devices per system.)


>	3. I'm not seeing how it would actually solve your problem.  If you
>	   want to have 2 CPUs handling NIC interrupts, they would both have
>	   to acquire the IPL_NET_slock, which would stall one of them.

Err, no, I must have phrased it carelessly: thats precisely
what I *dont* want. See below.

[..]


>What have I missed?

Interrupt handler woudn't aquire the IP_NET_slock. They acquire just
the lock for a given different softc, when they enter the device
interrupt handler _for that device_.  They dont need to get the
IPL_NET lock until they need to touch datastructures outside the
device softc itself (say for a NIC driver, dropping a packet on the ip
(or other) input queue.)

Reordered slightly:

>	4. Having to acquire several simplelocks in a hierarchy would increase
>	   the interrupt latency.
>

Nope, the `hierarchy' (modified above to `emulate whatever we have
now') was for IPLs *above* device interrupts.

I suggested the interrupt glue will know whether an interrupt routine
was SMP-safe before entering it (from a flag provided by the device,
when it registered the interrupt).

If we changed the argument to an (SMP-safe) interrupt routine from a
void * to a struct device * (or an extension of struct device), then
the per-device spinlock could be part of struct device, and the
interrupt glue could test or acquire the per-softc lock before
entering the driver. I'm agnostic about doing that versus spinning in
the very beginning of the interrupt hanlder.

If and when the driver needs IP_NET_slock, to touch something outside
the softc, it can attempt to obtain it.

Yamamoto-san already suggested doing away wiht per-IP locks and just
using kernel_lock. smp-intr-save drivers would then attempt to acquire
kernel_lock before they reach into data structures outside the softc.

I said I though that should work, too. but it seems a (small) step in
the wrong direction, too: we want finer-grained locks, not yet more
use of kernel_lock.