Subject: Re: SMP re-entrancy in kernel drivers/"bottom half?"
To: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
From: Jonathan Stone <jonathan@dsg.stanford.edu>
List: tech-kern
Date: 12/17/2003 17:52:17
>i know about current implementation.
>i'm asking how it works in your suggested way.

I' not sure which part you are unclear on.

Here is a true fact: we could replace the current spl()s with a
hierarchy of spinlocks.  In this scheme, instead of raising SPl by
manipulating mask of `observable' SPLs, raising SPL would be achieved
by acquiring all the new spl-locks in order (to avoid deadlock).

Everything would work pretty much as it does now.

I'm suggesting a slightly more subtle variant, where instead of a
`global' lock for IP_BIO and IPL _NET, each (SMP-save) device has its
own spinlock, in the softc, which is used to serialise entry to the
device-driver entrypoints. (Code at the start of each entrypoint has
to spin getting the lock.)

Hm. Data structures outside such SMP-save drivers still need a
(`global', in your term) spinlock corresponding to IPL_NET or IP_BIO.

If it helps, you could think of the proposed spinlocks in each
(proposed smp-safe) driver softc as being just infinitesimally lower
than the `global' splnet()/IPL_NET or splbio()/IPL_BIO spinlock.

When I think of a machine with a half-dozen gigabit nics, or (very
soon) a couple of ten-gigabit NICs, this sounds like a very useful
halfway house between a single-big-lock kernel and real fine-grained
SMP locking.