Subject: Re: splx() optimization [was Re: SMP re-eetrancy in "bottom half" drivers]
To: Bill Studenmund <wrstuden@netbsd.org>
From: Jonathan Stone <jonathan@dsg.stanford.edu>
List: tech-kern
Date: 06/06/2005 12:05:13
>On Mon, Jun 06, 2005 at 03:45:52PM +0900, YAMAMOTO Takashi wrote:
>> > Or, to put that another way: given where NetBSD is today, how would
>> > you propose to get from the current state, to a kernel with
>> > fine-grained SMP synchronization?
>>=20
>> tackle the highest IPL first.
>
>Just to make sure I understand this, your suggestion is to leave most IPLs
>alone (taking big lock) and convert individual IPLs one at a time?
>
>If we have the resources to do that, it probably is quicker.
>
>However I think the big questions are 1) if we do this transition to
>per-level locks, how much harder will it be to transition to full
>locking afterwards? i.e. once we have the per-IPL locking Jonathan
>suggested, will we be behind where we are now? I think not, but it's good
>to know. 2) Do we have folks who want to do the transition to per-IPL
>locking more than work on individual fixup? i.e. do we have developer
>resources that are more available for the fixup than the per-IPL work?

Per-IPL locking lets developers work on drivers piecemeal.  In
principle, nindividual developers can tackle individual drivers.
Driver SMP-safe changes might even be tested on real hardware before
they are committed.


"Whole-IPL-at-a-time" has a much higher, um, Least Publishable Unit,
if you'll forgive me.  Given that we're a volunteer organization, then
I'd expect per-IPL locking to make significantly sbetter use of
available developer cycles.



>As above, I think doing the per-IPL fixup will not make it harder to go to
>true fine-grained locking later. It effectively may be a distraction, but
>I don't think it's harder to turn a section from a big lock to
>fine-grained than it is to turn it from a per-IPL lock to fine-grained. Am
>I missing something?

The lock-per-IPL enables us make individual drivers SMP-safe, mark
them as such. locore/MD code can and will call SMP-safe drivers with*out*
acquiring the per-IPL lock.  So I think lock-per-IPL makes it *easier*
to move remove locking for the IPL-at-a-time approach: we will already
have some (maybe all?) drivers for that IPL which are SMP-safe.


>Given that, if someone WANTS to do the per-IPL fixup, I'd say let them. We
>will get some improvement now on SMP systems, and we won't loose anything.

Sure. I'm all for that, for anyone who has both time and expertise.