Subject: Re: New spl level: splsched()
To: Erik E. Fair <fair@clock.org>
From: Jonathan Stone <jonathan@DSG.Stanford.EDU>
List: tech-kern
Date: 03/03/1997 00:46:46
>Isn't it time we started using a new mutual exclusion technique that is
>semaphore based, instead of interrupt based? We're going to need that if we
>ever have any SMP aspirations.
Yes, that would be cool.
But IMHO it's fair to consider this change a step along that path.
It's well-known in the OS community that a ``single big lock'' is not
hot. There's a signficiant gain to going to two (or three) locks: one
for the process queue, and one for the `file system', and maybe other
things, depending. (And of course malloc).
I would be very happy to see interrupt-based mutual-exclusion changes
to the kernel, if those changes help partition the regions where
mutual exclusion is required, in a way that naturally corresponds to
separate mutexes for the relevant data structures.
I don't know if Charles has thought about that, but it wouldn't
surprise me.
The folklore in the field is that a common progression for OSes
from single-CPU to multiprocessor is something like:
"one big lock around the OS" ->
"a few locks" ->
Measurement that shows lots of performance loss due to inter-CPU
cache misses (on mbufs, CPU-switched processes, etc), ->
"per-CPU pools of memory, short-term process scheduling queus, etc.,
with longer-term scheduling of processes and memory to achieve
reasonable fairness" ->
screamingly fast CPUs that are so far away from memory (21264, R10K)
that more radical techniques, like non-blocking synchronization,
start to make sense
So, perhaps these changes can help us skip the first step and get
directly to the second. ;)