Subject: Re: New spl level: splsched()
To: Erik E. Fair <fair@clock.org>
From: Jonathan Stone <jonathan@DSG.Stanford.EDU>
List: tech-kern
Date: 03/03/1997 00:46:46
>Isn't it time we started using a new mutual exclusion technique that is
>semaphore based, instead of interrupt based? We're going to need that if we
>ever have any SMP aspirations.

Yes, that  would be cool.

But IMHO it's fair to consider this change a step along that path.
It's well-known in the OS community that a ``single big lock'' is not
hot.  There's a signficiant gain to going to two (or three) locks: one
for the process queue, and one for the `file system', and maybe other
things, depending.  (And of course  malloc).

I would be very happy to see interrupt-based mutual-exclusion changes
to the kernel, if those changes help partition the regions where
mutual exclusion is required, in a way that naturally corresponds to
separate mutexes for the relevant data structures.

    

I don't know if Charles has thought about that, but it wouldn't
surprise me.


The folklore in the field is that a common progression for OSes
from single-CPU to multiprocessor is something like: 

"one big lock around the OS" -> 

  "a few locks" ->

    Measurement that shows lots of performance loss due to inter-CPU
    cache misses (on mbufs, CPU-switched processes, etc), ->

    "per-CPU pools of memory, short-term process scheduling queus, etc.,
     with longer-term scheduling of processes and memory to achieve
    reasonable fairness" ->

      screamingly fast CPUs that are so far away from memory (21264, R10K)
      that more radical techniques,  like non-blocking synchronization,
      start to  make sense

So, perhaps these changes can help us skip the first step and get
directly to the second. ;)