I looked a some Solaris docs, dug out the last discussion on this topic,
reread (some of) the paper on scheduler activations, and thought some more
about thread models.  I have come up with a method that makes every thread
preemptable and kernel scheduled, uses a reasonable amount of kernel
resources, and should be somewhat faster then scheduler activations. 

  This is done by separating kernel threads that are currently associated
with processes from those processes and associating them only when needed. 
All user thread would be scheduled by the kernel (needing somewhere around
100 bytes each of per-thread kernel memory) and do the equivalent of the
many-to-many model inside the kernel.  Until a user thread makes a
syscall, you don't really need a kernel thread.  So don't allocate one
(from a pool) until it is needed.  Each process (see below for what
exactly this means) would have a configurable number of guaranteed kernel
threads available to do kernel work for it.  It could decide to allow
overcommitting of kernel threads if there are enough extra available or
require that all of its guaranteed kernel threads be reserved for its use
only.  If there was a shortage of kernel threads and a user thread needed
a guaranteed thread while it was in use, the system call of an
overcommitted process is interrupted and the kernel thread given back to
its rightful owner. 

  If a user thread in need was not able to procure a kernel thread, it
would get in line for the next one available or perhaps take various
measures against other lower priority threads in its process.  Whenever
reasonable (I'm not sure how often this is, if ever), a user thread
waiting for a kernel resource but not actually needing a kernel thread
while waiting would not be associated with a kernel thread (unless it was
reserved.  It might also be guaranteed one when it needs it).  As with the
Solaris model, a signal could be sent to a process when all of its kernel
threads are in use, allowing it to increase the number (subject to
applicable limit) if desired. 

  A process always has just one signal handler table, but may have
multiple address spaces and threads.  Pthread signal behavior is used. 
The userland PID uniquely identifies this process, though it is not in any
way related to thread IDs (TIDs) or address space IDs (VIDs).  That is, a
process is something that you can control as a unit.  Only a process has a
parent.  Processes are not directly scheduled, though they keep as much
scheduling (and other) info as possible to lower the cost for each thread.

  I think this would be a powerful thread model that would play well in a
variety of MP environments.  If this is considered a possibly reasonable
idea, I would like to develop it further (I already have a good idea of
what the userland API would look like) and (if it makes it that far) try
to implement it. 

  Comments?

  Am I completely nuts or completely misunderstanding something?

Matthew Orgass
darkstar@pgh.net