Subject: Re: SMP API things, lock debugging, etc.
To: None <tech-smp@netbsd.org>
From: Jason Thorpe <thorpej@nas.nasa.gov>
List: tech-smp
Date: 07/27/1999 10:48:25
[ NOTE: I am addressing several reposonses in this message, so everyone
  please pay attention. :-]

On Tue, 27 Jul 1999 09:56:00 -0700 (PDT) 
 "Eduardo E. Horvath" <eeh@one-o.com> wrote:

 > Locks need to have an spl() associated with them.  Otherwise code could
 > grab a lock and be interrupted, and the interrupt handler could try to
 > grab that lock once again.

Yes, I understand that.  However, currently several places in the kernel
already do:

	s = splfoo()
	simple_lock(...);
	.
	.
	.
	simple_unlock(...);
	splx(s);

So, this issue can be addressed later.  Yes, I agree its an issue, but
it touches a lot more code, affects every platform, and needs to have
a new, separate API in place first.  My goal right now is to make
incremental improvement in the right direction, mostly so that Big Lock
can be committed soon.

 > Now you could have these separately so you need to do an splfoo() before a
 > cpu_simple_lock(foo_lock), but it's simpler to simply add an spl field to
 > the lock structure itself and do:
 > 
 > void cpu_simple_lock_init(__volatile struct simplelock *alp, int spl);

 > Next thing:  Locks are owned by something, and what they're owned by must
 > be defined.  It could be a process, a thread, or a particular CPU, but it
 > must be defined or chaos will ensue.  

What a lock is owned by is defined by what sort of lock it is.  More on
this below.

 > Are all locks spinlocks?  What about stealing the adaptive mutex concept
 > from Solaris?

No.  This document is not meant to define what locking types exist in the
NetBSD kernel.  It is only meant to define what CPU primitives the MI locking
code (which already exists!) will use to implement the necessary atomic
operations.

Briefly (and I'm only going to describe these in this thread once, since
it's really beyond the scope of what I'm attempting to do RIGHT NOW, so
pay attention everyone :-), there are three types of locks in the NetBSD
kernel, which already exist, and have been there for quite some time:

	simple lock

		This is the fundamental atomic locking primitive.  It
		is a spinning mutex.  These locks are held by CPUs.

	read/write spin lock

		This is a more complicated spinning lock, which provides
		for multiple readers or single writers.  The functionality
		is provided by spinlockmgr().  These locks are held by CPUs.

	read/write sleep lock

		This is a reader/writer sleeping lock, which provides
		for multiple readers or single writers.  The functionality
		is provided by lockmgr().  These locks are held by processes
		(threads).

These locking types are inherited from 4.4BSD-Lite2, which took them from
Mach (and then heavily hacked them).  The read/write spinlock functionality
was added to the normal lockmgr() recently by Ross Harvey and myself.  It is
currently used by the proclist locking protocol.

 > > #if defined(MULTIPROCESSOR)
 > > #define	curproc		cpu_info[cpu_number()].ci_curproc
 > > #else
 > > extern struct proc	*curproc;
 > > #endif
 > > 
 > 
 > Why not just have a function that returns a pointer to the current
 > cpu_info structure?  Array indexing may require rather expensive
 > multiplication if the cpu_info structure is not a power of 2.

Okay, I can see that cpu_info[] needs to be more flexible.  The problem
with requiring per-CPU-page mapped to the same VA on all processors is
that it greatly complicates memory management on some architectures (for
example, the Alpha), whereas on others, this is very simple (for example,
the MIPS and SPARC).

How about the following:

#if defined(MULTIPROCESSOR)
#define	curproc		curcpu()->ci_curproc
#else
extern struct proc *curproc;
#endif

I.e. introduce a curcpu() macro which returns a pointer to the cpu_info
structure for the current CPU.  Then the platform can implement this
in any way it chooses (cpu_info[...] or per-CPU-page).

        -- Jason R. Thorpe <thorpej@nas.nasa.gov>