So, I've implemented semaphores in the kernel - for the purpose of
forcing the boot cpu (Hypersparc) to wait for other cpu's to flush
cache entries before unmapping the page.  This seems to do the trick,
an MP kernel on a dual hypersparc SS20 can get to start_init() now
(see my post to tech-smp)

There's still a lot of work to be done on the sparc, but I think the
semaphores achieved the desired goal.  So I'm wondering if its worth
committing kern/kern_sem.c and sys/ksem.h now - even though virtually
nothing uses them (other than my local arch/sparc/sparc/cache.c), so
that those with more kernel clue than I, can take a look and probably
fix the obvious bugs.

Note semaphores are not the answer to many issues that the MP kernels
face, but in the case of hypersparc cpu's semaphores or a moral equiv
are probably necessary. 

The implementation is currently very simple and MI.  If semaphores get
used enough it might be worth doing MD implementations but time will
tell. 

typedef struct sema_s {
	struct simplelock sem_interlock;
	const char *sem_wmesg;
	int	sem_flags;
	int	sem_sleepers;
	int	sem_count;
} sema_t;

#define SEMAF_VALID (1<<0)		/* initialized */
#define SEMAF_DEBUG (1<<1)

void	sema_init(sema_t *sp, int cnt, const char *wmesg, int flags);
void	sema_setflags(sema_t *sp, int flags);
void	sema_spinwait(sema_t *sp);	/* spin version of P() */
void	sema_wait(sema_t *sp);		/* sleep version of P() */
int	sema_signal(sema_t *sp);	/* V() */

In the case of the sparc smp_cache_flush() et al, the semaphore is
initialized on entry to the smp*flush routine, and each single cpu
version does a sema_signal() after flushing.  The smp*flush routine
then does a sema_wait() for each cpu, and when all done, marks the
semaphore as invalid.   This is the main use of sema_setflags().


sema_spinwait() does just that.

sema_wait() increments sleepers before sleeping, and sema_signal()
only calls wakeup_one() if sleepers > 0.  sema_signal() returns the
value of sem_count, so that one can know whether there might be anyone
sleeping on it.  When sema_setflags is being asked to invalidate the
semaphore, it calls sema_signal() to ensure no one is sleeping on it.

Thoughts?
--sjg