Re: Per-CPU Unit (PCU) interface

[Mindaugas Rasiukevicius <rmind%netbsd.org@localhost>]

Andrew Doran <ad%netbsd.org@localhost> wrote:
> > 
> > http://www.netbsd.org/~rmind/subr_pcu.c
> > http://www.netbsd.org/~rmind/pcu.h
> > http://www.netbsd.org/~rmind/pcu.diff
> 
> I have three problems with this:
> 
> - It uses our xcall interface which provides a mailbox to pass messages 
>   between CPUs.  The intent seems to be to change the above code to use
>   IPIs in the future. <...>

Right, see below.

> - When re-implementing the x86 lazy FPU switching for 5.0 I dropped
>   the notion of discarding state.  There's very little benefit, but
>   many drawbacks with potential race conditions.  It's easier just to
>   save whatever state is on the CPU.  You lose an optimization for a 
>   corner condition and gain code that's easier to maintain and prove.

Note that pcu_discard() is only for 'curlwp' and is basically for LWP
exit case.  There is one race condition when FPU state of other LWP on
remote CPU might be released - in such case LWP traps and re-loads the
state again (see description in "Concurrency notes").  Also, this can
be avoided by passing LWP pointer via xcall's mailbox.  However, do you
see any other races?

> - I would like to see immediate users of this interface, ideally at
>   the time of commit.  In its current form it seems not an acceptable
>   replacement for the x86
>   lazy FPU code because it uses xcalls and those are somewhat 
>   heavyweight.  Efficiency is important in this case because it's
>   the raison d'etre for lazy FPU context switch.

Right, that is my concern as well i.e. xcall's soft-interrupt being
quite more heavyweight than IPI.  However, adding MI IPI interface is
not trivial, as you already noted, because we need some mailbox.

An alternative might be to have MI IPIs for FPU_SAVE and FPU_RELEASE,
and support FPU only.  Multiple instances of PCU for PowerPC AltiVec,
SPE, etc can be supported later.

-- 
Mindaugas