Re: MI non-interlocked atomic ops?

[David Laight <david%l8s.co.uk@localhost>]

On Tue, Jan 24, 2012 at 11:16:00AM -0600, David Young wrote:
> Does NetBSD run on any processor architectures where it is difficult
> or impossible in the kernel[1] to provide non-interlocked atomic
> operations?  That is, operations that are atomic with respect to other
> operations *on the same processor*, but possibly divisible by operations
> on other processors?

On a lot of systems it will require interrupts be disabled
or restartable atomic sequences be used (which are expensive in
the interrupt path).

> On x86, creating non-interlocked variants of atomic operations is
> usually a small matter of leaving the LOCK prefix off of an instruction,
> because traps, interrupts, et cetera, only occur between instructions.
> (Do correct me if I am wrong about this!)  I figure that it is more
> complicated on a RISC machine where there may be no/few instructions
> that read-modify-write RAM.

On a 68020 a fault (not an interrupt) can occur mid-instruction, the fault
stack frame contains the state of the microcode engine.
(So don't restart on a differnt cpu!)

> I ask because the other day I stumbled on the statistics-counting
> function mbstat_type_add(),
> 
>         void
>         mbstat_type_add(int type, int diff)
>         {
>                 struct mbstat_cpu *mb;
>                 int s;
> 
>                 s = splvm();
>                 mb = percpu_getref(mbstat_percpu);
>                 mb->m_mtypes[type] += diff;
>                 percpu_putref(mbstat_percpu);
>                 splx(s);
>         }
> 
> which spends (on x86) a small but not immeasurable amount of time in
> splx(s)...

How did you measure the time spent in  splx(), you are likely to
include time exectuting any deferred interrupts.

I also had a feeling that the spl functions just acted on a per-cpu
variable.
When an interrupt actually happens the spl level is checked, and if
it sould have been masked it is masked then.
This does mean that splx() will have a check for 'did an interrupt
get masked' - butthat ought to be a single compare+branch that is
statically predicted not-taken.

For short sequences (like the above) completely disabling interrupts
might be an option. The single instructions to do that might
be quick, but on i386 I suspect they are slow (amd64's cr8 accesses
may be better, but may be synchronizing instructions).

        David

-- 
David Laight: david%l8s.co.uk@localhost