Re: Questions of lock usage in NetBSD kernel code

[Taylor R Campbell <campbell+netbsd-tech-kern%mumble.net@localhost>]

> Date: Sat, 24 Jun 2017 09:59:43 +0800
> From: Jia-Ju Bai <baijiaju1990%163.com@localhost>
> 
> I am a freshman in developing NetBSD drivers, and I have a question in
> lock usage in NetBSD kernel code.

Cool!

> I find three kinds of locks which are often used in OpenBSD drivers,
                                                      ^^^^^^^
I assume you mean NetBSD here!  I'm afraid I'm not familiar enough
with OpenBSD to answer any analogous questions over there.

> namely "adaptive mutex lock", "spin mutex lock" and "rw lock".
> I want to know  which locks can be held when the current thread can sleep.
> 
> I have read NetBSD man pages:
> http://netbsd.gw.com/cgi-bin/man-cgi?mutex+9+NetBSD-current
> http://netbsd.gw.com/cgi-bin/man-cgi?rwlock+9+NetBSD-current
> 
> I find two points:
> (1) The "adaptive mutex lock" *can* be used in this situation.
> (2) The "spin mutex lock" *can not* be used in this situation.

It is technically possible to sleep while holding an adaptive lock,
but the duration of sleep must be bounded -- you can't wait for
external I/O, and any other processes that are trying to acquire the
same adaptive lock are blocked uninterruptibly until you release it.

Sleeping with a spin lock held is absolutely prohibited and does not
work.

> But I still have three questions:
> (1) Are my points right?
> (2) What about "rw lock"?
> (3) What about other kinds of locks if they exist?

It is also technically possible to sleep while holding an rwlock, and
file systems do so even when waiting for I/O, but this is an abuse of
rwlocks because they're not interruptible and we should clean it up
one of these days.  In general, if something might ever wait for I/O
before returning to userland, it should be interruptible.

There are other kinds of synchronization devices with related rules:

- Blocking interrupts (splnet, &c.): Sleeping while interrupts are
  blocked works, but it reenables interrupts.  Most new code doesn't
  block interrupts directly -- instead uses mutex(9) or pserialize(9)
  or other higher-level constructs that use it as a primitive.

- Disabling preemption (kpreempt_disable): Currently the kernel
  doesn't crash if you sleep while preemption is disabled, but we may
  change that because doing so is very error-prone.  Also usually
  limited to the internals of higher-level constructs.

- Passive serialization (pserialize): Read sections run with
  interrupts blocked, and must not sleep.

- Passive references (psref): Users may sleep, but must be bound to a
  CPU.

- CPU-local reference counts (localcount): Readers may sleep and
  migrate across CPUs.