Re: pserialized reader/writer locks

[Taylor R Campbell <riastradh%NetBSD.org@localhost>]

   Date: Sat, 26 Jul 2014 16:48:27 +0100
   From: Mindaugas Rasiukevicius <rmind%netbsd.org@localhost>

   I would expect a better problem statement even if it is a brain dump (one
   sentence would have been enough).  Are you trying to solve sleepable reader
   problem?  In such case, messing up pserialize(9) with somewhat read-write
   lock semantics in ad-hoc way is a wrong approach.  If you are building a
   read-optimised *lock*, then it should be designed and implemented as such
   mechanism rather than built as an ad-hoc wrapper.

The object is scalable nestable transactions: entering and exiting a
transaction should not usually incur interprocessor synchronization,
and threads in transactions may sleep.

The main application I had in mind is fstrans(9), where every vnode
operation on a given file system must make sure the file system
doesn't go away, and may sleep -- many readers.  Infrequently, a file
system may be unmounted or suspended, which will require waiting for
all pending transactions to complete -- infrequent, expensive writer.

fstrans(9) basically encodes the logic of what I called an rrwlock --
I wrote the rrwlock code when I sat down to disentangle what it does
and understand it.  On doing so I realized it's the same as what zfs
does with ZFS_ENTER/ZFS_EXIT (and later found that zfs has another
abstraction also called rrwlock with ~the same semantics as mine, but
zfs's rrwlock is not scalable -- readers and writers both always
require interprocessor synchronization).

   Basically, there are two approaches:

   a) Implement read-optimised / read-mostly lock.  Years ago ad@ wrote an
   implementation of rdlock(9).  It was not published, but he added a BSD
   license so I guess it is okay to post here:

   http://www.netbsd.org/~rmind/kern_rdlock.c

That makes a single reader cheap; doesn't make multiple readers
scalable.

   Alternatively, there is FreeBSD's rmlock(9):

   http://nxr.netbsd.org/source/xref/src-freebsd/sys/kern/kern_rmlock.c

That doesn't allow sleeping during a transaction.

   b) Implement grace-period based synchronisation mechanism which allows
   readers to sleep, such as SRCU.  There are some dangers here:

   http://lists.freebsd.org/pipermail/freebsd-arch/2014-June/015435.html
   http://lists.freebsd.org/pipermail/freebsd-arch/2014-June/015454.html

   However, it can provide less expensive writer side and more granular
   garbage collection of objects.  I think it is worth to consider this way.

Less expensive writer side is not the point here.