Re: pserialize(9) vs. TAILQ

[Taylor R Campbell <campbell+netbsd-tech-kern%mumble.net@localhost>]

   Date: Tue, 18 Nov 2014 23:13:34 +0900
   From: Masao Uebayashi <uebayasi%gmail.com@localhost>

   In pserialize_perform(), context switches are made on all CPUs.  After
   pserialize_perform(), all readers on all CPUs see the update data.
   The data old data item is safely destroyed.

   In the TAILQ case, where readers iterate a list by TAILQ_FOREACH(),
   TAILQ_REMOVE() is safely used as the update operation, because:

   - Readers only see tqe_next in TAILQ_FOREACH(), and
   - Pointer assignment (done in TAILQ_REMOVE()) is atomic.

   If this is correct, pserialize(9) should be updated to be clearer;
   probably this TAILQ example is worth being added there.

That is correct.

The one tricky detail is that after a reader has fetched the tqe_next
pointer, it must issue a membar_consumer before dereferencing the
pointer: otherwise there is no guarantee about the order in which the
CPU will fetch the tqe_next pointer and its contents (which it may
have cached).  Whoever inserts entries must also issue a
membar_producer after initializing the entry and before inserting it,
to match the reader's membar_consumer.

Someone^TM should invent names for queue operations that are
pserialize-safe by virtue of automatically issuing these memory
barriers: TAILQ_FOREACH_PSZ, TAILQ_INSERT_HEAD_PSZ, &c., or something,
so that it is easier to use them correctly and spot incorrect use.