Re: rw_downgrade/tryupgrade semantics

[Jason Thorpe <thorpej%me.com@localhost>]

> On Feb 22, 2023, at 9:40 AM, Taylor R Campbell <campbell+netbsd-tech-kern%mumble.net@localhost> wrote:
> 
> I believe this is guaranteed as rwlock(9) is currently implemented.
> (Whether it is _intended_ to be guaranteed, I'm not sure.)

That guarantee certainly falls in line with the principle of least astonishment.  We should probably document it, but it doesn’t seem correct any other way.

> The reason these can fail is that rw_tryupgrade and rw_downgrade only
> issue store-before-store barriers (membar_producer), putting no
> restrictions on when the CPU can load foo->nreaders relative to
> rw_tryupgrade or rw_downgrade.

That’s a bug, full-stop.

> Even though the reader is not technically _just_ a `reader', in that
> it issues stores as well as loads, I think these possibilities are
> extremely counterintuitive and possibly dangerous, especially for a
> relatively high-level API like rwlock(9) that prioritizes ease of use
> over maximal parallelism -- merely taking a read lock can lead to
> shared memory contention, which is why we also have harder-to-use but
> cheaper options like pserialize(9), psref(9), and localcount(9).

I think it would be worth documenting that “reader” and “writer” are just colloquialisms … “shared” and “exclusive” is what is really meant, but nearly everyone uses the “reader” and “writer” names.

> I'm open to other opinions; perhaps it is intended that loads in a
> writer can bleed into an adjacent read section, and that readers
> aren't supposed to issue stores anyway, and perhaps there's a good
> reason for all this.
> 
> But absent a good reason, I think rw_downgrade should be a release
> operation, and rw_tryupgrade should be an acquire operation, just like
> rw_exit and rw_enter.  Which means they need to use membar_release and
> membar_acquire inside, not membar_producer (and we need to issue
> pullups).

I agree 100% … try-upgrade and downgrade are really just optimizations … their effect should be equivalent to dropping the writer lock and re-acquiring as a reader.

-- thorpej