Re: CPU-local reference counts

[David Young <dyoung%pobox.com@localhost>]

On Mon, Jun 13, 2016 at 04:54:32PM +0000, Taylor R Campbell wrote:
> Prompted by the discussion last week about making sure autoconf
> devices and bdevsw/cdevsw instances don't go away at inopportune
> moments during module unload, I threw together a generic sketch of the
> CPU-local reference counts I noted, which I'm calling localcount(9).

Hi Taylor,

Here are a couple of ideas that I probably picked up from other
NetBSDers and from contributors to an old thread,
<https://mail-index.netbsd.org/tech-kern/2013/01/17/msg014872.html>.

One way both to save some memory and to reduce the cache footprint of a
reference-counting scheme like this is to use "narrow" per-CPU counters
(uint16_t, say) and a wide, shared counter that's increased, using an
interlocked atomic operation, whenever a per-CPU counter rolls over.

To save some more memory, you can make struct localcount,

        struct localcount {
                int64_t         *lc_totalp;
                struct percpu   *lc_percpu; /* int64_t */
        };

into a handle,

        struct localcount {
                unsigned int    lc_slot;
        };

where the lc_slot indicates an index into a few arrays: a per-CPU array
of local counters, a global array of shared counters, and an array of
whichever flags ("draining") that you may require.

Depending how many localcount you expect to be extant in the system, you
can make lc_slot a uint16_t or a uint32_t.

Dave

-- 
David Young         //\ Trestle Technology Consulting
(217) 721-9981      Urbana, IL    http://trestle.tech