Re: select/poll optimization

[Andrew Doran <ad%netbsd.org@localhost>]

On Sun, Feb 24, 2008 at 08:42:56AM +0000, David Laight wrote:

> On Sun, Feb 24, 2008 at 02:51:40AM +0000, Andrew Doran wrote:
> > Hi,
> > 
> > Currently select() and poll() synchronize using a global mutex. The below
> > diff changes them to use per-CPU state, and removes the locking in
> > selrecord(). In the best case there is no lock contention internal to
> > select() or poll() with the patch, and in the worse case, all state is
> > hashed out by a factor of $ncpu so it should scale as CPUs are added.
> > 
> > In the few tests I have done there is either no lock contention or it's just
> > background noise. rmind@ and I have been discussing an approach using a
> > 'thread x descriptor' array to store state, and do fine grained locking
> > based on that, but I think the per-CPU approach is better because a lot less
> > synchronization is needed, and no changes are needed to drivers.
> > 
> >     http://www.netbsd.org/~ad/percpu-select.diff
> > 
> > Comments?
> 
> I think it doesn't work !
> 
> You've not allowed for more than one lwp (using different mutex)
> being in selrecord() for the same device at the same time.
> Also:
> +             /* Multiple waiters: record a collision. */
> +             sip->sel_collision |= sc->sc_mask;
> needs to be locked - or you lose bits in sel_collision.

It's not a problem because the caller (eg device driver) provides mutual
exclusion around calls to selrecord/selwakeup. See the comments at the top
of both functions. That might be kernel_lock or to use pipes as an example,
the pipe's mutex.

Andrew