Re: select/poll optimization

[Mindaugas Rasiukevicius <rmind%NetBSD.org@localhost>]

Hello,

Andrew Doran <ad%netbsd.org@localhost> wrote:
> Currently select() and poll() synchronize using a global mutex. The below
> diff changes them to use per-CPU state, and removes the locking in
> selrecord(). In the best case there is no lock contention internal to
> select() or poll() with the patch, and in the worse case, all state is
> hashed out by a factor of $ncpu so it should scale as CPUs are added.
> 
> In the few tests I have done there is either no lock contention or it's just
> background noise. rmind@ and I have been discussing an approach using a
> 'thread x descriptor' array to store state, and do fine grained locking
> based on that, but I think the per-CPU approach is better because a lot less
> synchronization is needed, and no changes are needed to drivers.
> 
>       http://www.netbsd.org/~ad/percpu-select.diff
> 
> Comments?

As discussed with Andrew, here is the per-thread approach with the array of
descriptors to store state:

http://www.netbsd.org/~rmind/perthread-select.diff

My arguments for this way:
- Locking is more fine-grained, and I think it would perform better in systems
  with >= 8 CPUs. That is a long-term solution.
- Only selector LWP is awoken, there are no case of collision like in current
  implementation, when all selecting LWPs are awoken to re-check the state.
- Our second goal is to avoid calling selscan/pollscan twice, for that we
  need per-thread storage anyway.

Also, hope I have done the changes for the drivers:
http://mail-index.netbsd.org/tech-kern/2008/02/24/msg000449.html
This patch depends on these changes.

Please comment :)

-- 
Best regards,
Mindaugas
www.NetBSD.org