Re: nfsd "serializing" patch

[Chuck Silvers <chuq%chuq.com@localhost>]

On Thu, Nov 22, 2012 at 01:18:09PM +0100, Manuel Bouyer wrote:
> Hello,
> while working on nfs performance issues with overquota writes (which
> turned out to be a ffs issue), I came up with the attached patch.
> What this does it, for nfs over TCP, restrict a socket buffer processing
> to a single thread (right now, all pending requests are processed
> by all threads in parallel). This has two advantages:
> - if a single client sends lots of request (like writes comming from a
>   linux client), a single thread is busy and other threads will be
>   available to serve other client's requests quickly
> - by avoiding CPU cache sharing and lock contention at the vnode level
>   (if all requests are for the same vnode, which is the common case),
>   we get sighly better performances.
> 
> My testbed is a linux box with 2 Opteron 2431 (12 core total) and 32GB RAM
> writing over gigabit ethernet to a NetBSD server (dual
> Intel(R) Xeon(TM) CPU 3.00GHz, 4 hyperthread cores total) running nfsd -tun4.
> Without the patch, the server processes about 1230 writes per second,
> with this patch it processes about 1250 writes/s.
> 
> Comments ?

this seems to be trying to work around our current limitation that all writes
to a given vnode are serialized, is that right?  eventually we'd like to
improve things such that I/O to non-overlapping ranges of the file can be
processed in parallel.  this change appears to help in the context of the
current limitation, but it sounds like the opposite direction for what we'd
like to do in the long run.  it would be better if we could find some way
improve things now that won't make life harder for ourselves later.

-Chuck