"Stephen M. Jones" wrote:

> I'm wondering if others with high volume nfs traffic see similar numbers to
> these.  There are 8 clients and one server that handles all disk
> requests (except for local root/tmp/swap on the clients).  Should the servers
> be used more evenly or is it typical for one nfs server to be pigged out?
> 
> The clients and server are all on the same 100mbit network.  I have
> 20 servers, the clients and server have vfs.nfs.iothreads set to 32
> (is that really even necessary on the server?)

As I understand it (admittedly at a simple level!), nfsd's are tried in
order.  If one is busy, then the next on the list is used and so on.

> I restarted the nfsd processes about a day and a half ago, and it looks
> like this:
> 
>     0 28003     1   0   2  0   128      0 netcon   IWs  ??    0:00.00 nfsd: mas
>     0 28004 28003   1  -5  0    88    808 biowait  DL   ??  199:01.50 nfsd: ser
>     0 28005 28003   0   2  0    88    808 nfsd     SL   ??   50:20.35 nfsd: ser
>     0 28006 28003   0   2  0    88    808 nfsd     SL   ??   16:03.24 nfsd: ser
>     0 28007 28003   0   2  0    88    808 nfsd     SL   ??    5:22.58 nfsd: ser
>     0 28008 28003   0   2  0    88    808 nfsd     SL   ??    2:07.91 nfsd: ser
>     0 28009 28003   0   2  0    88    808 nfsd     SL   ??    0:57.50 nfsd: ser
>     0 28010 28003   0   2  0    88    808 nfsd     SL   ??    0:28.92 nfsd: ser
>     0 28011 28003   0   2  0    88    808 nfsd     SL   ??    0:16.76 nfsd: ser
>     0 28012 28003   0   2  0    88    808 nfsd     SL   ??    0:00.85 nfsd: ser
>     0 28013 28003   0   2  0    88    808 nfsd     SL   ??    0:10.56 nfsd: ser
>     0 28014 28003   0   2  0    88    808 nfsd     SL   ??    0:07.71 nfsd: ser
>     0 28015 28003   0   2  0    88    808 nfsd     SL   ??    0:05.17 nfsd: ser
>     0 28016 28003   0   2  0    88    808 nfsd     SL   ??    0:03.35 nfsd: ser
>     0 28017 28003   0   2  0    88    808 nfsd     SL   ??    0:02.72 nfsd: ser
>     0 28018 28003   0   2  0    88    808 nfsd     SL   ??    0:02.05 nfsd: ser
>     0 28019 28003   0   2  0    88    808 nfsd     SL   ??    0:01.45 nfsd: ser
>     0 28020 28003   0   2  0    88    808 nfsd     SL   ??    0:01.18 nfsd: ser
>     0 28021 28003   0   2  0    88    808 nfsd     SL   ??    0:00.88 nfsd: ser
>     0 28022 28003   0   2  0    88    808 nfsd     SL   ??    0:01.06 nfsd: ser
>     0 28023 28003   0   2  0    88    808 nfsd     SL   ??    0:00.97 nfsd: ser

To me, that says you can get away with about 6 or 8 nfsd's with your
current load.  I'm sure it doesn't hurt to have more than that, as long
as you don't have a really small memory of RAM on your box.

> I've noticed that many of the clients will have not responding/responding
> issues at various busy times .. I've been playing with mount_nfs options
> including -r/-w, but have gone back to the defaults.   I've also attempted
> to use TCP mounts for one client, which I do not recommend because it 
> actually performed much worse.

Look at the output of netstat -i and look for errors and collisions, and
ping,etc for dropped packets.  Pretty unlikely on a local 100Mb network
though, but worth checking...

> My question is, is it a big deal to have a pigged out nfsd with all
> others nearly idle?  How can requests be a bit more distributed? If nfsd
> requests were more distributed would I have less not responding/responding
> issues?

I haven't looked under the hood to see if round-robin usage of the
nfsd's would be easy (or even possible).

Simon.
--
Simon Burge                            <simonb@wasabisystems.com>
NetBSD Support and Service:         http://www.wasabisystems.com/