Subject: Re: A clew to low NFS performance
To: Frank van der Linden <fvdl@wasabisystems.com>
From: John Franklin <franklin@elfie.org>
List: current-users
Date: 08/10/2002 12:12:04
On Mon, Aug 05, 2002 at 12:59:32AM +0200, Frank van der Linden wrote:
> On Sun, Aug 04, 2002 at 02:37:27AM -0400, John Franklin wrote:
> > On my NFS server, I recently updated to 1.6 BETA5.  Now my NFS
> > performance feels like the Dow.  Apps hang when they need to access
> > files (like Mozilla writing to cache or seti updating the state.sah.)
> 
> If you updated only your server, it's not an issue with read or
> write sizes, since they are limited by the clients. However,
> your server there is a pretty slow machine, so the changed
> disk I/O behavior of UBC may be bad for it.

Doubt it.  The problem with UBC was primarily with swap needed to be
touched when a big write was filling up the I/O queue for a particular
drive.  In this system, root, swap, /var and /usr are on wd0.  /home and
/usr/src (the two NFS exported volumes) are on wd1.

UPDATE:

I added options	BUFCACHE=30 to the kernel config and the system is
considerably more responsive.  The prior problems I had with NFS hanging
are gone.  Performance is noticably better.  I thought that UBC made
this option obsolete, but apparently I was wrong.

Still, / is using 55% of the metadata buffer space.  There is no way
that the system has 16M of metadata from a partition that has less than
40M of data data.  I notice / uses very little until hit with NFS
traffic.  I can only conclude that NFS buffers (response cache?) or
lower level network buffers are taking the space and charging it to /. 
Does this make sense?

Take from df -k: 

Filesystem  1K-blocks     Used     Avail Capacity  Mounted on
/dev/wd0a       39887    29548      8344    77%    /
/dev/wd0e       15583     7786      7017    52%    /var
/dev/wd0f      364567   324616     21722    93%    /usr
mfs:146         14847        1     14103     0%    /tmp
/dev/wd1f    10142259  1806854   7828292    18%    /usr/src
/dev/wd1g    55307073 48280583   4261136    91%    /home

Taken from systat buf:

There are  6144 metadata buffers using           29280 kBytes of memory.
There are  6616 pages for cached file data using 26464 kBytes of memory.
There are   682 pages for executables using       2728 kBytes of memory.

File System          Bufs used   %   kB in use   %  Bufsize kB   %  Util %
/usr/src                  3123  50        3349  11       12548  42      26
/                         2594  42       14055  48       16292  55      86
/usr                        45   0          94   0         204   0      46
/home                       32   0         128   0         180   0      71
/var                        10   0          18   0          44   0      40

Total:                    5804  94       17644  60       29268  99      60

[6]% uname -a
NetBSD dhcp-816-94 1.6_BETA5 NetBSD 1.6_BETA5 (BROADWAY) #0: Thu Aug  8 12:25:08 EDT 2002 franklin@deathmitten:/usr/src/16b/usr/src/sys/arch/i386/compile/BROADWAY i386



jf
-- 
John Franklin
franklin@elfie.org
ICBM: 35°43'56"N 78°53'27"W