Subject: Re: A clew to low NFS performance
To: Frank van der Linden <firstname.lastname@example.org>
From: John Franklin <email@example.com>
Date: 08/10/2002 12:12:04
On Mon, Aug 05, 2002 at 12:59:32AM +0200, Frank van der Linden wrote:
> On Sun, Aug 04, 2002 at 02:37:27AM -0400, John Franklin wrote:
> > On my NFS server, I recently updated to 1.6 BETA5. Now my NFS
> > performance feels like the Dow. Apps hang when they need to access
> > files (like Mozilla writing to cache or seti updating the state.sah.)
> If you updated only your server, it's not an issue with read or
> write sizes, since they are limited by the clients. However,
> your server there is a pretty slow machine, so the changed
> disk I/O behavior of UBC may be bad for it.
Doubt it. The problem with UBC was primarily with swap needed to be
touched when a big write was filling up the I/O queue for a particular
drive. In this system, root, swap, /var and /usr are on wd0. /home and
/usr/src (the two NFS exported volumes) are on wd1.
I added options BUFCACHE=30 to the kernel config and the system is
considerably more responsive. The prior problems I had with NFS hanging
are gone. Performance is noticably better. I thought that UBC made
this option obsolete, but apparently I was wrong.
Still, / is using 55% of the metadata buffer space. There is no way
that the system has 16M of metadata from a partition that has less than
40M of data data. I notice / uses very little until hit with NFS
traffic. I can only conclude that NFS buffers (response cache?) or
lower level network buffers are taking the space and charging it to /.
Does this make sense?
Take from df -k:
Filesystem 1K-blocks Used Avail Capacity Mounted on
/dev/wd0a 39887 29548 8344 77% /
/dev/wd0e 15583 7786 7017 52% /var
/dev/wd0f 364567 324616 21722 93% /usr
mfs:146 14847 1 14103 0% /tmp
/dev/wd1f 10142259 1806854 7828292 18% /usr/src
/dev/wd1g 55307073 48280583 4261136 91% /home
Taken from systat buf:
There are 6144 metadata buffers using 29280 kBytes of memory.
There are 6616 pages for cached file data using 26464 kBytes of memory.
There are 682 pages for executables using 2728 kBytes of memory.
File System Bufs used % kB in use % Bufsize kB % Util %
/usr/src 3123 50 3349 11 12548 42 26
/ 2594 42 14055 48 16292 55 86
/usr 45 0 94 0 204 0 46
/home 32 0 128 0 180 0 71
/var 10 0 18 0 44 0 40
Total: 5804 94 17644 60 29268 99 60
% uname -a
NetBSD dhcp-816-94 1.6_BETA5 NetBSD 1.6_BETA5 (BROADWAY) #0: Thu Aug 8 12:25:08 EDT 2002 franklin@deathmitten:/usr/src/16b/usr/src/sys/arch/i386/compile/BROADWAY i386
ICBM: 35°43'56"N 78°53'27"W