Subject: Re: Tuning NFS server performance?
To: None <arne@pegasus.esprit.ec.org>
From: Luke Mewburn <lukem@telstra.com.au>
List: current-users
Date: 02/17/1996 13:57:30
Arne Helme writes:
> Except for the parameter to nfsd to start more daemons, what other
> parameters or tricks can I use to tune NFS server peformance in
> NetBSD?  Is NBUF of any relevance for a machine with much memory?

> I have been asked to install NetBSD/i386 on a 200 MHz Pentium-Pro
> system with 128 MB of memory and a SCSI-2 in order to compare its
> performance against that of Solaris 2.4 on the same system.  In the
> initial tests NetBSD is outperformed rather badly although I run with
> 16 nfsds.

(I've mentioned this before, and PR kern/1903 is my official change-request
 for this.)

You can easily tune the amount of physical memory that is used for buffer
cache by applying the following patch, and putting
	options BUFCACHE=80
to get 80% of physmem (102MB in your case) as buffer cache.
This is much less stuffing around than trying to tweak NBUF.

We run our ULTRIX NFS servers at 80% - your milage may vary though.

Other ways to increase performance (for NFS in general):
- Multiple CPUs, a multithreaded NFSD (yes, solaris 2.x supports
  both rather well; it may well be that Solaris 2.x on a multi cpu
  machine with presto is a better long term solution...)
- PrestoServe (battery backed-up NVRAM and a driver that uses this
  to store synchronous writes (any NFS, local metadata) for flushing
  out later. I think DEC sell PCI presto cards for their Alpha's,
  if you had one of these and the inclination, you could write a driver
  for NetBSD :) There are a couple of other ports which have Presto
  simms/cards but no support yet (pmax, sparc)

There *is* a way to enable Asynchronous NFS writes on NetBSD.
>From memory, in /sys/nfs/nfs_serv.c, change the line in
nfsrv_write() (about line 550) from
	int ioflags = IO_SYNC | IO_NODELOCKED;
to
	int ioflags = IO_NODELOCKED;
*NOTE* This is dangerous; don't do this if you care about your data!


Here's the BUFCACHE patch:

--- cut here --- file: ~/unix/netbsd/bufcache.dif
*** /usr/osrc/sys/arch/i386/i386/machdep.c	Mon Oct 16 12:43:28 1995
--- i386/machdep.c	Wed Jan  3 18:46:22 1996
***************
*** 115,120 ****
--- 115,128 ----
  #else
  int	bufpages = 0;
  #endif
+ #if !defined(BUFCACHE)
+ #define	BUFCACHE	10
+ #endif
+ #if (BUFCACHE < 1) || (BUFCACHE > 95)
+ #undef	BUFCACHE
+ #define	BUFCACHE	10
+ #endif
+ int	bufcache = BUFCACHE;

  int	physmem;
  int	dumpmem_low;
***************
*** 313,329 ****
  #endif

  	/*
! 	 * Determine how many buffers to allocate.  We use 10% of the
! 	 * first 2MB of memory, and 5% of the rest, with a minimum of 16
! 	 * buffers.  We allocate 1/2 as many swap buffer headers as file
! 	 * i/o buffers.
  	 */
  	if (bufpages == 0)
! 		if (physmem < btoc(2 * 1024 * 1024))
! 			bufpages = physmem / (10 * CLSIZE);
! 		else
! 			bufpages = (btoc(2 * 1024 * 1024) + physmem) /
! 			    (20 * CLSIZE);
  	if (nbuf == 0) {
  		nbuf = bufpages;
  		if (nbuf < 16)
--- 321,332 ----
  #endif

  	/*
! 	 * Set size of buffer cache to physmem/buffercache * 100
! 	 * (i.e., bufcache % of physmem). We allocate 1/2 as many
! 	 * swap buffer headers as file I/O buffers.
  	 */
  	if (bufpages == 0)
! 		bufpages= physmem / (CLSIZE * 100) * bufcache;
  	if (nbuf == 0) {
  		nbuf = bufpages;
  		if (nbuf < 16)
--- cut here ---