Subject: Re: NFS performance
To: None <andreasc@netinsight.se, netbsd-users@netbsd.org>
From: Brian C. Grayson <bgrayson@marvin.ece.utexas.edu>
List: netbsd-users
Date: 10/19/1998 17:27:30
On Sun, Oct 18, 1998 at 08:48:28PM +0200, andreasc@netinsight.se wrote:
> 
> We have a NFS server running NetBSD 1.3.2 and three kind of clients,
> solaris, netbsd and linux. It works well but the performance for the
> linux clients is poor.
> 
> The read performance for linux is about 4Mb/s, but the write performance
> is less than 600kb/s.
> 
> I already usese 8kb/blocks. (rsize,wsize when mounting).
> 
> Can I do anything else to speed up the write performance for our linux
> clients?

  Take all this with a grain of salt, but here's my experience:

  Supposedly, the cause for such poor write performance is:
Linux does everything in 1K blocks.  So when it writes to an
NFS filesystem, it sends a write for the first 1K, which requires
the server to read 4K from disk, modify 1K, and write 4K back to
disk.  Then, Linux writes the next 1K, which requires 4K read, 1K
modify, 4K write to disk.  Etc.  So, there is 4x the
filesystem/disk traffic on the server than necessary.  This can
be observed via running xosview on the server -- on our setup,
the net traffic is a steady 160KB/sec, while the disk traffic is
~650K/sec.  From a NetBSD client, both net traffic and disk
throughput go to near 1MB/sec (we are on 10Mb/sec Ethernet).

  NFSv3 alleviates this 1K/4K problem by allowing writes to be
cached at the server's side, thus 4K is read, 1K modified, 1K
modified, 1K modified, 1K modified, 4K written back to disk. 
This problem can also be alleviated by using the wsize parameter.

  However, as far as I can tell, Linux does neither!  (Its NFSv3
support is a sham/empty husk, and it doesn't always honor
wsize, from what I've seen, or at least wsize=8192 seems to use
4096, and wsize=2048 seems to use 1024, IIRC.)

  Feel free to contradict me with examples.  I've looked at
the Linux NFSv3 source code and not been impressed.  tcpdump also
doesn't show the sync requests that ought to be seen when
exercising an NFSv3 mount.  But I know very little about NFS
innards, so I could be way wrong.

  BTW, I've also encountered occasional (repeatable) lossage with
NFS under Linux (admittedly with a rare corner case, but
still!), so I'd be a bit wary.  BSD NFS appears to be much more
solid, both from server and client point of view, IMHO.

  Hope this helps!

  Brian
-- 
"Infinity is greater than t-sub-2.  Infinity is when Saddam Hussein gets 
      one of those things (a missile that doesn't fall apart
      immediately after launch) built."  - Dr. Cyprus, Elec 427