Subject: nfsio behaviour on 2.0F
To: None <port-alpha@netbsd.org>
From: Stephen M. Jones <smj@cirr.com>
List: port-alpha
Date: 07/14/2004 13:10:05
I'm doing a little load test between FreeBSD 5.2.1 and NetBSD 2.0F machines
by creating a 500mb file of zeros over and over again on an NFS mounted
directory.  When I do this with two CS20s running FreeBSD 5.2.1, I do not
generate a load over 1.00 and I do not experience ethernet timeouts.

However, within a few hours of doing the same excerise using a NetBSD 2.0F
client writing to the NFS mounted filesystem I have a process table that
looks like this:
 
  PID USERNAME PRI NICE   SIZE   RES STATE      TIME   WCPU    CPU COMMAND
  601 root     -14    0     0K   15M nfsclock   3:45  0.00%  0.00% [nfsio]
  596 root     -14    0     0K   15M nfsclock   3:39  0.00%  0.00% [nfsio]
  603 root     -14    0     0K   15M nfsclock   3:36  0.00%  0.00% [nfsio]
  589 root       2    0     0K   15M netio      3:35  0.00%  0.00% [nfsio]
   11 root     -18    0     0K   15M aiodoned   1:08  0.00%  0.00% [aiodoned]
   10 root      18    0     0K   15M syncer     0:16  0.00%  0.00% [ioflush]
 4471 smj      -14    0  2096K 1688K nfsclock   0:01  0.00%  0.00% dd

 5:58PM  up 17:18, 1 user, load averages: 4.91, 4.82, 4.84

I initially tried 2.0F with the generic.mp kernel, but the test would lock
up the machine fairly quickly (I couldn't even break to a debugger on the
console and had to do a hardware reset), so I switched to the single CPU
generic.

The CS20 running FreeBSD 5.2.1 is SMP however and has a proc table that looks
like this:

  PID USERNAME PRI NICE   SIZE    RES STATE  C   TIME   WCPU    CPU COMMAND
 4970 smj       96    0  3320K  2504K CPU0   0   0:04 12.69%  9.72% top
  358 root       4    0  1728K  1128K -      1  27:37  0.10%  0.10% nfsd
  359 root       4    0  1728K  1128K -      1  61:13  0.00%  0.00% nfsd
  361 root       4    0  1728K  1128K -      0   3:02  0.00%  0.00% nfsd
  362 root       4    0  1728K  1128K -      1   2:06  0.00%  0.00% nfsd
  363 root       4    0  1728K  1128K -      0   1:35  0.00%  0.00% nfsd

 6:05PM  up 2 days, 17:08, 1 user, load averages: 0.02, 0.04, 0.00

(note this is what uptime reports while the dd is still writing to its
NFS file system ..)

I can probably kill off the nfsio processes and bring the load down, but
my question is, why is it happening?  When I have the scenario reversed
you might find it interesting that FreeBSD as a client has delays (nfsrcvlk)
during simple operations such as 'cat' and 'ls' while the 500mb file is 
being created.  When NetBSD is the client 'cat' and 'ls' are pretty snappy.