Subject: nfsio behaviour on 2.0F
To: None <port-alpha@netbsd.org>
From: Stephen M. Jones <smj@cirr.com>
List: port-alpha
Date: 07/14/2004 13:10:05
I'm doing a little load test between FreeBSD 5.2.1 and NetBSD 2.0F machines
by creating a 500mb file of zeros over and over again on an NFS mounted
directory. When I do this with two CS20s running FreeBSD 5.2.1, I do not
generate a load over 1.00 and I do not experience ethernet timeouts.
However, within a few hours of doing the same excerise using a NetBSD 2.0F
client writing to the NFS mounted filesystem I have a process table that
looks like this:
PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND
601 root -14 0 0K 15M nfsclock 3:45 0.00% 0.00% [nfsio]
596 root -14 0 0K 15M nfsclock 3:39 0.00% 0.00% [nfsio]
603 root -14 0 0K 15M nfsclock 3:36 0.00% 0.00% [nfsio]
589 root 2 0 0K 15M netio 3:35 0.00% 0.00% [nfsio]
11 root -18 0 0K 15M aiodoned 1:08 0.00% 0.00% [aiodoned]
10 root 18 0 0K 15M syncer 0:16 0.00% 0.00% [ioflush]
4471 smj -14 0 2096K 1688K nfsclock 0:01 0.00% 0.00% dd
5:58PM up 17:18, 1 user, load averages: 4.91, 4.82, 4.84
I initially tried 2.0F with the generic.mp kernel, but the test would lock
up the machine fairly quickly (I couldn't even break to a debugger on the
console and had to do a hardware reset), so I switched to the single CPU
generic.
The CS20 running FreeBSD 5.2.1 is SMP however and has a proc table that looks
like this:
PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND
4970 smj 96 0 3320K 2504K CPU0 0 0:04 12.69% 9.72% top
358 root 4 0 1728K 1128K - 1 27:37 0.10% 0.10% nfsd
359 root 4 0 1728K 1128K - 1 61:13 0.00% 0.00% nfsd
361 root 4 0 1728K 1128K - 0 3:02 0.00% 0.00% nfsd
362 root 4 0 1728K 1128K - 1 2:06 0.00% 0.00% nfsd
363 root 4 0 1728K 1128K - 0 1:35 0.00% 0.00% nfsd
6:05PM up 2 days, 17:08, 1 user, load averages: 0.02, 0.04, 0.00
(note this is what uptime reports while the dd is still writing to its
NFS file system ..)
I can probably kill off the nfsio processes and bring the load down, but
my question is, why is it happening? When I have the scenario reversed
you might find it interesting that FreeBSD as a client has delays (nfsrcvlk)
during simple operations such as 'cat' and 'ls' while the 500mb file is
being created. When NetBSD is the client 'cat' and 'ls' are pretty snappy.