Subject: Re: NetBSD-1.5 and NFS - any suggestions?
To: <>
From: David Laight <David@l8s.co.uk>
List: netbsd-users
Date: 01/26/2002 12:52:43
> > > Hm.. it sounds like this is related to a machine being both a client
> > > and a server, and having traffic for both functions. It's not
> > > a configuration that is used often. Try pushing a machine using
> > > it both as a client and a server, I suspect you may see the hangs
> > > more often.
> >
> 
> don't know whether it helps but i had similar effects (few seconds chokes)
> on 3c509 ISA client and 3c590 PCI server.
> 
> mounting with 1,2 or 4k blocks instead of default 32k helps.
> 

I wonder....

Maybe your network card doesn't have enough buffer space to receive
significant numbers of back to back ethernet packets.  NFS (over udp)
will be sending 32k (aarg - 8k is bad enough) as a single UDP datagram.
I make that 23 packets.  If a single one gets dropped the request is
repeated (by NFS).  Not only that, but the IP layer will be
holding on to the incomplete message hoping the missing fragment
will turn up - which it never will be.

Now since IP won't recognise the rerequest as being (effectively) the
same as the packet stuck in re-assembly (it has different RPC identifiers)
the old partial packet isn't discarded.  The net effect is that you run
out of memory for IP re-assembly. The memory won't be freed until the
re-assembly timeout expires.

In my experience you can't get data off ISA cards as fast as it arrives!
If this is working at all, your card probably has best part of 32k
available for rx buffering.  However as soon as you get concurrent NFS
traffic, the card has to buffer more data....

I see three options:

1) Mount with 1k blocks
2) Make NFS use shorter blocks for retries, and some dynamic sizing
3) Make the system dump buffers in IP reassembly when it runs out of
   space.
4) Don't use crappy ISA ethernet cards, and give the PCI ones enough
   rx buffer space.  (IIRC 'le' was running with 8 rx buffers on 1.5.2,
   not enough to cope with interrupt latency)
5) Don't expect NFS to work with ISA cards at 100M
(I can't count)

    David