Subject: Re: NetBSD and large pps
To: Miles Nordin <carton@Ivy.NET>
From: Jonathan Stone <jonathan@dsg.stanford.edu>
List: tech-net
Date: 12/03/2004 11:14:49
In message <oqis7k6ot9.fsf@castrovalva.Ivy.NET>,
Miles Nordin writes:

[... guess at NAPI ...]

Oh. Well, thanks for that. I've implemented something similar,
privately; but didn't give it a silly marketing name.


>    js> That said, you can get a dramatic increase in peak throughput
>    js> by trading off for increased latency.
>
>Great.  Where can we read about these sysctls?  It would help if you
>were more specific.  You don't mention the name of the sysctl, [...]

Yes, I did, but in another reply to the OP, just after the one you
quote from:

In message <E1CZxix-0003WU-00@smeg.dsg.stanford.edu>, Jonathan Stone
wrote:

[...]
>
>I suggest you try using sysctl to increase net.inet.ip.ifq.maxlen from
>its default value of 50 to 256 or threabouts, then try tweaking the
>undocumented(?) sysctl hw.bge.rx_lvl to values between 3 and 5.  If at
>all possible do the tweaking from a local serial line, not a network
>connection.

The bge sysctl values aren't well documented by _anyone_. The values
in the NetBSD if_bge source are chosen by ad-hoc experimental
tweaking.  Those sets of values happened to do well, for my network
workload, on the bcm5700 and 5704 devices I had to play with over a
year ago.  With the sysctl settings I recommended to Mihai, I can
happily sink one ttcp TCP stream, unidirectionally filling a gigabit
on one port of a bcm5704, and generate another unidirectional ttcp TCP
stream on the other port.  

(If dim memory serves, I think I could actually fill, one-way, four
ports on two bcm5704s; but don't hold me to it).


Re the IP input queue limits: I have a very clear recollection that
I've suggested raising the NetBSD default limits to signficantly
larger values, appropriate for gigabit (and for some, 10GbE) NICs with
significant interrupt mitigation. Perhaps it's time to Just Do That.
Or at least adding a `peak' counter. Or maybe both.