Subject: Re: NetBSD and large pps
To: Mihai CHELARU <kefren@netbastards.org>
From: Jonathan Stone <jonathan@dsg.stanford.edu>
List: tech-net
Date: 12/02/2004 11:26:19
In message <41AEEAB0.9000506@netbastards.org>Mihai CHELARU writes
>Hello,
>
>I observed that NetBSD is not able to route more than 30kpps. When it 
>reaches this limit it usually stays there for a while and then suddenly 
>reboots. I don't use NAPI nics so I get around 30k irqs/sec. Is this a 
>known issue ? What can I do about it ?

It woul help a lot if you could restate your question using
better-accepted terminology, and in rather more detail.

I assume that 'irq/sec' means interrupts per second, but its not clear
if that's just from your NICs, or all interrupts. It doesn't seem
woefully high to me, but you don't state what CPU architecture or what
model/speed of CPU you're using (x86 of some kind?), or even which
NetBSD version (1.6.2, 2.0, or -current?  I also don't know what NAPI
is; is it perhaps Microsoft- or LInux-centric jargon for some flavour
of TCP offload? That's unlikely to help for IP routing.

That said, you can get a dramatic increase in peak throughput by
trading off for increased latency. Some drivers have hooks to let you
experiment with that via sysctl. However, at large packet rates, you
will also need to increase the limits on packet-queue sizes: otherwise
the NICs may hand back more Ethernet frames in a single interrupt than
the IP input queue is prepared to hold, and you get persistent drastic
packet loss.

Also, what kind of forwarding do you require? Layer-2 bridging, normal
IP routing, or IP routing with firewall filtering as well?