Jonathan Stone wrote:
> In message <41AEEAB0.9000506@netbastards.org>Mihai CHELARU writes
> 
>>Hello,
>>
>>I observed that NetBSD is not able to route more than 30kpps. When it 
>>reaches this limit it usually stays there for a while and then suddenly 
>>reboots. I don't use NAPI nics so I get around 30k irqs/sec. Is this a 
>>known issue ? What can I do about it ?
> 
> 
> It woul help a lot if you could restate your question using
> better-accepted terminology, and in rather more detail.
> 
> I assume that 'irq/sec' means interrupts per second, but its not clear
> if that's just from your NICs, or all interrupts. It doesn't seem
> woefully high to me, but you don't state what CPU architecture or what
> model/speed of CPU you're using (x86 of some kind?), or even which
> NetBSD version (1.6.2, 2.0, or -current?  I also don't know what NAPI
> is; is it perhaps Microsoft- or LInux-centric jargon for some flavour
> of TCP offload? That's unlikely to help for IP routing.
> 
> That said, you can get a dramatic increase in peak throughput by
> trading off for increased latency. Some drivers have hooks to let you
> experiment with that via sysctl. However, at large packet rates, you
> will also need to increase the limits on packet-queue sizes: otherwise
> the NICs may hand back more Ethernet frames in a single interrupt than
> the IP input queue is prepared to hold, and you get persistent drastic
> packet loss.
> 
> Also, what kind of forwarding do you require? Layer-2 bridging, normal
> IP routing, or IP routing with firewall filtering as well?
> 

Hello Jonathan,

Ok, sorry, I forgot to mention, it's NetBSD-2-0 i386. The hardware 
configuration is this:

	- dual Xeon @ 2GHz
	- HT enabled
	- 1GB RAM
	- 2 * BCM5702X PCI-X nic

Software tweaks:
	- HZ 1000

So, I have 4000 IRQs/sec generated by scheduler. Rest of IRQs/sec is 
less than 200, except the NIC IRQ that is filling up to 30000 IRQs/sec.

NAPI means RX polling, meaning that not for every packet received there 
is an IRQ generated. Don't know more, this is what I understood from 
Google :) So it's not TCP TX cksum offload or such. Anyway, I don't 
beleive it has such feature (I only seen this on Intel e1000).

Routing/Bridging don't matter. It just won't accept more than 25k pps in 
both situations on ingress (this is corresponding to about 30000 
IRQs/sec). Even more, lowering HZ back to 100 makes it accept more 
packets up to the limit of 30k IRQs/sec.

-- 
Thanks
Mihai