Subject: Re: bpf/pcap performance
To: Darren Reed <darrenr@reed.wattle.id.au>
From: Jonathan Stone <jonathan@dsg.stanford.edu>
List: tech-net
Date: 04/07/2004 11:34:22
date:component="Your message dated",formatfield=""
body:component=">",overflowtext=">",overflowoffset=0
In message <200404071527.BAA00735@avalon.reed.wattle.id.au>Darren Reed writes
>
>Does anyone know why using bpf/pcap would drop packets when running
>netbsd as opposed to freebsd ?
>
>In testing on identical hardware, using an identical test suite, I've
>observed 100% packet capture with freebsd 5.1 but netbsd 1.6zk always
>returned some packet loss (in this instance, missing 1 packet is as bad
>as missing 10 or 100 or 1000.)  On this particular piece of hardware,
>freebsd 5.1 is returning 100% captures at upto 150Mb/s spread over 4
>NICs (2*Intel Pro100S, 2*Intel 1000MT.)
>
>Is anyone interested in looking at this ?

Darren,

Yes, there are interested parties.  Unfortunely, (wearing my
kernel-hacker's hat), your test changed three or more independent
variables at once: what libpcap requests the kernel for its bpf size,
kernel-internal limits (or default limits) on bpf buffer size; and the
NIC drivers (em vs wm is significant; fxp probably somewhat less so).

Some weeks ago, I committed increase to NetBSD-current's default
kernel limit, and the limit libpcap asks for. (I just checked a 2.0B
i386 kernel, and net.bpf.maxbufsize is 1048576).  Updating to just a
new NetBSD kernel would not pick up the userland changes to libpcap;
if you have old userland code, it won't ask for a nice large buffer.

Are you using polling on the FreeBSD side? If your machine is truly
gutless, one ~1Khz interrupt shared across all 4 NICs might
conceivably make a difference, compared to NetBSD (where all the NICs
interrupt independently).

Sometimes I regret how well the i386 port does at finding IRQ
mappings.  Your application might be better served with a single
shared interrupt across all four PCI NICs.

>Can anyone tell me if....gasp...wasabi are likely to have made any
>improvements here?

I can't speak for Wasabi, but nobody commented when the topic came up
a few months back.