Subject: Re: rfc: socketing it to gre
To: None <tech-net@NetBSD.org>
From: David Young <firstname.lastname@example.org>
Date: 10/02/2007 16:04:51
On Tue, Sep 18, 2007 at 12:23:07PM -0700, Bill Stouder-Studenmund wrote:
> On Fri, Sep 14, 2007 at 04:09:08PM -0500, David Young wrote:
> > On Fri, Sep 14, 2007 at 12:14:11PM -0700, Bill Stouder-Studenmund wrote:
> > >
> > > How much of a performance hit do we take for this?
> > I haven't measured. I will.
I have measured the TCP performance with iperf, now. Initial tests
showed a significant slowdown. A 133-MHz Soekris net4521,
cpu0: AMD Am5x86 W/B 133/160 (486-class), id 0x4f4
could sink a TCP stream over fully-socketized gre(4) at 75% the speed
of the gre(4) in -current today. The same box could source a TCP stream
over fully-socketized gre(4) at 90% the speed of gre(4) in -current today.
(The other end in all my tests is a Pentium 4 running -current from
The net4521 is very slow by today's standards. To get an idea of the
performance on a faster machine, I switched to a net4801,
cpu0: National Semiconductor Geode GX1 (586-class), 266.62 MHz, id 0x540
Performance was better, but dissatisfying. Taking some advice from
Matt Thomas and Andrew Doran, I made many optimizations: I stopped
waking a kernel thread to call sosend() and soreceive(); instead,
I use slimmed-down copies of sosend() and soreceive() that I can run
directly from a software interrupt and from a socket upcall, respectively.
The kernel thread is completely out of the xmit and recv paths, and on the
net4801, the slowdown of socketized gre(4) is IMO acceptable, at about 96%
of -current in both directions. I will test on the net4521 in a while.
> > I believe we will see an improvement in performance at the receiver when
> > there are hundreds or thousands of tunnel interfaces, because the socket
> > demux code uses a hash instead of walking a list.
I was wrong about this: while UDP sockets do use a hash, raw IP sockets
do not. Nevertheless, raw IP sockets *should* use a hash, and I have
been reading the code to find a way to do it where raw/UDP/TCP sockets
can share the most code.
David Young OJC Technologies
email@example.com Urbana, IL * (217) 278-3933 ext 24