Subject: Re: Request for comments: sharing memory between user/kernel space
To: der Mouse <mouse@Rodents.Montreal.QC.CA>
From: Bill Stouder-Studenmund <wrstuden@netbsd.org>
List: tech-kern
Date: 03/22/2007 13:01:23
--zx4FCpZtqtKETZ7O
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Thu, Mar 22, 2007 at 02:57:21PM -0400, der Mouse wrote:
> > Depends on how the socket buffer is being handled.  If we have the
> > pending info sitting around in mbufs, this change won't do much.
> > You're right that it won't save us anything.
>=20
> > If however we copy the incoming packets into the buffer, we save an
> > extra copy in the kernel.  Yes, we still have a copy to userland, but
> > we have one less copy in the kernel.
>=20
> But doesn't that require either (a) exposing *all* network traffic to
> the userland process or (b) knowing at DMA buffer setup time (ie, well
> in advance) which packets are destined for that particular connection?
>=20
> Now, (a) may be acceptable for some special-case applications, but the
> discussion I've seen makes this sound as though it's something intended
> for general-purpose use.  And I think (b) is even less generally
> suitable than (a) is....
>=20
> Perhaps I'm misunderstanding something.

Yes. This suggestion doesn't eliminate the data copy (DMA) from card to=20
mbuf, it eliminates a possible datacopy from mbuf to kernel socket buffer.=
=20
It also releases the mbuf sooner (if we are in the regime where we have=20
the data hang around in mbufs) because the to-userspace copy happens=20
sooner.

I agree that this is not the most agressive improvement possible. It=20
however would be an improvement and can happen with little disruption to=20
the TCP stack; all that has to change is we gain a way to retain these=20
buffers, to use them, and then to tell userland that we're done with=20
them. Also, there could be a cache impact effect. If we are doing TCP=20
checksumming manually, we probably filled the D Cache with the packet.=20
This way the packet gets to userland before it's flushed out of the cache=
=20
(or, technically, it's more likely that it's still in cache). This of=20
course assumes you're not using offload and have enough cache that the=20
whole thing fits anyway.

Take care,

Bill

--zx4FCpZtqtKETZ7O
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (NetBSD)

iD8DBQFGAu6jWz+3JHUci9cRAozAAJoCgiEmXdCrMSnSxBJFMF1XmWJ2NQCePCyc
KISly7Y1DAfajsDZzDk+JSI=
=HyNj
-----END PGP SIGNATURE-----

--zx4FCpZtqtKETZ7O--