Subject: Importing kqueue's from FreeBSD...
To: None <tech-kern@netbsd.org>
From: Luke Mewburn <lukem@wasabisystems.com>
List: tech-kern
Date: 03/15/2001 15:33:37
After I saw a presentation at BSDCon 2000 last year about the
implementation of kqueue in FreeBSD, a few NetBSD developers
present decided that it's definately worth a look for NetBSD.

kqueue (aka `kevent') is a generic and scalable event
notification facility, developed by Jonathan Lemon of FreeBSD.

It's generic: you can monitor socket events, filesystem events
such as `file ready to read', vnode operations such as `file has been
deleted', `directory has been modified', signals delivered to a
process, process creation/exit, etc.

It's scalable: time to process increases linearly with respect to
number of outstanding events (such as `socket waiting to be read'),
unlike select & poll where the time increases with respect to the
number of sockets listened to.

It supports multiple listeners for the same event. FreeBSD has had it
in a couple of releases, and I believe third party vendors such as
Apache are already using the API.  Other API's such as Solaris'
/dev/poll and Irix's /dev/imon provide a subset of what kqueue offers.

Jonathan has more information, both the original paper and the slides
he presented at BSDCon 2000, up at:
	http://people.freebsd.org/~jlemon/

It's also referenced in a great paper on server scalability:
	http://www.kegel.com/c10k.html

I think we need this in NetBSD; it would be of great benefit to
various applications that need scalability. If we do import this,
I feel that it's imperative that we maintain the same API as FreeBSD,
or if we feel it needs enhancing, at least work with Jonathan to
maintain compatibility with FreeBSD. There's no point in doing an API
like this for third-party application vendors that is gratuitiously
incompatible with FreeBSD where the API originated from.


I have ported his original diffs to FreeBSD into NetBSD, and
incorporated most of the patches and fixes made in FreeBSD after the
original import into FreeBSD; everything except the recent rework to
move the filesystem kqueue code lower down to make it more generic - I
wanted to get it working first.

I have this almost working, modulo a couple of panics in certain
conditions that I'm debugging. (These only occur if I create a kqueue
and try to trigger a certain event.)

I'd like to bring these into -current, #ifdef-ed appropriatly, to
allow others to `bang' on the code as well.

Thoughts/comments?

-- 
Luke Mewburn  <lukem@wasabisystems.com>  http://www.wasabisystems.com
Luke Mewburn     <lukem@netbsd.org>      http://www.netbsd.org
Wasabi Systems - providing NetBSD sales, support and service.