Subject: Re: HEADS UP: kqueue support merged
To: Jaromir Dolecek <jdolecek@netbsd.org>
From: Chuck Yerkes <chuck+nbsd@2002.snew.com>
List: tech-kern
Date: 10/24/2002 09:40:27
The paper on the kqueue stuff was delivered at Usenix/Boston, in
2001 IIRC.  It seemed promising to problems we had at my then-employer
dealing with mail oriented proxies (though *BSD support was ironically
never there).

Read up on it.  It's available in OpenBSD, FreeBSD and now NetBSD.
Sun and other commercial vendors, being as speedy as they are,
might have it by 2006 :)

It's a really useful thing when your heavily threaded apps
need to watch lots of connections.  Select(3) can be a killer.

The watching of files/dirs for changes is something I'd missed.
I'd look forward to things taking advantage of this.  Using this
on dirs served by ftpd/SAMBA/Netatalk jump to mind - I could
tell users that when the drag a file to a folder on the desktop,
it will trigger processing on the server.  I've only done this
with cron and various scripts of late.  Workable, but unideal.

Quoting Jaromir Dolecek (jdolecek@netbsd.org):
> 
> Support for kqueue has been merged to -current today. This code
> is based upon code written by Jonathan Lemon from FreeBSD
> project. The NetBSD port was done by Luke Mewburn, Jason Thorpe
> and Jaromir Dolecek.
> 
> Kqueue provides a generic method of notifying the user
> when an event happens or a condition holds, based on the results
> of small pieces of kernel code termed filters.
> 
> Since kqueue provides stateful interface to poll many descriptors
> at once, it's ideal for applications which need to watch large,
> mainly constant set of descriptors, for which poll(2)/select(2)
> has significant system overhead.  Example applications which might
> benefit from kqueue are web servers, proxies, ircd etc.
> 
> Kqueue also provides support for watching files and directories
> for changes. This means that e.g. file managers can use kqueue
> instead of periodic stat(2) on the directory nodes. It's my
> understanding that e.g. Samba could make a good use of this too;
> Samba needs to watch&report file/directory changes to support
> the Microsoft Windows change notification filters.
> This particular feature has been missing in Unix systems for a long
> time, and it's good it's finally coming to NetBSD too.
> 
> On NetBSD, kqueue is supported by every device driver supporting
> poll(2), so kqueue can be used as replacement for select(2)/poll(2)
> in all cases. Also, we support filesystem kevents for all writable
> local filesystems (including ffs, lfs, ext2fs, msdosfs) and also
> for NFS. Thus, file managers and file servers can depend on the
> functionality to be present for any filesystem likely to be used
> as file storage.
> 
> There are some plans to add glue for notification of e.g. USB
> devices additions/removals to userland or to support e.g. ifwatchd
> functionality better way in future. The code is extensible
> well and any additional filters like that are easy to add.
> 
> For more detailed information, see Jonathan's paper available at
> 	http://people.freebsd.org/~jlemon/papers/kqueue.pdf
> or
> 	http://people.freebsd.org/~jlemon/papers/kqueue.ps
> 
> and/or the NetBSD kqueue(2) manpage.