UNIX kernel notification system

[Erik Fair <fair%netbsd.org@localhost>]

Ah, Matt, now you've stepped in it: UNIX kernel notifications, and a model for 
that. A topic that I glossed over in my previous note.

There are three basic modes of UNIX use:

1. traditional multi-user timesharing system. We in NetBSD land still use our 
systems this way sometimes; cf. ftp.netbsd.org

2. single-user workstation (this includes laptops) and that user works on the 
"console" (or "graphics device" + "input devices").

3. lights-out server in some dark closet or data center. Console? Uh, yeah, 
it's over there, on top of the KVM or serial switch; keyboard's in the drawer. 
Be sure to blow the dust off it before you use it.

We ought to try and come up with a notification abstraction model that works 
reasonably well for each use case, and preferably one which permits automated 
userland software response to various common events.

So, historically, we have the venerable serial console (or some simulation 
thereof) onto which the kernel can printf, with the expectation (well, 
assumption) that an "operator" is watching (reading), and will respond; born of 
that time when UNIX systems were too heavy for a single human to lift, kept in 
well air-conditioned glass rooms, with noisy LA-120 DECwriter II consoles ... 
or if we go way back, *really* noisy TTY model ASR-33's. With real paper made 
from trees!

A little later, we faked that with xterm -C or xconsole. Or the kernel just 
blatted all over your frame buffer in large font, and you had to xrefresh(1), 
and then dmesg(8) in an xterm to see what it had said.

Or syslog(3), on the presumption that there's a userland daemon "watching" 
(well, logging the events) ... and ... someone maybe, eventually looks at it? 
Maybe that feeds into an SNMP network management console ... somewhere?

UNIX used to require more operator vigilance & intervention than it does now; 
users are so impolite - they keep sucking up all the CPU with runaway 
processes, and filling up the disks. Of course, now that we've given them all 
their own computers to use (and thus manage), that's their headache now, isn't 
it?

We've done a pretty good job in automating a range of what had been manual 
tasks (log rotations, etc.), but we're still working on more automated 
responses (OK, pre-coded software responses) to various system events like, 
"the network interface just changed IP addresses on us!" or if you're paranoid, 
how about intrusion detection systems ...

We set up sysctl(9) for kernel variables we wanted to read/tweak - that's 
better than poking at /dev/kmem with adb, as was the practice.

Why not a classification/taxonomy of kernel missives? This doesn't mean we 
can't continue to have relatively free form (and possibly amusing) text for 
those conditions we're not yet prepared to classify/codify yet ('cause they're 
rare, or debug, or ... whatever). The potential for win is in making (or 
retaining) software parse-ability to enable software response.

        just some thoughts on an old topic,

        Erik <fair%netbsd.org@localhost>