nsswitch

[David Holland <dholland-tech%netbsd.org@localhost>]

so I've been meaning for some time (usual peanut gallery comments
here) to look at the nsswitch and getpw* (and similar) code, for the
following reasons:
   - we want to be able to kick db 1.85 out of libc when the time for
     the mythical libc bump comes;
   - this should not involve losing the ability to use db 1.85 (or
     db4/db5/whatever, too) to hash the password file or other system
     databases;
   - this needs to work even with static linking;
   - therefore the nsswitch logic needs to support forking
     subprocesses as an alternative to loading .so plugin files;
   - and it can't currently do this.
and,
   - the "compat" mode should be generalized so you can use it with
     arbitrary backends, which currently AFAIK doesn't quite work;
   - the password file code is a big mess.

Also, cursory examination in the past has suggested that the nsswitch
code is also itself kind of a mess.

I have now done some preliminary investigation, and have the following
preliminary conclusions:

 - The architecture of the nsswitch code may have had noble
intentions, but it's far more general than really needed; and its data
structures are more general still. This has several consequences; one
is that it's messy to interface to (as readily observed in the
password code); one is that it's possible for an external nss module
to provide meaningless combinations of methods; another is that it
uses a lot of arbitrary strings internally and there's nothing to
crosscheck for typos; and finally, while the dispatching code does
seem to more or less be legal C it isn't remotely typesafe.

 - The interfacing, at least for the builtin data sources, is the
transpose of what it ought to be; for abstraction reasons there ought
to be first sources, and then methods; but it's set up with methods
first and then sources. This is a direct contributor to (though not
the only cause of) the level of mess in getpwent.c.

 - If we had cvs rename the first step I would take is to move both
the nsswitch code and its various clients to their own subdir of libc,
something like "sysdb" or "systemdb" (name to be bikeshedded later...)
Right now the code's spread across net/ and gen/ with no structure.
However, we don't have cvs rename so I guess this won't be happening
anytime soon.

 - Since we can't have code that's polymorphic over function types in
C, and cutpasting the logic for iterating over sources into every
method of every data feed is not a winner, we should have something
like the vnode dispatching code, which came about the way it is (with
argument structures) largely in order to be able to handle lots of
assorted methods uniformly. This is not as safe as one might like, but
it's a lot safer than what the nsswitch dispatching is currently
doing. And it gets rid of a lot of the use of arbitrary strings too.

 - Rather than dispatching by passing some default function pointers
to nsdispatch() and having it do magic, things should work by fetching
a source object from the nsswitch code and then calling methods on it
explicitly. (And, the source objects for the different data feeds
should be different types -- I don't think there's going to be much to
gain from having them all be the same and it offers the opportunity
for confusion. But this isn't entirely clear.)

 - As far as I can tell nsdispatch() is not actually an outside-facing
function; it may have been conceived that way originally (e.g. to be
used by amd to get amd maps) but currently there is no data feed it
supports that makes sense for anything outside libc to be calling.
Therefore, we shouldn't have to provide compat for it, which is good
because rearchitecting things and providing compat for it would be a
big headache.

 - Providing binary compat for existing 3rd-party nss modules looks
quite doable, although after rearchitecting the source interface
should probably be different. This will require patching the ones that
exist; but there aren't very many and this shouldn't be a problem. (I
only see two in pkgsrc.)

 - It ought to be possible to use db 1.85, or sqlite, or db[456] from
pkgsrc, or whatever else to hash the local password file. (And other
"files" data feeds, too.) At the moment I'm not sure how choosing one
of these should relate to nsswitch modules, although obviously there
has to be something external that's linked to whatever database
library. (I'm sort of wondering if there should be an independent
switch in libc for key/value databases.) Right now there's no
abstraction for this at all in the password code.

 - After the rearchitecting (and maybe after the mythical libc bump
too) I'd expect the NIS and Hesiod password lookup code to live in
external nss modules and not in libc itself. With luck this and other
cleanup will allow kicking a bunch of sunrpc stuff out of libc too
when the bump comes.

Comments?

I realize this is all very broad generalizations, but it's too early
to be very specific.

-- 
David A. Holland
dholland%netbsd.org@localhost