Subject: Re: A potential step towards modularisation
To: Pavel Cahyna <pavel.cahyna@st.ms.mff.cuni.cz>
From: Quentin Garnier <netbsd@quatriemek.com>
List: tech-kern
Date: 02/08/2004 22:14:02
Le Sun, 08 Feb 2004 21:04:50 +0100
Pavel Cahyna a ecrit :
[...]
> There should be a way to embed in the ex_pci.ko module the information
> that it depends on the ex attribute, and in a similar way, to embed in
> ex.ko the information that it:
> - provides ex
> - depends on arp, ether, ifnet, mii, mii_bitbang

That's the goal.

[...]
> There should be a way for the running kernel to tell all the attributes
> it has compiled in. That way you know the kernel has attribute ex and
> you can safely load ex_pci.ko which depends on it, without loading any
> other LKM.

Yes, this is where I want to go, too.

> If such information is embedded in the LKMs themselves (and in the
> kernel), there will be no need for a "depmod" utility (like in Linux,
> see below) and for separately maintained dependency information (which
> could get out of sync).

Well, there are some caveat with the 'need-count' and 'need-flag' options
for files, which control optional dependencies.  'need-count' might not be
an issue for long, since it would be very easy to add a variable to
contain it.  But 'need-flag' makes code be optionally compiled, which
doesn't get along well with LKMs.

I think that getting rid or working around need-flag uses might add a lot
of complexity to the kernel.  I'll take the example of the audio device,
and the file dev/audio.c.  That file might be compiled in if one of audio,
midi or midibus attributes are selected.  If audio is not selected, only a
single function of audio.c is compiled in.  Making this LKM-friendly will
not be easy, unless the path of creating an attribute (and a module) for
that single function is chosen.

Also, audio 'need-flag's aurateconv, which change the structure
audio_softc.  There, we will have to make a design choice: either adding a
#ifdef _LKM blocks (technically the module will be able to optionally
depend on another, obtaining the symbol values from ksyms) or making it
always possible for audio to depend on aurateconv.

> Also, the dependencies should be at attribute level (ex_pci.ko needs
> ex), not at module level (i.e. ex_pci.ko needs ex.ko - NOT!), because
> the functionality you need may be compiled in the kernel.

Certainly not at the module level.

> Is the ELF format extensible enough to have the dependency information
> in a special section?

Yes.  Besides, we will have much control over the way the modules will be
loaded.

> > My concern is that by having multiple module options, we create a
> > support morass. If this implementation is the direction we all want to
> > go, I'd rather have it than nothing.
> > 
> > But I think we'd be better served if we came up with some standard way
> > to carve the kernel up into modules. Among other things, drivers that
> 
> As I understand it, there would be a standard way - because for each
> attribute, there would be a module.

Mostly, yes.  As I said in another post, there is code to merge modules in
the unlikely case several selected attributes share a file, but I'd tend
to consider this a bug in the way the said file is used.

[...]
> > I'm mainly thinking of audio drivers, but usb would probably fit, and
> > I bet there will be more with time.
> > 
> > We will need a standard set of modules to be able to really support
> > 3rd-party modules. If I as a vendor can't be sure about how to tell
> 
> I think there should be a standard set of attributes. If the code for
> those attributes is statically linked into the kernel or provided in a
> module shouldn't matter, no?

Indeed, that's the goal.  But this is a design issue for the in-kernel
linker.  As my config(8) patch is made, it is easy to add various lists of
attributes to the resulting module, but first I have to know how to add
them.

> > I appreciate that you're working on this. Please keep doing it!
> 
> I also appreciate it. But please, do it well, not like Linux.

That's precisely why I try to gather comments before I spend too much time
on it :)  And thanks for yours.

[...]
> > they handle this?
> 
> As I understand it (if there are Linux gurus here, feel free to correct
> me, I'm not a kernel expert), the dependency in Linux LKMs are at symbol
> level. There is a tool (depmod) which computes dependencies from a set

This is my understanding, too.  But I will investigate precisely.  I'll
see what FreeBSD does, also.

[...]
> The result is that dynamic autoconfiguration DOES NO LONGER WORK. The
> configuration becomes stateful. There must be a program which scans your
> hardware at each change and writes correct entries to /ec/module.conf .
> (In RedHat, this program (called kudzu) is run on every boot, in case
> you change your HW).
> 
> Dynamic loading of network protocols and filesystems doesn't seem to
> have such problems, because in this case it's always clear what
> functionality you want.
> 
> The only solution is to load all the hardware drivers for the hardware
> you have. It wastes some memory, if you don't use this hardware, but
> this shouldn't do much harm. How this should be done is another
> question.

Well, we'll see about this.  I don't really believe in autoconfiguration
unless the kernel gets feeded with the probing functions.

-- 
Quentin Garnier - cube@cubidou.net - cube@NetBSD.org
"Feels like I'm fiddling while Rome is burning down.
Should I lay my fiddle down and take a rifle from the ground ?"
Leigh Nash/Sixpence None The Richer, Paralyzed, Divine Discontents, 2002.