Subject: Re: Request for comments: let config(1) generate LKMs
To: Bill Stouder-Studenmund <wrstuden@netbsd.org>
From: Hiroyuki Bessho <bsh@grotto.jp>
List: tech-toolchain
Date: 09/27/2007 16:42:55
Bill Stouder-Studenmund wrote:
> On Sun, Sep 23, 2007 at 01:54:37AM +0900, Hiroyuki Bessho wrote:
>   
>> At Tue, 18 Sep 2007 13:21:11 -0700,
>> Bill Stouder-Studenmund wrote:
>>     
>>>       
>>>>   I think we can allow users to say
>>>>
>>>> module       ne0	at isa? port 0x280 irq 9
>>>> module "nex" ne1	at isa? port 0x290 irq 10
>>>> module "nex" ne2	at isa? port 0x300 irq 11
>>>>
>>>>   In this case, module ne.o has ne and ne_isa driver codes and a
>>>> device instance ne0. Module nex.o will have only config(9) glue staff
>>>> to probe port 0x290 and 0x300 and attach ne1 and/or ne2 because ne and
>>>> ne_isa related codes are in module ne.o.  I'm not 100% sure this works
>>>> yet, but I guess that allowing users to specify module name for
>>>> devices will provide good way for some configuration.  And if it
>>>> works, config(1) can detect that module nex.o depends on module ne.o
>>>> and can submit some useful information.
>>>>         
>>> Why would we want to support this option? I don't see the benefit to 
>>> making something like the "nex" module.
>>>       
>>   I'm afraid this "nex" was not a good example for optional module
>> name after `module' keyword.  I thought we'd better to allow users to
>> change how to "group" the codes into modules. (Sorry that I can't
>> think of a better English word than "group".  I mean which codes go to
>> which module.)  I was afraid that whatever rules we define, users might
>> want the other way.
>>     
>
> Group is probably a good word. However I'm not sure why we want to support 
> this. Why will users care exactly what is in which module?
>
> If we're really supporting modules, we also need a way (beyond the scope
> of exactly what you're talking about here but rather directly enabled by
> what you're talking about here) of loading the right modules. A way that a
> daemon or startup tool can notice we need driver X loaded into the kernel.
> For direct-attach busses (PCI), we can match the locators in user space to
> get a candidate list, then let the modules in the kernel figure it out.
> For indirect busses (ISA), we can load modules and see if they attach.
>
> The upshot is I fail to see why users will care what code is in exactly 
> which module. Assuming the drivers the user wants are in there, what more 
> will be needed?
>   
  OK, I gave up to convince you to think allowing optional module names 
is a very useful feature, as I couldn't think
of stronger examples.  Anyway it's optional and if we make good enough 
rules to satisfy all users,  no one will use it.

> I also see a strong downside to permitting this customization. I think 
> that it will make modules more complicated to support at the project 
> level. Sure, for any one compile, it won't matter. But when someone asks 
> us for help on the list, variety in exactly what is in which module will 
> complicate our support on the lists. With a consistent set of modules, we 
> can more readily understand a situation a user is having a problem with.
>
> As a specific example, with a consistent configuration of what's in what 
> module, a user can post the output of modstat, and we all understand 
> what's loaded. With a variety in what goes in exactly what module, we 
> would realy need the output of nm on the modules to tell what's what.
>   
  I don't think  you need  to worry about such cases.  If I implement 
the feature (It is implemented for now,
but I'm not sure I'll keep it), it is just for users who want a special 
tweak (like me), and I don't expect many
users will use it and get confused and ask you to help on the list.
>>> I think a good rule is:
>>>
>>> a) If nothing depends on the attribute, leave it out.
>>>
>>> b) If something hard-coded into the kernel depends on the attribute, build 
>>> it into the kernel. Also mark it as present for whatever dependency system 
>>> we come up with.
>>>
>>> c) if modules depend on the attribute and nothing hard coded into the 
>>> kernel does (i.e. we aren't also in case (b)), make a module. And note all 
>>> the dependencies.
>>>
>>>       
>>   I'd say for c):
>>
>>   c) if one module depends on the attribute that is not in the
>>   kernel, the attribute is compiled into the depending module.
>>
>>   d) if more than one modules depend on the attribute that is not in
>>   the kernel, a module for the attribute is made. And note all the
>>   dependencies.
>>
>>   I hope this will reduce the number of modules and dependencies in
>> some cases.
>>     
>
> Why do we care about the number of modules?
>
> I dislike this suggestion for the reasons above. What exactly is in a 
> given module will depend on the kernel it was built for. That strikes me 
> as a support nightmare. :-)
>   
  I'm afraid you cannot be completely free from your nightmare :-)   
There always be some cases where
LKMs from one config file doesn't work with a kernel from the other 
config file.  `options'  can affect
both LKMs and kernel, for example.
> I admit I am assuming we have one thing that hasn't been mentioned, which 
> is a way to automatically load dependencies before loading a module. So if 
> I ask to load module X, which depends on Y and Z, Y depends on Z and P and 
> Q, and Z depends on R and S, then R & S get loaded, P & Q & Z get loaded, 
> Y gets loaded, then X gets loaded.
>   
  I  believe we are basically on the same assumption here.  I think the 
best place to put module dependencies is
LKM .o file using ELF sections.  Modload(8) can be extended to read 
those sections to find required
modules.  But you seem to assume one thing that I don't;

   LKMs are independent with kernels, and you can load any LKMs to the 
kernel built from the differenct config file.
 
  To support it, we'll need changes to kernel to provide some means to 
know which attributes are in the running kernel.
I feel it's easier to tell users to use LKMs from the same config file 
with the kernel, at least for the first step.
> But I don't think that's that hard to add once the other pieces are there. 
> And there's lots of prior art.
>
> Thus I do not understand why we want to reduce the number of dependencies.
>   
  For load-time efficiency.  Assume you have module A which depends on 
module B, and no other
 module needs module B.  If you always load module B when you need 
module A,  why don't you
link them together into one module?  I always prefer doing  as many 
things as possible  in  compile time
to  consuming  CPU cycles  in  run-time.

  And also I guess kernel memory consumption is smaller with one 
combined module than with multiple
modules with same functionality.

  But you made me realize that  it is a kind of optimization rather than 
a principal.  I think I should make it
 an option to config(1).  For your healthy sleep,  users may be 
discouraged to use the option  :-)

> I also see a strong advantage of modules for our release process. Right 
> now we build a number of kernels as part of building a release. With all 
> the drivers in modules, a number of the differences between kernels reduce 
> to a different set of drivers. So we can build the modules once and reuse 
> them. Or build only a few key sets of modules and reuse them as much as we 
> can. :-)
>
> Take care,
>
> Bill
>   

BR.
--
bsh.