Loading kernel modules with the boot loader

[Andrew Doran <ad%netbsd.org@localhost>]

Hi,

Someone asked me to write about how this works. There are a few things
required of MD code:

- Modules must be loaded to a page-aligned address, and must be mapped
  contiguously in VA space. Some combination of activity by the boot loader
  and MD kernel accomplishes this.

- If there are restrictions on the architecture's jump/call reach, they have
  to be mapped close to the kernel text. As examples, this means within 4GB
  (2GB? can't remember) on amd64, on i386 it doesn't matter.

- MD code provides a module_init_md() function. When called this should loop
  over all objects loaded by the boot loader and call module_prime() for
  each, passing in the address and size (if known, -1 if not). Example:
  http://nxr.homeunix.org/source/xref/sys/arch/x86/x86/x86_machdep.c#133

- The boot loader has to know about kernel versions in order to search
  for modules in the correct location. That means it first builds a list
  of modules, loads the kernel and gets the kernel version, and then tries
  to load the modules. Then the kernel starts. libsa pulls the kernel
  version from an ELF note so MD code doesn't need to do that. Example:
  http://nxr.homeunix.org/source/xref/sys/arch/i386/stand/lib/exec.c#323

- The x86 boot loader loads both i386 and amd64 kernels so it knows to look
  for modules in an arch directory matching the kernel image. Don't know if
  this matters for other platforms.

- Absolute paths given from the boot loader should also work.

-----

Here's how it works on x86 in brief. The boot loader passes in a bunch of
information to the kernel about the console, boot device, loaded modules,
and so on. The module related stuff is here:

http://nxr.homeunix.org/source/xref/sys/arch/x86/include/bootinfo.h#167

module_init_md() digests this and registers the loaded modules with
module_prime(). From that point on MI code takes care of them.

The x86 memory layout and how it's set up by the boot loader / kernel is
ugly and boring so I won't go into much detail. In short the kernel starts
with the MMU disabled, and the kernel image is contiguous in PA space.
Loaded modules start on PAGE_SIZE boundaries immediatley after the kernel
image. When the kernel starts running, pmap/locore code sets up page tables
and the kernel pmap. Part of that is mapping kernel/module text, the symbol
table, and so on. The kernel is mapped to the correct address in virtual
space, again all contiguous, and then the MMU is switched on. So what we end
up with is a series of ELF objects on PAGE_SIZE boundaries. The first is
always the kernel, and the remainder are described by BTINFO_MODULELIST.

I'll note that there is a 'type' field in bi_modulelist_entry so that the
loader can later pull in anything we want: a kernel config file, or
firmware, or whatever. Right now only modules are loaded.

Cheers,
Andrew