Re: replacing extent(9) with vmem(9): lessons learned

[Christoph Egger <Christoph_Egger%gmx.de@localhost>]

On Thursday 03 December 2009 13:17:49 David Young wrote:
> I've tried to replace the management of i386 bus space by extent(9) with
> management by vmem(9).  I've found some API/implementation problems in
> vmem(9) that make vmem(9) a dissatisfying replacement for extent(9).
>
> Problem 1: vmem(9) cannot manage a region that begins at 0.
>
>     vmem(9) reserves the address 0 for indicating errors.  For
>     example, vmem_xalloc(9) returns VMEM_ADDR_NULL (0) if it cannot
>     find a suitable region.
>
>     extent(9) indicates errors "out of band" instead of reserving
>     an address for error indications, so it can manage a region
>     starting at 0.
>
>     No workaround that I have found is very satisfying.
>
> Problem 2: vmem(9) cannot manage a region that includes the maximum
>     address, ~(vmem_addr_t)0.
>
>     This is an implementation problem.  Sometimes vmem(9) will add
>     the start of a region to the size of a region without checking
>     for overflow.  For example, 0x1 (start) + 0xffffffff (size) ==
>     0.  Every use of BT_END() needs some TLC.
>
> Problem 3: vmem(9) uses malloc(9)/pool(9)/RUN_ONCE(9), but
>     the bus space regions are initialized and used before
>     malloc(9)/pool(9)/RUN_ONCE(9) are available.
>
>     I have a rudimentary fix in my tree.  I added a new API routine,
>     vmem_create_storage(..., void *storage, size_t storage_size)
>     for creating a vmem_t with static storage.  I made the
>     implementation use the static storage if it was available.

Problem 4: vmem(9) does not allocate page aligned memory through kmem(9).
      This is a problem e.g. for Xen where page-aligned memory is required for
      the ring buffers. A workaround is to use uvm_km_alloc().

> By making several compromises, I did get NetBSD/i386 to boot to
> multiuser after an extent->vmem switch.

Great but I better don't ask how many hacks you had to put in. :)

Christoph