physical address space management

[David Young <dyoung%pobox.com@localhost>]

I have in mind an MI API for managing physical address space in NetBSD.
I invite your feedback.

Here are some uses that I envision for a physaddr space API:

        * abstract cache-control, execute- and write-protection features
          of x86 MTRR, AMD Elan SC520, et cetera

        * provide for dynamic allocation of bus space for ExpressCard,
          CardBus, et cetera.

        * express memory topology---e.g., NUMA, cluster

        * bad RAM management

        * replace multiple ad hoc, MD, broken and/or undocumented
          mechanisms for managing physical address space, including rbus.

The major entities in the API are an address space *arena*,
*interconnects* and *mappings* between arenas, address space *regions*
within each arena, and address region *type*, *use*, *protection*,
and *properties*.

The API lets us carve up address space into "arenas."  An arena is a
set of consecutive physical addresses with identical type and access
characteristics.  For example, arenas on a typical uniprocessor system may
be the system RAM, the system ROM, and a PCI bus.  In a NUMA system, there
may be more than one RAM arena.  Arenas are unified by "interconnects,"
objects that model the bus bridges, IOMMUs, HyperTransport links, et
cetera.  Each interconnect connects precisely two arenas.  Properties of
an interconnect include a mapping from the first arena to the second
(expresses IOMMU configuration), and a metric (expresses cost/delay/speed
of traversing the interconnect).

Within an arena are regions.  A region is a span of consecutive physical
addresses with identical ownership and access permissions, use, and
operational characteristics.  For example, a RAM arena may contain a
text region, a read-only data region, and a read/write data region.
A PCI bus arena may have both a prefetchable, write-combining region,
and several non-prefetchable regions.

Methods on arenas let the kernel load an arena with regions during
bootstrap, reserve regions, and map regions from one arena into
another.  Methods on regions let the kernel release a region, get/set
properties such as use (text, data, DMA buffer), access permissions
(read/write/execute), and operational characteristics (uncached,
prefetchable, write-back).

When device drivers for bus bridges, IOMMUs, CPUs, x86 MTRR, et cetera
attach, they register with the physaddr space manager.  When a region's
properties or mappings change, the manager will notify registered drivers
so that they can re-program MTRRs or IOMMUs, or adjust address windows
on bus bridges.

Here is the start of an API:

enum pmem_props {                       /* hardware implementation */
          PMEM_P_WTHRU          = 0x01  /* MTRR */
        , PMEM_P_WBACK          = 0x02  /* MTRR */
        , PMEM_P_WCOMB          = 0x04  /* MTRR */
        , PMEM_P_UNCACHED       = 0x08  /* MTRR, AMD Elan SC520 PAR */
        , PMEM_P_PREFETCH       = 0x10  /* PCI bus bridge */
};

enum pmem_prot {                        /* hardware implementation */
          PMEM_PROT_READ        = 0x01  /* PCI bus bridge, IOMMU */
        , PMEM_PROT_WRITE       = 0x02  /* PCI bus bridge, IOMMU, MTRR,
                                         * AMD Elan SC520 PAR
                                         */
        , PMEM_PROT_EXEC        = 0x04  /* AMD Elan SC520 PAR */
};

enum pmem_type {
        , PMEM_T_RAM            = 0x01
        , PMEM_T_ROM            = 0x02
        , PMEM_T_PCI            = 0x04
};

enum pmem_use {
          PMEM_T_TEXT           = 0x01
        , PMEM_T_DMABUF         = 0x02
        , PMEM_T_DATA           = 0x04
        , PMEM_T_DEVREGS        = 0x08
        , PMEM_T_FRAMEBUF       = 0x10
        , PMEM_T_BROKEN         = 0x20  /* bad RAM */
}

typedef enum pmem_props pmem_props_t;
typedef enum pmem_prot pmem_prot_t;
typedef enum pmem_type pmem_type_t;
typedef enum pmem_use pmem_use_t;

typedef struct pmem_arena *pmem_arena_t;

pmem_arena_t
pmem_arena_create(pmem_type_t);

static const pmem_mapping_t pmem_mapping_identity = NULL;

/* Connect two arenas. */
int
pmem_arena_connect(pmem_arena_t left, pmem_arena_t right,
    pmem_mapping_t m, pmem_metric_t metric);

/* Load arena `a' with physical addresses [start, end) having the given
 * default properties.
 */
int
pmem_arena_prime(pmem_arena_t a, paddr_t start, paddr_t end,
    pmem_use_t use, pmem_prot_t prot, pmem_props_t props);

/* Reserve a region in arena `a' that meets the given criteria.
 * The region is returned with a reference count of at least 1.
 */
pmem_region_t
pmem_alloc(pmem_arena_t a, paddr_t start, paddr_t end,
    pmem_prot_t prot, pmem_props_t props, pmem_use_t use,
    size_t align, size_t len, pmem_metric_t maxmetric);

/* Get/set properties on the region `r'. */
int
pmem_get(pmem_region_t r, pmem_prot_t *prot, pmem_props_t *props,
    pmem_use_t *use);

int
pmem_set(pmem_region_t r, pmem_prot_t prot, pmem_props_t props,
    pmem_use_t use);

/* Count another reference to region `r'. */
void
pmem_incref(pmem_region_t r);

/* Reduce the reference count on `r' by one.  pmem_decref may reclaim the
 * resources held by `r'.
 */
void
pmem_decref(pmem_region_t r);

/* Map region `r' into arena `a'.
 *
 * Returns NULL on failure.  `paddr' is undefined on failure.
 *
 * On success, return `r' if region `r' belongs to arena `a', or else
 * return an alias for region `r' in `a'.  The returned region's reference
 * count is increased by one.  Set `paddr' to the physical address of
 * the start of the region `r' in arena `a'.
 */
pmem_region_t
pmem_map(pmap_arena_t a, pmem_region_t r, paddr_t *paddr);

/* Remove a mapping of `r' from its arena.  Decrease the reference count
 * by one.
 */
void
pmem_unmap(pmem_region_t r);

Dave

-- 
David Young             OJC Technologies
dyoung%ojctech.com@localhost      Urbana, IL * (217) 278-3933 ext 24

Dave

-- 
David Young             OJC Technologies
dyoung%ojctech.com@localhost      Urbana, IL * (217) 278-3933 ext 24