Subject: Re: user mapping pci memory
To: Eric Hoffman <hoffman@caida.org>
From: Chris G. Demetriou <cgd@cs.cmu.edu>
List: port-alpha
Date: 03/11/1997 00:14:44
> I have a memory-mapped pci device that seems to work fine from kernel
> space via bus_mem_[read|write]_4 after use of bus_mem_map.
> 
> however, 32-bit aligned 32-bit writes from process space into the
> same region invariably panic the machine with a verbose
> "panic: machine check: vec 0x660, pc = 0x120000804, ra = 0x1200007d4"
> 
> the pcivga driver has the following code in mmap commented out:
> 
>    rv = alpha_btop(k0segtophys(sc->sc_dc->dc_crtat) + offset);
> 
> which shouldn't be necessary in my case given that I already have a
> physical address. looking at this memory region this address is in
> (7ffffe011000000) and the implementation of bus_mem_write_4 it would
> seem that this access should work. the only disturbing thing that I've
> found in poking around is that assumption that the physical page
> address as returned by the mmap call will fit in an int.
> 
> the machine in an apecs pci running 1.2. any hints would be greatly
> appreciated.

Whoa, where to begin.  8-)


(1) bus space handles _are not_ physical or virtual addresses, and
cannot be treated as such.  If you want to treat them as such, (a)
you're cheating, and (b) cheating is allowed, but you have to be very
careful and understand what they _actually_ are.

(2) the handle that you have 0x7ffffe011000000 is a sparse memory
handle, which is describing a mapping of the PCI bus physical address
0x20000000.

(3) The fact that your handle is a sparse memory handle indicates that
your device doesn't want its memory to be considered cacheable.
(Therefore it's mapped in sparse space.)  That means that (on current
Alphas) you can't just access it linearly.  You can map it, but trying
to access it via a structure, or in any way that doesn't take sparse
space address encoding into account, is going to lose very, very
badly (as you've discovered 8-).


Thoughts:

You can find out what sparse memory space handles 'really' are for
that particular bus by looking (carefully) at pcs_bus_mem_common.c.

You can find out more information on sparse memory space encoding by
looking at pcs_bus_mem_common.c, or by looking in the APECS docs
available from ftp://ftp.netbsd.org/pub/NetBSD/misc/dec-docs/index.html

You need to be using proper sparse space encoding to access the data,
or you will continue to get machine checks until the cows come home.
32-bit-aligned 32-bit accesses just won't cut it.

Your note about the potential inadequacy of using 'int's to describe
page frame numbers is a good observation, and one that may cause
problems in the future.  However, using ints gives 32 + (number of
bits of page offset; 13 on current alphas) bits of addressable space.
45 bits of physical address space is currently enough.  (However, the
time when it's _not_ enough is rapidly approaching.)

There's a reason that the ability to mmap the VGA frame buffer's
address space is disabled, namely, as you've discovered, there's no
good, sane interface by which to manipulate device memory or I/O space
data from user-land.  I intend to extend the current bus.h 'space'
interface functionality to work in user-land, but haven't actually
gotten around to doing so yet.


If you'd like to help out with the last item (extending bus.h's
'space' interface functionality to work in user-land), by all means
get in touch with me, because i'd love for somebody else to help out.
8-)


cgd