Subject: Re: DVMA on VME under NetBSD-current
To: None <pk@cs.few.eur.nl>
From: Rob Ginn <rob@olg.com>
List: port-sparc
Date: 10/04/1998 21:00:19
Paul,

>> I'm unclear as to how the VME code should determine that the card is
>> A16,A24, or A32 as the VMEMOD_A* parameters are not passed into
>> the bus_dmamap_create() function.
>
> This is the reason why that code is still "to do".
>
>> Perhaps a set of flags on the configuration
>> line which specify special capabilities/limitations of the specific
>> VME card could be used? 
>
> A couple of changes and enhancements to the VME bus implementation
> are being planned. These will also address the issue of specifying
> VME modifiers for drivers that need to know.
>
> In this case though, the VME modifier must be passed on through the
> bus_dmamap_*() functions, and it's not yet clear to me how that is
> best done.

Actually, the point I was trying to make is that if the driver writer
needs to specify this stuff inside the driver code, the driver is
inherently non-portable.  Whenever the card jumpers are changed, the
conf line must also be changed.  However, under the current scheme, if
the jumpers that were changed include A32/A24/A16 features, the driver
source must then also be changed -- this is not good.

Given that we want the driver source to remain unchanged when the
card jumpers are changed, there are then two options:  either let
the config line data be handled by the vme framework, or let the
individual drivers obtain the information through the vme framework
and then pass part of the information back to the vme framework.

It doesn't make sense to me for the individual drivers get info that
they only use to pass back to the vme framework -- thus it seems that
the other option (namely put the data on the config line and have the
framework handle it).  Note that if this option is taken, there is no
need to pass "mod" info to the bus_dmamap_*() functions or bus_space_*()
since that info could be added to the vma_dmatag and vma_bustag values.
The "mod" paramter could remain, as an optional override if non-zero.
Note that this would avoid any changes to existing vme drivers.

Presumably there is some objection to specifying something like "a24d32"
instead of "addr" in config, but I'm not sure what it is.

> You need to allocate in the top 8MB of DVMA space of which A24 devices
> take the first 1MB. In other words, the DVMA address range ff800000-
> ff900000 maps to VME/A24 space 0-00100000. I suppose A16 devices map
> into the first 64KB of this MB.

You've lost me here.

According to the SunOS manual I have, both the A32 and A24 address spaces
remap the low MB to the high one.  A16 is not mentioned.  I don't
know if this is programmable of if it is hard-wired.  What I get out
of the manual is that A24 (and A32) addresses of 00000000 - 000fffff
map to a virtual address (input to the MMU) of fff00000 - ffffffff.
Presumably there must be some similar support for A16 devices or you
couldn't do DMA with them, but I was unable to locate a reference.  The
reason given for this capability to be present in A32 devices is that
not all A32 cards can generate 32 bit DMA addresses, some are limited
to 20 bit (multibus device in an adapter) and 24 bit (some VME devices).

Note that the manual also says that the A16 space sits at the top
of the A24 space and that the A24 space sits at the top of the A32
space. 

The upshot is that I think I need DVMA addresses in the range of 
fff00000 - ffffffff (which would then be programmed as 0 - 0fffff).
The A24 card can only address 00000000 - 00ffffff, and fff00000 -
ffffffff via translation through the DVMA "slave decoder").  Unless
someone is programming the slave decoder to map the low megabyte to
some place other than the high megabyte.  If this is the case, please
tell me.

This interpretation seems to agree with the DVMA addresses given in
vaddrs.h in the include directory.  This file states that DVMA4M_BASE is
located at fc000000 and goes up to ffffffff.  Presumably A24 devices
would access the top MB out of this space and A32 could use the rest.
This assumes we go from DVMA4M_BASE - DVMA4M_TOP and not from
DVMA4M_START to DVMA4M_END.  Since the system is giving me fc000000
as a DVMA address, that's probably right.  I am not really clear
on the reason for the START/END constants, especially given the
names of the non-4M versions (BASE/END).  BASE/END and BASE4M/END4M
is used in machdep.c, the #defs START/TOP are not used at all.  Note
that this might be bad as it seems to imply that I can't get that top
MB that I need for DMA.  Also, it's unclear why I would ever want more
than 1 MB of DVMA space -- perhaps for A32 devices?

On the other hand, I may not thoroughly understand this.  As a
test, I tried bus_dmamem_alloc() -- this presumably allocates a 
page of memory and ties it to a virtual address in the DVMA range
(I got fc000000).  Then I called bus_dmamem_map() to map it into
kernal space -- this presumably redundently mapped the page and
locked it into the MMU page table.  I expected the second call to 
use the fc000000 value to get the physical address and make a 
redundant mapping.  Since the diagram in the Sun
manual shows the CPU going directly to the MMU, I expected that
both the kernel mapping and the original fc000000 would point to the
same page from the kernel - they don't, the fc000000 causes a fault.
Maybe bus_dmamem_alloc() doesn't lock the page table entry in the MMU.
Still, using the current scheme, I can access the VME cards
fine -- I just can't get the VME cards to access system memory.

At any rate, my quick and dirty workaround was going to be to just
raise DVMA4M_BASE from fc000000 to fff00000.  However, the comments
in vaddrs.h state that I must leave multiples of 16MB, so I'll
probably have to muck around with the dvma allocation code instead.
As mentioned above, I may need to change DVMA4M_END to ffffffff as
well.

- Rob

- Rob
rob@olg.com