Matthias Drochner <drochner@zel459.zel.kfa-juelich.de> writes:
> cgd@netbsd.org said:
> > Linear access should only be allowed if the region behaves like memory
> > from the kernel's point of view. 
> 
> Well - it _is_ memory in the particular case I had
> to deal with (the if_ti driver).
> It is shared memory between the adapter's CPU and PCI.
> 
> So far it can be write-combined and prefetched as long
> as memory barriers are used where needed to get coherence.

If that's true, it should have been marked prefetchable in the BAR.


> > and devmem is prefetchable (what the code means by cacheable there,
> 
> There is the problem: We can't express these "in-betweens".

Right, but really what's meant here _is_ prefetchable.  8-)


> The other way would be to map the range as "cacheable" from
> the device driver (even if this bit is not set in the
> PCI config register (*)).
> But that's not right either - the mapping needs memory barriers
> just like other device memory, not more. "Cacheable" would
> mean for me that the md code is free to put it behind
> a real cache, so cache flushes would be needed for coherence.

Right.  You ~never want this for PCI (or other) I/O cards.

ISTR that the bit's named "cacheable" because caching is actually an
option on the PC (and on device memory for (some, at least) TC
alphas).


> (*) Just had a look at the PCI specs - this bit is called
> "prefetchable", and it is explicitely stated that this
> memory is _not_ to be cached by the host CPU. So I guess
> this bit should have been set in the BAR.
> 
> So it just becoming clear that "prefetchable" and "cacheable"
> are different things and should get separate flag bits...

"eh."  I don't think i care that much.


It's worth noting that the lack of the prefetchable bit in the BAR for
non-BWX machines that some alpha systems' consoles will map the card
in sparse space, and that mapping _may not be accessible_ from dense
space...

Really, as anything other than a short-term hack, that driver should
be using the bus_space macros/fns and not linear mapping.  As a
short-term hack, well, hack your local kernel to allow
non-prefetchable ("non-cacheable") mappings to use linear mapping for
BWX.  I don't think it's correct that it become the default, unless
the conditions mentioned previously are true (the kernel is compiled
to use the BWX instructions, and the compiler actually generates them
in all of the places that it should, and doesn't "optimize" them
away).



cgd
-- 
Chris Demetriou - cgd@netbsd.org - http://www.netbsd.org/People/Pages/cgd.html
Disclaimer: Not speaking for NetBSD, just expressing my own opinion.