> > I haven't looked at NetBSD DMA code but one simple thing to
> > do is to use, for example, CACHE_FLUSH(addr,len) and
> > CACHE_INVALIDATE(addr,len) macros at appropriate points
> > around where DMA is set up or done.
> I believe that most (if not all) ix86 motherboards do not have a write-back
> second level cache, but a write-through one.  However, I also seem to have
> trouble with the external cache getting corrupted.  I have to disable it
> from the BIOS menu for NetBSD to work.  Does anyone have code to flush and
> invalidate lines in the external cache?  If so, could you send me a copy.

Since the problem seems to strike PC clones only, it's probably time to
direct this to port-i386 instead of current-users (unless someone with a
non-intel system with a misdesigned cache wants to speak up :-).

Most x86 motherboards have write-through caches, but many have write-back,
and according to the manual that came with mine, it has write-back.  The
i486 internal cache is write-through[1].

The Intel ISA doesn't define any way to invalidate particular lines in the
cache, certainly not the external cache (I sort of thought the 680x0 family
did, but the '20 manual I have makes no mention of such a thing).  What is
provided is a pair of instructions, INVD and WBINVD, which invalidate the
internal cache, and which provide external bus cycles to invalidate the
external cache.  WBINVD also provides an external bus cycle telling an
external write-back cache to flush its data out to memory.  It's possible
that there's a standard BIOS mechanism to arrange for individual cache lines
to be flushed, but (1) I take it using BIOS traps from NetBSD isn't easy, and
(2) "You said ``standard BIOS'', Butthead."  "Heh-heheh heh heh."

There isn't a central funnel through which all DMA requests go, hence no
obvious place to insert standardized cache-massasing code (which I guess is
part of why bounce-buffers haven't been made universal[2]).  As an experiment,
I added a wbinvd() function to locore.s, and arranged to call it from isa/dma.c
(where the DMA chip gets programmed for DMA-less devices) and from aha1542.c,
just before the scatter-gather list is prepared.  As a very gentle test, I
then re-enabled my cache, booted the resulting OS, and did a make in src/gnu.
I *think* that has generally been sufficient to cause disk curdling (I suppose
I ought to dump that partition to tape and try doing that without WBINVD), but
no curdling was observed when it was finished.

I'm not sure what to do with this (assuming it really works :-).  I doubt the
core team will enjoy decorating every DMA driver with

	#ifdef GODDAMNED_CUTRATE_CACHE_IMPLEMENTATIONS
			wbinvd();
	#endif

(and I *think*, but can't say for sure, that the #ifdef is needed; the i486
manual says that the special bus cycles generated for write-back and flush
have to be acknowledged by external hardware, and I don't think I trust PC
designers to acknowledge the write-back cycle if they don't anticipate seeing
it -- the question then is, does not acknowledging it save a gate? :-).
Maybe it should be a footnote to the INSTALL note about disabling the caches
(something along the lines of "If you feel really adventurous, study the
sources until you know where and why to add the following code...").

Who was it who was taking reports of broken cache implementations?


[1] And unlike Motorola, Intel is probably sensitive enough to the needs of
stunningly incompetant programmers not to change the cache from write-through
to write-back when advancing their product line (like the 68030->68040 change
that seemed to primarily affect one very large software firm in Redmond, WA).

[2] And the Adaptec driver makes that problem even more irritating by being
willing to do scatter-gather, hence potentially requiring several simultaneous
bounce-buffers.