Re: kern/60144: virtio(4) cache coherence issue

[Jason Thorpe <thorpej%me.com@localhost>]

> On Apr 11, 2026, at 3:35 AM, Taylor R Campbell <riastradh%NetBSD.org@localhost> wrote:
> 
>> Date: Fri, 10 Apr 2026 04:48:32 -0700
>> From: Jason Thorpe <thorpej%me.com@localhost>
>> 
>> Taylor, what's your take on the situation?
> 
> I don't know any details about m68k caches (let alone 68030 vs 68040).

Let me give a quick primer on them just so we’re all on the same page regarding now they work.

The 68010 does not have any caches.

The 68020 has a small instruction cache, and no data cache.  The instruction cache is a 256 byte direct-mapped cache arranged as 64 longword entries.  The cache is VIVT by virtue of the MMU being external to the CPU.  Some 68020 systems have external combined caches, which may be VIVT or PIPT, depending on how the system board is wired up.  If an external cache exists, it is always write-through.

The 68030, being an incremental update to the 68020, adds a small on-chip data cache.  That said, the 68030’s instruction cache is not the same as the 68020’s.  Rather both the instruction and data cache of the 68030 are both arranged as 16 lines of 16 bytes each (256 bytes for each cache).  Each longword in a cache line has a VALID bit.  Cache lines may be filled one longword at a time, or a burst-mode fill operation may be used to fill an entire cache line (one-at-a-time mode always works, and burst mode requires the machine’s memory subsystem to support the burst-mode bus protocol).  Despite the fact that the 68030 has a built-in MMU, the 68030 cache is VIVT.  This arrangement allows for some internal optimizations.  The ATC (address translation cache) on the 68030 is fairly small, so it is advantageous to perform cache lookups using virtual addresses; a cache hit isn’t very meaningful if the MMU has to do a table-walk in order to determine if you got a hit or not.  Despite the 68030 data cache being VIVT, if the Write-Allocate mode is used, it does not suffer from cache alias problems due  to the small size and internal cache arrangement.  The 68030 data cache is always write-though.  There are external caches on some 68030 systems, as well.  These are also always write-though, and due to the MMU being integrated into the CPU, these external caches are always PIPT.

The 68040 is a complete departure from the 68030’s MMU and cache implementation.  The 68040 has dual MMUs (one for instruction, one for data, although they share a single root table pointer).  There is also a 4KB instruction cache and a 4KB data cache on-chip.  Both caches are 4-way set-associative, arranged as 64 sets of 4 16-byte lines.  The major differences that matter to software: the caches are PIPT, and the MMU controls on a page-by-page basis: write-through or write-back (in addition to the non-cacheable, which is also something you can do with the other CPUs; I will skip the details of how that works with an external MMU).  I do not know of any 68040 systems that have external caches.  The 68040 has a snooping protocol that an external cache would need to participate in.

The 68060 is largely the same as the 68040, except the caches are larger (8K + 8K), and there is an addition of a Branch Target Cache for the branch predictor.

Here endeth the lesson.

-- thorpej