[src/trunk]: src/sys/arch/arm/arm32 On Cortex, speculative loads can cache li...

[matt <matt%NetBSD.org@localhost>]

details:   https://anonhg.NetBSD.org/src/rev/bf8af49cbfcc
branches:  trunk
changeset: 784062:bf8af49cbfcc
user:      matt <matt%NetBSD.org@localhost>
date:      Wed Jan 16 22:32:45 2013 +0000

description:
On Cortex, speculative loads can cache lines to be populated after then they've
been invalidated for a DMA read.  So after the DMA read we have to reinvalidate
them again.  We have to both invalidates since the former prevents dirty lines
overwriting just DMAed data.

diffstat:

 sys/arch/arm/arm32/bus_dma.c |  31 ++++++++++++++++++++++++++-----
 1 files changed, 26 insertions(+), 5 deletions(-)

diffs (73 lines):

diff -r d705a55f2fbc -r bf8af49cbfcc sys/arch/arm/arm32/bus_dma.c
--- a/sys/arch/arm/arm32/bus_dma.c      Wed Jan 16 21:48:56 2013 +0000
+++ b/sys/arch/arm/arm32/bus_dma.c      Wed Jan 16 22:32:45 2013 +0000
@@ -1,4 +1,4 @@
-/*     $NetBSD: bus_dma.c,v 1.66 2012/10/23 12:23:20 skrll Exp $       */
+/*     $NetBSD: bus_dma.c,v 1.67 2013/01/16 22:32:45 matt Exp $        */
 
 /*-
  * Copyright (c) 1996, 1997, 1998 The NetBSD Foundation, Inc.
@@ -33,7 +33,7 @@
 #define _ARM32_BUS_DMA_PRIVATE
 
 #include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: bus_dma.c,v 1.66 2012/10/23 12:23:20 skrll Exp $");
+__KERNEL_RCSID(0, "$NetBSD: bus_dma.c,v 1.67 2013/01/16 22:32:45 matt Exp $");
 
 #include <sys/param.h>
 #include <sys/systm.h>
@@ -758,6 +758,20 @@
                cpu_dcache_wb_range(va, len);
                cpu_sdcache_wb_range(va, pa, len);
                break;
+
+#ifdef CPU_CORTEX
+       /*
+        * Cortex CPUs can do speculative loads so we need to clean the cache
+        * after a DMA read to deal with any speculatively loaded cache lines.
+        * Since these can't be dirty, we can just invalidate them and don't
+        * have to worry about having to write back their contents.
+        */
+       case BUS_DMASYNC_POSTREAD|BUS_DMASYNC_POSTWRITE:
+       case BUS_DMASYNC_POSTREAD:
+               cpu_dcache_inv_range(va, len);
+               cpu_sdcache_inv_range(va, pa, len);
+               break;
+#endif
        }
 }
 
@@ -786,7 +800,7 @@
 
                if ((ds->_ds_flags & _BUS_DMAMAP_COHERENT) == 0)
                        _bus_dmamap_sync_segment(va + offset, pa, seglen, ops,
-                            false);
+                           false);
 
                offset += seglen;
                len -= seglen;
@@ -935,7 +949,9 @@
         *      we are doing a PREREAD|PREWRITE, we can collapse
         *      the whole thing into a single Wb-Inv.
         *
-        *      POSTREAD -- Nothing.
+        *      POSTREAD -- Re-invalidate the D-cache in case speculative
+        *      memory accesses caused cachelines to become valid with now
+        *      invalid data.
         *
         *      POSTWRITE -- Nothing.
         */
@@ -946,7 +962,12 @@
 #endif
 
        const int pre_ops = ops & (BUS_DMASYNC_PREREAD|BUS_DMASYNC_PREWRITE);
-       if (!bouncing && pre_ops == 0) {
+#ifdef CPU_CORTEX
+       const int post_ops = ops & (BUS_DMASYNC_POSTREAD|BUS_DMASYNC_POSTWRITE);
+#else
+       const int post_ops = 0;
+#endif
+       if (!bouncing && pre_ops == 0 && post_ops == BUS_DMASYNC_POSTWRITE) {
                return;
        }