Subject: Re: port-mips/31915: provide centralized wired_map logic
To: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
From: Garrett D'Amore <garrett_damore@tadpole.com>
List: port-mips
Date: 11/02/2005 08:55:58
This is a multi-part message in MIME format.
--------------090809060402080100090801
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Izumi Tsutsui wrote:

>In article <4368110F.5020905@tadpole.com>
>garrett_damore@tadpole.com wrote:
>
>  
>
>>To set up a full TLB the MD code will have to call this routine once 
>>each for PA0 and PA1.
>>    
>>
>
>Ok, it seems fine.
>
>  
>
>>+++ sys/arch/mips/mips/mips3_wired_map.c	2 Nov 2005 01:06:10 -0000
>>    
>>
>
>As I wrote before, this can be arch/mips/mips/wired_map.c.
>I'll rename arc/wired_map.c to wired_map_machdep.c or something.
>  
>
OK.  (Aside: what if someone ever wants to do a MIPS-I wired map?  I 
guess they have to #ifdef around it.  We can cross that bridge if we 
ever get to it, I suppose.)

>  
>
>>+#include <mips/wired_map.h>
>>    
>>
>
>Maybe we should have <machine/wired_map.h> for MD definitions,
>which includes <mips/wired_map.h> like <machine/vmparam.h>.
>  
>
That sounds like a reasonable idea.

>  
>
>>+static struct wired_map_entry {
>>+	paddr_t	pa0;
>>+	paddr_t	pa1;
>>+	vaddr_t	va;
>>+	vsize_t	pgmask;
>>+} wired_map[MIPS3_WIRED_ENTRIES];
>>    
>>
>
>This struct wired_map_entry and wired_map[] variable should be
>exported for MD functions.
>  
>
OK. 

>>+static boolean_t mips3_wire_down_page(vaddr_t va, paddr_t pa);
>>    
>>
>
>This should be removed (exported).
>  
>
Oops.  I meant to do that!

>  
>
>>+static int	nwired;
>>    
>>
>
>nwired also should be exported, though maybe it's better
>to rename it something like mips3_nwired_page.
>  
>
OK.

>>+boolean_t
>>+mips3_wire_down_page(vaddr_t va, paddr_t pa, vsize_t pgsz)
>>    
>>
>
>IMHO, "wire_down" seems a bit strange for this operation.
>It reminds me "pin down" memory (RAM) to avoid pageout.
>How about "mips3_wired_enter_page()" like pmap_enter(9)?
>  
>
OK, its just naming.  I'm not picky.  But if we do that, we should 
probably also rename mips3_wire_down to mips3_wired_enter_region.

>  
>
>>+	if ((va & (pgsz - 1)) || (pa & (pgsz - 1)))
>>+		panic("mips3_wire_down_page: not aligned");
>>    
>>
>
>This check could be wrapped by #ifdef DIAGNOSTIC.
>  
>
Yes, I suppose it could.  I think I'll just use KASSERT instead, though.

>  
>
>>+			return 0;
>>    
>>
> :
>  
>
>>+	return 1;
>>    
>>
>
>It's better to use TRUE and FALSE rather than 1 and 0?
>  
>
OK.

>  
>
>>+boolean_t
>>+mips3_wire_down(vaddr_t va, paddr_t pa, vsize_t size)
>>    
>>
>
>How about mips3_wired_enter() for this?
>  
>
How about mips3_wired_enter_region()?

>  
>
>>Index: sys/arch/mips/include/wired_map.h
>>    
>>
> :
>  
>
>>+/*
>>+ * Certain machines have peripheral busses which are only accessible
>>+ * using the TLB.
>>    
>>
> :
>  
>
>>+ * Note also that at the moment this is not supported on the MIPS-I
>>+ * ISA (but it shouldn't need it anyway.)
>>+ */
>>    
>>
>
>Isn't it better to move these comments to (mips3_)wired_map.c?
>  
>
Not sure about that.  I like having good comments in the header, because 
that is what is exposed to callers.

However, I'll move the meaty parts to the .c file, and leave a one-liner 
description in the header.

>  
>
>>+#define	MIPS3_SIZE_TO_PG_MASK(x)	(((x * 2) - 1) & ~0x1fff)
>>    
>>
>
>This is also defined in <mips/mips3_pte.h> later?
>  
>
Hmmm... I guess I left that in by mistake.  I'll remove it.

>  
>
>>+#ifndef	MIPS3_WIRED_ENTRIES
>>+#define MIPS3_WIRED_ENTRIES	8	/* upper limit */
>>+#endif	/* MIPS3_WIRED_ENTRIES */
>>    
>>
>
>How about "MIPS3_NWIRED_ENTRY" like VM_NFREELIST in vmparam.h?
>  
>
OK.

>  
>
>>+/*
>>+ * Wire down a mapping from a virtual to physical address.  The size
>>+ * of the region must be a multiple of MIPS3_WIRED_ENTRY_SIZE, with
>>+ * matching alignment.
>>+ */
>>    
>>
>
>  
>
>>+/*
>>+ * Lower layer API, to supply an explicit page size.  It only wires a
>>+ * single page at a time.
>>+ */
>>    
>>
>
>These function descriptions should be in (mips3_)wired_map.c.
>  
>
I'll make that change.

>
>Anyway, I'll try to adapt arc port to your patch shortly. Thanks!
>---
>Izumi Tsutsui
>  
>
See my attached diffs.


--------------090809060402080100090801
Content-Type: text/plain;
 name="wired-diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="wired-diff"

Index: mips/include/wired_map.h
===================================================================
RCS file: mips/include/wired_map.h
diff -N mips/include/wired_map.h
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ mips/include/wired_map.h	2 Nov 2005 16:52:31 -0000	1.2
@@ -0,0 +1,99 @@
+/*	$NetBSD: wired_map.h,v 1.2 2005/01/22 07:35:33 tsutsui Exp $	*/
+
+/*-
+ * Copyright (c) 2005 Tadpole Computer Inc.
+ * All rights reserved.
+ *
+ * Written by Garrett D'Amore for Tadpole Computer Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. The name of Tadpole Computer Inc. may not be used to endorse
+ *    or promote products derived from this software without specific
+ *    prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY TADPOLE COMPUTER INC. ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL TADPOLE COMPUTER INC.
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef	_MIPS_WIRED_MAP_H
+#define	_MIPS_WIRED_MAP_H
+
+/*
+ * Certain machines have peripheral busses which are only accessible
+ * using the TLB.
+ *
+ * For example, certain Alchemy parts place PCI and PCMCIA busses at
+ * physical address spaces which are beyond the normal 32-bit range.
+ * In order to access these spaces TLB entries mapping 36-bit physical
+ * addresses to 32-bit logical addresses must be used.
+ *
+ * Note that all wired mappings are must be 32 MB aligned.  This is
+ * because we use 32 MB mappings in the TLB.  Changing this might get
+ * us more effficent use of the address space, but it would greatly
+ * complicate the code, and would also probably consume additional TLB
+ * entries.
+ *
+ * Note that within a single 32 MB region, you can have multiple
+ * decoders, but they must decode uniquely within the same 32MB of
+ * physical address space.
+ *
+ * BEWARE: The start of KSEG2 (0xC0000000) is used by the NetBSD kernel
+ * for context switching and is associated with wired entry 0.  So you
+ * cannot use that, as I discovered the hard way.
+ *
+ * Note also that at the moment this is not supported on the MIPS-I
+ * ISA (but it shouldn't need it anyway.)
+ */
+
+#ifndef MIPS3_WIRED_SIZE
+#define	MIPS3_WIRED_SIZE	MIPS3_PG_SIZE_MASK_TO_SIZE(MIPS_PG_SIZE_16M)
+#endif
+
+/*
+ * This defines the maximum number of wired TLB entries that the wired
+ * map will be allowed to consume.  It can (and probably will!)
+ * consume fewer, but it will not consume more.  Note that NetBSD also
+ * uses one wired entry for context switching (see TLB_WIRED_UPAGES),
+ * and that is not included in this number.
+ */
+#ifndef	MIPS3_NWIRED_ENTRY
+#define MIPS3_NWIRED_ENTRY	8	/* upper limit */
+#endif	/* MIPS3_NWIRED_ENTRY */
+
+struct wired_map_entry {
+	paddr_t	pa0;
+	paddr_t	pa1;
+	vaddr_t	va;
+	vsize_t	pgmask;
+} wired_map[MIPS3_NWIRED_ENTRY];
+
+extern struct wired_map_entry	mips3_wired_map[];
+extern int			mips3_nwired_page;
+
+/*
+ * Wire down a region of the specified size.
+ */
+boolean_t mips3_wired_enter_region(vaddr_t, paddr_t, vsize_t);
+
+/*
+ * Wire down a single page using specified page size.
+ */
+boolean_t mips3_wired_enter_page(vaddr_t, paddr_t, vsize_t);
+
+#endif	/* _MIPS_WIRED_MAP_H */
Index: mips/include/mips3_pte.h
===================================================================
RCS file: /net/projects/meteor/cvs/netbsd/src/sys/arch/mips/include/mips3_pte.h,v
retrieving revision 1.1.1.2
retrieving revision 1.2
diff -u -r1.1.1.2 -r1.2
--- mips/include/mips3_pte.h	17 Oct 2005 16:48:25 -0000	1.1.1.2
+++ mips/include/mips3_pte.h	2 Nov 2005 16:52:31 -0000	1.2
@@ -181,6 +181,9 @@
 #define	MIPS3_PG_SIZE_MASK_TO_SIZE(pg_mask)	\
     ((((pg_mask) | 0x00001fff) + 1) / 2)
 
+#define	MIPS3_PG_SIZE_TO_MASK(pg_size)		\
+    ((((pg_size) * 2) - 1) & ~0x00001fff)
+
 /* NEC Vr41xx uses different pagemask values. */
 #define	MIPS4100_PG_SIZE_1K	0x00000000
 #define	MIPS4100_PG_SIZE_4K	0x00001800
@@ -190,3 +193,7 @@
 
 #define	MIPS4100_PG_SIZE_MASK_TO_SIZE(pg_mask)	\
     ((((pg_mask) | 0x000007ff) + 1) / 2)
+
+#define	MIPS4100_PG_SIZE_TO_MASK(pg_size)		\
+    ((((pg_size) * 2) - 1) & ~0x000007ff)
+
Index: mips/mips/wired_map.c
===================================================================
RCS file: mips/mips/wired_map.c
diff -N mips/mips/wired_map.c
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ mips/mips/wired_map.c	2 Nov 2005 16:52:31 -0000	1.3
@@ -0,0 +1,207 @@
+/*	$NetBSD: wired_map.c,v 1.8 2005/01/22 07:35:33 tsutsui Exp $	*/
+
+/*-
+ * Copyright (c) 2005 Tadpole Computer Inc.
+ * All rights reserved.
+ *
+ * Written by Garrett D'Amore for Tadpole Computer Inc.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. The name of Tadpole Computer Inc. may not be used to endorse
+ *    or promote products derived from this software without specific
+ *    prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY TADPOLE COMPUTER INC. ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+ * TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+ * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL TADPOLE COMPUTER INC.
+ * BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ */ 
+/*
+ * Copyright (C) 2000 Shuichiro URATA.  All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. The name of the author may not be used to endorse or promote products
+ *    derived from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
+ * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
+ * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
+ * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
+ * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
+ * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+/*
+ * This code is derived from similiar code in the ARC port of NetBSD, but
+ * it now bears little resemblence to it owing to quite different needs
+ * from the mapping logic.
+ */
+
+#include <sys/cdefs.h>
+__KERNEL_RCSID(0, "$NetBSD$");
+
+#include <sys/param.h>
+#include <sys/systm.h>
+#include <uvm/uvm_extern.h>
+#include <machine/cpu.h>
+#include <machine/locore.h>
+#include <machine/pte.h>
+#include <machine/vmparam.h>
+#include <machine/wired_map.h>
+
+struct wired_map_entry {
+	paddr_t	pa0;
+	paddr_t	pa1;
+	vaddr_t	va;
+	vsize_t	pgmask;
+} mips3_wired_map[MIPS3_NWIRED_ENTRY];
+
+int	mips3_nwired_page;
+
+/*
+ * Lower layer API, to supply an explicit page size.  It only wires a
+ * single page at a time.
+ */
+boolean_t
+mips3_wired_enter_page(vaddr_t va, paddr_t pa, vsize_t pgsz)
+{
+	struct tlb	tlb;
+	int		index;
+	int		found = 0;
+	vaddr_t		va0;
+
+	/* make sure entries are aligned */
+	KASSERT((va & (pgsize - 1)) == 0);
+	KASSERT((pa & (pgsize - 1)) == 0);
+
+	/* TLB entries come in pairs: this is the first address of the pair */
+	va0 = va & ~pgsz;
+
+	for (index = 0; index < mips3_nwired_page; index++) {
+		if (mips3_wired_map[index].va == va0) {
+			if ((va & pgsz) == 0) {
+				/* EntryLo0 */
+				mips3_wired_map[index].pa0 = pa;
+				found++;
+				break;
+			} else {
+				/* EntryLo1 */
+				mips3_wired_map[index].pa1 = pa;
+				found++;
+				break;
+			}
+		}
+	}
+
+	if (found == 0) {
+		/* we have to allocate a new wired entry */
+		if (mips3_nwired_page >= MIPS3_NWIRED_ENTRY) {
+#ifdef DIAGNOSTIC
+			printf("mips3_wired_map: entries exhausted\n");
+#endif
+			return FALSE;
+		}
+
+		index = mips3_nwired_page;
+		mips3_nwired_page++;
+		if (va == va0) {
+			/* EntryLo0 */
+			mips3_wired_map[index].pa0 = pa;
+			mips3_wired_map[index].pa1 = 0;
+		} else {
+			/* EntryLo1 */
+			mips3_wired_map[index].pa0 = 0;
+			mips3_wired_map[index].pa1 = pa;
+		}
+		mips3_wired_map[index].va = va0;
+		mips3_wired_map[index].pgmask = MIPS3_PG_SIZE_TO_MASK(pgsz);
+
+		/* Allocate new wired entry */
+		mips3_cp0_wired_write(MIPS3_TLB_WIRED_UPAGES +
+		    mips3_nwired_page + 1);
+	}
+
+	/* map it */
+	tlb.tlb_mask = mips3_wired_map[index].pgmask;
+	tlb.tlb_hi = mips3_vad_to_vpn(va);
+	if (mips3_wired_map[index].pa0 == 0)
+		tlb.tlb_lo0 = MIPS3_PG_G;
+	else
+		tlb.tlb_lo0 =
+		    mips3_paddr_to_tlbpfn(mips3_wired_map[index].pa0) |
+		    MIPS3_PG_IOPAGE(
+			    PMAP_CCA_FOR_PA(mips3_wired_map[index].pa0));
+	if (mips3_wired_map[index].pa1 == 0)
+		tlb.tlb_lo1 = MIPS3_PG_G;
+	else
+		tlb.tlb_lo1 = mips3_paddr_to_tlbpfn(
+			mips3_wired_map[index].pa1) |
+		    MIPS3_PG_IOPAGE(
+			    PMAP_CCA_FOR_PA(mips3_wired_map[index].pa1));
+	MachTLBWriteIndexedVPS(MIPS3_TLB_WIRED_UPAGES + index, &tlb);
+	return TRUE;
+}
+
+
+/*
+ * Wire down a mapping from a virtual to physical address.  The size
+ * of the region must be a multiple of MIPS3_WIRED_ENTRY_SIZE, with
+ * matching alignment.
+ *
+ * Typically the caller will just pass a physaddr that is the same as
+ * the vaddr with bits 35-32 set nonzero.
+ */
+boolean_t
+mips3_wired_enter_region(vaddr_t va, paddr_t pa, vsize_t size)
+{
+	vaddr_t	vend;
+	/*
+	 * This routine allows for for wired mappings to be set up,
+	 * and handles previously defined mappings and mapping
+	 * overlaps reasonably well.  However, caution should be used
+	 * not to attempt to change the mapping for a page unless you
+	 * are certain that you are the only user of the virtual
+	 * address space, otherwise chaos may ensue.
+	 */
+
+	/* offsets within the page have to be identical */
+	KASSERT((va & MIPS3_WIRED_OFFMASK) == (pa & MIPS3_WIRED_OFFMASK));
+
+	vend = va + size;
+	/* adjust for alignment */
+	va &= ~MIPS3_WIRED_ENTRY_OFFMASK;
+	pa &= ~MIPS3_WIRED_ENTRY_OFFMASK;
+
+	while (va < vend) {
+		if (!mips3_wired_enter_page(va, pa, MIPS3_WIRED_ENTRY_SIZE))
+			return FALSE;
+		va += MIPS3_WIRED_ENTRY_SIZE;
+		pa += MIPS3_WIRED_ENTRY_SIZE;
+	}
+	return TRUE;
+}
Index: mips/conf/files.mips
===================================================================
RCS file: /net/projects/meteor/cvs/netbsd/src/sys/arch/mips/conf/files.mips,v
retrieving revision 1.1.1.1
retrieving revision 1.6
diff -u -r1.1.1.1 -r1.6
--- mips/conf/files.mips	29 Sep 2005 16:42:46 -0000	1.1.1.1
+++ mips/conf/files.mips	2 Nov 2005 16:52:31 -0000	1.6
@@ -13,6 +13,7 @@
 					# ENABLE_MIPS_R3NKK
 defflag	opt_mips_cache.h		MIPS3_NO_PV_UNCACHED
 					ENABLE_MIPS4_CACHE_R10K
+defflag opt_mips3_wired.h		ENABLE_MIPS3_WIRED_MAP
 
 file	arch/mips/mips/locore_mips1.S		mips1
 file	arch/mips/mips/locore_mips3.S		mips3 | mips4 | mips32 | mips64
@@ -37,6 +38,7 @@
 file	arch/mips/mips/vm_machdep.c
 file	arch/mips/mips/process_machdep.c
 file	arch/mips/mips/cpu_exec.c
+file	arch/mips/mips/wired_map.c		enable_mips3_wired_map & !mips1
 
 file	arch/mips/mips/cache.c
 file	arch/mips/mips/cache_r3k.c		mips1
Index: evbmips/include/wired_map.h
===================================================================
RCS file: evbmips/include/wired_map.h
diff -N evbmips/include/wired_map.h
--- /dev/null	1 Jan 1970 00:00:00 -0000
+++ evbmips/include/wired_map.h	2 Nov 2005 16:52:31 -0000	1.1
@@ -0,0 +1,3 @@
+/*	$NetBSD$	*/
+
+#include <mips/wired_map.h>
Index: evbmips/include/Makefile
===================================================================
RCS file: /net/projects/meteor/cvs/netbsd/src/sys/arch/evbmips/include/Makefile,v
retrieving revision 1.1.1.1
retrieving revision 1.2
diff -u -r1.1.1.1 -r1.2
--- evbmips/include/Makefile	29 Sep 2005 16:40:06 -0000	1.1.1.1
+++ evbmips/include/Makefile	2 Nov 2005 16:52:31 -0000	1.2
@@ -19,6 +19,6 @@
 	setjmp.h signal.h stdarg.h \
 	trap.h types.h \
 	varargs.h vmparam.h \
-	wchar_limits.h
+	wchar_limits.h wired_map.h
 
 .include <bsd.kinc.mk>

--------------090809060402080100090801--