Izumi Tsutsui wrote:

>In article <20051025205402.7B93A63B8C2@narn.netbsd.org>
>garrett_damore@tadpole.com wrote:
>
>  
>
>> Here are the diffs.  Again, I've not conditionalized the inclusion of 
>> mips3_wired_map.c, but if folks feel strongly enough about it I can 
>> certainly do so.
>>    
>>
>
>IMO, currently only arc port uses wired map, so it might be better to
>have some proper option (like options MIPS3_WIRED_MAP etc.).
>  
>
That sounds reasonable.   The evbmips port (or at least some kernels in 
it) will be using it when I get my PCI support committed.  I need it for 
PCI and PCMCIA spaces on the Alchemy parts.

>  
>
>> Also, the ARC port could probably make use of my logic in 
>> mips3_wired_map.c, but I wanted to minimize change (and I don't have an 
>> ARC system to test against, anyway, so all I can do is verify the compile.)
>>    
>>
>
>I think the MI wired map function should also handle independent
>(non-contiguous) PA ranges for TLB_LO0 and TLB_LO1.
>arc port requires it to map PCI memory and I/O space at the early
>bootstrap stage, and it could save wired TLB entries in other case.
>  
>
An interesting idea.  I agree it would save entries.  Right now I am 
assuming that you can nearly always use a full 32MB of mapping, but 
obviously that isn't always the case.  (E.g. the PCI configuration space 
entry only really needs a single 4KB window.)  The complexity arises in 
trying to use these "efficiently".

However, if I make the logic such that we are using 16MB granular 
entries, I can pretty much hide the details of the fact that there are 
really 2 mappings per entry from the caller.

>I also have some more quick comments:
>
>- VA region for wired map should be defined in machine dependent
>  <machine/vmparam.h> and it should be managed by extent(9) etc.
>  
>
Agreed.  Right now my use defines it in a machine dependent header (not 
vmparam.h, but that sounds like a good place for it!) and it does use an 
extent.  I've intentionally taken the route of *not* exposing this 
detail to the MI wired mapping logic that I submitted.

>- pagemask for wired maps should be defined in machine dependent headers,
>  or maybe it should be passed from callers.
>  
>
I don't like passing it from callers, because I don't want it to be 
variable.  (This would make the code a lot more complex to make sure it 
was correct and you didn't have overlapping maps, etc.)   But defining 
it in the machine dependent header seems reasonable.

>Currently I use the attached wired_map.[ch] for ews4800mips ports,
>though it lacks some sanity checks and unmap (or remap) functions.
>
>Maybe we should discuss more about APIs for these wired map functions.
>  
>
This looks pretty darn close to the mapping logic that was in the ARC 
port.  I haven't run a diff to examine your deltas, but it has a lot 
more of the logic in it then mine does -- i.e. managment of the extents, 
and such.  I was foisting the address space management issues on the 
caller, and just providing bare-bones logic for establishing the mappings.

Is there a belief that we need to have centralized logic for managing 
these address spaces?  Or is my approach of having the caller figure it 
out better.  (E.g. in my Alchemy port I have the PCI code set up its own 
extent for the address spaces, and it knows what the physical to logical 
mappings are.)

I find that having the caller set it up works well for me, because the 
caller can do things to establish very simple translations.  For 
example, on my Alchemy port:

    * 0xD0000000 - 0xDFFFFFFF  == PCI memory virtual address space

I then set up the physical memory translation so that all I have to do 
to go from physical to virtual is OR in the upper bits (in this case 
physical addresses are 0x4D0000000 - 0x4DFFFFFFF (yes, those are 36-bit 
addresses).

I have different rules for PCI I/O and configuration space.

In order to establish these simple mappings, the caller needs to have 
some basic control over the VA space (i.e. I use different extents for 
PCI memory, IO, and configuration space.)

I suppose this is inefficient use of the limited VA space, but I find it 
simpler.  (I never have to inquire of the wired map to figure the VA->PA 
mapping.)

    -- Garrett

>---
>Izumi Tsutsui
>
>
>---
>/*	$NetBSD$	*/
>
>/*-
> * Copyright (C) 2005 Izumi Tsutsui.
> * Copyright (C) 2000 Shuichiro URATA.  All rights reserved.
> *
> * Redistribution and use in source and binary forms, with or without
> * modification, are permitted provided that the following conditions
> * are met:
> * 1. Redistributions of source code must retain the above copyright
> *    notice, this list of conditions and the following disclaimer.
> * 2. Redistributions in binary form must reproduce the above copyright
> *    notice, this list of conditions and the following disclaimer in the
> *    documentation and/or other materials provided with the distribution.
> * 3. The name of the author may not be used to endorse or promote products
> *    derived from this software without specific prior written permission.
> *
> * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
> * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
> * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
> * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
> * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
> * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> */
>
>#include <sys/cdefs.h>
>__KERNEL_RCSID(0, "$NetBSD$");
>
>#include <sys/param.h>
>#include <sys/systm.h>
>#include <sys/malloc.h>
>#include <sys/extent.h>
>
>#include <uvm/uvm_extern.h>
>
>#include <mips/pte.h>
>
>#include <machine/locore.h>
>#include <machine/vmparam.h>
>
>#include <ews4800mips/ews4800mips/wired_map.h>
>
>#define EWS4800MIPS_TLB_WIRED_ENTRIES	8	/* upper limit */
>
>static int ews4800mips_enter_wired(vaddr_t va, paddr_t pa0, paddr_t pa1,
>    uint32_t pg_size);
>
>static struct wired_map_entry {
>	paddr_t	pa0;
>	paddr_t	pa1;
>	vsize_t	size;
>	vaddr_t	va;
>} wired_map[EWS4800MIPS_TLB_WIRED_ENTRIES];
>
>static int nwired;
>static struct extent *wired_map_ex;
>static long wired_map_ex_storage[EXTENT_FIXED_STORAGE_SIZE(8) / sizeof(long)];
>
>void
>ews4800mips_init_wired_map(void)
>{
>
>	nwired = 0;
>	wired_map_ex = extent_create("wired_map",
>	    VM_MIN_WIRED_MAP_ADDRESS, VM_MAX_WIRED_MAP_ADDRESS, M_DEVBUF,
>	    (void *)wired_map_ex_storage, sizeof(wired_map_ex_storage),
>	    EX_NOWAIT);
>	if (wired_map_ex == NULL)
>		panic("ews4800mips_init_wired_map: can't create extent");
>}
>
>static int
>ews4800mips_enter_wired(vaddr_t va, paddr_t pa0, paddr_t pa1, uint32_t pg_size)
>{
>	struct tlb tlb;
>	psize_t size;
>
>	if (nwired >= EWS4800MIPS_TLB_WIRED_ENTRIES) {
>#ifdef DIAGNOSTIC
>		printf("ews4800mips_enter_wired: wired entry exausted.\n");
>#endif
>		return ENOMEM;
>	}
>
>	size = MIPS3_PG_SIZE_MASK_TO_SIZE(pg_size);
>	va = (va + (size - 1)) & ~(size - 1);
>
>	wired_map[nwired].va = va;
>	wired_map[nwired].pa0 = pa0;
>	wired_map[nwired].pa1 = pa1;
>	wired_map[nwired].size = size;
>
>	/* Allocate new wired entry */
>	mips3_cp0_wired_write(MIPS3_TLB_WIRED_UPAGES + nwired + 1);
>
>	/* Map to it */
>	tlb.tlb_mask = pg_size;
>	tlb.tlb_hi = mips3_vad_to_vpn(va);
>	if (pa0 == 0)
>		tlb.tlb_lo0 = MIPS3_PG_G;
>	else
>		tlb.tlb_lo0 = mips3_paddr_to_tlbpfn(pa0) | \
>		    MIPS3_PG_IOPAGE(PMAP_CCA_FOR_PA(pa0));
>	if (pa1 == 0)
>		tlb.tlb_lo1 = MIPS3_PG_G;
>	else
>		tlb.tlb_lo1 = mips3_paddr_to_tlbpfn(pa1) | \
>		    MIPS3_PG_IOPAGE(PMAP_CCA_FOR_PA(pa1));
>	mips3_TLBWriteIndexedVPS(MIPS3_TLB_WIRED_UPAGES + nwired, &tlb);
>
>	nwired++;
>
>	return 0;
>}
>
>/* Allocate new wired entries */
>int
>ews4800mips_map_wired(vaddr_t va, paddr_t pa0, paddr_t pa1, uint32_t pg_size)
>{
>	psize_t size;
>	int error;
>
>	size = MIPS3_PG_SIZE_MASK_TO_SIZE(pg_size);
>	va = (va + (size - 1)) & ~(size - 1);
>
>	if (va < VM_MIN_WIRED_MAP_ADDRESS ||
>	    va + size > VM_MAX_WIRED_MAP_ADDRESS) {
>#ifdef DIAGNOSTIC
>		printf("ews4800mips_enter_wired: invalid va range.\n");
>#endif
>		return EINVAL;
>	}
>
>	error = extent_alloc_region(wired_map_ex, va, size, EX_NOWAIT);
>	if (error) {
>#ifdef DIAGNOSTIC
>		printf("ews4800mips_enter_wired: cannot allocate region.\n");
>#endif
>		return error;
>	}
>
>	error = ews4800mips_enter_wired(va, pa0, pa1, pg_size);
>
>	return error;
>}
>
>vaddr_t
>ews4800mips_map_wired_region(paddr_t pa, int size, uint32_t pg_mask)
>{
>	vaddr_t va, rva;
>	vsize_t off;
>	psize_t pg_size;
>	int error;
>
>	pg_size = MIPS3_PG_SIZE_MASK_TO_SIZE(pg_mask);
>	off = pa & WIRED_ENTRY_OFFMASK(pg_size);
>	pa &= ~(paddr_t)WIRED_ENTRY_OFFMASK(pg_size);
>	size += off;
>
>	if ((size + WIRED_ENTRY_SIZE(pg_size) - 1) / WIRED_ENTRY_SIZE(pg_size) >
>	    EWS4800MIPS_TLB_WIRED_ENTRIES - nwired) {
>
>#ifdef DIAGNOSTIC
>		printf("ews4800mips_map_wired_region(0x%lx, 0x%lx)"
>		    ": %d is not enough\n", pa + off, size - off,
>		    EWS4800MIPS_TLB_WIRED_ENTRIES - nwired);
>#endif
>		return 0; /* free wired TLB is not enough */
>	}
>
>	error = extent_alloc(wired_map_ex, size, WIRED_ENTRY_SIZE(pg_size), 0,
>	    EX_NOWAIT, &va);
>	if (error) {
>#ifdef DIAGNOSTIC
>		printf("ews4800mips_map_wired_region: can't allocate region\n");
>#endif
>		return 0;
>	}
>	rva = va;
>
>	while (size > 0) {
>		ews4800mips_enter_wired(va, pa, pa + pg_size, pg_mask);
>		pa += WIRED_ENTRY_SIZE(pg_size);
>		va += WIRED_ENTRY_SIZE(pg_size);
>		size -= WIRED_ENTRY_SIZE(pg_size);
>	}
>
>	return rva + off;
>}
>---
>
>---
>/*	$NetBSD$	*/
>
>/*-
> * Copyright (C) 2005 Izumi Tsutsui.
> * Copyright (C) 2000 Shuichiro URATA.  All rights reserved.
> *
> * Redistribution and use in source and binary forms, with or without
> * modification, are permitted provided that the following conditions
> * are met:
> * 1. Redistributions of source code must retain the above copyright
> *    notice, this list of conditions and the following disclaimer.
> * 2. Redistributions in binary form must reproduce the above copyright
> *    notice, this list of conditions and the following disclaimer in the
> *    documentation and/or other materials provided with the distribution.
> * 3. The name of the author may not be used to endorse or promote products
> *    derived from this software without specific prior written permission.
> *
> * THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
> * IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
> * OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
> * IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
> * INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
> * NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
> * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
> * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
> * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
> */
>
>#define WIRED_ENTRY_SIZE(pg_size)	((pg_size) * 2) 
>#define WIRED_ENTRY_OFFMASK(pg_size)	(WIRED_ENTRY_SIZE(pg_size) - 1)
>
>void	ews4800mips_init_wired_map(void);
>int	ews4800mips_map_wired(vaddr_t va, paddr_t pa0, paddr_t pa1,
>    uint32_t pg_size);
>vaddr_t	ews4800mips_map_wired_region(paddr_t pa, int size, uint32_t pg_size);
>---
>  
>