Subject: "pmap_unwire: wiring ... didn't change!"
To: None <port-cobalt@netbsd.org>
From: Chuck Silvers <chuq@chuq.com>
List: port-cobalt
Date: 02/12/2005 16:32:47
--XsQoSWH+UP9D9v3l
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

hi,

I looked into the "pmap_unwire: wiring ... didn't change!" problem.
what's happening is that sometimes we're going through uvm_fault() again
for one of the wired pages, and the pmap_enter() which results from that
replaces the wired pmap entry with non-wired one.  so why would we go through
uvm_fault() again?  that's happening because sometimes the TLB entry is
recycled after the page mapped by one side of the entry is wired and before
the other page is wired.  and when MachTLBUpdate() is called for the second
page, the entry it creates only has the second side filled in.  this causes
us to get a "TLB invalid" trap when we access the other half of the TLB entry
instead of the "TLB miss" trap that we would get if there were no TLB entry
at all.  the handler for "TLB invalid" traps from usermode always calls
trap() instead of looking at the PTEs.

so the problem is that the PTEs and the TLB get out of sync, and the trap
handlers aren't expecting that.  I see several possible fixes, in
approximate decreasing order of preference:

 - change MachTLBUpdate() to take both PTEs for the TLB entry for MIPS3.

 - change the MIPS3 MachTLBUpdate() to only update existing TLB entries,
   not create new ones.

 - have the MIPS3 user "TLB invalid" trap handler try to reload from the
   PTEs before calling trap().

 - use MIPS3_TBIS() instead of MachTLBUpdate() for user mappings on MIPS3.


the last one was the easiest, so I implemented that one and it appears to
make the problem go away.  the patch is attached.  does anyone want to
implement one of the better solutions soon?  if not, I'll just check in
what I've got (after updating it to be friendly to MIPS1, etc).

-Chuck

--XsQoSWH+UP9D9v3l
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="diff.mips-pmap_unwire"

Index: arch/mips/mips/pmap.c
===================================================================
RCS file: /cvsroot/src/sys/arch/mips/mips/pmap.c,v
retrieving revision 1.156
diff -u -p -r1.156 pmap.c
--- arch/mips/mips/pmap.c	17 Jan 2005 04:54:14 -0000	1.156
+++ arch/mips/mips/pmap.c	13 Feb 2005 00:24:01 -0000
@@ -825,6 +825,7 @@ pmap_page_protect(pg, prot)
 	vm_prot_t prot;
 {
 	paddr_t pa = VM_PAGE_TO_PHYS(pg);
+	struct pmap *pmap;
 	pv_entry_t pv;
 	vaddr_t va;
 
@@ -859,10 +860,10 @@ pmap_page_protect(pg, prot)
 	default:
 		pv = pa_to_pvh(pa);
 		while (pv->pv_pmap != NULL) {
-			pmap_remove(pv->pv_pmap, pv->pv_va,
-				    pv->pv_va + PAGE_SIZE);
+			pmap = pv->pv_pmap;
+			pmap_remove(pmap, pv->pv_va, pv->pv_va + PAGE_SIZE);
+			pmap_update(pmap);
 		}
-		pmap_update(pv->pv_pmap);
 	}
 }
 
@@ -967,7 +968,7 @@ pmap_protect(pmap, sva, eva, prot)
 			 * Update the TLB if the given address is in the cache.
 			 */
 			if (needupdate)
-				MachTLBUpdate(sva | asid, entry);
+				MIPS_TBIS(sva | asid);
 		}
 	}
 }
@@ -1088,7 +1089,7 @@ pmap_page_cache(paddr_t pa, int mode)
 				entry = (entry & ~MIPS3_PG_CACHEMODE) | newmode;
 				pte->pt_entry = entry;
 				if (needupdate)
-					MachTLBUpdate(pv->pv_va | asid, entry);
+					MIPS_TBIS(pv->pv_va | asid);
 			}
 		}
 		pv = pv->pv_next;
@@ -1331,7 +1332,7 @@ pmap_enter(pmap, va, pa, prot, flags)
 	pte->pt_entry = npte;
 
 	if (pmap->pm_asidgen == pmap_asid_generation)
-		MachTLBUpdate(va | asid, npte);
+		MIPS_TBIS(va | asid);
 
 #ifdef MIPS3_PLUS	/* XXX mmu XXX */
 	if (MIPS_HAS_R4K_MMU && (prot == (VM_PROT_READ | VM_PROT_EXECUTE))) {

--XsQoSWH+UP9D9v3l--