Re: [Milkymist port] virtual memory management

[Eduardo Horvath <eeh%NetBSD.org@localhost>]

On Wed, 29 May 2013, Yann Sionneau wrote:

> Hello NetBSD fellows,
> 
> As I mentioned in my previous e-mail, I may need from time to time a little
> bit of help since this is my first "full featured OS" port.
> 
> I am wondering how I can manage virtual memory (especially how to avoid tlb
> miss, or deal with them) in exception handlers.

There are essentially three ways to do this.  Which one you chose depends 
on the hardware.

1) Turn off the MMU on exception

2) Keep parts of the address space untranslated

3) Lock important pages into the TLB


Turning off the MMU is pretty straight-forward.  ISTR the PowerPC Book E 
processors do this.  Just look up the TLB entry in the table and return 
from exception.  You just have to make sure that the kernel manages page 
tables using physical addresses.

Keeping parts of the address space untranslated is what MIPS does.  2GB 
goes through the MMU for user space, 1GB is kernel virtual addresses, and 
512MB is untranslated VA->PA (+offset).  In this scheme usually the 
kernel text is untranslated but kernel data is virtual.  If the hardware 
doesn't support untranslated parts of the address space, you can fake it 
by checking the address in the TLB miss handler and generating an entry 
based on the fault address.  (I did something similar to that for the 
PowerPC Book E port 'cause parts of the MD code make assumptions about 
accessing untranslated data.)  

This design does have some disadvantages.  User address space is reduced 
by both the kernel address space and the direct mapped address space.  You 
can't distinguish pages mapped into the kernel from random memory so 
kernel core dumps involve dumping all of RAM, not just kernel space.  It's 
also difficult to properly protect kernel text and data structures if you 
can scribble on random physical addresses.

If your MMU supports huge pages, you can lock the trap table and important 
bits of the kernel text and data segment in the TLB.  When I did the 
sparc64 port I used one 4MB TLB entry for kernel text, and another 4MB 
entry for kernel data.  After a while the kernel text overflowed the 4MB 
page and we needed to sacrifice a couple more TLB entries.  

I like this scheme since it allows 32-bit processes to use all 4GB of 
address space, but I did use some interesting features of the SPARC 
instruction set that allow load/store operations on alternate address 
spaces and physical addresses without needing to fiddle with the MMU.

So you can implement your VM subsystem several different ways.  Just 
remember that on a TLB-only machine you need to make sure the MMU handlers 
can access the page tables, either bypassing the MMU or using some trick 
that does not result in recursive TLB faults.

Eduardo