Subject: Re: compartmentalization of kernel memory
To: John Gordon <john_94501@yahoo.com>
From: Richard Earnshaw <rearnsha@buzzard.freeserve.co.uk>
List: tech-kern
Date: 04/09/2003 00:28:39
> Hello,
> 
> > > > Could the 16 protection bits (forgotten what they are called) be used??
> > >
> > > No, don't try to use the domains system.
> > 
> > Could you elaborate a little on this statement?
> 
> Taken out of context that sounds like a global suggestion not to use the domain
> mechanism... in context, it was simply a recommendation not to use them to try
> to protect the exception vectors (which was the topic of that particular
> sub-thread).

In fact, both in and out of context, I'm not keen on the domain model.

> 
> The problem with using the domain system is that the domain access control
> register is not changed by an exception. So, if the vectors were inaccessible
> at the time of the exception, they will remain inaccessible when the CPU tries
> to execute the vector instruction(s). That will result in another exception...
> and so on, ad infinitum.

All that is true, but there are also more general problems with the domain 
model.

The main problem is that each entry in the L1 can only belong to a single 
domain, and therefore use of the domain model fragments the address space 
available to a process.  That might be OK in some situations, but trying 
to cram n processes into a single L1 page is going to mean that each 
process can access at most 1/n th of the virtual memory map.  At a full 15 
domains (one domain must be reserved for the kernel) that means about 200M 
per process; which sounds quite a lot, but doesn't really amount to that 
much when you have laid out code, data, heap, shared libraries and stack 
in different areas of the virtual map.

If the processes are independent, then they cannot have any virtual 
address in common.  That leads to one of two possible consequences:
1) The program must be fully dynamically linked at load time (dll-style 
linking)
2) The program must be compiled PIC, so that only some data have to be 
linked.

Neither of these is especially palettable.  The first is slow to start up; 
the second imposes a run-time overhead when accessing data.  We already do 
the latter for shared libraries, so that would seem the most sensible, but 
I still don't really want to pay that sort of price.

If a program grows to exceed its share of the VM available in that page, 
then either it must be killed, or it must be given its own L1 entry.  In 
the latter case, the VM will be already badly fragmented by the layout 
imposed by previous allocation and it may not be possible to make full use 
of the dedicated L1 after that (since virtual addresses for a process 
cannot be changed once allocated).

Unless you are going to limit yourself to 15 processes maximum, then you 
are going to have to support multiple L1s.   So most of the benefit of 
domains (fast context switching, since the virtually-addressed cache 
doesn't need flushing) are lost as soon as you have 3 or 4 L1s that are 
active in the system.

Finally, domains are rarely used and there are rumours that some chips 
don't implement them properly -- I've no personal experience of using 
them, so I can't be sure of this.

R.