Subject: Re: compartmentalization of kernel memory
To: John Gordon <firstname.lastname@example.org>
From: Richard Earnshaw <email@example.com>
Date: 04/09/2003 00:28:39
> > > > Could the 16 protection bits (forgotten what they are called) be used??
> > >
> > > No, don't try to use the domains system.
> > Could you elaborate a little on this statement?
> Taken out of context that sounds like a global suggestion not to use the domain
> mechanism... in context, it was simply a recommendation not to use them to try
> to protect the exception vectors (which was the topic of that particular
In fact, both in and out of context, I'm not keen on the domain model.
> The problem with using the domain system is that the domain access control
> register is not changed by an exception. So, if the vectors were inaccessible
> at the time of the exception, they will remain inaccessible when the CPU tries
> to execute the vector instruction(s). That will result in another exception...
> and so on, ad infinitum.
All that is true, but there are also more general problems with the domain
The main problem is that each entry in the L1 can only belong to a single
domain, and therefore use of the domain model fragments the address space
available to a process. That might be OK in some situations, but trying
to cram n processes into a single L1 page is going to mean that each
process can access at most 1/n th of the virtual memory map. At a full 15
domains (one domain must be reserved for the kernel) that means about 200M
per process; which sounds quite a lot, but doesn't really amount to that
much when you have laid out code, data, heap, shared libraries and stack
in different areas of the virtual map.
If the processes are independent, then they cannot have any virtual
address in common. That leads to one of two possible consequences:
1) The program must be fully dynamically linked at load time (dll-style
2) The program must be compiled PIC, so that only some data have to be
Neither of these is especially palettable. The first is slow to start up;
the second imposes a run-time overhead when accessing data. We already do
the latter for shared libraries, so that would seem the most sensible, but
I still don't really want to pay that sort of price.
If a program grows to exceed its share of the VM available in that page,
then either it must be killed, or it must be given its own L1 entry. In
the latter case, the VM will be already badly fragmented by the layout
imposed by previous allocation and it may not be possible to make full use
of the dedicated L1 after that (since virtual addresses for a process
cannot be changed once allocated).
Unless you are going to limit yourself to 15 processes maximum, then you
are going to have to support multiple L1s. So most of the benefit of
domains (fast context switching, since the virtually-addressed cache
doesn't need flushing) are lost as soon as you have 3 or 4 L1s that are
active in the system.
Finally, domains are rarely used and there are rumours that some chips
don't implement them properly -- I've no personal experience of using
them, so I can't be sure of this.