Subject: Re: compartmentalization of kernel memory
To: Matthew Mondor <>
From: John Gordon <>
List: tech-kern
Date: 04/08/2003 11:29:02

The problem with the "magic number" based schemes is that they are only able to
detect the corruption after the fact. There are places in any kernel where
detecting the fact that something critical has been corrupted doesn't help you
that much - the only option you have is to reboot anyway. Even if it is not
critical, knowing that a data structure has been damaged doesn't give you much
clue about what damaged it.

The idea of truly compartmentalising the kernel is something that the true
microkernel folks have been doing for years (e.g. Mach, QNX, Enea's OSE). The
Windows development team will tell you all about the problems here though. To
keep things secure, run as much as possible outside the kernel, and keep things
as separated as possible. To keep things fast, keep things linked together, and
keep them in the kernel where possible.

Windows started life with drivers outside the kernel. It was too slow. So they
move them into the kernel, and we all know what happens when a driver
misbehaves. (Interesting side note: Windows XP has compromised a little here
and allows a driver to be installed outside the kernel for testing purposes -
neat idea.)

Of course, you could try to compartmentalise the kernel's data, and wrap each
block with a number of unmapped/inaccessible pages. That will probably prevent
many of the common programming errors in one space from damaging another. This
costs almost nothing in terms of performance, but it has some limitations. If
the bad code manages to "hop" over the guard pages, then you'll still get
corruption. It also eats virtual address space, and wastes more physical space
(each block needs to be a multiple of the page size).


--- Matthew Mondor <> wrote:
> Hmm since kernel space runs under the same MMU context I also see that
> self-memory protection can become tricky... There is a system which I
> used when writing Xisop (a microkernel which does not support MMU and
> can run on a plain motorola 68000) which allowed to perform some basic
> sanity checking on key system structures which are generally referenced
> by address/pointer...
> It uses a magic "cookie" which each special system structures which can
> be accessed using pointers have. This only consists of a unique number
> which is initialized when such an object is created/allocated, and set
> to zero when it is destroyed/freed. This allowed some critical code sections
> (i.e. message queuing to a port by port address and message address) to
> first verify if the pointer is non-null (it also would be possible to
> perform boundary checking there), and then make sure that the cookie
> corresponds to the expected value.
> Another aspect was that for instance the system had to never attempt to
> reply to a message via a message port which may not belong to the expected
> task (as the task may have died and it's memory recycled to allocate another
> task, same for ports)... Then for instance each task has a unique identifyer
> which is only generated by bumping a global counter and assigning it's
> new value to the object in question, making sure to prevent race conditions,
> and it allows to first verify port address validity using the previously
> mentionned technique, and then make sure that the port taskid corresponds
> to the unique task id... and then only attempt to queue the message.
> Using such simple methods did not significantly slow down the code, neither
> did it make it grow too much, and it works within the same MMU context...
> When an object validity is questionned a debugging message can be output
> without the operation actually being performed thus reducing memory
> corruption
> I'm not sure if this consists of what is needed, but I thought it would be
> worth describing the idea... So to use those an headerfile was created with
> various macros and unique cookie definitions to validate/unvalidate objects,
> associate an object to another and verify object association integrity,
> as simple as that... Not that it can fix everything, but it helped to make
> more stable a kernel which was inherently missing hardware assisted
> protection.
> Matt

Rate Corporate America at

Do you Yahoo!?
Yahoo! Tax Center - File online, calculators, forms, and more