Re: Placing lwp in uarea [was: Re: uarea swap-out]

[Mindaugas Rasiukevicius <rmind%netbsd.org@localhost>]

David Laight <david%l8s.co.uk@localhost> wrote:
> > 
> > struct lwp is approx 700-1000 bytes on our platforms.
> > that's a significant chunk to remove from kernel stacks isn't it?
> 
> My thoughts exactly ....

Right, e.g. on amd64, struct lwp is 900+ and struct pcb is 600+ bytes.
I do not think it takes too much from kernel stack.  Smallest USPACE
among architectures is 8k (and since they are not LP64, structures will
likely take less than 1.5k).

Rationale of this change: there would be a single allocation for LWP and
PCB, which I think would reduce some memory fragmentation, as well as some
allocation overhead, especially on architectures with 16k-pages or more,
like mips64.

> There are also lurking ideas to avoid needing a kernel stack for every LWP.
> Many of the 'normal' sleep points for lwp can handled by restarting the
> system call when the wakeup event happens.

Yes, matt@ has some radical ideas, not sure if that would happen soon (if
will be considered as a good way, at first).  Hopefully, he will elaborate.

> The next problem is that, as structures get larger, the memeory
> allocater gets less efficient (there is more waste at the end of the page
> relative to the number of items in the page). So allocating one big
> area will use more memory that two (or more) small ones.

Rules are slightly different in this case as structures in uarea share the
space with kernel stack, thus utilises memory better.

> We also have the lurking kernel DDI/DKI issue - where you want to expose
> pointers to small structures which can grow extra members (at the end)
> so maintaining binary compatibility, rather than a large structure that
> ought to be immutable.

Yes, it the reason why I did not put PCB into lwp_t::l_md as matt@ initially
suggested.  One memory block, but keeping structures separate, means that we
can add members without breaking ABI.  IMO, it would be useful to do the same
with struct mdlwp (i.e. take it out from struct lwp).

-- 
Mindaugas