On 28-Apr-98 Chuck Cranor wrote:

> can you post the details of what happens to the parts of the X server's
> map that happen between exit-exec-exit cycle?   i'm only interested in the 
> virtual addresses and object pointers that have changed between runs.

I'll have to hassle with it to get useful stuff there, but I can tell you the
only thing that changes is the unrelated allocation on top.  Under it we get
these mis-matched entries we're talking about, and they don't change once they
appear.  When they appear after the exec, they are already 1-page entries with
40-page objects.  After the exit they haven't changed.

> here is my guess as to what is happening, maybe you can confirm or 
> deny it:
>
>    in BSD/Mach VM when you mmap() an area of anonymous memory area 
>    it creates a zero fill vm_object with a reference count of 1 and 
>    maps it into the process' address space.
>
>    if you munmap() part, but not _all_ of the mapping, the memory in
>    the unmapped region is not actually freed (kind of like what you
>    are seeing with the active list).   this is because the reference
>    count on the backing object is still greater than zero, so the object
>    isn't terminated (even though it is now larger than needed).   this 
>    can create a "swap memory leak effect" if you use a certain pattern 
>    of mmap/munmap's... I am thinking that maybe the X server is doing 
>    this.

Or sbrk cycles.  Sbrk with negative change works just like munmap.  Sbrk calls
vm_deallocate(), which calls vm_map_remove() -- which munmap calls directly --
which calls vm_map_delete(), which gets into this business with
vm_map_clip_start(), which creates a truncated map entry to take the front of
the entry being partly deallocated (and/or vm_map_clip_end() to take the back). 
But that truncated entry is otherwise a clone of the original one, including the
object, which instead of being split or otherwise modified just has its
reference count bumped, so when the original entry gets tossed the object
remains with all of the resident pages (including those that backed the
deallocated part of the entry).

So anything that uses a heap-based malloc will probably do this sometimes, when
it calls realloc to reduce the size of an allocation, if that allocation
happens to be atop the heap.  Then realloc would optimize by just lopping off
the end of the allocation with sbrk, to avoid having to copy anything.

This seems likely in the case we're looking at, because the only allocation
over these on the heap is one that keeps changing location, with new entry and
object structures; the X server "re-implements" it frequently, I gather, to
open up the stuff under it for pruning to keep this stuff from piling up.  But
the old Mach vm doesn't quite cooperate in the effort because, as you point
out....

> this is basically a result of the Mach vm object design.   memory objects
> (files, devices, and in this case anonymous memory) have no idea how
> much of their range is mapped and how much us unmapped.   for files
> this isn't a problem because we can just write out (or toss) unused pages
> to backing store and be done with it.   but anonymous memory is backed
> by swap, thus we have to write it out to swap.    if you keep doing this
> sort of thing swap will eventually fill up.
>
> if this is indeed the problem, i'd like to use it as a practical example
> for my writeup on the uvm stuff.

I'm not sure how to absolutely prove it for you, but I'm pretty well convinced,
myself.

Then again, lest we forget, I've been absolutely convinced of some totally
erroneous things about this vm in the past....


--Jim Howard  <jiho@mail.c-zone.net>


----------------------------------
E-Mail: jiho@mail.c-zone.net
Date: 27-Apr-98
Time: 23:41:41

This message was sent by XFMail
----------------------------------