Subject: swap space leaks
To: None <tech-kern@NetBSD.ORG>
From: Onno van der Linden <onno@simplex.nl>
List: tech-kern
Date: 11/19/1995 10:43:07
 Here's my 2 cents about the leaks in swap space.

Appended to this message you'll see something relevant from the mach4-users
archive on jaguar.cs.utah.edu. It contains a pointer to a paper by
Randy Dean and Michelle Dominjani that describes a real fix to this problem.

Onno van der Linden (onno@simplex.nl)

>From mach4-request  Thu Apr 13 09:04:07 1995
	id JAA21370; Thu, 13 Apr 1995 09:04:07 -0600
	id IAA20948; Thu, 13 Apr 1995 08:59:08 -0600
	id KAA07948; Thu, 13 Apr 1995 10:58:26 -0400
To: Michael I Bushnell <mib@gnu.ai.mit.edu>
cc: mach3@cs.cmu.edu, mach4-users@cs.utah.edu
Subject: Re: [roland@frob.com: [lm@melb.cpr.itg.telecom.com.au: Re: -current memory leaks?]] 
In-reply-to: Your message of "Thu, 13 Apr 1995 09:45:59 EDT."
             <199504131345.JAA29155@geech.gnu.ai.mit.edu> 
Date: Thu, 13 Apr 1995 10:58:25 -0400
From: Steve Sears <sjs@osf.org>

>   There's a nasty problem with 386bsd's vm code (also present in
>   netrel2, mach 2.5 and even mach 3.0, I believe) where a vm_object
>   can't be collapsed if it has a pager.  pagers are typically
>   allocated to previously pager-less objects by the pageout daemon
>   when memory runs short.  This inability to collapse objects with
>   pagers is quite serious because pager resources (usually swap
>   space) eventually run out as a result of an enormous string of
>   un-collapsible vm_objects, each gobbling some amount of swap.

This is a problem if you are running Mach 3.0 without MACH_PAGEMAP
enabled.

The problem results from the VM systems inability to know what pages
are not in memory, and what the state of those missing pages is.  When
you collapse an object, you basically merge the pages from the two
objects and discard the overlaps from the old object.  If pages have
been sent to a pager, you cannot risk doing this as the EMMI does not
provide you a mechanism to outsmart the pager (i.e. if the pager
believes it has the only copy of a page and you page out yet another
copy, it has the right to panic).  This is problematic if pages are
marked precious by the pager, for example, when the pager absolutely
demands to know where the copy of the page is.

Violating this semantic requires the VM system to know something of the
pager, which is not a good thing.

MACH_PAGEMAP installs a bit array into temporary objects, with a bit
representing each page gone to the default pager.  The implementation
assumes some knowledge of the default pager, but allows the two
objects to be merged.  This is not a cheap operation, however.
Furthermore, it is limited by the size of the bit array, which I
believe represents up to a 24 Mbyte region (I don't recall the exact
number, but it is big, not huge).  And finally, it only works for
temporary objects (hence the default pager).

The real fix is in an overhaul to the EMMI which makes provisions for
merging.  This work has been researched and exists in a paper by Randy
Dean and Michelle Dominjani, but has not been implemented at OSF yet.


 -Steve