On Tue, Apr 03, 2001 at 10:47:03PM -0700, Chuck Silvers wrote:
> On Tue, Apr 03, 2001 at 08:05:21PM -0700, Todd Whitesel wrote:
> 
> what causes pages to be paged out is not that processes are dormant,
> it is that memory is low.  pages which have been used "recently" are
> not paged out but rather put back at the end of the list to consider.
> "recently" means "since the last time we checked to see if it was
> used recently".  this might be as short as a few seconds if we're
> churning through a lot of file data.  so processes which are "dormant"
> even for a few seconds are paged out, and the memory is reused for file
> data.  your suggestion describes what we do already.

I think the problem is that the buffer cache essentially acts like the
old hack used to "swap out" processes (I haven't looked at this in UVM
but I assume it's the same as it was in the Mach VM): we emulated
"swapping" by marking all of a process' pages inactive and then creating
a huge false demand for pages to ensure that the "swapped" pages actually
left physical memory.

The buffer cache currently would like to use all of physmem, if it could.
In other words, it acts like our old "false demand" except that in the
absence of processes that have been *deliberately* swapped, it 
*inadvertently* swaps out processes that probably shouldn't be.

The algorithms in the VM system ane clearly not designed to cope with a
consumer that has an infinite demand for pages -- we've always performed
poorly in this condition, and usually the answer is "get more RAM".  But
the current buffer cache implementation guarantees that you can't ever
get enough!

Two ways around this seem fairly obvious to me; one is Easy and one is
Hard.  The "easy" way is something that Kirk McKusick mentioned to me
a long time ago as the way he'd intended to constrain the page demand
from a merged buffer cache in 4.4 (obviously this never got done :-)):
make the likelihood that the buffer cache actually gets a given page
inversely proportional to the percentage of memory it's already got.
This has the nice property that the buffer cache *can* grow to use 
almost all of physmem, but that as it grows it is more and more 
"conservative" about pushing out pages that might belong to processes
you intend to use again soon.  In other words, if the buffer cache wants
a page, and it's got 10% of the pages on the system, give it a 9/10
chance that it gets one.  If it wants a page, and it's got 90% of the
pages on the system, give it a 10% chance that it gets one.  Obviously
some scaling needs to be done here, and you have to account for how often
you churn through the page scanner, but you get the basic idea.

The "hard" way is to notice that there already is something in the system
that is supposed to guess when a process will want to be run: the scheduler.
Any approach that tries to not touch "processes that are likely to run soon"
or "interactive processes" is ultimately, I think, going to require tight
integration with the scheduler.  I think this opens up a huge can of worms.

Thor