On Tue, Apr 03, 2001 at 08:05:21PM -0700, Todd Whitesel wrote:
> > I don't think this is the right answer.  if the only solution to the problem
> > is to limit the amount of data used to cache file data, then people can only
> > have good interactive response if they limit themselves to a tiny amount of
> > cached data.
> 
> I don't think you understood what he said:
> 
> > > I think there should be some minimal value of memory dedicated to
> > > buffer cache (i.e. what BUFCACHE/BUFPAGES does) and more memory
> > > would only be used if it's otherwise _unused at all_.
> 
> Note that the total cache size would be the minimal value PLUS whatever's
> not being used by the in-core processes. That could be quite large!
> 
> As long as dormant processes get paged/swapped out, this should lead to
> pretty sensible behavior since it means that any memory you're not actually
> using is made available to the buffer cache.
> 
> But if you fill the machine with programs, it falls back to something that
> should perform about as well as the old fixed-size buffer cache.

what causes pages to be paged out is not that processes are dormant,
it is that memory is low.  pages which have been used "recently" are
not paged out but rather put back at the end of the list to consider.
"recently" means "since the last time we checked to see if it was
used recently".  this might be as short as a few seconds if we're
churning through a lot of file data.  so processes which are "dormant"
even for a few seconds are paged out, and the memory is reused for file
data.  your suggestion describes what we do already.


> > we should be able to have good performance with a larger cache,
> > and this should be the default.
> 
> If UBC uses all memory not otherwise needed for in-core processes then
> that is good performance with a larger cache, is it not?

what does "needed for in-core processes" mean?  if we're on a machine
with 64 MB of RAM and we run a process which malloc()s 64 MB of memory,
do we "need" all 64 MB of RAM for that process?  no.  so how much of
the 64 MB is "needed"?  if the process is going to access a lot of that
64 MB again soon, we'd like the answer to be "most of it".  if the process
is going to sleep for the next week, we'd like the answer to be "none of it".


> The solution is not to limit the maximum size of the UBC, but instead to
> lower its priority when it already owns enough pages to get reasonable
> performance.
> 
> If we need to be more aggressive about swapping out processes to make
> room for UBC, then let's do that. But it's really stupid to have UBC
> kick out any process that is likely to run in the next few seconds.

ok, let's assume that processes which are runnable or "idle" (as opposed to
"sleeping" in the ps(1) terminology) are likely to run again soon.
when we're looking at a given page, we don't know what process or processes
have it mapped, or even whether it's mapped at all.  it would be possible to
add a bunch of code to keep track of this, but it would be prohibitively
expensive in terms of cpu usage.  so how would we know whether any process
is likely to need this page again in the next few seconds?

-Chuck