On Sat, Dec 10, 2005 at 01:39:44PM +0900, YAMAMOTO Takashi wrote:
> hi,
> 
> > > I agree that that change would keep pages around longer than the previous
> > > mechanism for clearing the referenced bit.  does changing that back
> > > make any detectable difference (either in measurable performance or
> > > in a double-blind test of subjective performance)?
> > 
> > i haven't done any serious comparison.
> 
> now i did a comparison.
> 
> ftp://ftp.netbsd.org/pub/NetBSD/misc/yamt/1/ld.ps.gz
> 
> - xen domainU with 40MB memory.
> - page type balancing is disabled for simplicity.

did having this enabled or disabled affect the results?


> - Y-axis is the number of major faults when linking a kernel.

do you really mean just the link step of building a kernel,
or the whole kernel build process?


>   (ru_majflt, average of 3 runs.)
> - X-axis is the percentage of inactive queue length.
>   (the current value is 33%.)
> - clrref=0 is the current behaviour.  clrref=1 means clearing
>   the reference bit when moving pages from active queue to inactive queue.
> - bln=N is the amount of memory in MB which is held by kernel to emulate
>   low memory machine.  ("balloon size")
> 
> ftp://ftp.netbsd.org/pub/NetBSD/misc/yamt/1/ld_e.ps.gz
> 
> - same as the above, except Y-axis is elapsed time.
> 
> so i'd like to:
> - put pmap_clear_reference back.  (clrref=1)
> - make inactive queue length longer, or at least tunable.
>   (i hesitate to to make it longer as it can have some negative impacts
>   esp. for ports without h/w reference bits.)
> 
> any comments?

I checked the access pattern of "ld" just now and it looks like ld makes
3 passes over all the object files, reading the whole file each time and
writing data to the output file during the last pass.  I would guess that
the desired paging behaviour for this access pattern would be to keep
most of the process data in memory and cycle the .o file contents through
the remaining memory.  can you tell if that's really what happens?

I guess that the effect of clearing the reference bit when moving a page
from active to inactive is to discount the initial pages access the occurs
when a page is first create.  only if a page is accessed again after some
time would we consider it important enough to keep.  it's interesting that
the shorter you make the period of "initial access" (ie. the longer you make
the inactive queue), the better the result.  so my notion that gathering
access info for longer was right, I just forgot that it's important to ignore
the initial access.

so yea, I think putting back the pmap_clear_reference() would be good.
for platforms that have a hardware reference bit, making the active queue
much smaller (maybe 20% of managed memory) seems fine. for other platforms
with no hardware reference bit, the benefits of a smaller active queue
probably still outweigh the cost of the extra page faults, but it would be
good to verify that.  either way, making the active/inactive split tunable
would be fine too.

-Chuck