Noriyuki Soda wrote:

> Two problems are observed here at least.
> 
> 1. unnecessary page-in/out.
>   Because a page-out usually only happens at anon pages, it seems anon
>   pages are needlessly considered inactive, and get paged out.
>   It is better to look at "anonymous pages", "cached file pages" and
>   "cached executable pages" in "vmstat -s" to see how many pages are
>   used for those 3 types of pages. (Perhaps "top" should show the size
>   of anon pages, too.)
> 
>   If the above guess is right, increasing vm.anonmax (and decreasing
>   vm.file{max,min}) may help.
> 
>   If the unnecessary page-in happened at exec pages, increasing
>   vm.execmax may help, too.

I was talking to Soda offline and we made the following observation:

$ vmstat -s | egrep 'managed|anonymous|cached' | grep -v scanned
   127992 pages managed
   490879 anonymous pages
    14622 cached file pages
    12453 cached executable pages

A little program that walks the kernel page lists says that there are
48595 anon pages and 27179 pages attached to objects.  The sum of file
and exec pages is close to the pages backed by objects.

Obviously there are not more anon pages than total pages.  These
numbers match gdb run on the live kernel and querying uvmexp directly,
so it's not some silly reporting bug in vmstat.

At least for the current knobs on this box the page daemon will think
that both anon and file pages are available for reclaim.

uvmexp.anonpages is only changed in a handful of places in
uvm/uvm_page.c and they all look right at a glance.  However, there
must be some accounting problem lurking there somewhere.

Simon.
--
Simon Burge                            <simonb@wasabisystems.com>
NetBSD Support and Service:         http://www.wasabisystems.com/