Problem with lots of (perhaps too much?) memory

[Paul Goyette <paul%whooppee.com@localhost>]

While running a long series of 'build.sh release' for various machines, 
I noticed that my system, despite having 128GB DRAM, was using some swap 
space!  (Not very much, just 50MB or so, but still, with that much 
memory, you wouldn't expect to need any swap at all!)

According to top(1), there was more than 110GB of File cache, however 
only 33 MB (yes, M, not G) of Inactive memory.  Despite the fact that 
the ioflush thread was running continuously at ~10% of one CPU (there 
are 8cores/16threads on this box), the system was never reaching goal of 
vm.inactivepct=33.

Changing inactivepct to 10 didn't really help.  Even though eventually 
the system reached 11GB of Inactive memory, the ioflush task continued 
to run at roughly the same level.

Some additional observations:

* xosview frequently reports that the system is running at near-100%
  in SYS time.
* There are periods where USB keyboard and mouse events are getting
  dropped, which makes it really difficult to play tetris(6)!
* There are also periods when the kernel's vdrain thread runs at up
  to 10% or so of a single CPU thread.
* The pgdaemon thread does not appear to run very often, and never
  for any sustained period of time.

Discussing this on IRC, the suggestion was made that perhaps something 
in the vm system is sensitive to my having so much memory.  It seems 
that 128GB is more than 2^24 pages, so maybe there's a 24-bit assumption 
left hanging around?  It was also suggested that perhaps using 
uvm_pdpolicy_clockpro might help, but I'm unfamiliar with what it might 
do.

Hopefully someone knowledgeable about this part of the system will have 
some insight on what might be wrong...  :)


+------------------+--------------------------+----------------------------+
| Paul Goyette     | PGP Key fingerprint:     | E-mail addresses:          |
| (Retired)        | FA29 0E3B 35AF E8AE 6651 | paul at whooppee dot com   |
| Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd dot org |
+------------------+--------------------------+----------------------------+