Subject: Compiling MLton for NetBSD
To: None <netbsd-users@netbsd.org>
From: Jesper Louis Andersen <jlouis@mongers.org>
List: netbsd-users
Date: 12/06/2004 13:59:51
www.mlton.org is a compiler for the Standard ML language. The 
compiler uses whole-program compilation, ie, it loads all source
files and compiles them all in one go. This, tied with the Garbage
collector it uses makes the compiler use quite a lot of anonymous
memory. 

This is not in itself a problem. My box has 640Mb, which should be
adequate since a self-compile runs perfectly in 512Mb of RAM. However,
I would like to know how I can tune it against swapping. First of all,
I lock the program with a ulimit -d 480000. This is adequate for the
heap size while compiling. The GC will ensure we keep below this line.
But the system keeps swapping, even when this is set.

The next tunable I set is sysctl -w vm.anonmax=99. This is agressive,
but it makes the system swap much less. I am wondering if there is
anything else I should tune? The application maps a very big heap, but
has no other things it needs. The compilers exec is a modest 11Mb. But
as of now, the line in top says:

PID   USERNAME PRI NICE SIZE RES  STATE TIME WCPU   CPU    COMMAND
16767 jlouis   39  0    11M  435M RUN   1:07 21.19% 21.19% mlton-compil

It swaps a damn lot while compiling, though it should keep within the
bounds of the systems memory.

vmstat -s output at the bottom of the mail. I wonder if there is
anything else good for diagnostics of a system.

vmstat -s:
     4096 bytes per page
       16 page colors
   159222 pages managed
      113 pages free
    90153 pages active
    44851 pages inactive
        0 pages paging
      208 pages wired
        6 zero pages
        1 reserve pagedaemon pages
        5 reserve kernel pages
   132811 anonymous pages
    18025 cached file pages
     4624 cached executable pages
       64 minimum free pages
       85 target free pages
    45017 target inactive pages
    53074 maximum wired pages
        1 swap devices
  262236 swap pages
   117454 swap pages in use
   202336 swap allocations
   411441 anons
   276068 free anons
 68531991 total faults taken
 81901595 traps
 28840522 device interrupts
 29777100 CPU context switches
 33913298 software interrupts
172034996 system calls
   202240 pagein requests
    15579 pageout requests
     1960 swap ins
     1984 swap outs
        0 pages swapped in
   238672 pages swapped out
   597495 forks total
   474886 forks blocked parent
   474919 forks shared address space with parent
  2305503 pagealloc zero wanted and avail
 37752689 pagealloc zero wanted and not avail
        0 aborts of idle page zeroing
 49255562 pagealloc desired color avail
   128602 pagealloc desired color not avail
      972 faults with no memory
        0 faults with no anons
       34 faults had to wait on pages
        0 faults found released page
   266552 faults relock (266359 ok)
  8157845 anon page faults
   202616 anon retry faults
  5526253 amap copy faults
  3136028 neighbour anon page faults
 72992375 neighbour object page faults
 24278995 locked pager get faults
    63927 unlocked pager get faults
  4995231 anon faults
  1301959 anon copy on write faults
 18255783 object faults
  6023027 promote copy faults
 36913274 promote zero fill faults
     6513 times daemon wokeup
     5239 revolutions of the clock hand
     5238 times daemon attempted swapout
  1419722 pages freed by daemon
  4043106 pages scanned by daemon
   239656 anonymous pages scanned by daemon
   613925 object pages scanned by daemon
  2528420 pages reactivated
   313534 anonymous pages reactivated
   200001 cached file pages reactivated
   118513 cached executable pages reactivated
    29057 pages found busy by daemon
        0 total pending pageouts
  5012874 pages deactivated
 48212376 total name lookups
 42181445 good hits
  1999119 negative hits
    96443 bad hits
        0 false hits
  3922848 miss
    12521 too long
  1033944 pass2 hits
  1068066 2passes
          cache hits (87% pos + 4% neg) system 2% per-process
          deletions 0%, falsehits 0%, toolong 0%

-- 
jlouis