Kazushi (Jam) Marukawa <jam@pobox.com>:
>I don't understand what the limit on the "UVM amap"
>allocation is.  Is it well known problem?  If so, why does
>the limit make KMEMSTATS wrong?  In addition, the system
>without UVM works fine with this KMEMSTATS.

i'm not sure if this problem can be called well known, but it
is understood.  the problem here is that UVM has a different 
kernel memory usage pattern than BSD VM and that has a bad 
interaction with the BSD malloc() code.

when you allocate and free memory in the kernel with malloc, 
you specify the "type" of memory you are going to allocate.
this "type" info is only used if KMEMSTATS is defined.
(KMEMSTATS is what is used to produce the "vmstat -m" output.)
in addition to keeping track of what malloc'd kernel memory 
is used for, KMEMSTATS also limits the amount of memory that
can be allocated for each "type."  this is causing malloc()'s
for the "UVM amap" data structure to hang if KMEMSTATS is defined
(but other types of mallocs would go through).  unfortunately, 
UVM needs amaps in order to do copy-on-write and zero-fill
memory, so if those allocations block you've got a problem.

that explains why turning of KMEMSTATS fixed your first problem.
however, you hit on a second problem, the amount of kernel virtual 
memory memory set aside for malloc() is small.   the size of this
memory can be adjusted by changing NKMEMCLUSTERS.   

basically, the more active anonymous virtual memory you've got 
on a system the more amap space you'll need.  in the design of 
an amap module there is a tradeoff between efficiency and amount 
of kernel memory allocated.   in the default amap module i've chosen
to allocate more kernel memory in the hopes of gaining efficiency.


>The system looks like good without KMEMSTATS.  But the
>system with KMEMSTATS should work, doesn't it?  

yes, a system with KMEMSTATS should work ... clearly some 
re-adjustment is called for.   i will look into it when i get
a chance.   in the mean time, i advise adjusting KMEMSTATS and
NKMEMCLUSTERS as needed.


chuck