Subject: Re: failing to keep a process from swapping
To: Chris Gilbert <chris@dokein.co.uk>
From: Arto Selonen <arto@selonen.org>
List: tech-kern
Date: 10/26/2004 14:55:14
Hi!

On Sat, 23 Oct 2004, Chris Gilbert wrote:

> On Fri, 22 Oct 2004 13:18:52 +0300 (EEST)
> Arto Selonen <arto@selonen.org> wrote:

> > Basically, what I am trying to achieve is to keep 'squid' in memory.
> > I've set the following limits (after having sent the above email):
> >
> > 	vm.anonmin = 65
> > 	vm.execmin = 2
> > 	vm.filemin = 10
> > 	vm.anonmax = 80
> > 	vm.execmax = 5
> > 	vm.filemax = 15
>
> They would seem reasonable values for what you're trying to do.

Except that all this time I've been assuming (yes, it is a bad verb)
that on a 1 GB system 50% would be ~500 MB. After checking sysctl(3):

"The percentage of physical memory"

So, why do the values get used like this (in uvm_pdaemon.c):

        t = uvmexp.active + uvmexp.inactive + uvmexp.free;
        anonunder = (uvmexp.anonpages <= (t * uvmexp.anonmin) >> 8);

uvmexp.active + uvmexp.inactive + uvmexp.free != physical memory ?

Even if one wanted to exclude all the memory that the kernel is using
(and not understanding UVM I don't see why one would want to do that),
shouldn't there still be at least uvmexp.wired included in 't'?

On my running 1GB system, active+inactive+free ~ 73%. With less than 1%
wired pages, that makes quite a difference to my limits. Is there any
other way to control that extra amount (kernel usage?) besides
BUFCACHE/BUFPAGES? (I had vm.bufcache as 15%, and reducing it via sysctl
down to the hard coded lower limit of 5% only caused that "missing" memory
usage to drop by ~50MB). What does this cover:

uvmexp.npages-uvmexp.active-uvmexp.inactive-uvmexp.wire-uvmexp.free=?

I haven't (yet) figured out the vm.bufcache usage, so I am open to
any help regarding that. What is it used for? According to code,
the size is calculated as a percentage of physical memory (or set
according to BUFPAGES), but how does that differ from "file cache"?
How much of physical memory does the kernel use *at minimum*, in
per cents?

> Might be worth watching the output of sysutils/xuvmstat see if it
> shows anything odd (although not sure how I'd define odd)  Probably

That was definately useful, as it showed a lot of 'allocd' pages, and
it took me a while to realize how the numbers are supposed to add up
(ie. look at xuvmstat.c to see what that 'allocd' is).

> The code that decides what to keep and what to throw out is in
> uvm_pdaemon, uvmpd_scan_inactive(), so reading that might help point you
> at what's going wrong.

I am starting to get a feeling that the numbers (related to UVM, either in
units of pages or bytes) don't add up the way they should according to
documentation, and that maybe there is confusion over *how* the numbers
should add up.

"And all I wanted was to set the limits so that the system would use RAM
in the most efficient manner, based on known usage profile. Why do I
need to look at the code, and somebody's dissertation thesis, to be able
to tune system memory usage?"


Artsi
-- 
#######======------  http://www.selonen.org/arto/  --------========########
Everstinkuja 5 B 35                               Don't mind doing it.
FIN-02600 Espoo        arto@selonen.org         Don't mind not doing it.
Finland              tel +358 50 560 4826     Don't know anything about it.