Subject: Re: Kernel resident memory size is quite large
To: None <antiright@gmail.com>
From: Havard Eidnes <he@NetBSD.org>
List: port-amd64
Date: 03/29/2006 15:20:28
> Hello.
> Is there any reason why the memory used by the kernel is so large?
> Top is showing that kernel processes use 233MB of memory.
> The system has 1GB, so the amount used by the kernel is a large perce=
ntage.
> Does this memory include the buffer cache, or is it occupied by the k=
ernel
> itself?

First I have to ask how you read top's output to come to the
conclusion you do.  E.g. you can't simply add the "RES" sizes of
the processes with [] around their names; as far as I know, these
processes all share the same virtual address space.

However, with that said, it is quite normal for the kernel's data
structures to occupy a major portion of physical memory, since this
includes the area for buffering file data, vnodes etc. etc.

Below I include three small shell scripts which are part of a not yet
committed NetBSD adaptation of the client portion of the Munin package
which might give some insight into what the memory is used for.

Best regards,

- H=E5vard

------------------------------


One way to look at the total memory consumption could be:

------------------------------
PAGESIZE=3D`/sbin/sysctl -n hw.pagesize`
MEMSIZE=3D`vmstat -s | awk '/pages managed/ { print $1 }'`
MEMMAX=3D`echo 2k $PAGESIZE $MEMSIZE '*p' | dc`

vmstat -s | awk -v bpp=3D$PAGESIZE '
/pages managed$/  { managed =3D $1; }
/pages free$/     { free =3D $1;     print "free.value "     $1 * bpp; =
}
/anonymous pages$/{ anon =3D $1;     print "anon.value "     $1 * bpp; =
}
/cached file pages$/{ file =3D $1;   print "file.value "     $1 * bpp; =
}
/cached executable pages$/{ exec =3D $1; print "exec.value " $1 * bpp; =
}
END { kernel =3D managed - anon - file - exec - free;
    print "kernel.value " kernel * bpp; }'
------------------------------

An alternative way to look at the total memory consumption:

------------------------------
PAGESIZE=3D`/sbin/sysctl -n hw.pagesize`
MEMSIZE=3D`vmstat -s | awk '/pages managed/ { print $1 }'`
MEMMAX=3D`echo 2k $PAGESIZE $MEMSIZE '*p' | dc`

vmstat -s | awk -v bpp=3D$PAGESIZE '
/pages managed$/  { managed =3D $1; }
/pages free$/     { free =3D $1;     print "free.value "     $1 * bpp; =
}
/pages active$/   { active =3D $1;   print "active.value "   $1 * bpp; =
}
/pages inactive$/ { inactive =3D $1; print "inactive.value " $1 * bpp; =
}
/pages wired$/    { wired =3D $1;    print "wired.value "    $1 * bpp; =
}
END { kernel =3D managed - wired - inactive - active - free;
    print "kernel.value " kernel * bpp; }'
------------------------------

A closer breakup showing some of the kernel's major memory areas:

------------------------------
PAGESIZE=3D`/sbin/sysctl -n hw.pagesize`

vmstat -m -W > /dev/null 2>&1
if [ $? -eq 0 ]; then
        vmstat -m -W 2>/dev/null | awk '
/^Memory resource pool statistics/ { inpool=3D1; next; }
/^In use/                          { inpool=3D0; next; }
/^Name/ { next; }
/^vnodepl/ &&  inpool   { vnode  =3D $10 * $11; next; }
/^ncachepl/ && inpool   { ncache =3D $10 * $11; next; }
/^ffsinopl/ && inpool   { ffsino =3D $10 * $11; next; }
/^dino1pl/ &&  inpool   { dinopl =3D $10 * $11; next; }
/^buf.*k / &&  inpool   { bufk  +=3D $10 * $11; next; }
/^bufpl/   &&  inpool   { bufpl  =3D $10 * $11; next; }
/^mbpl/    &&  inpool   { mbufs +=3D $10 * $11; next; }
/^mclpl/   &&  inpool   { mbufs +=3D $10 * $11; next; }
$16 !=3D "0x200" && $16 !=3D "0x600" && inpool { rest  +=3D $10 * $11; =
}
END {
        print "bufs.value " bufk;
        print "bufpl.value " bufpl;
        print "vnode.value " vnode;
        print "ncache.value " ncache;
        print "ffsino.value " ffsino;
        print "mbufs.value " mbufs;
        print "rest.value " rest;
}'
else
        # Need to guess size of pool pages, may not be correct
        # for small-memory machines (< 16MB)
        vmstat -m 2>/dev/null | awk -v bpp=3D$PAGESIZE '
/^Memory resource pool statistics/ { inpool=3D1; next; }
/^In use/                          { inpool=3D0; next; }
/^Name/                            { next; }
/^vnodepl/ &&  inpool   { vnode  =3D $8 * bpp; next; }
/^ncachepl/ && inpool   { ncache =3D $8 * bpp; next; }
/^ffsinopl/ && inpool   { ffsino =3D $8 * bpp; next; }
/^dino1pl/ &&  inpool   { dinopl =3D $8 * bpp; next; }
/^buf.*k / &&  inpool   { bufk  +=3D $8 * 65536; next; }
/^bufpl/    && inpool   { bufpl  =3D $8 * bpp; next; }
/^mbpl/    &&  inpool   { mbufs +=3D $8 * bpp; next; }
/^mclpl/   &&  inpool   { mbufs +=3D $8 * bpp; next; }
/^kvakernel/            { next; }
/^kvakmem/              { next; }
inpool                  { rest  +=3D $8 * bpp; }
END {
        print "bufs.value " bufk;
        print "bufpl.value " bufpl;
        print "vnode.value " vnode;
        print "ncache.value " ncache;
        print "ffsino.value " ffsino;
        print "mbufs.value " mbufs;
        print "rest.value " rest;
}'
fi
------------------------------