Subject: Re: Kernel memory leak?
To: None <tech-kern@NetBSD.org>
From: Matthias Scheler <tron@zhadum.org.uk>
List: tech-kern
Date: 06/25/2007 08:26:51
On Sat, Jun 16, 2007 at 12:53:56PM +0100, Matthias Scheler wrote:
> I've installed a new kernel with that option activated. But the reboot
> will have to wait until somebody is nearby to press the reset button.

Here is the output of "vmstat -m" a few minutes before the machine froze.
The machine was running "cvs update" at that point of time:

Memory statistics by bucket size
    Size   In Use   Free   Requests  HighWater  Couldfree
      16     2326   1258    4947975    1280          0
      32     1035    373     389987     640          0
      64     6271     65     481909     320        228
     128      677     59     201618     160          0
     256      189     51      95205      80          0
     512      374     50      58426      40       1825
    1024      371     81     188824      20      81540
    2048       42     24      25984      10      15623
    4096       50    118       5520       5       5387
    8192       22     34        922       5        780
   16384       23      0        692       5          0
   32768        5      0        765       5          0
   65536       32      0         38       5          0
  131072       12      0    3829970       5          0
  262144        7      0          7       5          0

Memory usage type by bucket size
    Size  Type(s)
      16  acpi, USB, VM swap, UVM aobj, UVM amap, ifaddr, vnodes, namecache,
	  soname, packet tags, pipe, sysctldata, in_multi, ip_moptions,
	  UFS mount, routetbl, temp, pcb, devbuf, exec, kernfs mount, RAIDframe,
	  xform
      32  acpi, USB, AGP, VM swap, UVM amap, ifmedia, ether_multi, ifaddr,
	  soname, proc, routetbl, temp, pcb, devbuf, exec, kevent, RAIDframe,
	  xform
      64  acpi, USB, UVM amap, vnodes, pipe, prop, in_multi, Export Host,
	  routetbl, temp, pcb, DMA map, devbuf, MFS node, NDP
     128  acpi, USB device, USB, AGP, VM swap, UVM amap, ifaddr, soname, ttys,
	  prop, sysctlnode, Export Host, routetbl, temp, devbuf, kevent,
	  RAIDframe, key mgmt, NDP
     256  acpi, USB, UVM amap, ifaddr, vnodes, subproc, proc, routetbl, temp,
	  DMA map, devbuf, exec, kevent, file desc, newblk, NFS daemon
     512  acpi, USB device, VM page, VM map, UVM amap, ifaddr, proc, UFS mount,
	  pcb, DMA map, devbuf, ioctlops, file desc, NFS srvsock, RAIDframe,
	  crypto
    1024  acpi, UVM aobj, UVM amap, ttys, sysctlnode, proc, UFS mount, temp,
	  devbuf, exec, kevent, NQNFS Lease, RAIDframe, key mgmt
    2048  UVM amap, ifaddr, sysctlnode, in_multi, UFS mount, temp, devbuf, exec,
	  RAIDframe
    4096  UVM amap, mount, sysctlnode, temp, devbuf, RAIDframe
    8192  UVM amap, sysctlnode, UFS mount, temp, RAIDframe
   16384  UVM amap, sysctlnode, UFS mount, devbuf
   32768  UVM amap, namecache, RAIDframe
   65536  UVM amap, UFS mount, temp, devbuf, pagedep, RAIDframe
  131072  devbuf, MSDOSFS mount, RAIDframe
  262144  namecache, UFS mount, UFS quota, inodedep, NFS node, ISOFS mount

Memory statistics by type                           Type  Kern
         Type  InUse MemUse HighUse  Limit Requests Limit Limit Size(s)
          acpi  1450    95K     95K157287K     2739    0     0  16,32,64,128,256,512,1024
    USB device     8     4K      4K157287K        8    0     0  128,512
           USB    31     3K      3K157287K       31    0     0  16,32,64,128,256
           AGP     2     1K      1K157287K        2    0     0  32,128
       VM swap     5     1K      1K157287K        5    0     0  16,32,128
       VM page     1     1K      1K157287K        1    0     0  512
        VM map     4     2K      2K157287K        4    0     0  512
      UVM aobj     2     2K      2K157287K        2    0     0  16,1024
      UVM amap  6207  1043K   1529K157287K  2045641    0     0  16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536
       ifmedia     6     1K      1K157287K        6    0     0  32
   ether_multi     9     1K      1K157287K        9    0     0  32
        ifaddr    47    14K     14K157287K       47    0     0  16,32,128,256,512,2048
         mount     8    32K     36K157287K       14    0     0  4096
        vnodes  1957   128K    128K157287K     9931    0     0  16,64,256
     namecache     4   545K    545K157287K        4    0     0  16,32768,262144
        soname     8     1K      3K157287K    48030    0     0  16,32,128
   packet tags     0     0K      1K157287K    21506    0     0  16
          ttys   356   231K    231K157287K      356    0     0  128,1024
          pipe     0     0K      1K157287K      197    0     0  16,64
          prop     2     1K      1K157287K        2    0     0  64,128
    sysctldata     0     0K      1K157287K        1    0     0  16
    sysctlnode    60    97K     97K157287K       85    0     0  128,1024,2048,4096,8192,16384
       subproc    26     7K     11K157287K    11053    0     0  256
          proc    10     2K      2K157287K     2488    0     0  32,256,512,1024
      in_multi    67     5K      5K157287K       67    0     0  16,64,2048
   ip_moptions     1     1K      1K157287K        1    0     0  16
   Export Host     6     1K      1K157287K        6    0     0  64,128
     UFS mount    22   519K    573K157287K       29    0     0  16,512,1024,2048,8192,16384,65536,262144
      routetbl    96     7K      7K157287K      204    0     0  16,32,64,128,256
          temp    24    62K     78K157287K   121290    0     0  16,32,64,128,256,1024,2048,4096,8192,65536
           pcb    36     5K      7K157287K    55060    0     0  16,32,64,512
       DMA map   410   139K    139K157287K      410    0     0  64,256,512
        devbuf   339   324K    324K157287K    17221    0     0  16,32,64,128,256,512,1024,2048,4096,16384,65536,131072
          exec     4     1K     11K157287K    39450    0     0  16,32,256,1024,2048
        kevent     9     4K      4K157287K    75265    0     0  32,128,256,1024
      ioctlops     0     0K      1K157287K       33    0     0  512
     file desc    28    10K     26K157287K    11205    0     0  256,512
     UFS quota     1   256K    256K157287K        1    0     0  262144
      MFS node     1     1K      1K157287K        1    0     0  64
        newblk     1     1K      1K157287K        1    0     0  256
      inodedep     1   256K    256K157287K        1    0     0  262144
       pagedep     1    64K     64K157287K        1    0     0  65536
   NFS srvsock     2     1K      1K157287K        2    0     0  512
    NFS daemon     1     1K      1K157287K        1    0     0  256
   NQNFS Lease     1     1K      1K157287K        1    0     0  1024
      NFS node     1   256K    256K157287K        1    0     0  262144
  kernfs mount     1     1K      1K157287K        1    0     0  16
 MSDOSFS mount     1   128K    128K157287K        1    0     0  131072
   ISOFS mount     1   256K    256K157287K        1    0     0  262144
     RAIDframe    65  2610K   2738K157287K  7659997    0     0  16,32,128,512,1024,2048,4096,8192,32768,65536,131072
      key mgmt   102    78K    156K157287K    93592    0     0  128,1024
           NDP    10     1K      2K157287K       15    0     0  64,128
        crypto     1     1K      1K157287K        1    0     0  512
         xform     0     0K      1K157287K    11824    0     0  16,32

Memory totals:  In Use    Free    Requests
                 7179K    954K    10227845

Memory resource pool statistics
Name        Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
phpool-0      40     4413    0     2435    22     1    21    22     0   inf    0
phpool-32     72      392    0       29     7     0     7     7     0   inf    0
pcgpool      140     1180    0     1163     1     0     1     1     0   inf    0
pmappl        76    39850    0    39792     2     0     2     2     0   inf    0
pdppl       4096      786    0      703   439   356    83   103     0   inf    0
kvakernel   4096    11329    0     1683   304     1   303   304     0     0    0
kvakmem     4096     4978    0     3085    88    28    60    60     0     0    0
inpcbpl      108    91872    0    91830     7     3     4     4     0   inf    2
inmltpl       32        2    0        0     1     0     1     1     0   inf    0
tcpipqepl     44    12474    0    12474   126   125     1     1     0   inf    1
synpl        184    29673    0    29673   100    99     1     1     0   inf    1
sackholepl    20       28    0       28     2     1     1     1     0   inf    1
tcpcbpl      360    38125    0    38084    25    14    11    11     0   inf    5
in6pcbpl     156     1692    0     1660     3     0     3     3     0   inf    0
ffsinopl     196  1889458    0  1825784  3194     0  3194  3194     0   inf    0
dino1pl      128  1889458    0  1825784  2061     0  2061  2061     0   inf    0
ufsdirpl     264     9835    0     9835    95    94     1     1     0   inf    1
rndsample    528   173026    0   173021     1     0     1     1     0   inf    0
filepl        56  1642604    0  1642332    10     3     7     7     0   inf    3
cwdipl        16    39851    0    39793     1     0     1     1     0   inf    0
fdescpl      284    39851    0    39793    11     4     7     8     0   inf    1
kqueuepl     260        3    0        0     1     0     1     1     0   inf    0
knotepl       72       25    0        0     1     0     1     1     0   inf    0
procpl       408    39866    0    39794    42    32    10    12     0   inf    1
lwppl        132    39904    0    39800    33    28     5     5     0   inf    0
pgrppl        16     4361    0     4333     1     0     1     1     0   inf    0
pcredpl       24    39866    0    39794     1     0     1     1     0   inf    0
plimitpl     176      285    0      274     1     0     1     1     0   inf    0
pstatspl     224    39866    0    39794    36    30     6     7     0   inf    0
rusgepl       72    39794    0    39794   115   114     1     1     0   inf    1
raspl         20      527    0      500     1     0     1     1     0   inf    0
sadatapl      40        5    0        1     1     0     1     1     0   inf    0
saupcpl     1612  1935577    0  1935568   184   173    11    11     0   inf    6
sastackpl     24       80    0       16     1     0     1     1     0   inf    0
savppl        52        5    0        1     1     0     1     1     0   inf    0
ptimerpl     124    11058    0    11028     2     0     2     2     0   inf    0
sessionpl     40     4287    0     4261     1     0     1     1     0   inf    0
credpl        84   130878    0   130844    28    26     2     2     0   inf    0
siginfo      128     8557    0     8557   148   147     1     1     0   inf    1
ksiginfo      44        5    0        1     1     0     1     1     0   inf    0
pipepl       144    64512    0    64475     4     2     2     3     0   inf    0
ttypl        264       71    0        0     5     0     5     5     0   inf    0
sockpl       216   148610    0   148459    20     7    13    13     0   inf    3
ncachepl      84  1620528    0  1556774  1331     0  1331  1331     0   inf    0
lockfpl       52 51421512    0 51421493     1     0     1     1     0   inf    0
vnodepl      168    71949    0     8264  2662     0  2662  2662     0   inf    0
rtentpl      132       48    0        0     2     0     2     2     0   inf    0
amappl        40   633655    0   631640    31     6    25    27     0   inf    1
aobjpl        52        1    0        0     1     0     1     1     0   inf    0
vmsppl       224    39850    0    39792     7     2     5     6     0   inf    0
vmmpepl       88  4905771    0  4901816   295   168   127   163     0   inf   36
ataspl        52 11073491    0 11073491     1     0     1     1     0   inf    1
bufpl        124 13385275    0 13328721  3401  1632  1769  1788     0   inf    1
buf1k       1024   101403    0    53719  1476   730   746   746     1     1    0
buf2k       2048     1903    0     1238    37    16    21    21     1     1    0
buf4k       4096      960    0      666    34    15    19    19     1     1    0
buf8k       8192    27346    0    19435  1866   877   989  1161     1     1    0
mbpl         256     5009    0     4699    66    45    21    29     1   inf    1
mclpl       2048     1148    0      989   434   350    84    84     4  1024    4
pnbufpl     1024      561    0      556    16    14     2     2     0   inf    0
extent        20       25    0        2     1     0     1     1     0   inf    0
raidpl       132 10371585    0 10371585    28    25     3     3     2     4    3
sigapl      2052    39851    0    39793   567   484    83   103     0   inf   25
rf_alloclist_pl 168 6395377  0  6395376    17    13     4     4     3    11    3
rf_asmhdr_pl  16 10118212    0 10118212     1     0     1     1     1     2    1
rf_asm_pl    288 10225334    0 10225334    39    33     6     6     5    14    6
rf_pda_pl     40 20450668    0 20450668     1     0     1     1     1     2    1
rf_vple_pl    12       30    0        0     1     0     1     1     1     2    0
rf_rad_pl    328  2458296    0  2458296   138   134     4     4     3    11    4
rf_stripelock_pl 28 2565418  0  2565418     1     0     1     1     1     2    1
rf_mcpair_pl  12  3829958    0  3829958     1     0     1     1     1     2    1
rf_dagnode_pl 444 29557713   0 29557713   568   547    21    22    15    57   21
rf_dagh_pl    76  6395376    0  6395376     1     0     1     1     1     3    1
rf_daglist_pl 236 2565418    0  2565418   212   209     3     3     2     8    3
rf_funclist_pl 12 2565418    0  2565418     1     0     1     1     1     2    1
rf_dqd_pl    116 10371585    0 10371585   201   198     3     3     2     8    3

In use 157751K, total allocated 160480K; utilization 98.3%

Does that shed light on the problem?

	Kind regards

-- 
Matthias Scheler                                  http://zhadum.org.uk/