tech-kern archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Running out of memory in 4.0?



Hi,

we have a system which runs as a Xen domU where we have been
experiencing a lack of stability.  It will appear to hang with
irregular frequency.  The domU uses NFS for all its I/O, including
swap.

We have had two separate types of symptoms, which probably both
have the same underlying cause.

In the first instance, the kernel would sometimes grind to a halt
endlessly spewing these messages on the console:

   xennet: rx no mbuf
   xennet: rx no mbuf
   xennet: rx no mbuf

One of the times it did this, I managed to get into DDB, and,
yes, indeed, it had run completely out of free pages, according
to the "show uvmexp" output ("0 free" in the "show uvm" output).

So...  Patterned after a patch I've applied to another 4.0 SMP
machine running directly on the iron (and where it appears to
have stabilized the host), we applied this:

--- uvm_page.c  27 Sep 2006 17:18:50 -0000      1.114
+++ uvm_page.c  1 Dec 2008 15:29:58 -0000
@@ -419,8 +419,8 @@
         * init various thresholds.
         */
 
-       uvmexp.reserve_pagedaemon = 1;
-       uvmexp.reserve_kernel = 5;
+       uvmexp.reserve_pagedaemon = 2;
+       uvmexp.reserve_kernel = 20;
 
        /*
         * determine if we should zero pages in the idle loop.

However, this didn't really fix the underlying cause for the
problems, just changed the symptom.  At one time I managed to
eyeball a window running "systat vm" before the system wedged, and I
saw system-cpu skyrocketing to 90% or more, with only 80KB of free
physical memory.

The symptom now is that the machine simply wedges.  Breaking into
DDB when it hangs shows that the number of free pages is always
below 20, and that the "woke" counter for the page daemon constantly
increases, but the "freed" count remains constant.  Also the
majority of the pages appear to be counted in the "pending" counter.
Some debugging info also displaying the output of "show all pools"
as well as some repeated "show uvm" and "ps" and "ps /l" follows
attached in the first attachment below.  I don't think there's
anything out of the ordinary in the pools output or the ps listings.

I've also attached some more "show uvm" output in the second
attachment below from a separate incident.


Now, this is where the facts stop and my guesswork and questions
begin.

It would appear that the system has managed to fill the entire
memory with dirty pages, and that it's unable to page out any of
them, possibly because the NFS code wants to allocate some memory
for writing out some pages?  (The old classical "you need to
allocate some memory to free memory" wedge in new(?) wrapping.)

It's as if there's some missing pushback against coming into this
situation which the system can't cope with, and I consider this to
be a bug.

Do we have some form of instrumentation which could be used to
show how many dirty pages are being held in memory?

I initially suspected that the memory was being overrun by dirty
file pages, but looking at the "show uvm" output, it actually
appears that anon memory dominates.

The VM susbsystem has a few knobs one can tweak, but we run with the
default limits, which according to "sysctl -a are

vm.anonmin = 10
vm.filemin = 10
vm.execmin = 5
vm.anonmax = 80
vm.filemax = 50
vm.execmax = 30
vm.inactivepct = 33
vm.bufcache = 15

However, I never did properly understand from an operator
perspective what these settings actually do -- from what I recall
they are apparently not "hard limits", only some form of "general
guidelines" for the VM subsystem(?)  So what effect adjusting any
of them would have is ... somewhat mysterious, and whether
adjusting some of them would prevent this problem from occurring
is dubious at best.  Since I'm after a "real fix" for this
problem, suggesting I adjust these probably falls into the
"half-way workaround which should not be necessary" category.

Isn't the syncer supposed to flush out at least dirty file pages, so
that they don't accumulate without bounds, and so that they end up
on stable storage within some reasonable time?

Isn't purely diskless systems supposed to work reliably without
falling into this hole which it can't crawl out of by itself?


We have a saying which translated is approximately "a fool can
ask more questions than ten wise men can answer", I suspect it
applies here...


Comments are welcome.

Best regards,

- Håvard
db> show all pools
POOL brtpl: size 32, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 126, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL ccdpl: size 140, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 29, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL nfsrvdescpl: size 180, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 22, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL nfsreqcachepl: size 36, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 112, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL pnbufpl: size 1024, align 4, ioff 0, roflags 0x00000000
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 4, nitems 3, nout 1, hardlimit 4294967295

        nget 980, nfail 0, nput 979
        npagealloc 45, npagefree 44, hiwat 4, nidle 0
POOL sigapl: size 2052, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 46
        itemsperpage 1, nitems 0, nout 46, hardlimit 4294967295

        nget 426864, nfail 0, nput 426818
        npagealloc 1157, npagefree 1111, hiwat 80, nidle 0
POOL mclpl: size 2048, align 4, ioff 0, roflags 0x00000000
        alloc 0xc0441b00
        minitems 8, minpages 4, maxpages 512, npages 4
        itemsperpage 2, nitems 8, nout 0, hardlimit 1024

        nget 7695, nfail 0, nput 7695
        npagealloc 794, npagefree 790, hiwat 44, nidle 4
POOL mbpl: size 256, align 4, ioff 0, roflags 0x00000000
        alloc 0xc04417c0
        minitems 16, minpages 1, maxpages 4294967295, npages 17
        itemsperpage 16, nitems 23, nout 249, hardlimit 4294967295

        nget 20171, nfail 0, nput 19922
        npagealloc 220, npagefree 203, hiwat 24, nidle 1
POOL buf32k: size 32768, align 4, ioff 0, roflags 0x00000000
        alloc 0xc0441d00
        minitems 1, minpages 1, maxpages 1, npages 1
        itemsperpage 1, nitems 1, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 1, npagefree 0, hiwat 1, nidle 1
POOL buf16k: size 16384, align 4, ioff 0, roflags 0x00000000
        alloc 0xc0441d00
        minitems 1, minpages 1, maxpages 1, npages 1
        itemsperpage 2, nitems 2, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 1, npagefree 0, hiwat 1, nidle 1
POOL buf8k: size 8192, align 4, ioff 0, roflags 0x00000000
        alloc 0xc0441d00
        minitems 1, minpages 1, maxpages 1, npages 308
        itemsperpage 4, nitems 33, nout 1199, hardlimit 4294967295

        nget 8893, nfail 0, nput 7694
        npagealloc 1517, npagefree 1209, hiwat 1465, nidle 0
POOL buf4k: size 4096, align 4, ioff 0, roflags 0x00000000
        alloc 0xc0441d00
        minitems 1, minpages 1, maxpages 1, npages 1
        itemsperpage 8, nitems 8, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 1, npagefree 0, hiwat 1, nidle 1
POOL buf2k: size 2048, align 4, ioff 0, roflags 0x00000000
        alloc 0xc0441d00
        minitems 1, minpages 1, maxpages 1, npages 1
        itemsperpage 16, nitems 16, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 1, npagefree 0, hiwat 1, nidle 1
POOL buf1k: size 1024, align 4, ioff 0, roflags 0x00000000
        alloc 0xc0441d00
        minitems 1, minpages 1, maxpages 1, npages 8
        itemsperpage 32, nitems 27, nout 229, hardlimit 4294967295

        nget 550, nfail 0, nput 321
        npagealloc 16, npagefree 8, hiwat 10, nidle 0
POOL buf512b: size 512, align 4, ioff 0, roflags 0x00000000
        alloc 0xc0441d00
        minitems 1, minpages 1, maxpages 1, npages 1
        itemsperpage 64, nitems 64, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 1, npagefree 0, hiwat 1, nidle 1
POOL kmem-4: size 4, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 15
        itemsperpage 128, nitems 308, nout 1612, hardlimit 4294967295

        nget 22066, nfail 0, nput 20454
        npagealloc 18, npagefree 3, hiwat 16, nidle 0
POOL kmem-8: size 8, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 8
        itemsperpage 64, nitems 89, nout 423, hardlimit 4294967295

        nget 8853, nfail 0, nput 8430
        npagealloc 9, npagefree 1, hiwat 9, nidle 0
POOL kmem-12: size 12, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 4
        itemsperpage 42, nitems 36, nout 132, hardlimit 4294967295

        nget 1926, nfail 0, nput 1794
        npagealloc 9, npagefree 5, hiwat 5, nidle 0
POOL kmem-16: size 16, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 3
        itemsperpage 32, nitems 22, nout 74, hardlimit 4294967295

        nget 1484, nfail 0, nput 1410
        npagealloc 9, npagefree 6, hiwat 5, nidle 0
POOL kmem-20: size 20, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 2
        itemsperpage 25, nitems 20, nout 30, hardlimit 4294967295

        nget 1141, nfail 0, nput 1111
        npagealloc 16, npagefree 14, hiwat 3, nidle 0
POOL kmem-24: size 24, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 4
        itemsperpage 21, nitems 45, nout 39, hardlimit 4294967295

        nget 1196, nfail 0, nput 1157
        npagealloc 5, npagefree 1, hiwat 5, nidle 0
POOL kmem-28: size 28, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 12
        itemsperpage 18, nitems 39, nout 177, hardlimit 4294967295

        nget 4368, nfail 0, nput 4191
        npagealloc 33, npagefree 21, hiwat 16, nidle 0
POOL kmem-32: size 32, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 11
        itemsperpage 16, nitems 47, nout 129, hardlimit 4294967295

        nget 3907, nfail 0, nput 3778
        npagealloc 21, npagefree 10, hiwat 15, nidle 0
POOL kmem-36: size 36, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 3
        itemsperpage 14, nitems 15, nout 27, hardlimit 4294967295

        nget 786, nfail 0, nput 759
        npagealloc 9, npagefree 6, hiwat 4, nidle 0
POOL kmem-40: size 40, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 6
        itemsperpage 12, nitems 35, nout 37, hardlimit 4294967295

        nget 1247, nfail 0, nput 1210
        npagealloc 18, npagefree 12, hiwat 9, nidle 0
POOL kmem-44: size 44, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 2
        itemsperpage 11, nitems 13, nout 9, hardlimit 4294967295

        nget 499, nfail 0, nput 490
        npagealloc 2, npagefree 0, hiwat 2, nidle 0
POOL kmem-48: size 48, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 10, nitems 7, nout 3, hardlimit 4294967295

        nget 481, nfail 0, nput 478
        npagealloc 3, npagefree 2, hiwat 2, nidle 0
POOL kmem-56: size 56, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 9, nitems 3, nout 6, hardlimit 4294967295

        nget 441, nfail 0, nput 435
        npagealloc 14, npagefree 13, hiwat 2, nidle 0
POOL kmem-64: size 64, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 63
        itemsperpage 8, nitems 75, nout 429, hardlimit 4294967295

        nget 9824, nfail 0, nput 9395
        npagealloc 147, npagefree 84, hiwat 85, nidle 0
POOL kmem-72: size 72, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 7, nitems 0, nout 0, hardlimit 4294967295

        nget 676, nfail 0, nput 676
        npagealloc 20, npagefree 20, hiwat 3, nidle 0
POOL kmem-84: size 84, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 6, nitems 0, nout 0, hardlimit 4294967295

        nget 860, nfail 0, nput 860
        npagealloc 27, npagefree 27, hiwat 4, nidle 0
POOL kmem-100: size 100, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 2
        itemsperpage 5, nitems 4, nout 6, hardlimit 4294967295

        nget 526, nfail 0, nput 520
        npagealloc 30, npagefree 28, hiwat 4, nidle 0
POOL kmem-128: size 128, align 4, ioff 0, roflags 0x00000c00
        alloc 0xc0dd10b4
        minitems 0, minpages 0, maxpages 4294967295, npages 2
        itemsperpage 4, nitems 5, nout 3, hardlimit 4294967295

        nget 755, nfail 0, nput 752
        npagealloc 45, npagefree 43, hiwat 11, nidle 0
POOL proparay: size 72, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 56, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL propdata: size 20, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 203, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL propdict: size 72, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 56, nitems 49, nout 7, hardlimit 4294967295

        nget 7, nfail 0, nput 0
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL pdict128: size 160, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 25, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL pdict32: size 64, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 63, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL pdict16: size 48, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 84, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL propnmbr: size 36, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 112, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL propstng: size 20, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 203, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL rttmrpl: size 32, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 126, nitems 0, nout 0, hardlimit 4294967295

        nget 1419, nfail 0, nput 1419
        npagealloc 15, npagefree 15, hiwat 1, nidle 0
POOL rtentpl: size 136, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 2
        itemsperpage 29, nitems 29, nout 29, hardlimit 4294967295

        nget 1548, nfail 0, nput 1519
        npagealloc 2, npagefree 0, hiwat 2, nidle 0
POOL vnodepl: size 172, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1422
        itemsperpage 23, nitems 1493, nout 31213, hardlimit 4294967295

        nget 54919, nfail 0, nput 23706
        npagealloc 1424, npagefree 2, hiwat 1424, nidle 0
POOL lockfpl: size 52, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 78, nitems 72, nout 6, hardlimit 4294967295

        nget 378764292, nfail 0, nput 378764286
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL ncachepl: size 84, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 651
        itemsperpage 48, nitems 179, nout 31069, hardlimit 4294967295

        nget 3743733, nfail 0, nput 3712664
        npagealloc 922, npagefree 271, hiwat 665, nidle 0
POOL biopl: size 124, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 32, nitems 30, nout 2, hardlimit 4294967295

        nget 374793547, nfail 0, nput 374793545
        npagealloc 78, npagefree 77, hiwat 11, nidle 0
POOL bufpl: size 124, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 53
        itemsperpage 32, nitems 268, nout 1428, hardlimit 4294967295

        nget 9290, nfail 0, nput 7862
        npagealloc 185, npagefree 132, hiwat 184, nidle 0
POOL sockpl: size 216, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 8
        itemsperpage 18, nitems 3, nout 141, hardlimit 4294967295

        nget 327658, nfail 0, nput 327517
        npagealloc 62, npagefree 54, hiwat 21, nidle 0
POOL ttypl: size 264, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 15, nitems 9, nout 6, hardlimit 4294967295

        nget 6, nfail 0, nput 0
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL shmmp: size 12, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 338, nitems 0, nout 0, hardlimit 4294967295

        nget 100322, nfail 0, nput 100322
        npagealloc 39, npagefree 39, hiwat 1, nidle 0
POOL pipepl: size 100, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 40, nitems 24, nout 16, hardlimit 4294967295

        nget 1205428, nfail 0, nput 1205412
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL vmembtpl: size 28, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 71
        itemsperpage 145, nitems 176, nout 10119, hardlimit 4294967295

        nget 23673, nfail 0, nput 13554
        npagealloc 72, npagefree 1, hiwat 71, nidle 0
POOL ptimerspl: size 140, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 29, nitems 15, nout 14, hardlimit 4294967295

        nget 5657, nfail 0, nput 5643
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL ptimerpl: size 124, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 32, nitems 17, nout 15, hardlimit 4294967295

        nget 6061, nfail 0, nput 6046
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL ksiginfo: size 44, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 92, nitems 0, nout 0, hardlimit 4294967295

        nget 2244975, nfail 0, nput 2244975
        npagealloc 9, npagefree 9, hiwat 1, nidle 0
POOL siginfo: size 128, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 31, nitems 0, nout 0, hardlimit 4294967295

        nget 62489, nfail 0, nput 62489
        npagealloc 64, npagefree 64, hiwat 1, nidle 0
POOL savppl: size 52, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 78, nitems 77, nout 1, hardlimit 4294967295

        nget 404, nfail 0, nput 403
        npagealloc 2, npagefree 1, hiwat 1, nidle 0
POOL sastackpl: size 24, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 169, nitems 153, nout 16, hardlimit 4294967295

        nget 6464, nfail 0, nput 6448
        npagealloc 2, npagefree 1, hiwat 1, nidle 0
POOL saupcpl: size 1612, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 2, nitems 1, nout 1, hardlimit 4294967295

        nget 59746984, nfail 0, nput 59746983
        npagealloc 82, npagefree 81, hiwat 3, nidle 0
POOL sadatapl: size 40, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 101, nitems 100, nout 1, hardlimit 4294967295

        nget 404, nfail 0, nput 403
        npagealloc 2, npagefree 1, hiwat 1, nidle 0
POOL raspl: size 20, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 203, nitems 192, nout 11, hardlimit 4294967295

        nget 307343, nfail 0, nput 307332
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL sessionpl: size 40, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 101, nitems 77, nout 24, hardlimit 4294967295

        nget 21793, nfail 0, nput 21769
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL rusgepl: size 72, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 56, nitems 0, nout 0, hardlimit 4294967295

        nget 426821, nfail 0, nput 426821
        npagealloc 23, npagefree 23, hiwat 1, nidle 0
POOL pstatspl: size 224, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 4
        itemsperpage 18, nitems 16, nout 56, hardlimit 4294967295

        nget 426877, nfail 0, nput 426821
        npagealloc 44, npagefree 40, hiwat 5, nidle 0
POOL plimitpl: size 176, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 23, nitems 16, nout 7, hardlimit 4294967295

        nget 872, nfail 0, nput 865
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL pgrppl: size 16, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 253, nitems 223, nout 30, hardlimit 4294967295

        nget 22406, nfail 0, nput 22376
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL procpl: size 420, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 7
        itemsperpage 9, nitems 7, nout 56, hardlimit 4294967295

        nget 426877, nfail 0, nput 426821
        npagealloc 83, npagefree 76, hiwat 10, nidle 0
POOL lwpucpl: size 776, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 5, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL lwppl: size 148, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 3
        itemsperpage 27, nitems 22, nout 59, hardlimit 4294967295

        nget 428090, nfail 0, nput 428031
        npagealloc 28, npagefree 25, hiwat 4, nidle 0
POOL ktepl: size 56, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 72, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL knotepl: size 72, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 2
        itemsperpage 56, nitems 53, nout 59, hardlimit 4294967295

        nget 13390, nfail 0, nput 13331
        npagealloc 2, npagefree 0, hiwat 2, nidle 0
POOL kqueuepl: size 260, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 15, nitems 9, nout 6, hardlimit 4294967295

        nget 1018, nfail 0, nput 1012
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL fdescpl: size 284, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 4
        itemsperpage 14, nitems 10, nout 46, hardlimit 4294967295

        nget 426864, nfail 0, nput 426818
        npagealloc 47, npagefree 43, hiwat 6, nidle 0
POOL cwdipl: size 16, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 253, nitems 207, nout 46, hardlimit 4294967295

        nget 426864, nfail 0, nput 426818
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL filepl: size 60, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 4
        itemsperpage 67, nitems 56, nout 212, hardlimit 4294967295

        nget 105346834, nfail 0, nput 105346622
        npagealloc 20, npagefree 16, hiwat 4, nidle 0
POOL kauthcredpl: size 100, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 20
        itemsperpage 40, nitems 637, nout 163, hardlimit 4294967295

        nget 663152594, nfail 0, nput 663152431
        npagealloc 89, npagefree 69, hiwat 89, nidle 0
POOL rndsample: size 528, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 7, nitems 6, nout 1, hardlimit 4294967295

        nget 43099084, nfail 0, nput 43099083
        npagealloc 58, npagefree 57, hiwat 2, nidle 0
POOL swp vnd: size 128, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 31, nitems 0, nout 0, hardlimit 4294967295

        nget 36329, nfail 0, nput 36329
        npagealloc 67, npagefree 67, hiwat 11, nidle 0
POOL swp vnx: size 20, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 203, nitems 0, nout 0, hardlimit 4294967295

        nget 35062, nfail 0, nput 35062
        npagealloc 40, npagefree 40, hiwat 2, nidle 0
POOL ractx: size 24, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 76
        itemsperpage 169, nitems 552, nout 12292, hardlimit 4294967295

        nget 426228, nfail 0, nput 413936
        npagealloc 229, npagefree 153, hiwat 174, nidle 0
POOL vmmpepl: size 88, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 91
        itemsperpage 46, nitems 199, nout 3987, hardlimit 4294967295

        nget 513097702, nfail 0, nput 513093715
        npagealloc 397, npagefree 306, hiwat 98, nidle 0
POOL vmsppl: size 224, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 3
        itemsperpage 18, nitems 8, nout 46, hardlimit 4294967295

        nget 426840, nfail 0, nput 426794
        npagealloc 23, npagefree 20, hiwat 5, nidle 0
POOL aobjpl: size 56, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 72, nitems 70, nout 2, hardlimit 4294967295

        nget 2, nfail 0, nput 0
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL uaoeltpl: size 84, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 48, nitems 44, nout 4, hardlimit 4294967295

        nget 36, nfail 0, nput 32
        npagealloc 4, npagefree 3, hiwat 1, nidle 0
POOL anonpl: size 16, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 807
        itemsperpage 253, nitems 2382, nout 201789, hardlimit 4294967295

        nget 565896, nfail 0, nput 364107
        npagealloc 1434, npagefree 627, hiwat 811, nidle 0
POOL amappl: size 48, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 32
        itemsperpage 84, nitems 161, nout 2527, hardlimit 4294967295

        nget 8945439, nfail 0, nput 8942912
        npagealloc 60, npagefree 28, hiwat 32, nidle 0
POOL ufsdirpl: size 264, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 15, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL lfslbnpool: size 16, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 253, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL lfsinoextpl: size 104, align 8, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 39, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL lfsdinopl: size 128, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 31, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL lfsinopl: size 200, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 20, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL dino2pl: size 256, align 4, ioff 0, roflags 0x00000000
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 16, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL dino1pl: size 128, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 31, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL ffsinopl: size 200, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 20, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL newdirblkpl: size 20, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 203, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL dirrempl: size 44, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 92, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL mkdirpl: size 32, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 126, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL diraddpl: size 40, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 101, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL freefilepl: size 40, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 101, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL freeblkspl: size 176, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 23, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL freefragpl: size 44, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 92, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL allocindirpl: size 64, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 63, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL indirdeppl: size 32, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 126, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL allocdirectpl: size 80, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 50, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL bmsafemappl: size 36, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 112, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL newblkpl: size 36, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 112, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL inodedeppl: size 92, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 44, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL pagedeppl: size 72, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 56, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL sdpcpool: size 124, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 32, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL ext2dinopl: size 128, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 31, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL ext2fsinopl: size 200, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 20, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL nfsvapl: size 112, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 869
        itemsperpage 36, nitems 79, nout 31205, hardlimit 4294967295

        nget 3023896, nfail 0, nput 2992691
        npagealloc 910, npagefree 41, hiwat 910, nidle 0
POOL nfsnodepl: size 308, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 2403
        itemsperpage 13, nitems 34, nout 31205, hardlimit 4294967295

        nget 3023896, nfail 0, nput 2992691
        npagealloc 2518, npagefree 115, hiwat 2518, nidle 0
POOL smbfsnopl: size 148, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 27, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL msdosnopl: size 160, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 25, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL cd9660nopl: size 168, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 24, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL smbt2pl: size 116, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 35, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL smbrqpl: size 140, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 29, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL in6pcbpl: size 156, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 26, nitems 11, nout 15, hardlimit 4294967295

        nget 54697, nfail 0, nput 54682
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL tcpcbpl: size 392, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 3
        itemsperpage 10, nitems 0, nout 30, hardlimit 4294967295

        nget 35796, nfail 0, nput 35766
        npagealloc 97, npagefree 94, hiwat 28, nidle 0
POOL sackholepl: size 20, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 203, nitems 0, nout 0, hardlimit 4294967295

        nget 20924, nfail 0, nput 20924
        npagealloc 20, npagefree 20, hiwat 1, nidle 0
POOL synpl: size 184, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 22, nitems 0, nout 0, hardlimit 4294967295

        nget 17045, nfail 0, nput 17045
        npagealloc 93, npagefree 93, hiwat 1, nidle 0
POOL tcpipqepl: size 44, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 92, nitems 0, nout 0, hardlimit 4294967295

        nget 12963, nfail 0, nput 12963
        npagealloc 115, npagefree 115, hiwat 1, nidle 0
POOL ipqepl: size 44, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 92, nitems 0, nout 0, hardlimit 4294967295

        nget 35500384, nfail 0, nput 35500384
        npagealloc 14, npagefree 14, hiwat 1, nidle 0
POOL inmltpl: size 32, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 126, nitems 124, nout 2, hardlimit 4294967295

        nget 4, nfail 0, nput 2
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL inpcbpl: size 108, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 37, nitems 9, nout 28, hardlimit 4294967295

        nget 175594, nfail 0, nput 175566
        npagealloc 18, npagefree 17, hiwat 8, nidle 0
POOL igmppl: size 20, align 4, ioff 0, roflags 0x00000040
        alloc 0xc04417c0
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 203, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL kvakmem: size 4096, align 4, ioff 0, roflags 0x00000600
        alloc 0xc0458a30
        minitems 0, minpages 0, maxpages 4294967295, npages 5
        itemsperpage 32, nitems 50, nout 110, hardlimit 4294967295

        nget 989, nfail 0, nput 879
        npagealloc 7, npagefree 2, hiwat 5, nidle 0
POOL kvakernel: size 4096, align 4, ioff 0, roflags 0x00000600
        alloc 0xc0457a10
        minitems 0, minpages 0, maxpages 4294967295, npages 206
        itemsperpage 32, nitems 33, nout 6559, hardlimit 4294967295

        nget 10235, nfail 0, nput 3676
        npagealloc 206, npagefree 0, hiwat 206, nidle 0
POOL pdppl: size 4096, align 4, ioff 0, roflags 0x00000000
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 46
        itemsperpage 1, nitems 0, nout 46, hardlimit 4294967295

        nget 1374, nfail 0, nput 1328
        npagealloc 376, npagefree 330, hiwat 80, nidle 0
POOL pmappl: size 76, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441800
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 53, nitems 7, nout 46, hardlimit 4294967295

        nget 426840, nfail 0, nput 426794
        npagealloc 18, npagefree 17, hiwat 2, nidle 0
POOL pcgpool: size 140, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441840
        minitems 0, minpages 0, maxpages 4294967295, npages 8
        itemsperpage 29, nitems 186, nout 46, hardlimit 4294967295

        nget 31635, nfail 0, nput 31589
        npagealloc 552, npagefree 544, hiwat 91, nidle 0
POOL phpool-128: size 164, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441840
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 24, nitems 9, nout 15, hardlimit 4294967295

        nget 18, nfail 0, nput 3
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL phpool-64: size 100, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441840
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 40, nitems 28, nout 12, hardlimit 4294967295

        nget 18, nfail 0, nput 6
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL phpool-32: size 68, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441840
        minitems 0, minpages 0, maxpages 4294967295, npages 4
        itemsperpage 59, nitems 4, nout 232, hardlimit 4294967295

        nget 276, nfail 0, nput 44
        npagealloc 4, npagefree 0, hiwat 4, nidle 0
POOL phpool-16: size 52, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441840
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 78, nitems 52, nout 26, hardlimit 4294967295

        nget 69, nfail 0, nput 43
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL phpool-8: size 44, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441840
        minitems 0, minpages 0, maxpages 4294967295, npages 2
        itemsperpage 92, nitems 76, nout 108, hardlimit 4294967295

        nget 267, nfail 0, nput 159
        npagealloc 2, npagefree 0, hiwat 2, nidle 0
POOL phpool-4: size 40, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441840
        minitems 0, minpages 0, maxpages 4294967295, npages 1
        itemsperpage 101, nitems 74, nout 27, hardlimit 4294967295

        nget 70, nfail 0, nput 43
        npagealloc 1, npagefree 0, hiwat 1, nidle 0
POOL phpool-2: size 40, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441840
        minitems 0, minpages 0, maxpages 4294967295, npages 0
        itemsperpage 101, nitems 0, nout 0, hardlimit 4294967295

        nget 0, nfail 0, nput 0
        npagealloc 0, npagefree 0, hiwat 0, nidle 0
POOL phpool-0: size 36, align 4, ioff 0, roflags 0x00000040
        alloc 0xc0441840
        minitems 0, minpages 0, maxpages 4294967295, npages 9
        itemsperpage 112, nitems 619, nout 389, hardlimit 4294967295

        nget 2973, nfail 0, nput 2584
        npagealloc 14, npagefree 5, hiwat 14, nidle 0
db> show uvm
Current UVM status:
  pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12
  127927 VM pages: 76357 active, 38080 inactive, 2422 wired, 14 free
  pages  73990 anon, 37343 file, 4129 exec
  freemin=64, free-target=85, wired-max=42642
  faults=699027877, traps=1139900538, intrs=1896317095, ctxswitch=1996705683
  softint=1413173146, syscalls=-1667333796, swapins=22181, swapouts=22204
  fault counts:
    noram=80487, noanon=0, pgwait=18, pgrele=0
    ok relocks(total)=91353234(91353238), anget(retrys)=336073187(88949), 
amapcopy=13217717
    neighbor anon/obj pg=22602255/122425262, 
gets(lock/unlock)=314336598/91264289
    cases: anon=310994581, anoncow=5802729, obj=308430853, prcopy=5905702, 
przero=61146438
  daemon and swap counts:
    woke=132815, revs=129824, scans=7916927, obscans=4454472, anscans=1952588
    busy=0, freed=4602612, reactivate=44541, deactivate=6643836
    pageouts=23872, pending=124072, nswget=11190
    nswapdev=1, swpgavail=128000
    swpages=128000, swpginuse=128000, swpgonly=127997, paging=0
db> c
Stopped in pid 10.1 (pagedaemon) at     netbsd:cpu_Debugger+0x4:     popl    
%ebp
db> show uvm
Current UVM status:
  pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12
  127927 VM pages: 76357 active, 38080 inactive, 2422 wired, 11 free
  pages  73990 anon, 37343 file, 4129 exec
  freemin=64, free-target=85, wired-max=42642
  faults=699027877, traps=1139900539, intrs=1896348813, ctxswitch=1996763626
  softint=1413186702, syscalls=-1667333796, swapins=22181, swapouts=22204
  fault counts:
    noram=99797, noanon=0, pgwait=18, pgrele=0
    ok relocks(total)=91372544(91372548), anget(retrys)=336073187(108259), 
amapcopy=13217717
    neighbor anon/obj pg=22602255/122425262, 
gets(lock/unlock)=314336598/91264289
    cases: anon=310994581, anoncow=5802729, obj=308430853, prcopy=5905702, 
przero=61146438
  daemon and swap counts:
    woke=152125, revs=149134, scans=8225887, obscans=4454472, anscans=2261548
    busy=0, freed=4602612, reactivate=44541, deactivate=6643836
    pageouts=23872, pending=124072, nswget=11190
    nswapdev=1, swpgavail=128000
    swpages=128000, swpginuse=128000, swpgonly=127997, paging=0
db> tra
cpu_Debugger(0,1,0,64,1) at netbsd:cpu_Debugger+0x4
xencons_tty_input(c0e31e80,c0573064,1,cb2b0ae8,c039a6f1) at 
netbsd:xencons_tty_input+0xa6
xencons_handler(c0e31e80,cb2b0b80,cb2b0b4c,3e2,cb2b0b3c) at 
netbsd:xencons_handler+0xf2
evtchn_do_event(2,cb2b0b80,cb2b0b4c,cb2b0b9c,493145a5) at 
netbsd:evtchn_do_event+0xbd
do_hypervisor_callback(cb2b0b80,0,cb2b0011,49310031,a0011) at 
netbsd:do_hypervisor_callback+0xfd
hypervisor_callback(4a,1,c040c350,0,c047fd20) at netbsd:hypervisor_callback+0x63
uvm_pageout(ca94ba68,0,c01001e7,c01001df,c01001e7) at netbsd:uvm_pageout+0x19e
db> c
Stopped in pid 10.1 (pagedaemon) at     netbsd:cpu_Debugger+0x4:     popl    
%ebp
db> ps
 PID           PPID     PGRP        UID S   FLAGS LWPS          COMMAND    WAIT
 13402          369      369          0 2       0    1             sshd
 20052         8594    20052       1001 2  0x4002    1               cp uvn_fp1
 23460        19909    23460       1612 2  0x4102    1           systat
 19909        28031    19909       1612 2  0x4002    1             tcsh   pause
 28031        26560    26560       1612 2  0x4000    1            xterm
 26560        20689    26560       1612 2  0x4000    1             tcsh   pause
 20689        14156    14156       1612 2   0x100    1             sshd
 14156          369    14156          0 2  0x4100    1             sshd   netio
 6864         24746     6864       1612 2  0x4102    1     screen-4.0.3
 24746        22473    24746       1612 2  0x4002    1             tcsh   pause
 22473        24311    24311       1612 2  0x4000    1            xterm
 24311        28069    24311       1612 2  0x4000    1             tcsh   pause
 28069        23001    23001       1612 2   0x100    1             sshd
 23001          369    23001          0 2  0x4100    1             sshd   netio
 8594          2398     8594       1001 2  0x4002    1             tcsh   pause
 2398         25593     2398       1612 2  0x4002    1             tcsh   pause
 25593        26311    26311       1612 2  0x4000    1            xterm
 26311        28730    26311       1612 2  0x4000    1             tcsh   pause
 28730         5912     5912       1612 2   0x100    1             sshd
 5912           369     5912          0 2  0x4100    1             sshd   netio
 26913          528      528         12 2  0x4108    1           pickup
 19497        15892    19497       1001 2  0x4002    1             tcsh   ttyin
 15892        17598    15892       1612 2  0x4002    1             tcsh   pause
 17598            1    17598       1612 2   0x100    1     screen-4.0.3  select
 19393          472      472       1000 2   0x100    1            httpd
 8635           472      472       1000 2   0x100    1            httpd    poll
 23280          472      472       1000 2   0x100    1            httpd semwait
 23218          472      472       1000 2   0x100    1            httpd semwait
 23324          472      472       1000 2   0x100    1            httpd semwait
 10219          472      472       1000 2   0x100    1            httpd
 9268           472      472       1000 2   0x100    1            httpd semwait
 11502          472      472       1000 2   0x100    1            httpd semwait
 6031             1     1293       1001 2  0x4402    2     scotty2.1.11       *
 663              1      663          0 2  0x4002    1            getty flt_nor
 472              1      472          0 2       0    1            httpd
 629              1      629          0 2       0    1             cron
 636              1      628          0 2       0    1        python2.4
 559            528      528         12 2  0x4108    1             qmgr
 569              1      569          0 2       0    1            inetd  kqread
 528              1      528          0 2  0x4108    1           master
 369              1      369          0 2       0    1             sshd
 382              1      382         19 2   0x100    1            rwhod
 351              1      351          0 2       0    1           powerd  kqread
 349              1      349          0 2       0    1             ntpd
 182              1      182          0 2       0    1          syslogd
 12               0        0          0 2 0x20200    1         aiodoned aiodone
 11               0        0          0 2 0x20200    1          ioflush
>10               0        0          0 2 0x20200    1       pagedaemon
 9                0        0          0 2 0x20200    1            nfsio  nfsidl
 8                0        0          0 2 0x20200    1            nfsio  nfsidl
 7                0        0          0 2 0x20200    1            nfsio
 6                0        0          0 2 0x20200    1            nfsio
 5                0        0          0 2 0x20200    1           xenbus
 4                0        0          0 2 0x20200    1         xenwatch   evtsq
 3                0        0          0 2 0x20200    1        cryptoret crypto_
 1                0        1          0 2  0x4001    1             init    wait
 0               -1        0          0 2 0x20200    1          swapper schedul
db> ps /l
 PID         LID S     FLAGS       STRUCT LWP *            UAREA * WAIT
 13402         1 2       0x4         0xcb46e788         0xcd2b8ce0
 20052         1 3       0x4         0xcb46ecbc         0xcd2a8ce0 uvn_fp1
 23460         1 2       0x4         0xcbc81b24         0xcd369ce0
 19909         1 3      0x80         0xcbc81f30         0xcd366ce0 pause
 28031         1 2 0x8000084         0xcbc818d4         0xcd35cce0
 26560         1 3      0x80         0xcbc81ce0         0xcd355ce0 pause
 20689         1 2       0x4         0xcbc81c4c         0xcd34cce0
 14156         1 3      0x80         0xcbc81d74         0xcd35fce0 netio
 6864          1 2      0x84         0xcb46e2e8         0xcd2a5ce0
 24746         1 3      0x80         0xcb46eb00         0xcd2a2ce0 pause
 22473         1 2       0x4         0xcb46ec28         0xcd23ace0
 24311         1 3      0x80         0xcb46e254         0xcd233ce0 pause
 28069         1 2       0x4         0xcb46eb94         0xcd230ce0
 23001         1 3      0x80         0xcb46e6f4         0xcd14ace0 netio
 8594          1 3      0x80         0xcb46e81c         0xcd22dce0 pause
 2398          1 3      0x80         0xcb46e410         0xcd22ace0 pause
 25593         1 2       0x4         0xcb46e8b0         0xcd14dce0
 26311         1 3      0x80         0xcb46ef0c         0xcd142ce0 pause
 28730         1 2       0x4         0xca94b094         0xcd136ce0
 5912          1 3      0x80         0xca94b2e4         0xcd15ece0 netio
 26913         1 2       0x4         0xca94b818         0xccf02ce0
 19497         1 3      0x80         0xcbc81a90         0xcd5cbce0 ttyin
 15892         1 3      0x80         0xcbc81e9c         0xcd5b9ce0 pause
 17598         1 3      0x80         0xcbc81e08         0xcd5dbce0 select
 19393         1 2       0x4         0xcb46e004         0xcd13fce0
 8635          1 3      0x80         0xcb46ea6c         0xcd001ce0 poll
 23280         1 3      0x80         0xcb46e538         0xcce86ce0 semwait
 23218         1 3      0x80         0xcb46e37c         0xcb503ce0 semwait
 23324         1 3      0x80         0xcb46e098         0xcd5a0ce0 semwait
 10219         1 2       0x4         0xcb46ee78         0xccafdce0
 9268          1 3      0x80         0xcb46e660         0xcb51dce0 semwait
 11502         1 3      0x80         0xca94b250         0xcb552ce0 semwait
 6031          2 2  0x800404         0xcb46ede4         0xcb41fce0
               3 3  0xc00480         0xcb46e5cc         0xcb4face0 select
 663           1 3       0x4         0xca94b8ac         0xcb2b9ce0 flt_noram1
 472           1 2       0x4         0xca94b000         0xcb41cce0
 629           1 2       0x4         0xca94b40c         0xcb36dce0
 636           1 2       0x4         0xca94b128         0xcb412ce0
 559           1 2 0x8000084         0xca94b1bc         0xcb37ece0
 569           1 3      0x80         0xca94b378         0xcb374ce0 kqread
 528           1 2 0x8000084         0xca94b784         0xcb2e2ce0
 369           1 2       0x4         0xca94b4a0         0xcb36ace0
 382           1 2       0x4         0xca94b534         0xcb2f6ce0
 351           1 3      0x80         0xca94b6f0         0xcb2edce0 kqread
 349           1 2      0x84         0xca94b5c8         0xcb2f3ce0
 182           1 2       0x4         0xca94b65c         0xcb2f0ce0
 12            1 3       0x4         0xca94b940         0xcb2b6ce0 aiodoned
 11            1 2 0x8000004         0xca94b9d4         0xcb2b3ce0
>10        >   1 7       0x4         0xca94ba68         0xcb2b0ce0
 9             1 3      0x84         0xca94bafc         0xcb2adce0 nfsidl
 8             1 3      0x84         0xca94bb90         0xcb2aace0 nfsidl
 7             1 2      0x84         0xca94bc24         0xcb2a7ce0
 6             1 2      0x84         0xca94be74         0xca933ce0
 5             1 2       0x4         0xca94bcb8         0xca93cce0
 4             1 3       0x4         0xca94bd4c         0xca939ce0 evtsq
 3             1 3       0x4         0xca94bde0         0xca936ce0 crypto_wait
 1             1 3      0x80         0xca94bf08         0xca930ce0 wait
 0             1 3       0x4         0xc0481f00         0xc056cce0 scheduler
db> 
db> show uvm
Current UVM status:
  pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12
  127927 VM pages: 76357 active, 38080 inactive, 2422 wired, 7 free
  pages  73990 anon, 37343 file, 4129 exec
  freemin=64, free-target=85, wired-max=42642
  faults=699027877, traps=1139900541, intrs=1896398826, ctxswitch=1996854326
  softint=1413207991, syscalls=-1667333796, swapins=22181, swapouts=22204
  fault counts:
    noram=130024, noanon=0, pgwait=18, pgrele=0
    ok relocks(total)=91402771(91402775), anget(retrys)=336073187(138486), 
amapcopy=13217717
    neighbor anon/obj pg=22602255/122425262, 
gets(lock/unlock)=314336598/91264289
    cases: anon=310994581, anoncow=5802729, obj=308430853, prcopy=5905702, 
przero=61146438
  daemon and swap counts:
    woke=182352, revs=179361, scans=8709519, obscans=4454472, anscans=2745180
    busy=0, freed=4602612, reactivate=44541, deactivate=6643836
    pageouts=23872, pending=124072, nswget=11190
    nswapdev=1, swpgavail=128000
    swpages=128000, swpginuse=128000, swpgonly=127997, paging=0
db> 
db> show uvm
Current UVM status:
  pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12
  127927 VM pages: 78586 active, 38378 inactive, 2461 wired, 6 free
  pages  64195 anon, 49602 file, 4232 exec
  freemin=64, free-target=85, wired-max=42642
  faults=27062034, traps=35485029, intrs=60722702, ctxswitch=54725993
  softint=40880845, syscalls=82019467, swapins=2422, swapouts=2454
  fault counts:
    noram=5221613, noanon=0, pgwait=3, pgrele=0
    ok relocks(total)=2834251(2834251), anget(retrys)=8684680(1412556), 
amapcopy=337101
    neighbor anon/obj pg=2148608/5607364, gets(lock/unlock)=7154372/1421696
    cases: anon=4350398, anoncow=3928091, obj=6807667, prcopy=346385, 
przero=14372435
  daemon and swap counts:
    woke=5227236, revs=5221906, scans=84184241, obscans=443661, anscans=83645406
    busy=0, freed=573291, reactivate=7482, deactivate=755813
    pageouts=18756, pending=110913, nswget=1492
    nswapdev=1, swpgavail=128000
    swpages=128000, swpginuse=128000, swpgonly=127694, paging=0
db> ps
 PID           PPID     PGRP        UID S   FLAGS LWPS          COMMAND    WAIT
 28794          631      631          0 2       0    1             cron
 27946          381      381          0 2       0    1             sshd
 26157        29223      631          0 2  0x4100    1         postdrop   netio
 29223        27332      631          0 2  0x4000    1         sendmail  piperd
 25           29412    29585       1001 2  0x4000    1            egrep  piperd
 25178        29412    29585       1001 2  0x4001    1          rrdtool  piperd
 29412        27982    29585       1001 2  0x4000    1               sh    wait
 27982        28245    29585       1001 2  0x4000    1             perl
 28245        29585    29585       1001 2  0x4000    1               sh    wait
 29585        27332    29585       1001 2  0x4000    1               sh    wait
 27332          631      631          0 2       0    1             cron  piperd
 28958          511      511         12 2  0x4108    1           pickup
 13527          475      475       1000 2   0x100    1            httpd
 1344           475      475       1000 2   0x100    1            httpd
 1598           475      475       1000 2   0x100    1            httpd semwait
 952            106      952       1612 2  0x4102    1           systat
 365            571      365       1001 2  0x4002    1             tcsh   ttyin
 596              1      848       1001 2  0x4402    2     scotty2.1.11       *
 463              1      848       1001 2  0x4402    2     scotty2.1.11       *
 235            723      235       1001 2  0x4002    1             tcsh   ttyin
 723            782      723       1612 2  0x4002    1             tcsh   pause
 782            164      164       1612 2  0x4000    1            xterm
 164            165      164       1612 2  0x4000    1             tcsh   pause
 165            663      663       1612 2   0x100    1             sshd
 571            214      571       1612 2  0x4002    1             tcsh   pause
 663            381      663          0 2  0x4100    1             sshd   netio
 214            754      754       1612 2  0x4000    1            xterm
 754            689      754       1612 2  0x4000    1             tcsh   pause
 689            104      104       1612 2   0x100    1             sshd
 106            103      106       1612 2  0x4002    1             tcsh   pause
 104            381      104          0 2  0x4100    1             sshd   netio
 103             99       99       1612 2  0x4000    1            xterm
 99             482       99       1612 2  0x4000    1             tcsh   pause
 482             96       96       1612 2   0x100    1             sshd
 96             381       96          0 2  0x4100    1             sshd   netio
 686            475      475       1000 2   0x100    1            httpd semwait
 658            475      475       1000 2   0x100    1            httpd semwait
 565            475      475       1000 2   0x100    1            httpd semwait
 688            475      475       1000 2   0x100    1            httpd semwait
 666            475      475       1000 2   0x100    1            httpd semwait
 668              1      668          0 2  0x4002    1            getty
 475              1      475          0 2       0    1            httpd
 631              1      631          0 2       0    1             cron
 638              1      627          0 2       0    1        python2.4
 593            511      511         12 2  0x4108    1             qmgr
 569              1      569          0 2       0    1            inetd  kqread
 511              1      511          0 2  0x4108    1           master
 381              1      381          0 2       0    1             sshd flt_nor
 386              1      386         19 2   0x100    1            rwhod
 303              1      303          0 2       0    1           powerd  kqread
 344              1      344          0 2       0    1             ntpd
 182              1      182          0 2       0    1          syslogd
 12               0        0          0 2 0x20200    1         aiodoned aiodone
 11               0        0          0 2 0x20200    1          ioflush
>10               0        0          0 2 0x20200    1       pagedaemon
 9                0        0          0 2 0x20200    1            nfsio  nfsidl
 8                0        0          0 2 0x20200    1            nfsio  nfsidl
 7                0        0          0 2 0x20200    1            nfsio  nfsidl
 6                0        0          0 2 0x20200    1            nfsio  nfsidl
 5                0        0          0 2 0x20200    1           xenbus    rdst
 4                0        0          0 2 0x20200    1         xenwatch   evtsq
 3                0        0          0 2 0x20200    1        cryptoret crypto_
 1                0        1          0 2  0x4001    1             init    wait
 0               -1        0          0 2 0x20200    1          swapper
db> c
Stopped in pid 10.1 (pagedaemon) at     netbsd:cpu_Debugger+0x4:   popl    %ebp
db> show uvm
Current UVM status:
  pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12
  127927 VM pages: 78587 active, 38379 inactive, 2459 wired, 9 free
  pages  64195 anon, 49602 file, 4232 exec
  freemin=64, free-target=85, wired-max=42642
  faults=27062034, traps=35485030, intrs=60742101, ctxswitch=54750485
  softint=40888982, syscalls=82019467, swapins=2422, swapouts=2455
  fault counts:
    noram=5233859, noanon=0, pgwait=3, pgrele=0
    ok relocks(total)=2834251(2834251), anget(retrys)=8696926(1412556), 
amapcopy=337101
    neighbor anon/obj pg=2148608/5607364, gets(lock/unlock)=7154372/1421696
    cases: anon=4350398, anoncow=3940337, obj=6807667, prcopy=346385, 
przero=14372435
  daemon and swap counts:
    woke=5239482, revs=5234152, scans=84380177, obscans=443661, anscans=83841342
    busy=0, freed=573291, reactivate=7482, deactivate=755814
    pageouts=18756, pending=110913, nswget=1492
    nswapdev=1, swpgavail=128000
    swpages=128000, swpginuse=128000, swpgonly=127694, paging=0
db> c
Stopped in pid 10.1 (pagedaemon) at     netbsd:cpu_Debugger+0x4:   popl    %ebp
db> show uvm
Current UVM status:
  pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12
  127927 VM pages: 78587 active, 38379 inactive, 2459 wired, 7 free
  pages  64195 anon, 49602 file, 4232 exec
  freemin=64, free-target=85, wired-max=42642
  faults=27062034, traps=35485031, intrs=60750058, ctxswitch=54760981
  softint=40892467, syscalls=82019467, swapins=2422, swapouts=2455
  fault counts:
    noram=5239107, noanon=0, pgwait=3, pgrele=0
    ok relocks(total)=2834251(2834251), anget(retrys)=8702174(1412556), 
amapcopy=337101
    neighbor anon/obj pg=2148608/5607364, gets(lock/unlock)=7154372/1421696
    cases: anon=4350398, anoncow=3945585, obj=6807667, prcopy=346385, 
przero=14372435
  daemon and swap counts:
    woke=5244730, revs=5239400, scans=84464145, obscans=443661, anscans=83925310
    busy=0, freed=573291, reactivate=7482, deactivate=755814
    pageouts=18756, pending=110913, nswget=1492
    nswapdev=1, swpgavail=128000
    swpages=128000, swpginuse=128000, swpgonly=127694, paging=0
db> c
Stopped in pid 10.1 (pagedaemon) at     netbsd:cpu_Debugger+0x4:   popl    %ebp
db> show uvm
Current UVM status:
  pagesize=4096 (0x1000), pagemask=0xfff, pageshift=12
  127927 VM pages: 78587 active, 38379 inactive, 2459 wired, 8 free
  pages  64195 anon, 49602 file, 4232 exec
  freemin=64, free-target=85, wired-max=42642
  faults=27062034, traps=35485032, intrs=60758559, ctxswitch=54772133
  softint=40896170, syscalls=82019467, swapins=2422, swapouts=2455
  fault counts:
    noram=5244683, noanon=0, pgwait=3, pgrele=0
    ok relocks(total)=2834251(2834251), anget(retrys)=8707750(1412556), 
amapcopy=337101
    neighbor anon/obj pg=2148608/5607364, gets(lock/unlock)=7154372/1421696
    cases: anon=4350398, anoncow=3951161, obj=6807667, prcopy=346385, 
przero=14372435
  daemon and swap counts:
    woke=5250306, revs=5244976, scans=84553361, obscans=443661, anscans=84014526
    busy=0, freed=573291, reactivate=7482, deactivate=755814
    pageouts=18756, pending=110913, nswget=1492
    nswapdev=1, swpgavail=128000
    swpages=128000, swpginuse=128000, swpgonly=127694, paging=0
db> 


Home | Main Index | Thread Index | Old Index