Subject: Re: Large ipf Rule Sets - Memory Usage and NetBSD 2.1_Stable
To: Manuel Bouyer <bouyer@antioche.eu.org>
From: None <yancm@sdf.lonestar.org>
List: tech-net
Date: 03/27/2006 19:07:12
> On Sun, Mar 26, 2006 at 07:56:40AM -0500, yancm@sdf.lonestar.org wrote:
>> > On Fri, Mar 24, 2006 at 04:31:25PM -0500, yancm@sdf.lonestar.org
>> wrote:
>> >>
>> >> Question 2: If I flush the rulesets, I do not seem to get this
>> >> kernel memory back. How can I determine if this is a NetBSD kernel
>> >> issue or an ipf issue?
>> >
>> > Does ipf -D get it back ?
>> >
>>
BTW: After I executed the ipf -D/ipf -E sequence, my rules
appeared to load but were apparently getting ignored?!?!?
I guess it's statistically possible that my rules would
generate 10-100 blocks per hour would go quiet for 24 hours,
but there were no blocks recorded at all. And yes, they
did show up when I issued ipfstat -i (or -o)...
>> No. AFAICT. Is there a better way to look at memory usage?
>> I'm using 'systat vmstat'...
>
> if you have 'options KMEMSTATS', you can use vmstat -m
>
OK, I can do this.
What shoud I be looking for???
I don't see any concise statement of
kernel memory usage and/or availability.
None of the keywords look like they belong to the ipf kernel process...
******
Memory statistics by bucket size
Size In Use Free Requests HighWater Couldfree
16 1274 262 134536 1280 0
32 1379 925 8539 640 0
64 11618 30 21942 320 0
128 403 13 12085 160 0
256 196183 9 208658 80 0
512 198489 351 206463 40 6
1024 274 6 785 20 0
2048 59 5 504 10 0
4096 23 3 53 5 0
8192 9 3 20 5 0
16384 4 0 24 5 0
32768 9 0 51 5 0
65536 30 0 78 5 0
131072 12 0 122292 5 0
Memory usage type by bucket size
Size Type(s)
16 VM swap, UVM aobj, UVM amap, ifaddr, namecache, soname, pipe, proc,
in_multi, routetbl, temp, pcb, devbuf, exec, kernfs mount,
RAIDframe,
NETSMBDEV, xform
32 AGP, VM swap, UVM amap, ifmedia, ether_multi, ifaddr, vnodes,
sysctldata, routetbl, temp, pcb, devbuf, exec, RAIDframe, xform
64 UVM amap, ifaddr, vnodes, lockf, prop, session, in_multi, routetbl,
temp, pcb, DMA map, devbuf, exec, MFS node, NDP
128 AGP, VM swap, UVM amap, ifaddr, soname, ttys, prop, sysctlnode,
cred,
proc, routetbl, temp, devbuf, kevent, RAIDframe, NDP
256 VM map, UVM amap, ifaddr, vnodes, subproc, proc, routetbl, temp,
pcb,
DMA map, devbuf, exec, kevent, file desc, newblk, NFS daemon
512 UVM amap, ifaddr, mount, proc, UFS mount, temp, pcb, devbuf,
ioctlops,
NFS srvsock, RAIDframe, crypto
1024 VM page, UVM aobj, UVM amap, ttys, sysctlnode, proc, temp, exec,
kevent, NQNFS Lease, RAIDframe
2048 UVM amap, ifaddr, sysctlnode, in_multi, UFS mount, temp, devbuf,
exec,
RAIDframe
4096 UVM amap, sysctlnode, UFS mount, temp, devbuf, RAIDframe
8192 UVM amap, namecache, sysctlnode, UFS mount, temp, RAIDframe
16384 UVM amap, sysctlnode, UFS mount, pagedep
32768 UVM amap, temp, devbuf, RAIDframe
65536 UVM amap, namecache, UFS mount, temp, inodedep, NFS node, ISOFS
mount,
RAIDframe
131072 UVM amap, devbuf, RAIDframe
Memory statistics by type Type Kern
Type InUse MemUse HighUse Limit Requests Limit Limit Size(s)
AGP 2 1K 1K236045K 2 0 0 32,128
VM swap 5 1K 1K236045K 5 0 0 16,32,128
VM page 1 1K 1K236045K 1 0 0 1024
VM map 4 1K 1K236045K 4 0 0 256
UVM aobj 2 2K 2K236045K 2 0 0 16,1024
UVM amap 2350 415K 809K236045K 25718 0 0
16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536,131072
ifmedia 12 1K 1K236045K 12 0 0 32
ether_multi 7 1K 1K236045K 7 0 0 32
ifaddr 47 12K 12K236045K 49 0 0
16,32,64,128,256,512,2048
mount 9 5K 5K236045K 11 0 0 512
vnodes 1906 124K 124K236045K 1928 0 0 32,64,256
lockf 11 1K 2K236045K 4468 0 0 64
namecache 4 137K 137K236045K 4 0 0 16,8192,65536
soname 2 1K 1K236045K 42 0 0 16,128
ttys 326 212K 212K236045K 326 0 0 128,1024
pipe 0 0K 1K236045K 14 0 0 16
prop 2 1K 1K236045K 2 0 0 64,128
sysctldata 0 0K 1K236045K 1 0 0 32
sysctlnode 47 79K 79K236045K 67 0 0
128,1024,2048,4096,8192,16384
cred 21 3K 3K236045K 244 0 0 128
subproc 14 4K 4K236045K 15 0 0 256
session 24 2K 2K236045K 26 0 0 64
proc 6 2K 2K236045K 9 0 0
16,128,256,512,1024
in_multi 35 4K 4K236045K 35 0 0 16,64,2048
UFS mount 27 146K 146K236045K 28 0 0
512,2048,4096,8192,16384,65536
routetbl 74 5K 6K236045K 107 0 0
16,32,64,128,256
temp404425149006K 149190K236045K 433847 0 0
16,32,64,128,256,512,1024,2048,4096,8192,32768,65536
pcb 45 6K 6K236045K 94 0 0
16,32,64,256,512
DMA map 19 2K 2K236045K 19 0 0 64,256
devbuf 233 179K 179K236045K 444 0 0
16,32,64,128,256,512,2048,4096,32768,131072
exec 6 1K 7K236045K 853 0 0
16,32,64,256,1024,2048
kevent 6 3K 3K236045K 2951 0 0 128,256,1024
ioctlops 0 0K 1K236045K 17 0 0 512
file desc 3 1K 1K236045K 14 0 0 256
MFS node 1 1K 1K236045K 1 0 0 64
newblk 1 1K 1K236045K 1 0 0 256
inodedep 1 64K 64K236045K 1 0 0 65536
pagedep 1 16K 16K236045K 1 0 0 16384
NFS srvsock 2 1K 1K236045K 2 0 0 512
NFS daemon 1 1K 1K236045K 1 0 0 256
NQNFS Lease 1 1K 1K236045K 1 0 0 1024
NFS node 1 64K 64K236045K 1 0 0 65536
kernfs mount 1 1K 1K236045K 1 0 0 16
ISOFS mount 1 64K 64K236045K 1 0 0 65536
RAIDframe 66 2736K 2736K236045K 244633 0 0
16,32,128,512,1024,2048,4096,8192,32768,65536,131072
NETSMBDEV 1 1K 1K236045K 1 0 0 16
NDP 12 1K 2K236045K 15 0 0 64,128
crypto 1 1K 1K236045K 1 0 0 512
xform 0 0K 1K236045K 14 0 0 16,32
Memory totals: In Use Free Requests
153291K 267K 716041
Memory resource pool statistics
Name Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg
Maxpg Idle
phpool 40 217 0 12 3 0 3 3 0
inf 0
pcgpool 140 24 0 13 1 0 1 1 0
inf 0
pmappl 76 620 0 567 3 1 2 2 0
inf 1
pdppl 4096 63 0 8 55 0 55 55 0
inf 0
vmsppl 220 620 0 567 4 0 4 4 0
inf 1
vmmpepl 88 99896 0 98412 35 1 34 34 0
inf 1
vmmpekpl 88 759 0 755 1 0 1 1 0
inf 0
uaoeltpl 84 27 0 9 1 0 1 1 0
inf 0
aobjpl 52 1 0 0 1 0 1 1 0
inf 0
amappl 40 8019 0 7239 8 0 8 8 0
inf 0
bufpl 120 428342 0 427664 33 6 27 28 0
inf 6
buf1k 1024 109 0 12 2 0 2 2 1
1 0
buf2k 2048 73 0 3 3 0 3 3 1
1 0
buf4k 4096 9 0 1 1 0 1 1 1
1 0
buf8k 8192 576 0 155 72 7 65 72 1
1 0
buf16k 16384 82 0 2 21 0 21 21 1
1 1
mbpl 256 190 0 75 10 1 9 9 1
inf 1
mclpl 2048 114 0 76 42 4 38 42 4
32784 19
kqueuepl 260 2 0 0 1 0 1 1 0
inf 0
knotepl 72 1288 0 1248 1 0 1 1 0
inf 0
sockpl 200 831 0 722 6 0 6 6 0
inf 0
ttypl 264 65 0 0 5 0 5 5 0
inf 0
rndsample 528 5447 0 5439 2 0 2 2 0
inf 0
procpl 408 631 0 567 7 0 7 7 0
inf 0
lwppl 132 631 0 567 3 0 3 3 0
inf 0
pgrppl 16 56 0 29 1 0 1 1 0
inf 0
pcredpl 24 631 0 567 1 0 1 1 0
inf 0
plimitpl 156 18 0 9 1 0 1 1 0
inf 0
pstatspl 224 631 0 567 4 0 4 4 0
inf 0
rusgepl 72 567 0 567 1 0 1 1 0
inf 1
raspl 20 150 0 140 1 0 1 1 0
inf 0
ptimerpl 124 15 0 1 1 0 1 1 0
inf 0
filepl 56 6603 0 6412 3 0 3 3 0
inf 0
cwdipl 12 620 0 567 1 0 1 1 0
inf 0
fdescpl 280 620 0 567 4 0 4 4 0
inf 0
pnbufpl 1024 12 0 8 2 0 2 2 0
inf 1
vnodepl 164 2853 0 241 109 0 109 109 0
inf 0
ncachepl 84 2869 0 258 55 0 55 55 0
inf 0
ffsinopl 176 2831 0 232 114 0 114 114 0
inf 0
dino1pl 128 2831 0 232 84 0 84 84 0
inf 0
wdcspl 52 348160 0 348158 1 0 1 1 0
inf 0
extent 20 6403 0 6321 1 0 1 1 0
inf 0
raidpl 128 348044 0 348042 3 0 3 3 2
4 2
rtentpl 132 36 0 1 2 0 2 2 0
inf 0
in6pcbpl 156 21 0 14 1 0 1 1 0
inf 0
inpcbpl 108 732 0 659 3 0 3 3 0
inf 0
inmltpl 32 3 0 0 1 0 1 1 0
inf 0
tcpcbpl 316 383 0 324 6 0 6 6 0
inf 0
tcpipqepl 44 250 0 250 1 0 1 1 0
inf 1
synpl 184 223 0 222 1 0 1 1 0
inf 0
sigapl 2052 620 0 567 55 0 55 55 0
inf 2
rf_alloclist_pl 168 219996 0 219994 4 0 4 4 3
11 3
rf_asmhdr_pl 16 322059 0 322057 1 0 1 1 1
2 0
rf_asm_pl 288 342279 0 342277 6 0 6 6 5
14 5
rf_pda_pl 40 684558 0 684554 1 0 1 1 1
2 0
rf_vple_pl 8 30 0 0 1 0 1 1 1
2 0
rf_rad_pl 328 77491 0 77491 4 0 4 4 3
11 4
rf_stripelock_pl 28 97711 0 97711 1 0 1 1 1
2 1
rf_mcpair_pl 12 122285 0 122284 1 0 1 1 1
2 0
rf_dagnode_pl 444 1008034 0 1008029 22 0 22 22 15
57 21
rf_dagh_pl 76 219996 0 219995 1 0 1 1 1
3 0
rf_daglist_pl 236 97711 0 97711 3 0 3 3 2
8 3
rf_funclist_pl 8 97711 0 97711 1 0 1 1 1
2 1
rf_dqd_pl 116 348046 0 348044 3 0 3 3 2
8 2
pipepl 140 808 0 798 1 0 1 1 0
inf 0
In use 7183K, total allocated 8752K; utilization 82.1%
*******