Subject: Re: 3.0.1: softdep + ffsv2 + 'heavy' load = pauses
To: None <netbsd-users@netbsd.org>
From: Mark Cullen <mark.r.cullen@gmail.com>
List: netbsd-users
Date: 07/23/2006 20:17:24
This is a multi-part message in MIME format.
--------------010005080800080602060001
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Greg A. Woods wrote:
> At Sat, 22 Jul 2006 19:53:06 +0100,
> Mark Cullen wrote:
>> I shall at some point. Going single user would be a bit of hassle, so
>> it might have to be a multi user test. Regardless of the results, I am
>> forced to run without softdep anyway so...
>
> It is very difficult, next to impossible, to believe benchmark results
> when they're run in conjunction with any other activity that might use
> the same resources. :-)
>
> (unless you can _guarantee_ exactly the same background activity occurs
> at exactly the same time during each run :-))
>
>
Oh, I know :-) In my defense, the speed benchmarking is just for me
really, and I am quite happy comparing rough numbers with a benchmark
run several times for each different configuration, just to get a
general feel of the difference in performance. If I were to be doing any
serious benchmarking for the NetBSD folks, which might help them improve
performance, then I would certainly do it properly! :-)
>> Well, the system wasn't swapping and still had idle CPU to spare, I
>> can say that much! I don't really know what else to look for.
>
> Look at the current and changing usages of all relevant resources, not
> just externally observable activity. Look at accumulations of CPU time
> for each and every process over a relatively long period of time (say an
> hour). Look at various kernel table sizes. Look at all I/O activity.
>
> Post the output of "vmstat -w 1" for a while during the time when the
> load average is ~1 and you don't see anything else going on. Make sure
> to list all your active disk devices on the command-line too. Post a
> full "ps -auxww" after running for a while.
>
> Build your kernel with "options KMEMSTATS" and post "vmsstst -m" after
> running for a while.
>
By externally observable do you mean listening for hard disk activity
and such? I've been looking at the numbers from various tools like
vmstat and things. I've not really had to do any of this before though,
so it may very well be I am missing something blindingly obvious :-)
I have attached the output of vmstat (both, but I dont think I have
KMEMSTATS so I'm not sure if that's going to be useful?), ps and a
simple iostat.
I'm seeing something happen on the 'sys' column every couple of seconds
according to the vmstat numbers? They seem to rise and fall a little
bit, from ~20 to ~100. This seems to just be apcupsd (I think I
mentioned that before, along with lcdproc causing very high numbers).
Stopping both of these doesn't affect the load average though. The
numbers from vmstat with apcupsd stopped still have a few bumps, which
is likely caused by Samba (which unless I *really* have to, I wouldn't
like to stop).
Those results were after two days of running:
"8:09PM up 2 days, 4:11, 1 user, load averages: 1.00, 0.94, 0.90"
Also, I messed about with the vm sysctl's (another thread on here
recently, 'top memory output', using Johnny's numbers as a base
(thanks!) and I dont seem to be seeing any swap usage at all now, unlike
before where I was seeing about ~20MB - ~40MB. I'd imagine this is quite
generally a good thing?
"Memory: 111M Act, 55M Inact, 1408K Wired, 19M Exec, 104M File, 7084K Free
Swap: 1024M Total, 1024M Free"
Quite nice. I always hear people say how the BSD's tend to not use swap
all that much, but this is the first time *I* have seen this on my
machine :-)
Anyway, I hope someone can make something of the output I have provided.
If it's not quite what you were after / expecting, give me a shout :-)
--
Mark Cullen <mark.r.cullen@gmail.com>
--------------010005080800080602060001
Content-Type: text/plain;
name="iostat.out"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="iostat.out"
tty wd0 wd2 wd1 wd3 raid1 raid0 CPU
tin tout KB/t t/s MB/s KB/t t/s MB/s KB/t t/s MB/s KB/t t/s MB/s KB/t t/s MB/s KB/t t/s MB/s us ni sy in id
18 28 13.70 7 0.10 49.52 7 0.34 15.39 6 0.09 51.77 7 0.33 6.86 8 0.05 11.44 2 0.02 1 0 2 1 97
0 0 11.56 9 0.10 0.00 0 0.00 11.56 9 0.10 0.00 0 0.00 11.56 9 0.10 0.00 0 0.00 0 0 0 0 100
60 12 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
51 9 0.00 0 0.00 7.60 10 0.00 0.00 0 0.00 7.60 10 0.00 0.00 0 0.00 7.60 10 0.00 4 0 0 1 95
0 0 0.00 0 0.00 15.00 2 0.00 0.00 0 0.00 15.00 2 0.00 0.00 0 0.00 15.00 2 0.00 0 0 0 0 100
0 0 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
0 0 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
0 0 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 1 99
45 9 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
67 12 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 1 0 99
0 0 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
0 0 0.00 0 0.00 7.60 10 0.07 0.00 0 0.00 7.60 10 0.07 0.00 0 0.00 7.60 10 0.07 1 0 0 0 99
0 0 11.00 4 0.00 0.00 0 0.00 11.00 4 0.00 0.00 0 0.00 11.00 4 0.00 0.00 0 0.00 0 0 0 0 100
0 0 0.00 0 0.00 9.00 2 0.00 0.00 0 0.00 9.00 2 0.00 0.00 0 0.00 9.00 2 0.00 0 0 0 0 100
27 7 0.00 0 0.00 16.00 7 0.00 0.00 0 0.00 16.00 7 0.00 0.00 0 0.00 16.00 7 0.00 0 0 0 0 100
85 14 16.00 1 0.00 0.00 0 0.00 16.00 1 0.00 0.00 0 0.00 16.00 1 0.00 0.00 0 0.00 0 0 0 0 100
tty wd0 wd2 wd1 wd3 raid1 raid0 CPU
tin tout KB/t t/s MB/s KB/t t/s MB/s KB/t t/s MB/s KB/t t/s MB/s KB/t t/s MB/s KB/t t/s MB/s us ni sy in id
0 0 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
0 0 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 1 0 0 1 98
0 0 5.00 2 0.00 0.00 0 0.00 5.00 2 0.00 0.00 0 0.00 5.00 2 0.00 0.00 0 0.00 0 0 0 0 100
0 0 0.00 0 0.00 4.00 2 0.00 0.00 0 0.00 4.00 2 0.00 0.00 0 0.00 4.00 2 0.00 0 0 0 1 99
9 3 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
86 14 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
17 4 16.00 1 0.00 0.00 0 0.00 16.00 1 0.00 0.00 0 0.00 16.00 1 0.00 0.00 0 0.00 0 0 0 0 100
0 0 6.00 1 0.00 0.00 0 0.00 6.00 1 0.00 0.00 0 0.00 6.00 1 0.00 0.00 0 0.00 0 0 0 0 100
0 0 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
0 0 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
0 0 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
76 14 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
36 7 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 1 99
0 0 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100
--------------010005080800080602060001
Content-Type: text/plain;
name="ps.out"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="ps.out"
USER PID %CPU %MEM VSZ RSS TTY STAT STARTED TIME COMMAND
root 0 0.0 18.8 0 48712 ? DKs Fri03PM 0:00.29 [swapper]
root 1 0.0 0.1 72 220 ? Is Fri03PM 0:00.02 init
root 2 0.0 18.8 0 48712 ? DK Fri03PM 0:00.00 [cryptoret]
root 3 0.0 18.8 0 48712 ? DK Fri03PM 0:00.00 [sysmon]
root 4 0.0 18.8 0 48712 ? DK Fri03PM 0:00.00 [atabus0]
root 5 0.0 18.8 0 48712 ? DK Fri03PM 0:00.00 [atabus1]
root 6 0.0 18.8 0 48712 ? DK Fri03PM 0:00.02 [usb0]
root 7 0.0 18.8 0 48712 ? DK Fri03PM 0:00.00 [usbtask]
root 8 0.0 18.8 0 48712 ? DK Fri03PM 0:00.02 [usb1]
root 9 0.0 18.8 0 48712 ? DK Fri03PM 0:34.07 [raid1]
root 10 0.0 18.8 0 48712 ? DK Fri03PM 3:30.84 [raidio1]
root 11 0.0 18.8 0 48712 ? DK Fri03PM 0:14.13 [raid0]
root 12 0.0 18.8 0 48712 ? DK Fri03PM 1:13.68 [raidio0]
root 13 0.0 18.8 0 48712 ? DK Fri03PM 0:04.03 [pagedaemon]
root 14 0.0 18.8 0 48712 ? DK Fri03PM 1:47.54 [ioflush]
root 15 0.0 18.8 0 48712 ? DK Fri03PM 0:05.81 [aiodoned]
root 172 0.0 0.0 300 4 ? Is Fri04PM 0:02.05 /usr/sbin/sshd
root 227 0.0 1.7 1096 4304 ? Ss Fri04PM 0:07.86 /usr/pkg/sbin/httpd
www 230 0.0 1.7 4896 4448 ? I Fri04PM 0:04.08 /usr/pkg/sbin/httpd
www 233 0.0 1.5 1488 3836 ? I Fri04PM 0:02.77 /usr/pkg/sbin/httpd
www 234 0.0 1.5 1516 3820 ? I Fri04PM 0:02.02 /usr/pkg/sbin/httpd
www 235 0.0 1.7 2204 4524 ? I Fri04PM 0:04.14 /usr/pkg/sbin/httpd
root 237 0.0 0.0 132 4 ? IWs Fri04PM 0:00.06 apcupsd: apcmain
www 238 0.0 1.5 4968 3768 ? I Fri04PM 0:03.72 /usr/pkg/sbin/httpd
www 239 0.0 1.5 1488 3796 ? I Fri04PM 0:02.00 /usr/pkg/sbin/httpd
www 240 0.0 1.3 1488 3340 ? I Fri04PM 0:01.32 /usr/pkg/sbin/httpd
root 280 0.0 0.4 2432 1072 ? Is Fri04PM 0:08.82 /sbin/dhclient fxp1
root 295 0.0 0.0 64 4 ? IW Fri04PM 0:00.01 /usr/pkg/sbin/courierlogger -pid=/var/run/imapd.pid -start -name=imapd /usr/pkg/sbin/couriertcpd -address=0 -maxprocs=40 -maxperip=4 -nodnslookup -noidentlookup 143 /usr/pkg/libexec/courier/imaplogin /usr/pkg/bin/imapd Maildir
root 296 0.0 0.0 88 4 ? I Fri04PM 0:00.04 /usr/pkg/sbin/couriertcpd -address=0 -maxprocs=40 -maxperip=4 -nodnslookup -noidentlookup 143 /usr/pkg/libexec/courier/imaplogin /usr/pkg/bin/imapd Maildir
root 306 0.0 0.0 4788 3040 ? IWsa Fri04PM 0:38.46 /usr/sbin/named -4
root 320 0.0 0.3 136 760 ? S Fri04PM 1:44.34 apcupsd: apcser
postfix 322 0.0 0.2 248 480 ? I Fri04PM 0:00.35 qmgr -l -t fifo -u
root 323 0.0 0.0 2700 4 ? Is Fri04PM 0:00.09 /usr/pkg/sbin/smbd -D
root 337 0.0 0.0 1116 4 ? IWs Fri04PM 0:00.01 pflogd: [priv]
root 345 0.0 0.2 188 524 ? Ss Fri04PM 0:05.05 /usr/sbin/syslogd -s
_pflogd 349 0.0 0.3 1168 820 ? S Fri04PM 0:11.64 pflogd: [running] -s 116 -f /var/log/pflog (pflogd)
root 353 0.0 0.2 84 392 ? I Fri04PM 0:00.09 /usr/pkg/libexec/courier-authlib/authdaemond
root 354 0.0 0.2 84 392 ? I Fri04PM 0:00.12 /usr/pkg/libexec/courier-authlib/authdaemond
root 356 0.0 0.2 84 392 ? I Fri04PM 0:00.07 /usr/pkg/libexec/courier-authlib/authdaemond
root 416 0.0 0.1 160 260 ? Ss Fri04PM 0:00.33 uptimec
root 422 0.0 0.0 76 4 ? IWs Fri04PM 0:00.01 /usr/sbin/inetd -l
root 425 0.0 0.2 264 592 ? Ss Fri04PM 0:00.79 /usr/sbin/cron
ant 431 0.0 1.0 3500 2512 ? I Fri04PM 0:53.90 /usr/pkg/sbin/smbd -D
root 537 0.0 0.1 60 372 ? Is Fri04PM 0:00.03 /usr/pkg/bin/mbmon -P 6666 -r
root 546 0.0 0.3 136 680 ? Ss Fri04PM 0:03.82 /usr/local/bin/newdbcpuusage
root 691 0.0 0.5 1176 1160 ? Is Fri04PM 0:00.24 /usr/sbin/dhcpd -q
root 693 0.0 0.0 260 4 ? Is Fri04PM 0:00.03 pure-ftpd: -pure-ftpd (SERVER)
root 703 0.0 0.2 84 396 ? I Fri04PM 0:00.15 /usr/pkg/libexec/courier-authlib/authdaemond
root 732 0.0 0.2 84 392 ? I Fri04PM 0:00.09 /usr/pkg/libexec/courier-authlib/authdaemond
root 743 0.8 1.2 3612 3204 ? S Fri04PM 9:20.86 /usr/pkg/sbin/smbd -D
root 762 0.0 0.0 64 4 ? IW Fri04PM 0:00.01 /usr/pkg/sbin/courierlogger -pid=/var/authdaemon/pid -start /usr/pkg/libexec/courier-authlib/authdaemond
root 851 0.0 0.2 84 392 ? I Fri04PM 0:00.07 /usr/pkg/libexec/courier-authlib/authdaemond
nobody 858 0.0 0.2 184 564 ? Is Fri04PM 0:00.19 /usr/local/bin/noip2
root 869 0.0 0.0 2700 4 ? IW Fri04PM 0:00.00 /usr/pkg/sbin/smbd -D
root 889 0.0 0.6 956 1456 ? Ss Fri04PM 0:05.64 /usr/pkg/sbin/nmbd -D
mrboo 1023 0.0 0.0 100 4 ? Is Fri04PM 0:00.04 ssh-agent -s
root 1265 0.0 0.2 196 480 ? Is Fri04PM 0:02.14 /usr/pkg/libexec/postfix/master
mrboo 11130 0.0 0.4 1500 996 ? I 1:42PM 0:07.13 /usr/pkg/bin/imapd Maildir
donna 11722 0.0 1.0 3192 2688 ? S 11:08AM 0:11.86 /usr/pkg/sbin/smbd -D
admin_xp 16273 0.0 1.2 3176 3084 ? S Sat11AM 0:10.80 /usr/pkg/sbin/smbd -D
postfix 20400 0.0 0.4 212 1008 ? I 7:41PM 0:00.02 pickup -l -t fifo -u
mrboo 22501 0.0 0.5 432 1240 ? S 11:08PM 0:07.06 sshd: mrboo@ttyp1
root 26164 0.0 0.4 424 1100 ? Is 11:08PM 0:00.45 sshd: mrboo [priv]
root 13192 0.0 0.3 116 712 ttyp1 R+ 7:58PM 0:00.00 ps -auxww
mrboo 23911 0.0 0.0 144 4 ttyp1 IWs 11:08PM 0:00.04 -zsh
root 25918 0.0 0.6 224 1640 ttyp1 S 11:08PM 0:00.80 zsh
root 429 0.0 0.0 56 4 tty00 IWs+ Fri04PM 0:00.01 /usr/libexec/getty std.57600 console
root 828 0.0 0.0 156 4 tty00- IW Fri04PM 0:00.02 /bin/sh /usr/pkg/bin/mysqld_safe --user=mysql --datadir=/var/mysql --pid-file=/var/mysql/bone.bone.servebeer.com.pid
mysql 1115 0.0 0.0 34980 4056 tty00- IWa Fri04PM 16:07.05 /usr/pkg/libexec/mysqld --basedir=/usr/pkg --datadir=/var/mysql --user=mysql --pid-file=/var/mysql/bone.bone.servebeer.com.pid
root 430 0.0 0.0 56 4 ttyE1 IWs+ Fri04PM 0:00.01 /usr/libexec/getty Pc ttyE1
root 427 0.0 0.0 56 4 ttyE2 IWs+ Fri04PM 0:00.01 /usr/libexec/getty Pc ttyE2
root 426 0.0 0.0 56 4 ttyE3 IWs+ Fri04PM 0:00.01 /usr/libexec/getty Pc ttyE3
--------------010005080800080602060001
Content-Type: text/plain;
name="vmstat.out"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="vmstat.out"
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr m0 w0 w2 in sy cs us sy id
0 0 0 113524 6024 40 3 0 0 2 7 0 7 7 202 10209 142 1 2 97
0 0 0 113528 6020 6 0 0 0 0 0 0 2 0 157 76 42 0 0 100
0 0 0 113528 6020 5 0 0 0 0 0 0 2 0 250 213 108 0 0 100
0 0 0 113528 6020 5 0 0 0 0 0 0 0 0 139 73 17 0 0 100
0 0 0 113528 6020 8 0 0 0 0 0 0 2 10 163 114 77 2 0 98
0 0 0 113528 6020 5 0 0 0 0 0 0 0 0 127 19 12 0 0 100
0 0 0 113528 6020 5 0 0 0 0 0 0 0 0 139 20 10 0 0 100
0 0 0 113528 6020 5 0 0 0 0 0 0 3 7 157 36 51 0 1 99
0 0 0 113528 6020 5 0 0 0 0 0 0 0 0 248 212 98 0 0 100
0 0 0 113528 6020 5 0 0 0 0 0 0 0 0 162 68 32 0 0 100
0 0 0 113528 6020 5 0 0 0 0 0 0 8 0 148 16 39 0 0 100
0 0 0 113528 6020 5 0 0 0 0 0 0 0 0 138 166 12 1 0 99
0 0 0 113528 6020 5 0 0 0 0 0 0 0 0 138 18 9 0 1 99
0 0 0 113528 6020 5 0 0 0 0 0 0 0 0 146 23 11 0 1 99
0 0 0 113528 6020 6 0 0 0 0 0 0 1 0 216 189 89 0 0 100
0 0 0 113528 6020 6 0 0 0 0 0 0 0 0 190 109 51 0 0 100
0 0 0 113528 6020 7 0 0 0 0 0 0 0 0 117 142 29 0 0 100
0 0 0 113528 6020 5 0 0 0 0 0 0 0 2 130 19 22 0 0 100
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr m0 w0 w2 in sy cs us sy id
0 0 0 113528 6020 5 0 0 0 0 0 0 0 0 146 16 8 0 2 98
0 0 0 113528 6020 5 0 0 0 0 0 0 0 0 130 19 11 0 0 100
0 0 0 113528 6020 5 0 0 0 0 0 0 0 0 202 146 64 0 0 100
0 0 0 113528 6020 5 0 0 0 0 0 0 2 0 200 150 81 0 1 99
0 0 0 113528 6020 5 0 0 0 0 0 0 0 0 145 16 8 0 0 100
0 0 0 113528 6020 5 0 0 0 0 0 0 0 0 134 19 11 0 0 100
0 0 0 113528 6020 5 0 0 0 0 0 0 0 0 133 159 9 0 0 100
0 0 0 113528 6020 5 0 0 0 0 0 0 2 0 147 32 20 0 1 99
0 0 0 113528 6020 11 0 0 0 0 0 0 0 10 285 1847 147 0 3 97
0 0 0 113528 6020 5 0 0 0 0 0 0 2 2 243 234 113 0 0 100
0 0 0 113528 6020 5 0 0 0 0 0 0 0 5 153 16 25 0 0 100
0 0 0 113528 6020 5 0 0 0 0 0 0 0 2 144 23 20 0 0 100
--------------010005080800080602060001
Content-Type: text/plain;
name="vmstat.out.noapc"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="vmstat.out.noapc"
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr m0 w0 w2 in sy cs us sy id
0 0 0 113180 7252 40 3 0 0 2 7 0 7 7 202 10179 142 1 2 97
0 0 0 113184 7248 11 0 0 0 0 0 0 0 12 240 1758 116 0 3 97
0 0 0 113184 7248 5 0 0 0 0 0 0 1 9 160 19 52 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 9 0 169 211 49 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 0 142 20 12 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 27 719 7574 430 13 9 78
0 0 0 113184 7248 5 0 0 0 0 0 0 0 0 126 21 12 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 0 144 16 8 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 0 140 25 12 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 1 136 17 14 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 3 154 67 33 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 3 146 20 26 0 1 99
0 0 0 113184 7248 5 0 0 0 0 0 0 1 0 145 19 17 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 1 141 16 14 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 2 0 149 19 22 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 3 151 16 21 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 0 137 19 11 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 1 0 136 16 14 0 0 100
procs memory page disks faults cpu
r b w avm fre flt re pi po fr sr m0 w0 w2 in sy cs us sy id
0 0 0 113184 7248 5 0 0 0 0 0 0 0 0 152 23 11 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 0 144 16 8 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 0 149 19 11 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 0 156 20 10 0 1 99
0 0 0 113184 7248 5 0 0 0 0 0 0 0 2 141 19 20 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 0 142 16 8 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 0 146 19 11 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 1 0 168 226 15 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 0 136 19 11 0 1 99
0 0 0 113184 7248 5 0 0 0 0 0 0 0 0 144 16 8 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 1 0 144 70 48 0 0 100
0 0 0 113184 7248 5 0 0 0 0 0 0 0 0 141 29 9 0 0 100
--------------010005080800080602060001
Content-Type: text/plain;
name="vmstat_m.out"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="vmstat_m.out"
Memory resource pool statistics
Name Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
phpool-0 40 6996 0 4038 30 0 30 30 0 inf 0
phpool-32 72 207 0 9 4 0 4 4 0 inf 0
pcgpool 140 141 0 131 2 1 1 2 0 inf 0
pmappl 76 34040 0 33983 6 4 2 4 0 inf 0
pdppl 4096 469 0 388 369 286 83 205 0 inf 2
kvakernel 4096 8374 0 2696 191 8 183 185 0 0 0
kvakmem 4096 751 0 321 16 1 15 15 0 0 0
inpcbpl 108 4137 0 4099 2 0 2 2 0 inf 0
inmltpl 32 4 0 1 1 0 1 1 0 inf 0
tcpipqepl 44 601 0 601 6 5 1 1 0 inf 1
synpl 184 276 0 276 6 5 1 1 0 inf 1
sackholepl 20 6 0 6 2 1 1 1 0 inf 1
tcpcbpl 360 1166 0 1141 6 2 4 4 0 inf 0
in6pcbpl 156 660 0 657 1 0 1 1 0 inf 0
ffsinopl 196 2998484 0 2982671 1769 235 1534 1583 0 inf 708
dino2pl 256 2998484 0 2982671 2231 333 1898 1978 0 inf 878
ufsdirhash 2048 12685 0 12196 638 369 269 269 0 inf 24
ufsdirpl 264 185300 0 185300 2 1 1 1 0 inf 1
rndsample 528 141590 0 141583 2 0 2 2 0 inf 0
filepl 56 2588955 0 2588546 11 4 7 7 0 inf 0
cwdipl 16 34040 0 33983 1 0 1 1 0 inf 0
fdescpl 284 34040 0 33983 30 24 6 16 0 inf 0
kqueuepl 260 2 0 0 1 0 1 1 0 inf 0
knotepl 72 12 0 0 1 0 1 1 0 inf 0
procpl 408 34056 0 33985 43 33 10 23 0 inf 1
lwppl 132 34066 0 33985 13 9 4 8 0 inf 0
pgrppl 16 1154 0 1122 1 0 1 1 0 inf 0
pcredpl 24 34056 0 33985 4 3 1 2 0 inf 0
plimitpl 176 81 0 70 1 0 1 1 0 inf 0
pstatspl 224 34056 0 33985 25 19 6 13 0 inf 1
rusgepl 72 33985 0 33985 7 6 1 1 0 inf 1
raspl 20 619 0 617 1 0 1 1 0 inf 0
sadatapl 40 2 0 0 1 0 1 1 0 inf 0
saupcpl 1612 936531 0 936528 7 4 3 3 0 inf 1
sastackpl 24 32 0 0 1 0 1 1 0 inf 0
savppl 52 2 0 0 1 0 1 1 0 inf 0
ptimerpl 124 578 0 554 1 0 1 1 0 inf 0
sessionpl 40 538 0 511 1 0 1 1 0 inf 0
credpl 84 63168 0 63049 7 3 4 4 0 inf 1
siginfo 128 6995 0 6995 9 8 1 1 0 inf 1
ksiginfo 44 3 0 1 1 0 1 1 0 inf 0
pipepl 144 31828 0 31801 3 1 2 2 0 inf 0
shmmp 12 136 0 134 1 0 1 1 0 inf 0
ttypl 264 11 0 0 1 0 1 1 0 inf 0
sockpl 216 20679 0 20530 11 0 11 11 0 inf 1
ncachepl 84 3025245 0 3010057 1036 706 330 409 0 inf 0
lockfpl 52 4634896 0 4634860 1 0 1 1 0 inf 0
vnodepl 168 179057 0 163234 1468 158 1310 1319 0 inf 2
rtentpl 132 138146 0 136593 185 112 73 111 0 inf 19
amappl 40 479671 0 478752 41 25 16 22 0 inf 4
aobjpl 52 9 0 6 1 0 1 1 0 inf 0
vmsppl 224 34040 0 33983 23 18 5 12 0 inf 1
vmmpepl 88 2063540 0 2061427 155 92 63 100 0 inf 9
ataspl 52 5061224 0 5061224 1 0 1 1 0 inf 1
bufpl 124 5500959 0 5496966 260 80 180 198 0 inf 0
buf1k 1024 174 0 174 1 0 1 1 1 1 1
buf2k 2048 116795 0 113945 571 430 141 168 1 1 0
buf4k 4096 1588 0 1554 35 31 4 4 1 1 0
buf8k 8192 1052 0 1029 41 37 4 5 1 1 0
buf16k 16384 79658 1 78572 2711 2411 300 335 1 1 0
mbpl 256 1008 0 681 31 4 27 28 1 inf 0
mclpl 2048 745 0 467 218 56 162 203 4 1024 9
nfsreqcachepl 36 64 0 64 1 0 1 1 0 inf 1
nfsrvdescpl 216 4564 0 4564 1 1 0 1 0 inf 0
pnbufpl 1024 181 0 178 29 27 2 13 0 inf 1
extent 20 170 0 150 1 0 1 1 0 inf 0
raidpl 132 4915495 0 4915495 12 9 3 3 2 4 3
pfrulepl 556 77 0 43 10 4 6 10 0 inf 0
pfstatepl 256 175322 0 175304 76 21 55 65 0 625 43
pfpooladdrpl 68 12 0 6 1 0 1 1 0 inf 0
pfrktable 1240 11 0 6 3 1 2 3 0 inf 0
pfrkentry 152 6 0 2 1 0 1 1 0 inf 0
pfiaddrpl 92 8 0 4 1 0 1 1 0 inf 0
pfosfpen 108 1035 0 690 10 0 10 10 0 inf 0
pfosfp 28 564 0 376 2 0 2 2 0 inf 0
pffrent 16 22 0 22 2 1 1 1 0 20 1
pffrag 48 11 0 11 2 1 1 1 0 12 1
sigapl 2052 34040 0 33983 405 322 83 214 0 inf 26
rf_alloclist_pl 168 2814207 0 2814205 12 8 4 4 3 11 3
rf_asmhdr_pl 16 3894277 0 3894277 1 0 1 1 1 2 1
rf_asm_pl 288 3905172 0 3905172 14 8 6 6 5 14 6
rf_pda_pl 40 7810344 0 7810344 1 0 1 1 1 2 1
rf_vple_pl 8 60 0 0 1 0 1 1 1 2 0
rf_rad_pl 328 1712343 0 1712343 11 7 4 4 3 11 4
rf_stripelock_pl 28 1723238 0 1723238 1 0 1 1 1 2 1
rf_mcpair_pl 12 1090967 0 1090967 1 0 1 1 1 2 1
rf_dagnode_pl 444 13358110 0 13358110 69 44 25 25 15 57 25
rf_dagh_pl 76 2814205 0 2814205 1 0 1 1 1 3 1
rf_daglist_pl 236 1723238 0 1723238 9 6 3 3 2 8 3
rf_funclist_pl 8 1723238 0 1723238 1 0 1 1 1 2 1
rf_dqd_pl 116 4915495 0 4915495 10 7 3 3 2 8 3
In use 37571K, total allocated 54896K; utilization 68.4%
--------------010005080800080602060001--