Subject: Re: Are tlp[01] weak for big traffic?
To: None <port-cobalt@netbsd.org>
From: NAKAJI Hiroyuki <nakaji@jp.freebsd.org>
List: port-cobalt
Date: 01/21/2007 22:07:49
>>>>> In <070109010537.M0122134@mirage.ceres.dti.ne.jp>
>>>>> Izumi Tsutsui <tsutsui@ceres.dti.ne.jp> wrote:
> Maybe you could try:
> - disable CPUFLAGS (if there is some improper optimization)
> - increase NMBCLUSTERS (if there is some resource shortage)
> - check some more network stats (as discussed in tech-net,
> http://mail-index.netbsd.org/tech-net/2007/01/08/0001.html )
> etc?
netstat -m when network stops:
393 mbufs in use:
390 mbufs allocated to data
3 mbufs allocated to packet headers
0 calls to protocol drain routines
I logged 'vmstat -m' in my blog (in Japanese).
http://heimat.jp/~nakaji/diary/?200701c&to=200701212#200701212
I got the "WARNING: mclpool limit reached; increase NMBCLUSTERS"
message, and found the kernel FAQ.
http://netbsd.org/Documentation/kernel/#mclpool-limit
I checked NMBCLUSTERS. Because I enabled the kernel option GATEWAY,
the value of kern.mbuf.nmbclusters is 2048.
Following the instruction described in this URL, I tried
kern.mbuf.nmbclusters=8192. But it was not helpful.
I added 'kern.mbuf.nmbclusters=16384' in /etc/sysctl.conf and
rebooted. And then started the download Windows Vista ISO image from
the TechNet subscriber download site with Microsoft File Transfer
Manager.
At first, the speed was 300-400kB/sec but about one hour later it
decreased to 10-20kB/sec. But the network seems stable.
And, netstat -m shows that mbuf usage is as same as before.
$ netstat -m
395 mbufs in use:
389 mbufs allocated to data
6 mbufs allocated to packet headers
0 calls to protocol drain routines
$ vmstat -m
vmstat: Kmem statistics are not being gathered by the kernel.
Memory resource pool statistics
Name Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
extent 24 7 0 5 1 0 1 1 0 inf 0
pnbufpl 1024 4 0 0 1 0 1 1 0 inf 0
sigapl 2056 464 0 433 35 0 35 35 0 inf 4
mclpl 2048 277 0 0 143 0 143 143 4 8192 4
mbpl 256 409 0 0 27 0 27 27 1 inf 1
buf16k 16384 139 0 0 35 0 35 35 1 1 0
buf8k 8192 13 0 0 2 0 2 2 1 1 0
buf4k 4096 7 0 0 8 0 8 8 1 1 1
buf2k 2048 117 0 0 59 0 59 59 1 1 0
buf1k 1024 3 0 0 1 0 1 1 1 1 0
buf512b 512 3 0 0 1 0 1 1 1 1 0
kmem-8 8 790 0 0 7 0 7 7 0 inf 0
kmem-16 16 62 0 0 1 0 1 1 0 inf 0
kmem-24 24 27 0 0 1 0 1 1 0 inf 0
kmem-32 32 222 0 0 7 0 7 7 0 inf 0
kmem-40 40 17 0 0 1 0 1 1 0 inf 0
kmem-48 48 15 0 0 1 0 1 1 0 inf 0
kmem-56 56 12 0 0 1 0 1 1 0 inf 0
kmem-64 64 213 0 0 14 0 14 14 0 inf 0
kmem-72 72 9 0 0 1 0 1 1 0 inf 0
kmem-80 80 12 0 0 1 0 1 1 0 inf 0
kmem-88 88 33 0 0 3 0 3 3 0 inf 0
kmem-96 96 12 0 0 2 0 2 2 0 inf 0
kmem-112 112 7 0 0 1 0 1 1 0 inf 0
kmem-128 128 4 0 0 1 0 1 1 0 inf 0
kmem-144 144 16 0 0 3 0 3 3 0 inf 0
kmem-168 168 7 0 0 2 0 2 2 0 inf 0
kmem-200 200 15 0 0 3 0 3 3 0 inf 0
kmem-256 256 4 0 0 1 0 1 1 0 inf 0
propdict 72 25 0 6 1 0 1 1 0 inf 0
pdict32 64 3 0 0 1 0 1 1 0 inf 0
pdict16 48 5 0 0 1 0 1 1 0 inf 0
propnmbr 40 7 0 2 1 0 1 1 0 inf 0
propstng 24 4 0 3 1 0 1 1 0 inf 0
ataspl 56 2717 0 2717 1 0 1 1 0 inf 1
rtentpl 136 51 0 3 2 0 2 2 0 inf 0
vnodepl 168 2295 0 94 92 0 92 92 0 inf 0
lockfpl 56 9248 0 9233 1 0 1 1 0 inf 0
ncachepl 88 2320 0 120 48 0 48 48 0 inf 0
biopl 136 1765 0 1765 1 0 1 1 0 inf 1
bufpl 136 282 0 0 10 0 10 10 0 inf 0
sockpl 216 567 0 412 9 0 9 9 0 inf 0
ttypl 264 3 0 0 1 0 1 1 0 inf 0
pipepl 104 288 0 272 1 0 1 1 0 inf 0
vmembtpl 32 195 0 0 2 0 2 2 0 inf 0
ptimerspl 144 30 0 18 1 0 1 1 0 inf 0
ptimerpl 128 30 0 18 1 0 1 1 0 inf 0
ksiginfo 48 8034 0 8034 1 0 1 1 0 inf 1
sessionpl 40 30 0 9 1 0 1 1 0 inf 0
rusgepl 72 433 0 433 1 0 1 1 0 inf 1
pstatspl 224 471 0 433 3 0 3 3 0 inf 0
plimitpl 176 8 0 3 1 0 1 1 0 inf 0
pgrppl 16 38 0 13 1 0 1 1 0 inf 0
procpl 424 471 0 433 5 0 5 5 0 inf 0
lwppl 160 471 0 433 2 0 2 2 0 inf 0
knotepl 80 12 0 0 1 0 1 1 0 inf 0
kqueuepl 296 2 0 0 1 0 1 1 0 inf 0
fdescpl 288 464 0 433 3 0 3 3 0 inf 0
cwdipl 16 464 0 433 1 0 1 1 0 inf 0
filepl 64 5067 0 4819 5 0 5 5 0 inf 1
kauthcredpl 104 1073 0 997 3 0 3 3 0 inf 1
rndsample 528 45 0 42 1 0 1 1 0 inf 0
ractx 32 380 0 4 3 0 3 3 0 inf 0
vmmpepl 96 18348 0 17071 33 0 33 33 0 inf 2
vmsppl 240 464 0 433 3 0 3 3 0 inf 1
aobjpl 56 1 0 0 1 0 1 1 0 inf 0
anonpl 16 3825 0 0 16 0 16 16 0 inf 0
amappl 48 4833 0 4374 6 0 6 6 0 inf 0
ufsdirpl 264 131 0 131 1 0 1 1 0 inf 1
dino1pl 128 2287 0 94 71 0 71 71 0 inf 0
ffsinopl 208 2287 0 94 116 0 116 116 0 inf 0
in6pcbpl 160 54 0 31 1 0 1 1 0 inf 0
tcpcbpl 392 62 0 39 3 0 3 3 0 inf 0
synpl 184 24 0 24 1 0 1 1 0 inf 1
tcpipqepl 48 6 0 6 1 0 1 1 0 inf 1
inmltpl 32 3 0 0 1 0 1 1 0 inf 0
inpcbpl 120 195 0 168 1 0 1 1 0 inf 0
pvpl 16 106587 0 98298 39 0 39 39 1 inf 6
pmappl 32 464 0 433 1 0 1 1 0 inf 0
pcgpool 144 65 0 0 3 0 3 3 0 inf 0
phpool-128 168 7 0 0 1 0 1 1 0 inf 0
phpool-64 104 2 0 0 1 0 1 1 0 inf 0
phpool-32 72 10 0 0 1 0 1 1 0 inf 0
phpool-16 56 23 0 0 1 0 1 1 0 inf 0
phpool-8 48 23 0 0 1 0 1 1 0 inf 0
phpool-4 40 10 0 0 1 0 1 1 0 inf 0
phpool-0 40 279 0 0 3 0 3 3 0 inf 0
In use 5180K, total allocated 5583K; utilization 92.8%
--
NAKAJI Hiroyuki