Subject: exceeded mb_map sensibility?
To: None <current-users@NetBSD.ORG>
From: C. Ewen MacMillan <ilixi@tezcat.net>
List: current-users
Date: 09/17/1996 23:55:54
 Hello All,

 First of all, the id-ing type stuff - this is as you will see, not 
 in fact current (rather, based on current circa August 19):

tepe#> uname -a
NetBSD tepe.tezcat.com 1.2_BETA NetBSD 1.2_BETA (tepe-new) #1: 
Mon Aug 26 12:40:10 CDT 1996     
ilixi@tepe.tezcat.com:/usr/src/sys/arch/i386/compile/tepe-new i386
(the above has been reformatted for human-oriented mail-readers).

 Hardware: ASUS P90,  ASUS PCI200 SCSI Controller, 64MB.

 Dmesg output:
 NetBSD 1.2_BETA (tepe-new) #1: Mon Aug 26 12:40:10 CDT 1996
    ilixi@tepe.tezcat.com:/usr/src/sys/arch/i386/compile/tepe-new
CPU: Pentium (GenuineIntel 586-class CPU)
real mem  = 66715648
avail mem = 61054976
using 840 buffers containing 3440640 bytes of memory
mainbus0 (root)
isa0 at mainbus0
ep0 at isa0 port 0x300-0x30f irq 10: 3Com 3C509 Ethernet
ep0: aui/utp address 00:20:af:18:d6:67
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
npx0 at isa0 port 0xf0-0xff: using exception 16
pc0 at isa0 port 0x60-0x6f irq 1: color
fdc0 at isa0 port 0x3f0-0x3f7 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec
pci0 at mainbus0 bus 0: configuration mode 1
vendor 0x8086 product 0x122d (class bridge, subclass host, revision 0x02) at pci
0 dev 0 function 0 not configured
vendor 0x8086 product 0x122e (class bridge, subclass ISA, revision 0x02) at pci0
 dev 7 function 0 not configured
vendor 0x10a8 product 0x0000 (class display, subclass VGA, revision 0x00) at pci
0 dev 10 function 0 not configured
ncr0 at pci0 dev 12 function 0: NCR 53c810 SCSI
ncr0: interrupting at irq 11
ncr0: restart (scsi reset).
scsibus0 at ncr0
sd0 at scsibus0 targ 0 lun 0: <SEAGATE, ST51080N, 0943> SCSI2 0/direct fixed
sd0: sd0(ncr0:0:0): FAST SCSI-2 100ns (10 Mb/sec) offset 8.
1030MB, 4826 cyl, 4 head, 109 sec, 512 bytes/sec
sd1 at scsibus0 targ 1 lun 0: <SEAGATE, ST15230N, 0638> SCSI2 0/direct fixed
sd1: sd1(ncr0:1:0): FAST SCSI-2 100ns (10 Mb/sec) offset 8.
4095MB, 3992 cyl, 19 head, 110 sec, 512 bytes/sec
sd2 at scsibus0 targ 2 lun 0: <SEAGATE, ST15230N, 0638> SCSI2 0/direct fixed
sd2: sd2(ncr0:2:0): FAST SCSI-2 100ns (10 Mb/sec) offset 8.
4095MB, 3992 cyl, 19 head, 110 sec, 512 bytes/sec
biomask 840 netmask c40 ttymask c52
changing root device to sd0a

 And the relevant parts of the kernel config (the rest of the configuration
 simply reflects deletions from generic, for devices we do not have):
machine         i386            # architecture, used by config; REQUIRED

options         I586_CPU

# Some BIOSes don't get the size of extended memory right.  If you
# have a broken BIOS, uncomment the following and set the value
# properly for your system.
#options        EXTMEM_SIZE=... # size of extended memory

options DUMMY_NOPS      # speed hack; recommended
options         NMBCLUSTERS=512
options         UCONSOLE
#options                SCSIDEBUG
#options                SCSI_DEBUG_FLAGS
#options                NCR_IOMAPPED
options         INSECURE        # insecure; allow /dev/mem writing for X
options         MACHINE_NONCONTIG

maxusers        64              # estimated number of users
options         TIMEZONE=0      # time zone to adjust RTC time by
options         DST=0           # daylight savings time used by RTC

options         SWAPPAGER       # paging; REQUIRED
options         VNODEPAGER      # mmap() of files

 Back around the end of June I think it was, I first started running
 into mb_map full errors - Chris D. (I think) suggested the increase
 to 512 for NMBCLUSTERS which has worked until:

Sep 16 00:21:25 tepe /netbsd: mb_map full
Sep 16 00:30:12 tepe /netbsd: mb_map full
Sep 16 01:28:53 tepe /netbsd: mb_map full
Sep 16 03:36:48 tepe load: 15 minute load average = 5.05
Sep 16 03:39:02 tepe load: 15 minute load average = 5.18

 We have been getting the error periodically for about the last
 two weeks - for background that might make some sense of all
 of this, we climbed for some reason from ~#350 to #115 in the
 Freenix ratings (i.e. Top 1000 Usenet sites.)

 Obviously,  we have experienced a commensurate increase
 in network traffic. Unfortunately, increasing NMBCLUSTERS does
 not do what one might think it would - it seems to panic the machine
 (completely wiping the dmesg buffer for some reason!). I have tried
 both 768 and 1024 as values, and the problem is decidedly worse
 at 1024. Due to the lack of a 1.1 oriented compatibilty interface for
 if_name/if_unit, I cannot even use SNMP it seems to generate a trap
 at the time the problem occurs. Since this is a production machine, I 
 cannot leave it set to drop to the debugger either, though my intuition
 tells me whatever kills it, kills it HARD.

 I realize that what I am providing is subjective - I cannot unfortunately
 provide you with dmesg outputs from the increased buffer sizes, as
 I cannot produce them. I do recall an experiment with a P75 (we dropped
 it in to test the hardware) generating a weird pmap error in the same
 situation.

 Lastly, the performance curve goes down as the mb_map full errors 
 increase - it seems as though (based on top) whenever the  threshold
 is reached the machine is effectively non-responsive for a period of at
 least 45 seconds. We have effectively dropped to about 25% of
 performance (based on userland NNTP statistics, the only thing relevant
 for this machine) of what we had about one week ago, with only a 2-3%
 total increase in network traffic to the machine.

 I have tried the PR 1903 bufcache patch under a slightly older revision
 but it panicked on me a great deal, and would not boot above 7% bufcache.

 Any suggestions, aside from the obvious (i.e. go back to 1.1, or some other
 more i386 arch friendly OS)?

 Regards,

 CEM


 

 
--
C. Ewen MacMillan

E-mail: ilixi@tezcat.net
Phone:  (312)850-0181