A quick summary for tech-kern readers:  I tried building mozilla on
1.5W/sparc the other day only to have it panic with "out of space in
kmem_map".  I tried increasing NKMEMPAGES_MAX to about 24MB worth, and
ran into a crash during the initial boot load:

	Rebooting with command:                                               
	Boot device: /iommu/sbus/espdma@f,400000/esp@f,800000/sd@0,0  File and args: net
	bsd
	>> NetBSD/sparc Secondary Boot, Revision 1.9
	>> (woods@sometimes, Tue Mar 27 17:22:23 EST 2001)
	Booting netbsd
	2578629+90240+799880 [68+157936Trap # 9 sfsr 327 sfar 28 
	Data Access Exception
	Type  help  for more information
	ok 

Sometimes the trap was #7.

So, 22 kernels later I've got one that works and which is mostly tuned
the way I tried to tune it in the first place the other day.  Here are
the important highlights of the kernel config (which has also been tuned
down to be for a headless Sun4m only, and only with the deviced I have
or think I have):

	maxusers 	128

	## Size of kernel VM map kmem_map, in PAGE_SIZE-sized chunks (the VM
	## page size; this value may be read from the sysctl(8) variable
	## hw.pagesize ).  This VM map is used to map the kernel malloc arena.
	## The kernel attempts to auto-size this map based on the amount of
	## physical memory in the system.  The result may be viewed with the
	## sysctl(8) variable vm.nkmempages.
	## 
	## The related options `NKMEMPAGES_MIN' and `NKMEMPAGES_MAX' allow the
	## bounds to be overridden.  These options are provided in the event
	## the computed value is insufficient resulting in an ``out of space
	## in kmem_map'' panic.
	##
	## On Sparc KMEMPAGES_MAX (and NKMEMPAGES) must be less than 57344.
	## 
	## XXX There should also be an option to change the 1/4 factor instead
	## of having to hard-code NKMEMPAGES in a custom kernel!
	##
	## XXX should we use btoc() instead of >>PAGE_SHIFT?
	## 
	#options 	NKMEMPAGES="((24 * 1024 * 1024) >> PAGE_SHIFT)"		# default 1/4 physmem
	#options 	NKMEMPAGES_MIN="((6 * 1024 * 1024) >> PAGE_SHIFT)"	# default 6MB worth
	options 	NKMEMPAGES_MAX="((24 * 1024 * 1024) >> PAGE_SHIFT)"	# default 6MB worth

	options 	KMEMSTATS
	options 	SHMMAXPGS=16384
	options 	DIAGNOSTIC

	## The static buffer cache is defined by two kernel parameters:  NBUF
	## BUFCACHE, and BUFPAGES.
	## 
	## NBUF defines the number of buffer headers, i.e. the number of files
	## that may have a buffer cache entry.  Each buffer header includes
	## space for an I/O buffer of MAXBSIZE (which defaults to 65536) so
	## take care not to allocate too many
	## 
	## The size of the actual buffer data cache is defined by the kernel
	## parameter BUFPAGES.  This size is given in 4kB pages.  If the
	## parameter BUFCACHE is set then BUFPAGES is automatically calculated
	## as the specified percentage of total available RAM.
	## 
	## By default the number of pages available for the buffer cache is
	## calculated as 10% of available RAM if there's 2MB or less RAM, else
	## 5% of the available RAM.
	## 
	#options 	NBUF=1200	# buffer headers
	options 	BUFCACHE=10	# use up to 10% of RAM
	#options 	BUFPAGES="btoc(10*1024*1024)"	# ignored if BUFCACHE is set

	options 	NMBCLUSTERS=32768

(anyone please let me know if the text in the above comments is wrong)

dmesg for the latest follows my .sig....

Indeed sysctl confirms that I now have more kernel virtual memory:

	vm.nkmempages = 6144

The only other thing I did was re-start the kernel build in a clean
directory (well, no, it wasn't even clean -- I just did a "make clean"
with the un-tuned but device-trimmed config).

What can cause Trap#7 or trap#9 from a fresh-built kernel?

Did I have a corrupt object file and not know it?  How could I get such
a corrupt file without any other error appearing (and with ECC RAM and
all)?


So, why isn't NKMEMPAGES_MAX_DEFAULT set to some more reasonable size?
Why is it locked down to match NKMEMPAGES_MIN_DEFAULT?  Why isn't it at
least the same as it is on i386, etc.?

I guess I can answer part of that because I tried this overly optimistic
value:

    options NKMEMPAGES_MAX="(((KERNEND - KERNBASE) >> PAGE_SHIFT) - 10000)"

which crapped out on boot after probing the devices with:

	panic: ubc_init: failed to map ubc_object

	db> examine/d nkmempages
	nkmempages:     18387

Hmmm... so kmem_map can definitely get too big, and when it's not really
that big overall too -- is 1/4 of physmem really the best factor to use
to calculate nkmempages?  With a minimum of 6MB worth to get us going,
I'd say the calculation should probably be 1/10'th, though that'll blow
if you've got one or more GB of RAM.

So, how about NKMEMPAGES_MAX_DEFAULT?  What's a more reasonable limit?
Is there any way to better dynamically calculate the upper bound (or
even the initial value) based on other tuning parameters?



BTW, Mozilla does now build (at least when built as root), and it works!
(and with NKMEMPAGES_MAX set to 12MB worth....)

# cd /usr/pkgsrc/www/mozilla
# time make package
[[... blah ...]]
28915.11s real 22420.38s user  4240.67s system
# 

(that's unpacking onto a mostly clean filesystem that covers most of a
ST32430N drive)

Oh, and DO NOT try to run a sparc mozilla with the same (NFS-mounted)
profile as you run your i386 variant from.  If you make this mistake
they'll both stop working as soon as you quit.  The fix is to either
wipe out ~/.mozilla and start again, or if you're really not wanting to
toss your carefully configured settings, cookie and image controls,
etc., to just remove the ~/.mozilla/YOURPROFILE/*/secmodule.db file.

Damn, but do I ever hate stupid architecture specific on-disk storage
formats!  :-)

-- 
							Greg A. Woods

+1 416 218-0098      VE3TCP      <gwoods@acm.org>     <woods@robohack.ca>
Planix, Inc. <woods@planix.com>;   Secrets of the Weird <woods@weird.com>


[ using 264552 bytes of netbsd ELF symbol table ]

Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001
    The NetBSD Foundation, Inc.  All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
    The Regents of the University of California.  All rights reserved.

NetBSD 1.5W (SOMETIMES) #23: Wed Aug 22 19:36:42 EDT 2001
    woods@sometimes:/build/NetBSD-obj/arch/sparc/compile/SOMETIMES
total memory = 287 MB
nbuf at 7350 is too large for VM_MAX_KERNEL_BUF... adjusted to 896
avail memory = 248 MB
using 896 buffers containing 29400 KB of memory
bootpath: /iommu@f,e0000000/sbus@f,e0001000/espdma@f,400000/esp@f,800000/sd@0,0
mainbus0 (root): SUNW,Axil-320
cpu0 at mainbus0: TMS390Z50 v0 or TMS390Z55 @ 75 MHz, on-chip FPU
cpu0: physical 20K instruction (64 b/l), 16K data (32 b/l), 1024K external (32 b/l): cache enabled
obio0 at mainbus0
clock0 at obio0 slot 0 offset 0x200000: mk48t08: hostid 72971942
timer0 at obio0 slot 0 offset 0x300000 delay constant 35
zs0 at obio0 slot 0 offset 0x100000 level 12 softpri 6
zstty0 at zs0 channel 0 (console i/o)
zstty1 at zs0 channel 1
zs1 at obio0 slot 0 offset 0x0 level 12 softpri 6
kbd0 at zs1 channel 0
ms0 at zs1 channel 1
fdc0 at obio0 slot 0 offset 0x700000 level 11 softpri 4: chip 82077
fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec
auxreg0 at obio0 slot 0 offset 0x800000
power0 at obio0 slot 0 offset 0xa01000 level 2
iommu0 at mainbus0 ioaddr 0xe0000000: version 0x1/0x1, page-size 4096, range 64MB
sbus0 at iommu0: clock = 25 MHz
dma0 at sbus0 slot 15 offset 0x400000: dma rev 2
esp0 at dma0 slot 15 offset 0x800000 level 4: ESP200, 40MHz, SCSI ID 7
scsibus0 at esp0: 8 targets, 8 luns per target
ledma0 at sbus0 slot 15 offset 0x400010: dma rev 2
le0 at ledma0 slot 15 offset 0xc00000 level 6: address 00:00:3b:80:3c:56
le0: 8 receive buffers, 2 transmit buffers
bpp0 at sbus0 slot 15 offset 0x4800000 level 2 (ipl 3): dma rev 2
SUNW,DBRIe at sbus0 slot 15 offset 0x8010000 level 9 not configured
eccmemctl0 at mainbus0: version 0x1/0x1
scsibus0: waiting 2 seconds for devices to settle...
esp0: wide mode 0
sd0 at scsibus0 target 0 lun 0: <WDIGTL, ENTERPRISE, 1.61> SCSI2 0/direct fixed
sd0: 4157 MB, 5720 cyl, 8 head, 186 sec, 512 bytes/sect x 8515173 sectors
sd0: sync (100.0ns offset 15), 8-bit (10.000MB/s) transfers, tagged queueing
sd1 at scsibus0 target 1 lun 0: <SEAGATE, ST32430N, 0510> SCSI2 0/direct fixed
sd1: 2049 MB, 3992 cyl, 9 head, 116 sec, 512 bytes/sect x 4197405 sectors
sd1: sync (100.0ns offset 15), 8-bit (10.000MB/s) transfers, tagged queueing
st0 at scsibus0 target 4 lun 0: <EXABYTE, EXB-4200, 216> SCSI2 1/sequential removable
st0: drive empty
st0: async, 8-bit transfers
cd0 at scsibus0 target 6 lun 0: <PLEXTOR, CD-ROM PX-4XCH, 1.23> SCSI2 5/cdrom removable
cd0: sync (248.0ns offset 15), 8-bit (4.032MB/s) transfers
Kernelized RAIDframe activated
root on sd0a dumps on sd0b
root file system type: ffs