On Mon, 4 Dec 95 21:58:15 CST 
 Chuck Cranor <chuck@maria.wustl.edu> wrote:

 >     it seems to me that we need to somehow limit maxproc to be
 > less than or equal to sizeof(pt_map)/sizeof(user page table) to
 > avoid this hang, in the general case.  where should this be done?
 > comments?   

Chuck, as you and I discussed privately, I'd much rather have fork() 
return EAGAIN than just see and random hang.  "Truncating" seems to be 
safe; it's used to initialize some hash tables, etc., but then, AFAICT, 
is basically only used to check if fork() should fail.  Seeing as how 
sysctl(2) can raise it, I certainly think it should be safe for 
pmap_init() to lower it.  Attached below is the patch I plan on 
committing to the NetBSD/hp300 pmap.c unless someone else has a better 
idea that's relatively feasible.  (Note, I have a pending PR on this 
that's O(1.5 years) old, so I'm somewhat motivated to get some sort of fix 
into the tree :-).

Index: pmap.c
===================================================================
RCS file: /usr/og/devsrc/netbsd/src/sys/arch/hp300/hp300/pmap.c,v
retrieving revision 1.1.1.3
diff -c -r1.1.1.3 pmap.c
*** pmap.c	1995/10/09 21:32:58	1.1.1.3
--- pmap.c	1995/12/04 03:22:39
***************
*** 457,463 ****
  	 * map where we want it.
  	 */
  	addr = HP_PTBASE;
! 	s = min(HP_PTMAXSIZE, maxproc*HP_MAX_PTSIZE);
  	addr2 = addr + s;
  	rv = vm_map_find(kernel_map, NULL, 0, &addr, s, TRUE);
  	if (rv != KERN_SUCCESS)
--- 457,473 ----
  	 * map where we want it.
  	 */
  	addr = HP_PTBASE;
! 	if ((HP_PTMAXSIZE / HP_MAX_PTSIZE) < maxproc) {
! 		s = HP_PTMAXSIZE;
! 		/*
! 		 * XXX We don't want to hang when we run out of
! 		 * page tables, so we lower maxproc so that fork()
! 		 * will fail instead.  Note that root could still raise
! 		 * this value via sysctl(2).
! 		 */
! 		maxproc = (HP_PTMAXSIZE / HP_MAX_PTSIZE);
! 	} else
! 		s = (maxproc * HP_MAX_PTSIZE);
  	addr2 = addr + s;
  	rv = vm_map_find(kernel_map, NULL, 0, &addr, s, TRUE);
  	if (rv != KERN_SUCCESS)

I would suggest that each port declare a MAXMAXPROC for param.h so that 
sysctl(2) would have an upper bound.

I've run this on my hp380 and hp319, and ran a fork-bomb on both.  It 
appeared to do the right thing ... the fork failed before everything 
wedged.  The bomb basically did the following:

 top:
	if fork fails
		exit with error message;

	if parent
		wait for child to exit;
	else {
		malloc NBPG
		bzero page
		goto top;
	}

I don't particularly like the idea of arbitrarily limiting a processes' 
vmspace, so the above patch seems like a reasonable solution, at least 
until something better can be found.

--------------------------------------------------------------------------
Jason R. Thorpe                                       thorpej@nas.nasa.gov
NASA Ames Research Center                               Home: 408.866.1912
NAS: M/S 258-6                                          Work: 415.604.0935
Moffett Field, CA 94035                                Pager: 415.428.6939