Subject: port-i386/1635: GDT allocation code does not prevent user forks from panic'ing system
To: None <gnats-bugs@gnats.netbsd.org>
From: John Kohl <jtk@kolvir.arlington.ma.us>
List: netbsd-bugs
Date: 10/17/1995 21:59:43
>Number:         1635
>Category:       port-i386
>Synopsis:       GDT allocation code does not prevent user forks from panic'ing system
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    gnats-admin (GNATS administrator)
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Oct 18 07:05:01 1995
>Last-Modified:
>Originator:     John Kohl
>Organization:
NetBSD Kernel Hackers `R` Us
>Release:        NetBSD-current, 1995/10/17
>Environment:
	
System: NetBSD pattern 1.0A NetBSD 1.0A (PATTERN) #191: Wed Oct 11 21:53:46 EDT 1995 jtk@pattern:/u1/NetBSD-current/src/sys/arch/i386/compile/PATTERN i386


>Description:
The GDT allocation scheme does not limit maxproc to insure that a fork
operation will succeed.  By the time it discovers that the GDT slot
allocation failed, it cannot complete the process fork and will
currently panic.

>How-To-Repeat:
Fork off enough processes (a bit more than 4000) and watch it die.

>Fix:
(a) clamp maxproc in system startup code.
(b) [not provided here yet] make MI kern_sysctl() call MD-code to pass
judgment on maxproc 
*** /sys/arch/i386/i386/gdt.c	Sun Oct 15 06:35:45 1995
--- gdt.c	Tue Oct 17 21:27:21 1995
***************
*** 9,15 ****
  #include <machine/gdt.h>
  
  #define	GDTSTART	64
- #define	MAXGDTSIZ	8192
  
  union descriptor *dynamic_gdt = gdt;
  int gdt_size = NGDT;		/* total number of GDT entries */
--- 39,44 ----
***************
*** 156,161 ****
--- 185,200 ----
  		if (gdt_next != gdt_count)
  			panic("gdt_get_slot botch 1");
  		if (gdt_next >= gdt_size) {
+ 			/*
+ 			 * gdt_size is clamped by maxproc, set in
+ 			 * /sys/conf/param.c and clamped in init386().
+ 			 * It's held there to (MAXGDTSIZ - NGDT) if no
+ 			 * user LDTs, or half that if user LDTs are
+ 			 * allowed. It's important to count that
+ 			 * correctly, because by the time we get here,
+ 			 * it's too late to abort the fork operation
+ 			 * -- we must have a GDT slot available.
+ 			 */
  			if (gdt_size >= MAXGDTSIZ)
  				panic("gdt_get_slot botch 2");
  			if (dynamic_gdt == gdt)
*** /sys/arch/i386/include/gdt.h	Fri Oct 13 21:57:38 1995
--- gdt.h	Wed Oct 11 18:01:20 1995
***************
*** 1,3 ****
--- 1,4 ----
+ #define	MAXGDTSIZ	8192		/* max # entries in an i386 GDT */
  void tss_alloc __P((struct pcb *));
  void tss_free __P((struct pcb *));
  void ldt_alloc __P((struct pcb *, union descriptor *, size_t));
*** /sys/arch/i386/i386/machdep.c	Sun Oct 15 22:43:28 1995
--- machdep.c	Tue Oct 17 21:25:11 1995
***************
*** 1130,1135 ****
--- 1154,1169 ----
  
  	/* call pmap initialization to make new kernel address space */
  	pmap_bootstrap((vm_offset_t)atdevbase + IOM_SIZE);
+ 
+ #ifdef USER_LDT
+ #define MAXPROC ((MAXGDTSIZ-NGDT)/2)
+ #else
+ #define MAXPROC (MAXGDTSIZ-NGDT)
+ #endif
+ 	if (maxproc > MAXPROC) {
+ 		printf("reducing maxproc to %d to fit into gdt\n", MAXPROC);
+ 		maxproc = MAXPROC;
+ 	}
  }
  
  struct queue {
>Audit-Trail:
>Unformatted: