Enormous scheduler bug

[Maxime Villard <max%m00nbsd.net@localhost>]

Hi,
(not related to my recent spl question)
I've found a bug in kern/sys_pset.c::sys_pset_assign():

	LIST_FOREACH(t, &alllwp, l_list) {
		if (t->l_affinity == NULL) {
			continue;
		}
		lwp_lock(t);
		if (t->l_affinity == NULL) {
			lwp_unlock(t);
			continue;
		}
		if (kcpuset_isset(t->l_affinity, cpu_index(ci))) {
			lwp_unlock(t);
			mutex_exit(proc_lock);
			mutex_exit(&cpu_lock);
			return EPERM;
		}
	}

You can easily see that this loop tries to make sure no LWP is
already assigned to the cpu (ci). But you can also easily see
that if an LWP is already scheduled on a cpu, but not the one
we are requesting, the loop does not unlock that LWP.

How to trigger:

- - - - - - - - - - - - - - loop.c - - - - - - - - - - - - - -

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
int main() {
	while (1) {
		sleep(2);
		printf("ALIVE!\n");
	}
}

- - - - - - - - - - - - - - set.c - - - - - - - - - - - - - -
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/pset.h>
int main() {
	psetid_t psid;
	cpuid_t ci = 1;
	pset_create(&psid);
	pset_assign(psid, ci, NULL);
	pset_bind(psid, P_PID, P_MYID, NULL);
	while (1);
}

- - - - - - - - - - - - - - - - -  - - - - - - - - - - - - - -

	# gcc -o loop loop.c
	# gcc -o set set.c -lrt
	# [tty0] ./loop
	# [tty1] schedctl -p PID_OF_LOOP -A 0
	# [tty1] ./set

Here, 'loop' is scheduled on cpu0, and 'set' is assigned to cpu1.
Panics on DIAGNOSTIC, and I guess on release kernels the LWP
deadlocks.

Fix:

Index: sys_pset.c
===================================================================
RCS file: /cvsroot/src/sys/kern/sys_pset.c,v
retrieving revision 1.18
diff -u -r1.18 sys_pset.c
--- sys_pset.c	25 Feb 2014 18:30:11 -0000	1.18
+++ sys_pset.c	29 Jul 2015 20:35:07 -0000
@@ -380,6 +380,7 @@
 				mutex_exit(&cpu_lock);
 				return EPERM;
 			}
+			lwp_unlock(t);
 		}
 		/*
 		 * Set the processor-set ID.


Ok?