Re: pserialize_switchpoint() question

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

On Thu, Jul 19, 2018 at 01:43:56PM +0200, Manuel Bouyer wrote:
> Hello,
> while debugging why XEN domU kernels hang on psrlz, I found that, when
> this happens, pserialize_switchpoint() seems to not be called for one of the
> CPUs (not always the same). The CPU in question doens't show up in
> psz->psz_pass. I also noticed that sometimes, pserialize_perform() seems
> to take much longer (several seconds) than it should.
> 
> Now my question: how is pserialize_switchpoint() supposed to be called ?
> I see it's called from softint_dispatch() but Xen doesn't use it
> (no fast soft interrupts).
> I see it's called from lwp_startup() and mi_switch(), but only if
> there is a context switch. If the CPU is idle, it will never be called.
> So how should an idle CPU be handled ?

The attached patch makes mi_switch() always call pserialize_switchpoint(),
even if a LWP switch didn't take place. It fixes the hang I've been seeing
with Xen domUs, and I didn't notice any side effects, but I don't really
know what I'm doing.
Can someone please explain how pserialize is supposed to work ?
Especially what is the point of the broadcast cross call ?
It seems to be there only do cause an interrupt on the remote CPU, but
there is no pserialize calls in the interrupt path AFAIK, so it looks
like it's useless. The comment above the xc_broadcast() call suggest that
this is to force a context switch, but this won't: only a timer tick will
cause a mi_switch() call, and without my patch mi_switch() won't
call pserialize_switchpoint() if a lwp switch is not needed.

-- 
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
     NetBSD: 26 ans d'experience feront toujours la difference
--

Index: kern_synch.c
===================================================================
RCS file: /cvsroot/src/sys/kern/kern_synch.c,v
retrieving revision 1.316
diff -u -p -u -r1.316 kern_synch.c
--- kern_synch.c	12 Jul 2018 10:46:48 -0000	1.316
+++ kern_synch.c	20 Jul 2018 16:27:30 -0000
@@ -745,6 +745,7 @@ mi_switch(lwp_t *l)
 		retval = 1;
 	} else {
 		/* Nothing to do - just unlock and return. */
+		pserialize_switchpoint();
 		mutex_spin_exit(spc->spc_mutex);
 		lwp_unlock(l);
 		retval = 0;