Re: Xen MP panics in cpu_switchto()

[Andrew Doran <ad%netbsd.org@localhost>]

On Mon, Jan 13, 2020 at 03:16:05PM +0100, Manuel Bouyer wrote:

> On Mon, Jan 13, 2020 at 12:02:13PM +0000, Andrew Doran wrote:
> > Ah yes it does, I saw something that made me think it affected x86_64 only. 
> > I'll make the change on i386 too.
> 
> thanks.
> 
> Now I get a different panic:
> [   1.0000000] vcpu0 at hypervisor0
> [   1.0000000] vcpu0: 64 page colors
> [   1.0000000] vcpu0: Intel(R) Core(TM)2 Duo CPU     E6550  @ 2.33GHz, id 0x6fb
> [   1.0000000] vcpu0: node 0, package 0, core 1, smt 0
> [   1.0000000] vcpu1 at hypervisor0
> [   1.0000000] vcpu1: 2 page colors
> [   1.0000000] vcpu1: starting
> [   1.0000000] vcpu1: is started.
> [   1.0000000] vcpu1: Intel(R) Core(TM)2 Duo CPU     E6550  @ 2.33GHz, id 0x6fb
> [   1.0000000] vcpu1: node 0, package 0, core 0, smt 0
> [...]
> [   1.0000030] UVM: using package allocation scheme, 1 package(s) per bucket
> [   1.0000030] Xen vcpu1 clock: using event channel 7
> [   1.8809493] vcpu1: running
> [   1.8809493] panic: kernel diagnostic assertion "prev != NULL" failed: file "/dsk/l1/misc/bouyer/HEAD/clean/src/sys/kern/kern_lwp.c", line 1021
> [   1.8809493] cpu1: Begin traceback...
> [   1.8809493] vpanic(c057f868,d77abf74,d77abf98,c03cc3e5,c057f868,c057f802,c05b0f71,c05b0ce4,3fd,0) at netbsd:vpanic+0x134
> [   1.8809493] kern_assert(c057f868,c057f802,c05b0f71,c05b0ce4,3fd,0,0,0,c13a6900,c03c60c0) at netbsd:kern_assert+0x23
> [   1.8809493] lwp_startup(0,c13a6900,8b1000,c0674200,0,c010007a,0,0,0,0) at netbsd:lwp_startup+0x155
> [   1.8809493] cpu1: End traceback...
> 
> If I remove the call to cpu_switchto() in cpu_hatch() it boots, but it seems
> that all user processes are running on cpu0 only ...

I looked and the only thing cpu_switchto() is doing there is setting curlwp,
but that's already set in cpu_start_secondary(), so it's not needed.

> I can't see what extra work the cpu_switchto() could be doing that would
> matters, execpt maybe the %epb/rbp init. Any idea ?

Right I don't think cpu_switchto() matters there.  The strategy for
assigning LWPs to CPUs in the scheduler has changed.  If the machine is not
busy everything is likely to stay on CPU0.  Are you putting much load on it?

Andrew