Re: Xen MP panics in cpu_switchto()

[Andrew Doran <ad%netbsd.org@localhost>]

On Mon, Jan 13, 2020 at 09:17:28PM +0100, Manuel Bouyer wrote:

> On Mon, Jan 13, 2020 at 07:11:21PM +0000, Andrew Doran wrote:
> > On Mon, Jan 13, 2020 at 07:36:41PM +0100, Manuel Bouyer wrote:
> > 
> > > On Mon, Jan 13, 2020 at 06:33:08PM +0000, Andrew Doran wrote:
> > > > On Mon, Jan 13, 2020 at 05:43:51PM +0100, Manuel Bouyer wrote:
> > > > 
> > > > > On Mon, Jan 13, 2020 at 04:59:50PM +0100, Manuel Bouyer wrote:
> > > > > > It also sets rsp and rbp. I think rbp is not set by anything else, at last
> > > > > > in the Xen case.
> > > > > > The different rbp value would explain why in one case we hit a KASSERT()
> > > > > > in lwp_startup later.
> > > > > > But I don't know what pcb_rbp contains; I couldn't find where the pcb for
> > > > > > idlelwp is initialized.
> > > > > 
> > > > > I tried the attached patch, which should set rsp/rbp as cpu_switchto()
> > > > > does. It doens't cause the lwp_startup() KASSERT as calling cpu_switchto()
> > > > > does; it also doesn't change the scheduler behavior.
> > > > 
> > > > Wait - do you mean that everything works now?  Or that everything still runs
> > > > on CPU0?
> > > 
> > > No, everything still runs on CPU0
> > 
> > Hmm, I don't understand why.  I'll set up Xen and try it out.  It might take
> > me a day or two.
> 
> OK thanks. 

I reproduced it on native x86.  It's a bug in the CPU topology code.  Now
fixed with revision 1.11 src/sys/kern/subr_cpu.c - sorry about that.

> OK, so I removed the call to cpu_switchto() before idle_loop(),
> and added a few KASSERTS.
> I guess you can back out the prev == NULL case from cpu_switchto().

Will do.  Thank you Manuel.

Andrew