SMP/preempt bug in matt-nb5-mips64

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

Hello,
I found a problem in matt-nb5-mips64's mips/mips/spl.S, regarding
curcpu() and preemption. In both _splraise and _splsw_splhigh,
curcpu() is loaded from L_CPU(MIPS_CURLWP) in a register early,
especially before disabling interrupt. If the current IPL is 0,
the current thread can be preempted and rescheduled on another CPU,
and the new SPL is written back to the wrong cpu_info.
From there, bad things happens (what I've seen is an infinite loop
from the interrupt handler on the victim CPU, because _splsw_splhigh
thinks we're already at splhigh and do nothing, when interrupts are
really enabled).
The attached patch seems to fix it for me: it's enough to reload
curcpu() before writing back the new IPL, as for the above senario to
happen the old IPL of both CPUs has to be 0.

I suspect there's a similar issue with the use of L_CPU(MIPS_CURLWP)
in stub_lock.S, but I've not looked in details, as right now I'm
running with LOCKDEBUG and this code isn't used.

-- 
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
     NetBSD: 26 ans d'experience feront toujours la difference
--

Index: spl.S
===================================================================
--- spl.S       (revision 100)
+++ spl.S       (revision 101)
@@ -76,6 +76,10 @@
        DYNAMIC_STATUS_MASK(a0,v0)              # machine dependent masking
        mtc0    zero, MIPS_COP_0_STATUS         ## disable interrupts
        COP0_SYNC
+           #reload curcpu() we may have been preemted while IPL was 0
+           #we know IPL is 0 on the new CPU or we would not be there,
+           #so no need to reload CPU_INFO_CPL in v0
+       PTR_L   a3, L_CPU(MIPS_CURLWP)
        INT_S   a1, CPU_INFO_CPL(a3)            ## save IPL in cpu_info
        mtc0    a0, MIPS_COP_0_STATUS           ## store back
        COP0_SYNC
@@ -220,6 +224,10 @@
        DYNAMIC_STATUS_MASK(a0,a2)              # machine dependent masking
        mtc0    a0, MIPS_COP_0_STATUS           ## store back
        COP0_SYNC
+           #reload curcpu() we may have been preemted while IPL was 0
+           #we know IPL is 0 on the new CPU or we would not be there,
+           #so no need to reload CPU_INFO_CPL in v0
+       PTR_L   a3, L_CPU(MIPS_CURLWP)
        INT_S   a1, CPU_INFO_CPL(a3)            ## save IPL in cpu_info
 #ifdef PARANOIA
        j       ra                              ## return