Re: NetBSD in SIMH vax with 512 MB RAM

[Kalvis Duckmanton <kalvisd%gmail.com@localhost>]

On 8/3/23 08:26, Johnny Billquist wrote:

> On
>       2023-03-07 20:02, Anders Magnusson wrote:
>       
> >         
> > > I have a little hard time believing it's
> > >           the CAS handling that is having a problem with large memories
> > >           though. Why do you think it's there?
> > >           
> > >           
> > >         
> >         % grep CASMAGIC include/trap.h
> >         
> >         #define    CASMAGIC    0xBEDABABE /* highend of S0 space */
> >         
> >         
> >                  .globl  cas32_ras_start, cas32_ras_end
> >         
> >         cas32_ras_start:
> >         
> >                  movl    %r1,(%r4)
> >         
> >                  movl    *(%r4), %r0
> >         
> >                  cmpl    %r2,%r0
> >         
> >                  bneq    1f
> >         
> >                  movl    %r3,*(%r4)
> >         
> >         cas32_ras_end:
> >         
> >         1:
> >         
> >                  movl    $CASMAGIC, (%r4)
> >         
> >                  rsb
> >         
> >       
>       
>       Hmm. Good point. But that thing is only loading CASMAGIC if the
>       address did not contain the value expected. I can't see how any
>       size of memory would be an issue in here.
>       
>       
>       However, it does remind me that there is code elsewhere that
>       checks if we're currently executing between cas32_ras_start and
>       cas32_ras_end, to ensure the atomicity of the CAS implementation.
>       I wonder if there might be some stupid signed issue with the
>       comparison of those to addresses... This could be something...?
>       
>       
>         Johnny
>       
>       
>

I'm not sure if this will help or indeed whether the conclusion or solution below is the right one.

I spent some time with SIMH this week debugging this - it appears that if _do_cas is interrupted between cas32_ras_start and cas32_ras_end, and the interrupt handler calls an atomic_*() function, then when execution returns to _do_cas, (%r4) will contain $CASMAGIC and indirecting through (%r4) will fault.

The case I saw was when a hardware clock interrupt was received.  The call chain from the handler to _do_cas went something like this

    hardclock() via Xhardclock
    calls clockrnd_sample()
    calls atomic_dec_uint_nv() before returning; atomic_dec_uint_nv
    appears to be an alias for atomic_dec_32_nv()
    calls atomic_cas_32(); that appears to be an alias for
    _atomic_cas_32() on VAX
    which branches to _do_cas with %r4 = &curcpu()->ci_cas_addr
    which returns setting curcpu()->ci_cas_addr to $CASMAGIC

I could boot NetBSD in SIMH if IPL was raised and restored (borrowing code from the multiprocessor case).  Modified code below:

cas32_ras_start:
    /*
     * Lock everyone out on this cpu.
     */
    mfpr    $PR_IPL, %r5        /* save IPL */
    mtpr    $IPL_HIGH, $PR_IPL    /* block everything */
    
    movl    %r1, (%r4)
    movl    *(%r4), %r0
    cmpl    %r2, %r0
    bneq    1f
    movl    %r3, *(%r4)
1:
    mtpr    %r5, $PR_IPL        /* restore IPL */
cas32_ras_end:
    movl    $CASMAGIC, (%r4)
    rsb

I have no explanation for why this might be a problem only for large memory configurations, though.

cheers

kalvis