port-vax/60101: vax: __lwp_getprivate_fast() inline asm uses GCC-specific register variable pattern, not portable

["pyhirricturtle%gmail.com@localhost via gnats" <gnats-admin%NetBSD.org@localhost>]

>Number:         60101
>Category:       port-vax
>Synopsis:       vax: __lwp_getprivate_fast() inline asm uses GCC-specific register variable pattern, not portable
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    port-vax-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Mar 19 01:45:00 +0000 2026
>Originator:     FireTurtle
>Release:        NetBSD 10.1_STABLE
>Organization:
>Environment:
VAX (any), sys/arch/vax/include/mcontext.h
>Description:
The __lwp_getprivate_fast() function in sys/arch/vax/include/mcontext.h uses a GCC-specific inline asm pattern to read the syscall return value from r0:

    register void *tcb __asm("r0");
    __asm("chmk %0" :: "i"(SYS__lwp_getprivate) : "r0");
    return tcb;

This relies on GCC-specific behavior where register variables bound to a specific register survive that register being listed as a clobber. The C standard and the GCC documentation for clobbers do not guarantee this â?? clobbering r0 tells the compiler that r0 is destroyed by the asm statement.

Compilers that implement clobbers strictly (such as Clang) treat r0 as destroyed after the asm, making the subsequent read of tcb undefined. At optimization levels above -O0, the compiler may eliminate code that depends on the return value, since the value is provably undefined.

Because __lwp_getprivate_fast() is inlined into pthread__self() (pthread_int.h), which is called from pthread_mutex_lock() and many other libpthread functions, this can cause those functions to be miscompiled â?? the optimizer removes the function body after the chmk, leaving no ret instruction. Execution falls through into the next function in the text segment, hitting a HALT (opcode 0x00 from the .word entry mask) and causing SIGILL.

Symptoms: any program linked against libpthread.so crashes with SIGILL during library initialization (before main() is reached), if libpthread was compiled with a non-GCC compiler at -O1 or higher.
>How-To-Repeat:
Compile libpthread (or any code that includes <machine/mcontext.h> and calls __lwp_getprivate_fast()) with a compiler other than GCC, such as Clang, at -O1 or higher. The compiled code will be missing instructions after the chmk syscall.

To see the issue directly, compile a file containing:

    #include <sys/syscall.h>
    static inline void *test(void) {
        register void *tcb __asm("r0");
        __asm("chmk %0" :: "i"(316) : "r0");
        return tcb;
    }
    int use(void) {
        void *p = test();
        if (p == (void *)0) return 1;
        return 0;
    }

With GCC, the comparison against NULL is preserved. With Clang (or any compiler that treats clobbers strictly), the comparison is optimized away because tcb has no defined value after r0 is clobbered.
>Fix:
Change the inline asm to use a proper output constraint instead of the clobber pattern:

--- a/sys/arch/vax/include/mcontext.h
+++ b/sys/arch/vax/include/mcontext.h
@@ -86,7 +86,7 @@
 __lwp_getprivate_fast(void)
 {
 register void *tcb __asm("r0");
-__asm("chmk %0" :: "i"(SYS__lwp_getprivate) : "r0");
+__asm volatile("chmk %1" : "=r"(tcb) : "i"(SYS__lwp_getprivate));
 return tcb;
 }

The =r output constraint tells the compiler that tcb receives a value from the asm statement. Combined with register ... __asm("r0"), this forces r0 as the output register. The volatile qualifier prevents the asm from being optimized away or reordered. This is compatible with both GCC and Clang.