Re: kern/60426 (Signal handler corrupts AVX (YMM) registers)

[riastradh%NetBSD.org@localhost]

Synopsis: Signal handler corrupts AVX (YMM) registers

Responsible-Changed-From-To: kern-bug-people->riastradh
Responsible-Changed-By: riastradh%NetBSD.org@localhost
Responsible-Changed-When: Wed, 08 Jul 2026 23:48:44 +0000
Responsible-Changed-Why:
mine


State-Changed-From-To: open->analyzed
State-Changed-By: riastradh%NetBSD.org@localhost
State-Changed-When: Wed, 08 Jul 2026 23:48:44 +0000
State-Changed-Why:
Well, that's embarrassing.  Thought I was done with the menagerie of
x86 extended CPU state for a while until Intel or AMD invents another
new extension, after all the work to save and restore it properly on
thread switch.

But the signal delivery logic, and return-from-signal logic, simply
doesn't save and restore anything beyond the xmm registers.  And
there's no place to put them in ucontext_t, which is only around
700-800 bytes, while the extended CPU states range into the tens of
kilobytes.

So what we'll have to do is:

1. Add proper test cases for all this (with a template for adding new
   architectures and new extended CPU states).

2. Make mcontext_t::__fpregs a union (or extend the existing union for
   i386) of which one variant is simply a pointer to an extended save
   area.  Allocate a bit from _UC_MD_BIT<n> to mark this.

   https://nxr.netbsd.org/xref/src/sys/arch/amd64/include/mcontext.h?r=1.24#53

   For i386, we already use _UC_MD_BIT5 to indicate FXSAVE instead of
   FSAVE:

   https://nxr.netbsd.org/xref/src/sys/arch/i386/include/mcontext.h?r=1.19#76
   https://nxr.netbsd.org/xref/src/sys/arch/amd64/include/mcontext.h?r=1.24#119

3. Teach signal delivery (sendsig_siginfo -> cpu_getmcontext) to check
   whether:

   (a) anything beyond the xmm registers are in use,
   AND
   (b) there is space in the signal stack for XSAVEing whatever else is
       in use.

   If so, it will use the new variant.

   https://nxr.netbsd.org/xref/src/sys/arch/amd64/amd64/machdep.c?r=1.379#2126

   NOTE: We might want to change MINSIGSTKSZ, because the default of
   8192 is not enough for, e.g., Intel AMX.  But I'm not sure what we
   should bump this to because it's dynamically sized!

   https://nxr.netbsd.org/xref/src/sys/sys/signal.h?r=1.78#265

4. Teach return-from-signal (i.e., setcontext -> cpu_setmcontext) to
   understand the new _UC_MD_BIT<n> to recover the extended save state.

This way:

- Applications that don't use anything beyond xmm registers won't see
  any changes.

- Applications that do use registers beyond the xmm registers at least
  have them saved and restored by signal handlers, and can have access
  to them via siginfo.

- getcontext() is still busted, and I don't see any way to fix that
  without versioning it so we can expand mcontext_t and therefore
  ucontext_t -- but how large do we expand it?  Just like the OS needs
  to negotiate extended save area sizes with the CPU, userland will
  need to negotiate mcontext_t sizes with the kernel, and that
  basically requires another API.

We also need to create ptrace accessors for this stuff, but that's not
as urgent as avoiding corrupting CPU state beyond the xmm registers
when a signal is delivered.