Port-sparc64 archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: Tracing NetBSD sun4v issues
On Sat, 10 Oct 2026 15:11:44 +0000 (UTC)
Palle Lyckegaard <palle%netbsd.org@localhost> wrote:
> Debugging the issue over the past year or so convinces me that the
> scenario happends from time to time when returning to user space from a system call.
> The stack of the user process gets messed up somehow with incorrect values from the saved
> register windows (PCB).
With my test program I can see when SEGV crash occurs, but there are no
system calls at those locations, apart from a potential window registers
fill trap.
As mentioned previously, I'm observing the crash when returning from
nsdispatch() function in libc. The core dump shows the code starts
executing in a completely wrong function, so I wanted to check the
state of the registers prior to the return statement.
I added invalid pointer de-reference just before the function return,
in order to get a good stack trace:
nsdispatch(void *retval, const ns_dtab disp_tab[], const char *database,
const char *method, const ns_src defaults[], ...)
{
...
if (getenv("TEST_BAD_DEREF") != NULL)
{
*(unsigned int *)12345 = 0;
}
return (result ? result : NS_NOTFOUND);
}
Running my test program with TEST_NULL_DEREF env variable set
Core was generated by `test'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x000000004062f28c in _nsdispatch (retval=<optimized out>, disp_tab=0x4080dd60 <dtab>, database=0x406cd508 "passwd_compat", method=0x406cd560 "setpassent", defaults=0x4080c698 <__nsdefaultnis_forceall>) at /usr/src/lib/libc/net/nsdispatch.c:739
warning: 739 /usr/src/lib/libc/net/nsdispatch.c: No such file or directory
(gdb) bt
#0 0x000000004062f28c in _nsdispatch (retval=<optimized out>, disp_tab=0x4080dd60 <dtab>, database=0x406cd508 "passwd_compat", method=0x406cd560 "setpassent", defaults=0x4080c698 <__nsdefaultnis_forceall>) at /usr/src/lib/libc/net/nsdispatch.c:739
#1 0x0000000040661bdc in _passwdcompat_setpassent (stayopen=<optimized out>) at /usr/src/lib/libc/gen/getpwent.c:1887
#2 _compat_setpassent (nsrv=0x0, nscb=0x0, ap=0xffffffffffffce38) at /usr/src/lib/libc/gen/getpwent.c:2238
#3 0x000000004062f1e4 in _nsdispatch (retval=0x0, disp_tab=0x4080d9a0 <dtab>, database=0x406cd528 "passwd", method=0x406cd560 "setpassent", defaults=0x4080c718 <__nsdefaultcompat_forceall>) at /usr/src/lib/libc/net/nsdispatch.c:711
#4 0x0000000040662f58 in _setpassent (stayopen=<optimized out>) at /usr/src/lib/libc/gen/getpwent.c:2626
#5 0x0000000000100bc0 in fn (name=0x100e30 "root", uid=0xffffffffffffd784) at test.c:29
#6 0x0000000000100c48 in main () at test.c:52
Disassembly below shows the instruction that caused the crash and this
is just before the "return %i7 + 8" instruction, which returns to the
caller
(gdb) disas
Dump of assembler code for function _nsdispatch:
0x000000004062f278 <+568>: call 0x40821a00 <getenv%got.plt@localhost>
0x000000004062f27c <+572>: ldx [ %l7 + %o0 ], %o0
0x000000004062f280 <+576>: brz,pn %o0, 0x4062f29c <_nsdispatch+604>
0x000000004062f284 <+580>: sethi %hi(0xc0e400), %g1
0x000000004062f288 <+584>: srlx %g1, 0xa, %g1
=> 0x000000004062f28c <+588>: clrb [ %g1 ]
0x000000004062f290 <+592>: clrb [ %g1 + 1 ]
0x000000004062f294 <+596>: clrb [ %g1 + 2 ]
0x000000004062f298 <+600>: clrb [ %g1 + 3 ]
0x000000004062f29c <+604>: cmp %i0, 0
0x000000004062f2a0 <+608>: move %icc, 4, %i0
0x000000004062f2a4 <+612>: return %i7 + 8
0x000000004062f2a8 <+616>: sra %o0, 0, %o0
Let's see what value is in the %i7 register, the value is 0x40661bd4:
(gdb) info reg
g0 0x0 0
g1 0x3039 12345
g2 0x46 70
g3 0x46 70
g4 0x0 0
g5 0x46 70
g6 0x2d2d2d2d2d2d2d2d 3255307777713450285
g7 0x40216ac0 1075931840
o0 0xffffffffffffdec6 -8506
o1 0x406cc088 1080869000
o2 0x4093c918 1083427096
o3 0x0 0
o4 0x25 37
o5 0xf 15
sp 0xffffffffffffc211 0xffffffffffffc211
o7 0x4062f278 1080226424
l0 0x4020e310 1075897104
l1 0x4020e310 1075897104
l2 0xffffffffffffcc08 -13304
l3 0x40937938 1083406648
l4 0x408266d0 1082287824
l5 0x4080dd90 1082187152
l6 0xffffffffffffcb38 -13512
l7 0x40810cc8 1082199240
i0 0x1 1
i1 0x4080dd60 1082187104
i2 0x406cd508 1080874248
i3 0x406cd560 1080874336
i4 0x4080c698 1082181272
i5 0x40202080 1075847296
fp 0xffffffffffffc361 0xffffffffffffc361
i7 0x40661bd4 1080433620
pc 0x4062f28c 0x4062f28c <_nsdispatch+588>
npc 0x4062f290 0x4062f290 <_nsdispatch+592>
state 0x4482000206 294238814726
fsr 0x0 [ ]
fprs <unavailable>
y 0x0 0
cwp 0x6 6
pstate 0x2 [ IE ]
asi 0x82 130
ccr 0x44 [ icc.z xcc.z ]
Let's look at the code at 0x40661bd4, it is:
"call 0x40819060 <_nsdispatch%got.plt@localhost>" in "_compat_setpassent" so it
matches with the instruction sequencing and there is no %i7 register
corruption which would return execution to the wrong location.
(gdb) disas 0x40661bd4
Dump of assembler code for function _compat_setpassent:
0x0000000040661b80 <+0>: save %sp, -208, %sp
0x0000000040661b84 <+4>: sethi %hi(0x6000), %o4
0x0000000040661b88 <+8>: sethi %hi(0x4400), %o3
0x0000000040661b8c <+12>: ld [ %i2 + 0xc ], %i5
0x0000000040661b90 <+16>: sethi %hi(0x4400), %o2
0x0000000040661b94 <+20>: sethi %hi(0x4400), %o1
0x0000000040661b98 <+24>: ldx [ %i2 ], %i4
0x0000000040661b9c <+28>: sethi %hi(0x1af000), %l7
0x0000000040661ba0 <+32>: call 0x406c3a80 <__sparc_get_pc_thunk.l7>
0x0000000040661ba4 <+36>: add %l7, 0x128, %l7 ! 0x1af128
0x0000000040661ba8 <+40>: or %o4, 0x3d8, %o4
0x0000000040661bac <+44>: or %o3, 0x3c0, %o3
0x0000000040661bb0 <+48>: or %o2, 0x390, %o2
0x0000000040661bb4 <+52>: or %o1, 0x298, %o1
0x0000000040661bb8 <+56>: ldx [ %l7 + %o4 ], %o4
0x0000000040661bbc <+60>: ldx [ %l7 + %o3 ], %o3
0x0000000040661bc0 <+64>: sra %i5, 0, %g1
0x0000000040661bc4 <+68>: add %fp, 0x7fb, %o5
0x0000000040661bc8 <+72>: ldx [ %l7 + %o2 ], %o2
0x0000000040661bcc <+76>: clr %o0
0x0000000040661bd0 <+80>: ldx [ %l7 + %o1 ], %o1
0x0000000040661bd4 <+84>: call 0x40819060 <_nsdispatch%got.plt@localhost>
0x0000000040661bd8 <+88>: stx %g1, [ %sp + 0x8af ]
0x0000000040661bdc <+92>: sethi %hi(0x4400), %g1
0x0000000040661be0 <+96>: or %g1, 0x278, %g1 ! 0x4678
0x0000000040661be4 <+100>: ldx [ %l7 + %g1 ], %o0
0x0000000040661be8 <+104>: call 0x4065f980 <_compat_start>
0x0000000040661bec <+108>: st %i5, [ %o0 ]
0x0000000040661bf0 <+112>: xor %o0, 1, %g1
0x0000000040661bf4 <+116>: cmp %g0, %g1
0x0000000040661bf8 <+120>: subc %g0, -1, %g1
0x0000000040661bfc <+124>: st %g1, [ %i4 ]
0x0000000040661c00 <+128>: ret
0x0000000040661c04 <+132>: restore %g0, %o0, %o0
End of assembler dump.
I can only conclude that when executing "return %i7 + 8" instruction, it
potentially results in a window fill trap (as this instruction has
similar semantics to the "restore" instruction) and the trap handler
corrupts program counter register or similar, which causes execution to
jump to jemalloc function, instead of _compat_setpassent.
PS. you should probably ignore 0xbadcafe values in %l registers if you
see them. This value is explicitly set in locore.s when kernel DEBUG
option is set. I assume this was done to catch any code that relies on
zeroed registers, but is not the source of these sun4v issues.
Home |
Main Index |
Thread Index |
Old Index