lib/59567: dladdr(3) doesn't work properly especially when main executable is loaded at high memory address

[sagawa.aki+nbsd%gmail.com@localhost]

>Number:         59567
>Category:       lib
>Synopsis:       dladdr(3) doesn't work properly especially when main executable is loaded at high memory address
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    lib-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Jul 31 13:40:01 +0000 2025
>Originator:     Akihiro Sagawa
>Release:        NetBSD 10.1
>Organization:
>Environment:
NetBSD xiaolan.local 10.1 NetBSD 10.1 (GENERIC) #0: Mon Dec 16 13:08:11 UTC 2024  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/i386/compile/GENERIC i386
>Description:
I'm trying to run Wine on NetBSD/i386. When using ALSA lib (alisa-lib
and alsa-plugins-oss) to play a wav audio file, wine complains with the
following message and no sound is output.
> ALSA lib conf.c:4029:(snd_config_hooks_call) Cannot open shared library (null) (mmap of executable at correct address failed)
> ALSA lib conf.c:4658:(snd_config_update_r) hooks failed, removing configuration

Upon investigating the issue, I found that dladdr(3) appears to be the
source of the problem.

My Investigation:
_rtld_obj_from_addr function, called from dladdr(3), traverses
_rtld_objlist and returns the Obj_Entry object which satisfies
mapbase < addr < (mapbase + mapsize). However, the main executable's
mapsize is currently broken and has too large value. Thus, its result is
the main executable object. As a result, given addr is incorrectly
interpreted as belonging to the main executable rather than another shared library.
>How-To-Repeat:
As the compilation procedure for Wine is highly involved, a simplified
example is presented below.

[demo.c]----------------------------------------------------------------
#include <stdio.h>
#include <dlfcn.h>

int main(int argc, char* argv[])
{
  Dl_info dlinfo;
  int ret;

  ret = dladdr(printf, &dlinfo);
  printf("ret=%d, name=%s\n", ret, dlinfo.dli_fname);

  return 0;
}
------------------------------------------------------------------------

To compile the file, run:
% gcc -fno-plt -g -O -Wl,-Ttext-segment=0x78000000 -o demo demo.c

Expected Result:
% $PWD/demo
ret=1, name=/usr/lib/libc.so.12

Actual Result:
% $PWD/demo
ret=1, name=/home/sagawa/dladdr/demo

Debug procedure:
% gdb ./demo
(gdb) b dladdr
Function "dladdr" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (dladdr) pending.
(gdb) run
Starting program: /home/sagawa/dladdr/demo 

Breakpoint 1, dladdr (addr=0xb138a557 <printf>, info=0xbfb4753c)
    at /usr/src/libexec/ld.elf_so/rtld.c:1339
1339    {
(gdb) info proc mappings 
process 7029
Mapped address spaces:

        Start Addr   End Addr       Size     Offset   Flags   File
        0x78000000 0x78001000     0x1000        0x0  r-x C-PD /home/sagawa/dladdr/demo
        0x78001000 0x78002000     0x1000        0x0  rw- C-PD
(*snip*)
(gdb) p/x _rtld_objlist->mapsize
$1 = 0x78001000
mapsize should be 0x1000 at this point as shown in gdb's "info proc mapping" Size column.
>Fix:
Here is the proposed patch for NetBSD-current which fixes the main executable's mapsize in _rtld_objlist.
Otherwise, the value of obj->vaddrbase is zero when the line "size =
roundup(vaddr + ph->p_memsz) - obj->vaddrbase;" is executed at the first time.

Index: ld.elf_so/headers.c
===================================================================
RCS file: /cvsroot/src/libexec/ld.elf_so/headers.c,v
retrieving revision 1.75
diff -u -r1.75 headers.c
--- ld.elf_so/headers.c	2 May 2025 23:03:16 -0000	1.75
+++ ld.elf_so/headers.c	31 Jul 2025 12:33:07 -0000
@@ -551,14 +551,15 @@
 			break;
 
 		case PT_LOAD:
-			size = round_up(vaddr + ph->p_memsz) - obj->vaddrbase;
 			if (first_seg) {	/* First load segment */
 				obj->vaddrbase = round_down(vaddr);
 				obj->mapbase = (caddr_t)(uintptr_t)obj->vaddrbase;
+				size = round_up(vaddr + ph->p_memsz) - obj->vaddrbase;
 				obj->textsize = size;
 				obj->mapsize = size;
 				first_seg = false;
 			} else {		/* Last load segment */
+				size = round_up(vaddr + ph->p_memsz) - obj->vaddrbase;
 				obj->mapsize = MAX(obj->mapsize, size);
 			}
 			dbg(("headers: %s %p phsize %" PRImemsz,