lib/57831: Memory leaks in libpthread/libc

[marcotte%panix.com@localhost]

>Number:         57831
>Category:       lib
>Synopsis:       Memory leaks in libpthread/libc
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    lib-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Jan 08 23:15:00 +0000 2024
>Originator:     Brian Marcotte
>Release:        10.0
>Organization:
Public Access Networks, Corp
>Environment:
NetBSD mail1.panix.com 10.0_RC2 NetBSD 10.0_RC2 (PANIX-STD) #1: Sat Jan  6 00:27:58 EST 2024  root%xxx.panix.com@localhost:/misc/obj64/misc/devel/NO-BACKUPS/release-10/src/sys/arch/amd64/compile/PANIX-STD amd64

>Description:
After upgrading NetBSD-10, I've noticed that some daemons using pthread
have slow memory leaks. I've seen the problem in these daemons:

    gpg-agent
    opendmarc
    dkimpy_milter (python3)
    syslog-ng (only in some cases)

Also, we're getting segfaults in nnrpd (from INN) linked with libperl
when the perl code is called.

I attempted to track down when the problem began but it looks like there
may be more than one issue. I think problems may have been introduced in
these places:

2020/01/27:

  1.156 -> 1.157 src/lib/libpthread/pthread.c
  1.98  -> 1.99  src/lib/libpthread/pthread_int.h

2020/06/04:

  1.2   -> 1.3   external/bsd/jemalloc/dist/src/tsd.c
  1.12  -> 1.13  external/bsd/jemalloc/include/jemalloc/internal/jemalloc_internal_defs.h
  1.173 -> 1.174 lib/libpthread/pthread.c

joerg and ad were working on these libraries at those times.

The second set of commits were easy to back out from modern sources. This
fixed the nnrpd/libperl problem. It made the memory leak issue better,
but not fixed.

I also tried enabling JEMALLOC_PROF, but that made all programs segfault
on startup.

I'm in over my head at this point, so I decided to submit what little I
have.

Thanks.

- Brian

>How-To-Repeat:
I found that when running "gpg-agent --daemon", it's memory usage will
grow even if it's not being used in any way. The other programs required
that they were being used.


>Fix: