Re: Memory leaks in c/pthread libraries

[Paul Ripke <stix%stix.id.au@localhost>]

On Thu, Jun 06, 2024 at 04:05:03PM +0200, J. Hannken-Illjes wrote:
> > On 6. Jun 2024, at 00:55, Greg Troxel <gdt%lexort.com@localhost> wrote:
> > 
> > Brian Marcotte <marcotte%panix.com@localhost> writes:
> > 
> >> Since upgrading to NetBSD-10, we've seen memory leaks in several
> >> daemons which use libpthread:
> <snip>
> > 
> > 
> > I am seeing the leak with gpg-agent and mariadbd.  I don't run the rest
> > on 10.
> 
> For me it is spamass-milter.  On an amd64, 10.0, 56 days up:
> 
>   PID USERNAME PRI NICE   SIZE   RES STATE       TIME   WCPU    CPU COMMAND
>   387 root      85    0    26G  236M poll/0      0:50  0.00%  0.00% spamass-mil
> 
> On amd64 a vsize of 26G is only nasty, on my i386 mail server the
> milter dies every few days at ~2G :-)
> 
> I have a small test (p2.c, attached) that shows the leak.  It is
> always a combination of pthread_create()/pthread_detach().  Using
> pthread_join() instead of pthread_detach() doesn't have the problem.
> 
> All started with this commit:
> 
> https://mail-index.netbsd.org/source-changes/2020/01/27/msg113341.html
> 
> Undoing part of this commit (diff attached) makes the leak disappear
> for me.  I'm not sure if this is fix or it just plasters another bug,
> I'm not very familiar with the lwp lifecycle.
> 
> At least I'm sure pthread__deadqueue fills with dead pthreads and
> the mapped stacks are the memory leak observed.

Ah-ha! Now I know why my toy code didn't replicate the issue - there's
a required race between thread creates and exits. My code was effectively
waiting for the child thread to exit before creating the next thread...

Thanks for filling in the gap!

-- 
Paul Ripke
"Great minds discuss ideas, average minds discuss events, small minds
 discuss people."
-- Disputed: Often attributed to Eleanor Roosevelt. 1948.