Subject: Re: Problems trying to debug pkgsrc/mail/milter-greylist
To: None <port-sparc64@netbsd.org>
From: Gert Doering <gert@greenie.muc.de>
List: port-sparc64
Date: 11/22/2007 09:01:45
Bill Stouder-Studenmund wrote:

>On Wed, Nov 21, 2007 at 10:17:38AM -0500, Chris Ross wrote:
>>
>>   Hi there.  I have a sparc64 running 4.0_RC3, and I built pkgsrc/
>> mail/milter-greylist.  I notice that it sometimes just dies.  I 
>> upgraded the pkgsrc tree to the current release (4.0, vs the 3.0 
>> that's in pkgsrc currently), but it seems to fail in about the same way.

I have been plagued by this since quite a while.  Originally I assumed
"bug in milter-greylist" or "bug in NetBSD 2.x" or even "defective hardware
on my Ultra5".

Counter-Arguments to these assumptions:

 - Upgrading from NetBSD 2 to NetBSD 3.1 did not change the symptoms
   (so: not a problem in NetBSD 2)

 - moving the mail system (sendmail + milter-sender + clamav-milter) to
   a different Ultra10 did not change the symtoms (so: not a hardware
   problem)

 - clamav-milter dies as well on me, with similar effects - which points 
   more to a problem in libmilter or libpth.

 - I did try NetBSD 4 as well (about 4 months ago), uncovered other bugs :) 
   - but the milter-crashes still happened.

[..]
>Are you running on an SMP system w/ an SMP kernel? libpthread in 4.0 also
>has issues with concurrency.

Over here, it's single-CPU sparcs only (as sparc64 SMP is, as far as I can
see, still not production-ready).

>We actually have a branch, wrstuden-fixsa, which is dedicated to fixing
>the libpthread and Scheduler Activations issues in 4.0. I think it's
>caught up with NetBSD-4.0_RC3. Feel free to try it. 

So just updating sources to "netbsd-4" should get these fixes, then?

[..]
>>   milter-greylist doesn't have many threads running ever.  It just 
>> spawns off new threads for synchronization (which I'm not using), and 
>> dumping of data.  One for reading the config file too, I think, but I 
>> suspect that's not happening repeatedly.

As far as I understand, libmilter will spawn a new thread for every
incoming milter connection.

Which correlates with what I see: if I more SPAM comes in, milter-greylist
crashes more often.  And if milter-greylist crashes, the SPAM falls through
to clamav-milter (because thats's the way it's set up right now), and this
usually leads to clamav-milter crashes.  The amount of mail traffic that gets
through a non-crashed milter-greylist is little enough that it doesn't
trigger the issues in clamav-milter.

gert
-- 
gert@greenie.muc.de   fax: +49-89-35655025   http://alpha.greenie.net/mgetty/

With the press, it is safest to assume that there is no "off the
record."
                                -- Donald Rumsfeld