Re: httpd: _lwp_ctl: Cannot allocate memory

[Andrew Doran <ad%netbsd.org@localhost>]

On Fri, Mar 14, 2008 at 10:07:32PM +1100, Sarton O'Brien wrote:

> Andrew Doran wrote:
> >On Fri, Mar 14, 2008 at 01:30:31PM +1100, Sarton O'Brien wrote:
> >  
> >>Is this a known issue?
> >>
> >>/etc/rc.d/apache stop
> >>Stopping apache.
> >>httpd: _lwp_ctl: Cannot allocate memory
> >>
> >>NetBSD www.internal 4.99.55 NetBSD 4.99.55 (XEN3_DOMU) #0: Tue Mar 11 
> >>19:53:46 EST 2008  
> >>root%mx0.internal@localhost:/usr/obj/sys/arch/i386/compile/XEN3_DOMU i386
> >>    
> >
> >How much memory did you give the domain? Was it under memory pressure?
> >How many threads do your apache processes have each?
> >  
> 
> I'd only just booted, so no pressure outside of starting/booting.
> 
> This is what top reports after being up for over 8 hours:
> 
> load averages:  0.02,  0.01,  0.00                  up 0 days,  8:35   
> 21:51:19
> 35 processes:  34 sleeping, 1 on CPU
> CPU states:  0.0% user,  0.0% nice,  0.0% system,  0.0% interrupt,  100% 
> idle
> Memory: 298M Act, 12M Wired, 16M Exec, 135M File, 168M Free
> Swap: 512M Total, 512M Free
> 
> www# ps ax|grep httpd
> 5214 ?        I    0:00.01 /usr/pkg/sbin/httpd -k start
> 5382 ?        I    0:00.01 /usr/pkg/sbin/httpd -k start
> 5441 ?        I    0:00.01 /usr/pkg/sbin/httpd -k start
> 6237 ?        I    0:00.00 /usr/pkg/sbin/httpd -k start
> 6331 ?        I    0:00.02 /usr/pkg/sbin/httpd -k start
> 6939 ?        I    0:00.01 /usr/pkg/sbin/httpd -k start
> 7074 ?        Ss   0:00.25 /usr/pkg/sbin/httpd -k start
> 7362 ?        I    0:00.00 /usr/pkg/sbin/httpd -k start
> 7468 ?        I    0:00.00 /usr/pkg/sbin/httpd -k start
> 8677 ?        I    0:00.00 /usr/pkg/sbin/httpd -k start
> 8758 ?        I    0:00.01 /usr/pkg/sbin/httpd -k start

ps 'axs' will show threads within the httpd processes.

> 
> www# httpd -l|grep pre
>  prefork.c
> 
> <IfModule mpm_prefork_module>
>    StartServers         10
>    MinSpareServers       5
>    MaxSpareServers      10
>    MaxClients          150
>    MaxRequestsPerChild   0
> </IfModule>

Hmm, looks like httpd is not using threads to service requests.

> www# pkg_info |grep -i apache
> apr-1.2.12          Apache Portable Runtime
> apr-util-1.2.10     Apache Portable Runtime utilities
> apache-2.2.8        Apache HTTP (Web) server, version 2
> ap22-php5-5.2.5nb1  Apache (apache22) module for PHP5
> ap22-bw-0.6nb2      Class-based bandwidth limiting module for Apache 2
> 
> I'll note that stopping and starting now does not seem to cause the problem.
> 
> With the other one I reported for webmin, that is 100% repeatable as it 
> starts from rc.local ... well attempts to but never actually does. 
> Attempting to start it _after_ boot seems to work. The error is always 
> generated at boot. The same domu that fails with apache, starts webmin 
> fine at boot. I'm guessing if I had of left it for a while rather than 
> rebooting, apache probably would have stopped. There seems to be a time 
> factor and it seems to occur early on in boot/after boot.
> 
> If there is more I can provide please ask.

Could you try a ktrace -di on the failing command? Hopefully it does not
seteuid() which is likely to break the trace.

Thanks,
Andrew