Subject: Re: problem with free/malloc?
To: Manuel Bouyer <bouyer@antioche.eu.org>
From: Lista de NetBSD current <list11@sepc.edu.mx>
List: current-users
Date: 08/23/2002 13:15:30
I am sorry...
2 minutes after my last mail, the problem started again.
The file /var/log/httpd/error_log is 1.13 GB

dragon# uniq /var/log/httpd/error_log | tail
[Thu Aug 22 22:00:02 2002] [notice] Apache/1.3.24 (Unix) PHP/4.1.2
configured -- resuming normal operations
[Thu Aug 22 22:00:02 2002] [notice] Accept mutex: flock (Default: flock)
[Fri Aug 23 10:42:07 2002] [notice] child pid 8422 exit signal
Segmentation fault (11)
[Fri Aug 23 10:55:56 2002] [notice] child pid 8759 exit signal
Segmentation fault (11)
[Fri Aug 23 10:59:29 2002] [notice] child pid 8776 exit signal
Segmentation fault (11)
httpd in free(): warning: chunk is already free.
[Fri Aug 23 10:59:48 2002] [notice] child pid 8777 exit signal
Segmentation fault (11)
[Fri Aug 23 10:59:49 2002] [notice] child pid 8423 exit signal
Segmentation fault (11)
[Fri Aug 23 11:06:30 2002] [notice] child pid 8780 exit signal
Segmentation fault (11)
httpd in free(): warning: chunk is already free.
[Fri Aug 23 11:07:35 2002] [notice] child pid 8896 exit signal
Segmentation fault (11)
[Fri Aug 23 11:07:35 2002] [notice] child pid 8763 exit signal
Segmentation fault (11)
[Fri Aug 23 11:07:35 2002] [notice] child pid 8431 exit signal
Segmentation fault (11)
httpd in free(): warning: chunk is already free.
[Fri Aug 23 11:08:23 2002] [notice] child pid 7595 exit signal
Segmentation fault (11)
httpd in free(): warning: chunk is already free.
httpd in free(): warning: page is already free.
[Fri Aug 23 11:08:31 2002] [notice] child pid 8430 exit signal
Segmentation fault (11)
httpd in free(): warning: chunk is already free.
[Fri Aug 23 11:09:36 2002] [notice] child pid 8428 exit signal
Segmentation fault (11)
httpd in free(): warning: chunk is already free.
httpd in free(): warning: [Fri Aug 23 11:09:52 2002] [notice] child pid
7593 exit signal Segmentation fault (11)
httpd in malloc(): warning: recursive call.
FATAL:  emalloc():  Unable to allocate 2 bytes
httpd in free(): warning: chunk is already free.
[Fri Aug 23 11:10:40 2002] [notice] child pid 8895 exit signal
Segmentation fault (11)
httpd in free(): warning: chunk is already free.
httpd in free():[Fri Aug 23 11:44:59 2002] [notice] child pid 9097 exit
signal Segmentation fault (11)
httpd in free(): warning: chunk is already free.
httpd in malloc(): warning: recursive call.
FATAL:  emalloc():  Unable to allocate 2 bytes
httpd in free(): warning: chunk is already free.
httpd in free(): warning: modified (chunk-) pointer.
httpd in free(): warning: chunk is already free.
[Fri Aug 23 11:57:31 2002] [notice] child pid 7641 exit signal Abort (6)
httpd in free(): warning: chunk is already free.
httpd in free(): warning: page is already free.
[Fri Aug 23 12:09:42 2002] [notice] child pid 9101 exit signal
Segmentation fault (11)
httpdhttpd in free(): warning: chunk is already free.
 in free(): warning: chunk is already free.
httpd in free(): warning: chunk is already free.
[Fri Aug 23 12:16:45 2002] [notice] child pid 8779 exit signal
Segmentation fault (11)
httpd in free(): warning: chunk is already free.

At this point tha machine was freezed and I lost my ssh session.
The following is before power off/on

[Fri Aug 23 12:39:10 2002] [notice] Apache/1.3.24 (Unix) PHP/4.1.2
configured -- resuming normal operations
[Fri Aug 23 12:39:10 2002] [notice] Accept mutex: flock (Default: flock)

and here I stopped the httpd service
[Fri Aug 23 12:40:13 2002] [notice] caught SIGTERM, shutting down

I was working with MinSpareServers 5 and MaxSpareServers 24.

Please somebody can give me tips in order to find the problem?

Thanks in advance

Heron Gallegos
--------------
I will return to min/max of 5/16 and I will put
some ulimit for datasize (here I dont know exactly
what am I doing... the server have 512 MB RAM) and
restart apache
----------------------


On Fri, 23 Aug 2002, Lista de NetBSD current wrote:

> Date: Fri, 23 Aug 2002 10:40:36 -0500 (CDT)
> From: Lista de NetBSD current <list11@sepc.edu.mx>
> To: Manuel Bouyer <bouyer@antioche.eu.org>
> Cc: current-users@netbsd.org
> Subject: Re: problem with free/malloc?
>
> On Thu, 22 Aug 2002, Manuel Bouyer wrote:
> > On Thu, Aug 22, 2002 at 01:44:54PM -0500, Lista de NetBSD current wrote:
> > >
> > > This server works very fine with this values of httpd.conf:
> > >
> > > MinSpareServers 5
> > > MaxSpareServers 16 <- too low for num of users
> > >
> > > When I increase MaxSpareServers to 32 and restart apache,
> > > and the load of users begins, I have hundreds of messages
> > > in /var/log/httpd/error_log:
> > >
> > > httpd[Thu Aug 22 09:17:02 2002] [notice] child pid 6676 exit signal Segmentation fault (11)
> > > [Thu Aug 22 09:20:32 2002] [notice] child pid 6671 exit signal Segmentation fault (11)
> > > httpd in malloc(): warning: recursive call.
> > > FATAL:  emalloc():  Unable to allocate 2 bytes
> > > httpd in malloc(): warning: recursive call.
> > > FATAL:  emalloc():  Unable to allocate 2 bytes
> > > httpd in free(): warning: chunk is already free.
> > > httpd in free(): warning: chunk is already free. <---- hundreds like this
> > > httpd in free(): warning: chunk is already free.
> > > [Thu Aug 22 09:26:56 2002] [notice] child pid 6673 exit signal Segmentation fault (11)
> > >
> > > I modified the startup scripts of apache and mysql with
> > > this line: "ulimit -n 2048"
> >
> > Maybe you also need to bump maxproc and/or datasize for apache.
>
> THANKS!!!!!
> I added the line "ulimit -p 512" to the apache startup script
> and all is working well now. See the difference:
>
> with sysctl -a
> proc.curproc.rlimit.maxproc.soft = 80
> proc.curproc.rlimit.maxproc.hard = 1044
> proc.curproc.rlimit.descriptors.soft = 64
> proc.curproc.rlimit.descriptors.hard = 3404
>
> with sysctl proc.pid
> proc.7591.rlimit.maxproc.soft = 512
> proc.7591.rlimit.maxproc.hard = 512
> proc.7591.rlimit.descriptors.soft = 2048
> proc.7591.rlimit.descriptors.hard = 2048
> (the proc 7591 is father of all httpd's and owned by root)
>
> Experience 1
> To compile the kernel with "exclusively the hardware found"
> is not enough. Is necessary read and understand the options
> of the kernel config file. I would like to know the impact
> of changing maxusers to 128 or 256, or understand if NMBCLUSTERS
> are applicable for this case, etc. All this for have a
> "special kernel" for apache+php+mysql and taking into
> acount the hardware, num of tables in db, num of httpd's,
> etc.
>
> Experience 2
> In this server php, imp/horde are configured to send the
> messages via local1 of syslogd. It has been working well
> except for 3 or 4 messages that were sent to my screen
> (ssh session) instead the log file. It was in the 3 days
> of the problem. I dont know if it is a problem of syslogd
> or php, but under our circunstances, had an erratic way
> of work.
>
> Thanks again
>
> Heron Gallegos
>
>