On Wed, 21 Feb 2001 sudog@sudog.com wrote:

> What I've noticed is that there are three thresholds that I'm
> currently trying to "feel" out.
>
> 1. The threshold of maximum processes the machine is sysctl'd to.
>
> 2. The maximum number of apache processes/cgi processes, controlled by
> shell ulimits, that the server software is currently limited to.
>
> 3. The actual number of apache processes that apache limits itself to.
>
> I've noticed some patterns.
>
> #2 must not exceed #1-x% where x is some small single-digit number. If
> it exceeds #1-x%, things choke and I need to reboot the machine. (This
> load is driven mostly by users clicking and clicking and hitting
> refresh etc.)
>
	If it chokes can you pull the network cable and get control of
	the machine back, or has it really gone out to lunch?

> If #3 (and corresponding perl cgi children) exceeds #2, apache stalls
> and is no longer capable of handling cgi. It must be SIGHUP'd to be
> corrected.
>
	Hmm - have you ha a chance to try to debug what apache is doing
	in this case?

	It would be nice if apache could be told to 'tune down' the
	max processes when hitting a fork failed condition (presumably
	to another 'MinMaxProcesses' limit).

> It's a juggling act once the kernel is tuned to be capable of dealing
> with the huge limits and doesn't panic any more under the loads.
>
	Were the panics purely down to tuning the kernel params? Are
	there PRs open on them?

>[raidframe]
>
> 2. With RAM at a premium, another balance must be achieved: That of
> file system caching to actual RAM. I could increase the amount of
> memory dedicated to file system caching; but this needs to be
> considered in conjunction with how many users are connected at once.
> Too much, and there's too much wastage and machinery isn't living up
> to its potential. Too little and the users can starve the system.
>
	NetBSD -current supports a unified buffer cache, which will
	hopefully eliminate the need to tune this at boottime, though
	I'd imagine it will have some parameters that can be tweaked.

> I really need to figure out some specific equations to decide where to
> set these limits..  do some testing or profiling to find out how long
> on average each perl child takes, how much it accesses the disk and in
> what patterns, and then how much the system can actually handle of
> this..  how much users use the CGI as opposed to normal HTML links as
> a ratio.. and work this into the daily traffic.
>
> This is so complicated, I was hoping another user here could offer
> some common server configurations based on available RAM and disk type
> (IDE vs SCSI) that they've been able to work with and not have to
> worry about things getting out of control.

	Are you using mod_perl or fastcgi on the server? Switching
	a heavy perl/postgresql cgi client to fastcgi increased its
	apachebench top rate from 1.8 hits per second to over 80.

	It would be really nice at the end to have a doc to put up on
	www.netbsd.org - would you be willing to put one together? :)


		David/absolute		-- www.netbsd.org: No hype required --