Re: Finding bottlenecks on a proxy server

[Brett Lymn <blymn%internode.on.net@localhost>]

On Mon, Mar 04, 2019 at 03:42:05PM +0000, Stephen Borrill wrote:
> 
> The second squid process runs as user nobody and is (currently) configured
> to do no caching or access logging. I'll refer to it as squidnc (nc = no
> cache). It listens on port 8123. It has the same process limits as the first
> squid process.
> 

It may help slightly if squidnc was caching too, at least there may be
some content that can come from cache instead of an internet fetch
though with the push to https this is less likely these days...

> At busy times, the dansguardian processes stack up and hit the 250 limit.
> Web access then slows to a crawl as requests queue waiting for a free child.
> At that point users start shouting and time to investigate is short.
> 

OK - are you able to bump up the number of dansguardian processes so
there are enough to cope with the backlog of requessts at busy times?

> 
> My theory is that squidnc is the bottleneck (even though it is doing the
> least work), however I do not have any hard evidence of this. I am looking
> for help on finding and fixing any such bottlenecks or, if I'm looking in
> entirely the wrong place, suggestions of better places to look.
> 

I suspect it is more likely that squidnc is simply waiting for responses
from the remote servers which can be a slow process.  This is, of
course, assuming your internet connection is not saturated.  You didn't
mention it but I am taking it as read that this has been checked :)

I haven't used dansguardian but have structured a very similar chain
using a commercial filtering product and had a similar issue as you are
having.  I think you just need to keep bumping the dansguardian child
pool until there are enough processes to handing the peak request rate.

-- 
Brett Lymn
"We are were wolves",
"You mean werewolves?",
"No we were wolves, now we are something else entirely",
"Oh"