Re: Network problems on busy server

[Greg Oster <oster%cs.usask.ca@localhost>]

On Sat, 2 Jun 2012 19:56:41 +0400
Dima Veselov <i%kab00m.ru@localhost> wrote:

> Hello!
> 
> I have several NetBSD boxes, working for yrs, but one of them seem to
> be more loaded than others and expriencing some network issues.
> 
> Issues I have today:
> 1. IMAP4 mailbox, opened in mutt sometimes appear to be closed by
> itself 2. SSH window can close even if you work on server (when much
> people in the office working)
> 3. Asterisk drop calls (rare, but sometimes)
> 4. Internet going through sometimes get slow.
> 
> Problems do not put me in trouble, however I like thing being clear
> and want to configure it properly.
> 
> Main load of server caused by:
> 1. All the internet load of the company
> 2. Postfix/dovecot mail server
> 3. Asterisk
> 4. Ipfilter
> 5. DNS server
> 6. Number of IPSEC tunnels
> 7. Small web-server
> 
> It never has more than 1Gb memory used, never has more than 3% CPU
> load and don't use swap much (59M of 4096M currently), so I assume i
> just meet some limitations (network connections or something like
> that).
> 
> At weekend ipf state table show up 500 states, netstat -an shows
> about 500 lines (mostly about connected phones).  Ipf rules table
> says 130 lines.
> 
> Also in kernel config file i have:
> maxusers        128
> options         CHILD_MAX=1024   # max simultaneous processes
> options         OPEN_MAX=512    # max open files per process
> options         SHMMAXPGS=59400
> options         SHMSEG=512
> options         SEMMNI=512      # Maximum number of sets of IPC semaph
> options         SEMMNS=1024     # Sys-wide max number of individual
> IPC options         SEMMNU=512
> options         SEMMAP=512 
> 
> options         NMBCLUSTERS=4096

My guess is that this is where you're running out of
resources...  I used to run with 4096 too, but find now that a)
-current and netbsd-6 auto-tune this value to something typically much
larger than 4096 and b) 4096 just wasn't cutting it on my firewall any
more.

You can use 'netstat -m' to see how many of these clusters are in
use...  What I noticed on my systems is that I'd hit the 4096 limit
about the same time that performance went south.. Turns out that for
what I was doing I really needed about 15000 to operate in the 'normal'
range, and about 20000 to have a bit of a cushion.

My bet is that if you crank this up to 65536 or something that
your network performance issues will disappear.. (mine did...)

Later...

Greg Oster