Subject: Re: ipfilter performance with 'count' rules on NetBSD-1.4/i386
To: None <firstname.lastname@example.org, email@example.com>
From: Ross Harvey <firstname.lastname@example.org>
Date: 09/14/1999 14:27:59
> From: Erik Rungi <email@example.com>
> On Tue, 14 Sep 1999, David Maxwell wrote:
> > Additionally, you specifically stated that you didn't use 'quick' in
> > any of the rules. Would you mind running the test with quick on all
> > the rules? Depending on the distibution of destination, it should cut
> > the workload in half. If the addresses for your rules don't overlap,
> > you could then optimize the most used rules to the top of the list.
> In the case of optimization, you have to keep track of hits, but I'm sure it
> could be done. It would would be nice if the kernel could do this.
> > If you have rules that do overlap subnets, you could generate the
> > total by adding the smaller networks at 'stats' time. (Assuming you
> > aren't collecting for a portion of a subnet, and the whole net,
> > but not another portion - if so, change the whole subnet one to collect
> > the other portion with a quick rule, and sum the counts later.)
> In my case I'm adding up individual IPs. With this case it has also been
> suggested that I break the rules down using the HEAD/GROUP keywords which
> could bring the processing time down by a factor of LOG(N) in theory. In this
> case, optimization wouldn't really work (same time to get to the bottom of the
> tree in any case). But the configuration file would be soooo ugly.
Well, using a tree structure with head/group was going to be my suggestion
too. This should drastically reduce the time required. I was fairly impressed,
BTW, with how fast ipf worked in the test.
I'm really confused about ``soooo ugly''. Using and head and group to make
a tree isn't in any intrinsic way ugly.
For a really complex job like this, you very definitely want a meta-
configuration file that is processed by some tool. I suspect you are already
expanding something else with a perl or sed script, right? I mean, I presume
you didn't type in thousands of rules.
So you just need a trickier one. You need some kind of meta term like
``... count 192.67/16'', that will recursively expand it bit by bit,
creating the obvious binary tree.