On Tue, 26 Aug 2003, Jun-ichiro itojun Hagino wrote:

> > > 	sorry, which part of the cost is the concern?  why it is so bad to
> >
> > so->so_proto->pr_domain->dom_family
> >
> > That's a lot of cache churn. On IA32, we can kinda get by. But on other,
> > less-beefy architectures, that really hurts.
> >
> > The thing to remember is that with caches, the first de-reference of a
> > pointer is expensive. The cost I've heard is around 1000 clock cycles.
> > That will vary with architecture, but suffice it to say it'll be
> > expensive.
> >
> > And if dom_family is all that we're getting out of pr_domain, then we're
> > poisoning the cache for little value.
>
> 	the pointer dereference is really a concern, you can keep so-to-family
> 	mapping table cache.

Ok. Why not just include said cache in the in{,6}pcb? Having multiple
caches doing the same thing doesn't seem like a good idea. Assuming I'm
understanding your suggestion right.

Note also that so->so_proto->pr_domain->dom_family is three pointer
de-references. So that's 3x pointer cost.

> > > 	using a flag to distinguish between in{,6}pcb and sharing the structure
> > > 	means that you need to update every kmem grovellers which looks at
> > > 	inpcb structure.  it's not only netstat, but also all other thirdparty
> > > 	packages including trafshow and such.  this is the major (i guess the
> > > 	most important) reason why i did not touch inpcb structure when we
> > > 	integrated KAME into netbsd.
> > Well, adding a common pcb header will probably break things anyway. I'd
> > say do the common PCB trick, and shove in a flag. It's much cleaner.
>
> 	where did the "principle of least surprise" go?  we should be more
> 	careful when proposing a change that would affect every thirdparty
> 	software (well, those who grovels inpcb).

Careful, yes. But stagnant, no. If the change is really useful (and
getting rid of so->so_proto->pr_domain->dom_family and cleaning things up
sounds like it is), then we should make the change.

We also have pkgsrc, where we can fix things, so that our users don't end
up hosed. And to be honest, we have changed a number of other structures
that have had (AFAICT) comparable impact on kvm-grovelers.

Also, totally unrelated note, what happens if you're an IPsec gateway? Say
you're doing ESP tunnel mode for a number of protected boxes. And you have
nothing running that has an open port covered by the IPsec policy (or you
have port-specific policy and nothing's open on that port). Do you really
have a socket sitting around to hang the IPsec info off of, even though
there's nothing in userland around to hook to it?

Take care,

Bill