Subject: Re: Skipping TCP / UDP / IP checksums on loopback traffic
To: Nathan J. Williams <nathanw@wasabisystems.com>
From: Jonathan Stone <jonathan@dsg.stanford.edu>
List: tech-net
Date: 10/25/2004 10:33:19
In message <mtuwtxelod6.fsf@contents-vnder-pressvre.mit.edu>,
"Nathan J. Williams" writes:

>Can you give a more exact pointer for those of us who don't generally
>keep up with the networking literature?

Its folklore. I can give you [just you, personally] letters-of-introduction
to the relevant greybeards, if you wish.


>> Some of us who tweak the network stack use lo0 as a
>> measurement-proxy for the stack itself, independent of the vagaries
>> of any particular NIC or LAN.  In my own little corner of the world,
>> that's the primary purpose for lo0.


>[...]
>
>> If you *want* a high-speed, low-overhead local IPC transport, then use
>> PF_LOCAL. That's what it's for.
>
>This seems like a philosophical point as much as a practical one, and
>I can imagine [but will not make] an argument that lo0 is the
>"natural" local transport and that PF_LOCAL is redundant, and that
>your benchmarking usage is not the case we should be accomodating.

I wouldn't care to see anyone make that argument, given Jason's stated
rationale was *specifically* for performance relative to Linux on
certain [again, we seem to agree such benchmarks are bogus?].

I'm still not aware of any real-world case where lo0 throughput is a
significant bottleneck?  Speed up something that's well under 10% of
your application, by some 10%[*]? Not a big win.


Quite, honestly, I think I'd have less objection to code which
performed lo0 checksums in the outbound direction, for all protocols;
and then honoured the existing hw-checksum-offload flags in the
receive direction.

But then we'd also want to fix any remaining abuses of lo0 to deliver
post-IPsec-inbound-transform packets via lo0. (KAME gets this right
now, for IPv6, dunno about IPv4; FAST_IPSEC still does the wrong thing.)

[*] assuming all the data-touching for checksums hits in L1 CPU cache,
after the original copyin(). Not a bad bet, given packet sizes and
today's CPU cache sizes.