Re: RFC: softint-based if_input

[Ryota Ozaki <ozaki-r%netbsd.org@localhost>]

On Thu, Jan 28, 2016 at 2:44 PM, Taylor R Campbell
<campbell+netbsd-tech-kern%mumble.net@localhost> wrote:
>    Date: Thu, 28 Jan 2016 11:58:05 +0900
>    From: Ryota Ozaki <ozaki-r%netbsd.org@localhost>
>
>    On Thu, Jan 28, 2016 at 12:17 AM, Taylor R Campbell
>    <campbell+netbsd-tech-kern%mumble.net@localhost> wrote:
>    >             So vanilla might make less efficient use of the CPU cache,
>    > and vanilla might leave the rxq full for longer so that the device
>    > cannot fill it as quickly with incoming packets.
>
>    That might be true. If so, the real question may be why the old
>    implementation isn't efficient compared to the new one.
>
> By `the old implementation', do you mean the one with pcq instead of
> ifq, softint-rx?

Yes.

>  It's only a little bit slower than softint-rx-ifq.
> My wild guess is that all the atomics and memory barriers in pcq slow
> it down.

Perhaps. Though, I have no motivation to investigate it that I already
threw up :-/

>
>    > Another experiment that might be worthwhile is to bind the interrupt
>    > to a specific CPU, and then use splnet instead of WM_RX_LOCK to avoid
>    > acquiring and releasing a lock for each packet.
>
>    In the measurements, all interrupts are already delivered to CPU#0.
>    Removing the lock doesn't change the results. I guess acquiring and
>    releasing a lock (w/o contentions) are low overhead. Note that
>    wm has a RX lock per HW queue, so RX processing can be done with no
>    lock contention basically.
>
> OK, makes sense.
>
>    >  (On Intel >=Haswell,
>    > we should use transactional memory to avoid bus traffic for that
>    > anyway (and maybe invent an MD pcq(9) that does the same).  But the
>    > experiment with wm(4) is easier, and not everyone has transactional
>    > memory.)
>
>    How does transactional memory help?
>
> It doesn't.  I was just blathering aimlessly like I often do about
> things I don't understand well enough but am idly thinking about.  (It
> also turns out -- I hadn't heard until today -- that transactional
> memory is broken on Haswell!)

NP. I'm also interested in how it (and persistent memory) affects
OS design and facilities :)

  ozaki-r