Re: TCP and NET_MPSAFE

[Jason Thorpe <thorpej%me.com@localhost>]

Hey Kevin - sorry, I was sitting on this email thinking about it, and finally had some spare brain cycles to really pay it some proper attention.

> On Apr 19, 2026, at 5:17 PM, Kevin Bowling <kevin.bowling%kev009.com@localhost> wrote:
> 
>> 2) TCP becomes fast enough that I can starve out SOFTINT_CLOCK on a
>> single core iperf3 -R on the ER4 system (requires driver improvements
>> I will publish later), and it is likely possible on other hardware
>> sizes.  The effects of this can be somewhat dire, for instance TCP
>> timers stop working, or if you are running a watchdog it won't get
>> poked in time.  This needs either a rethink of softint priority, or
>> moving some of the work out of softint to something the scheduler can
>> rotate.

I definitely agree that a re-think of the software interrupt priorities are in order.  It seems to me that the logical ordering should be more like:

    clock > net == serial > bio

And that where should be another “general” (because “clock” has served as “general” historically) at the bottom below bio (would hate for some random thing to get in the way of a page-in completion), so:

    clock > net == serial > bio > gen

My logic for equating “net” and “serial” goes a little something like this: today’s high-throughput network interfaces have tight performance constraints that might bump them above the servicing of UARTs, but there are cases where UARTs (or, serial line devices generally) are part of the networking stack, and putting the below networking in the priority order seems like an icky inversion.

If a platform really wanted to use hardware support as the scheduling trigger and had 2 hardware levels to throw at it, I would say:

    clock > net == serial >= bio >= gen

I.e. clock always on top, net and serial always equal, and an opportunistic (or software-only) prioritization below that.  But I’m willing to be convinced otherwise if there’s a good argument for a different ordering.

-- thorpej