Subject: Re: SMP re-eetrancy in "bottom half" drivers
To: Stephan Uphoff <ups@tree.com>
From: Jonathan Stone <jonathan@dsg.stanford.edu>
List: tech-net
Date: 05/17/2005 16:21:59
In message <1116370425.7597.824.camel@palm>,
Stephan Uphoff writes:

>No need to.
>We both agree on the cost of context switches and talked to some of the
>same people. This being said there is no need to change the MMU for an
>interrupt thread (the kernel is always mapped on x86 - gentle
>clue-by-four as requested) but FreeBSD's context switch is not as light
>as it could be.

I haven't looked in a few months; but last time I did look, I asked
Nathan Williams, and Nathan confirmed the i386 code was thwacking the
CR-whatever register rather more than we needed to (possibly for
kernel-only to kernel-only cases?) I think that's been addressed; tho'
obviously I need to check in detail if I'm going to work in this
stuff.

[... sun tricks...]


>I spend quite some time recently looking at the NetBSD x86 spl code.
>Some of the recent changes to fix race conditions added some extra
>interrupt enable/disable operations to common code paths.
>These are relatively expensive operations and I would like to eventually
>rewrite the regular spl code paths to be able to run with interrupts
>always enabled. This being said my track record on actually getting any
>time for my NetBSD projects is not great this year :-(.
>I am still recovering from BSDCan but should be able to think more
>constructively about the SMP driver problem in a few days.
>
>> One might even make a case that the work done in NetBSD so
>> far, to improve parallelism for userspace code just might be
>> detrimental to improved SMP parallelism in the kernel.  I don't want
>> to go that far myself, though.
>
>Mhhh - I only recall vaguely that I was not to happy about threads being
>pinned to a CPU to keep SA happy on SMP. However I have not looked at SA
>for a long,long time and may be totally wrong here.
>Is this what you are referring to?
>( If this is not the problem then could you please return the
>clue-by-four )


I'm thinking of ancient history, so I may well be misremembering.  But
as I recall, when this last came up, circa Jan 2004, there was some
opposition to introducing a strict hierarchy of SPLs and locks.  OTOH,
for one, Jason just said he's no longer opposed. I don't recall
who else was strongly against the idea.

My own grip of the NetBSD's current locking protocol for the biglock
vs. scheduler locks, lacks opposable thumbs. But (though I may be
hopelessly out-of-date), I could easily see that needing some re-work,
to fit into the kind of strict deadlock-free hierarchy Bill and I (and
others?) are talking about.

Personally, I'd sooner see some progress on making some MI drivers
SMP-safe, so that there's something to measure, whereby different
SPL/lock hierarchies and implementations (or other completely
different schemes) can be compared and evaluated.

Right now we're in a vacuum. Unless we start somewhere, we'll
never get out of that vacuum....