Subject: Re: splx() optimization [was Re: SMP re-eetrancy in "bottom half" drivers]
To: Stephan Uphoff <ups@tree.com>
From: Bill Studenmund <wrstuden@netbsd.org>
List: tech-kern
Date: 06/02/2005 18:39:01
--p4qYPpj5QlsIQJ0K
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Thu, Jun 02, 2005 at 02:56:28PM -0400, Stephan Uphoff wrote:
> On Thu, 2005-06-02 at 14:36, Jonathan Stone wrote:
> > Bill STudenmund writes:
> >=20
> > >On Wed, Jun 01, 2005 at 03:01:11PM +0900, YAMAMOTO Takashi wrote:
> > [[ Jonathan claims a consensus for ordering of SPLs]]
> >=20
> > >> i don't see any needs to have all interrupts ordered.
> > >> i won't object if you want to make them ordered as
> > >> a transient convention, tho.
> >=20
> > >I thought the point was that interrupt handlers now will use spin lock=
s to
> > >protect their data. When an interrupt handler runs, it will run with S=
PL
> > >set to block its interrupt on the CPU it's on, and it will also grab a
> > >spinlock so that no other CPU services this interrupt (and also keep o=
ther
> > >processing that would now block interrupts out of the data).
> > >
> > >The locks will need to be ordered to avoid deadlock. As there will be a
> > >correspondence between locks and SPL, we will then need to order SPL to
> > >avoid deadlock.
> > >
> > >Thus we will have a hierarchy to SPL levels.
> > >
> > >Am I missing something?
>=20
> Mhhh - not all locks need to be strongly ordered.
> Maybe we can do the same for spl levels?

Well, the problem is that we'd then have to figure out exactly what locks=
=20
are at the same level, and document it. While I'm not against doing it, I=
=20
think Jonathan is right that it's an optimization.

> > Hi Bill,
> >=20
> > That's pretty much the argument I was making in late 2003/early 2004;
> > thank you for a clear concise statement.   I see Yamamoto-san said, in =
reply:
> >=20
> >=20
> > >i have objection if you implies global IPLs.
> > >(like your original proposal in 2003)
> >=20
> >=20
> > Now that leaves me confused! I thought what Bill outlines is, in
> > essence, what I proposed in late 2003. Yamamoto-san, do they sound
> > like the same thing to you? Do we have a consensus for it (or for one
> > or other, if they are different)?
>=20
> Global as in contrast to per CPU?
> I think we all agree that we want per CPU. Right?
> Do you have a link handy to your 2003 proposal?=20

Oh, I see what's wrong. I'm talking about a global IPL ORDERING, but not=20
necessarily a global IPL state. It was only now that I noted that yamt=20
omitted the word "ordering" in his objection.

How certain are we that all our code has been changed to not abuse spl=20
ordering? It used to be that some spl levels implied all the lower ones=20
too. Translated into locks, that means that if a cpu wants to set a=20
certain slp, it would need to implicitly grab a whole lot of locks (the=20
one for this spl and all lower ones). I _think_ we cleaned that out, but=20
I'm not sure.

If our code is clean, then I think we're fine, and we can have a given=20
spl only grab one level of locks. So you could have different heads in the=
=20
system operating at different spl levels.

Take care,

Bill

--p4qYPpj5QlsIQJ0K
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (NetBSD)

iD8DBQFCn7S1Wz+3JHUci9cRAn/fAJ9hfaB5BguFr0+vaotMZwVyyRIxNwCeNexe
nDJ76B9UTo7c6udjuRK6z8U=
=TeMX
-----END PGP SIGNATURE-----

--p4qYPpj5QlsIQJ0K--