Subject: Re: Bug in x86 ioapic interrupt code for devices with shared interrupts?
To: None <jonathan@dsg.stanford.edu>
From: Quentin Garnier <cube@cubidou.net>
List: tech-kern
Date: 03/03/2006 23:29:45
--wshT/1ut16otUOEK
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Fri, Mar 03, 2006 at 01:35:41PM -0800, jonathan@dsg.stanford.edu wrote:
>=20
> In message <20060303211035.GA943@antioche.eu.org>, Manuel Bouyer writes:
> >On Fri, Mar 03, 2006 at 04:02:19PM -0500, Thor Lancelot Simon wrote:
> >> On Fri, Mar 03, 2006 at 09:53:15PM +0100, Manuel Bouyer wrote:
> [...]
>=20
> Thor, Manuel, Jason: =20
>=20
> Can I add one further piece to the puzzle?

I'll add another.  A story not involving bge.

> I've never seen problems quite like Thor describes.  But something I
> *have* seen is that when I put a multi-port PCI-Express bge into an
> amd64+nforce4 NetBSD machine, and I ifconfig one of the bge's up,
> NetBSD reports about 70,000 interrupts/sec, but none of the interrupts
> actually get to the bge driver.
>=20
> Ever since that, I've been quietly suspecting we won't actually fix
> all the amd64/nforce4 problem without implementing MSI.
>=20
>=20
> I'm really *not* looking forward to what I'll find, once I get to try
> an Opteron system with the Broadcom/Serverworks HT-2000, which
> (according to Broadcom's product sheet) has an on-chip version of the
> same multi-port bge core.=20
>=20
> Will the HT-2000 blow up the same way as nforce4, or won't it?
> Inquiring minds want to know. Or don't want to. Or something.

I've experienced a weird issue on an Intel motherboard a few weeks ago.
The mobo is the D945PSN, equipped with a Pentium D.  I think I've
mentioned that experience before.  I had that issue that the BIOS
wouldn't fill in the IRQ line register in a device's configuration
space.  Using ioapic allowed me to receive interrupts, though.  I was
merely testing interrupt generation, so the device was a very basic
mode where once triggered, it tries as often as possible to generate
interrupts until it is stopped.  My straightforward driver had a handler
that stopped it the first time it received the interrupt.

Although the irq number was left to 0xff, using ioapic(4) made it just
work, except in one case, and that's where things got weird.  I told
Manuel about this already, but I've not figured why it happens.

The ICH7 can be set in several modes for the Legacy/SATA/Whatever crap,
and in one of these modes, one of the slot got connected to same pin of
the ioapic as the SATA controller.  wd0 appeared on that controller, so
the piixide(4) device was obviously receiving interrupts.

Yet my interrupt handler (which printf()'d something no matter what)
never received the interrupts in that configuration, and moreover,
according to vmstat -i, the line was indeed getting ~230k
interrupts/sec, which means the device, as it was supposed to, was
frantically issuing interrupts.

And that's on i386.  How bizarre is that?

--=20
Quentin Garnier - cube@cubidou.net - cube@NetBSD.org
"When I find the controls, I'll go where I like, I'll know where I want
to be, but maybe for now I'll stay right here on a silent sea."
KT Tunstall, Silent Sea, Eye to the Telescope, 2004.

--wshT/1ut16otUOEK
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (NetBSD)

iQEVAwUBRAjDWdgoQloHrPnoAQJrsgf7BzjUMYp+iqBeWeN95iUhz2Rkn9rHfOX3
IJfJ10S0T/Rz3q4muDGEC4zfCuxe4joJWJKvMa2fYyGDNjQ8D0+1vXAphbuXI1fQ
6CMbE4Xa+wg5dSJ338zW1Ice+lKegxMPPpf20zAwUj4/WabFO0v72mW879WEoxOT
8pCtRj/ksMg1c4iHoUAq4OuL7j5pa59Ud4Bv3D8Tm3bs0QbNaDgABEa7J9YZLEAX
Xl8Z0jQy42vR41O2klvRZ/+R+7KyFN+2XFzm/b7rA4cWvN6LHDlDpaanaGekB/T1
XgWvZpil46yxwpCLnuwBEEDHPYlJ7PLsI5lwhQ3aNWKupW+B5Uo5Ow==
=bNRn
-----END PGP SIGNATURE-----

--wshT/1ut16otUOEK--