NetBSD-Bugs archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
Re: port-i386/53894: "ehci_sync_hc: timed out" with ACPI/SMP enabled on eBox 3352DX3-AP
The following reply was made to PR port-i386/53894; it has been noted by GNATS.
From: Andrius V <vezhlys%gmail.com@localhost>
To: gnats-bugs%netbsd.org@localhost
Cc:
Subject: Re: port-i386/53894: "ehci_sync_hc: timed out" with ACPI/SMP enabled
on eBox 3352DX3-AP
Date: Wed, 18 Mar 2026 19:34:43 +0200
On Tue, Mar 5, 2024 at 9:57=E2=80=AFAM Andrius V <vezhlys%gmail.com@localhost> wrote:
>
> On Wed, Mar 16, 2022 at 10:25=E2=80=AFAM Andrius V <vezhlys%gmail.com@localhost> wr=
ote:
> >
> > Hi,
> >
> > I did some investigation on this issue in recent days and here are my f=
indings:
> > * I can confirm, that it is interrupts issue, more specifically level
> > type PCI interrupts are not working (thus, USB, network, SCI, probably
> > audio too (untested)) interrupts doesn't work (checked by intrctl and
> > vmstat).
> > * The interrupt establishing flow seems OK though, did some debugging,
> > which shows that the flow doesn't seem to be different from similar
> > system which has no issues and visually everything is "established".
> > * By advice I checked MADT overrides: system reports two overrides,
> > and indeed, it seems that IRQ 0 NetBSD do not override. Unfortunately,
> > I have no idea, where to do that in NetBSD code (FreeBSD seems to go
> > through all MADT table and do overrides, in NetBSD case only SCI
> > interrupt individually comes to override code). Would that fix the
> > issue, honestly, I am not certain, I did try silly hacks, like not
> > masking interrupts by default, which added "i82489_ipi_init: ESR
> > 00000040" to dmesg, but none helped to make PCI interrupts work.
> > acpi0: [MADT] INTR OVERRIDE : BUS 0, IRQ 0, GSI 2, FLAGS 0x00
> > acpi0: [MADT] INTR OVERRIDE : BUS 0, IRQ 9, GSI 9, FLAGS 0x0D
> > * SoC does not support MSI.
> > * current kernel doesn't report "ehci_sync_hc: timed out" anymore
> > compared to my original dmesg report, just silently boots without any
> > uhidev devices attached, likely related to recent USB changes.
> > * FreeBSD 12.0 (the last release which boots out of the box on the
> > system), and recent Linux kernels has no problems with these
> > interrupts. OpenBSD have likely the same issue as NetBSD though, since
> > USB and network also doesn't work.
> > * Booting without ACPI/SMP enabled, makes interrupts work (it doesn't
> > go through apic in this case of course).
> >
> > Unfortunately, I am out of ideas for now on next steps, but will keep
> > investigating once I have some, any help would be welcome. Thank you.
> >
>
> Hi,
>
> I had few additional attempts to investigate the issue, unfortunately
> without any results which would lead to understanding the issue. I
> would be willing to give up my eBox 3360 with RS-485 serial ports if
> anybody would be interested to work on this.
>
> From the latest attempts I was still focussing on interrupt
> initialization, which increasingly feels like a wrong place to look
> at. Likely level type interrupts themselves should be investigated,
> but I don't know how to do that effectively.
>
> I did debug the flow from lapic.c, ioapic.c, pci_machdep.c, intrc.c
> and recently mpacpi.c, compared to my VIA system which is somewhat
> similar. Compared to FreeBSD code, which has differences, but I can't
> pinpoint anything of importance, extINT being the most visible
> difference, but it doesn't seem to affect level type interrupts in any
> way (14 now can boot as well). In general, from side it looks like all
> those attempts show "flawless" initialization, without errors or going
> to erroneous path. System boots without issues as well, however the
> end result is that USB, Ethernet doesn't work since no level type
> interrupts are acknowledged from them (or at least it looks like so).
> ISA interrupts though work, thus serial can be used and system itself
> seems to be stable otherwise.
>
> System BIOS also has "RDC engineer mode" by clicking F4 while in BIOS.
> It provides few additional options like disabling APIC bus, which
> allows to boot system with ACPI enabled and interrupts working,
> however they are wired through PIC IRQs and second core needs to be
> disabled to boot.
>
> In recent attempts I enabled MPDEBUG which gives more log messages
> about configuring the interrupts, also shows that mpacpi takes care of
> both MATD overrides, all logs are expected and matches working
> systems:
> ....
> mpacpi: found root PCI bus 0
> mpacpi: found subordinate bus 1
> mpacpi: found subordinate bus 2
> mpacpi: 3 PCI busses
> mpacpi: ISA interrupt override 0 -> 2 (0/0)
> mpacpi: ISA interrupt override 9 -> 9 (1/3)
> mpacpi: configuring PCI bus 0 int routing
> pciroute: dev 13 INTA on globint 22
> pciroute: done adding entry
> pciroute: dev 1 INTA on globint 16
> pciroute: done adding entry
> pciroute: dev 1 INTB on globint 17
> pciroute: done adding entry
> pciroute: dev 1 INTC on globint 18
> pciroute: done adding entry
> pciroute: dev 1 INTD on globint 19
> pciroute: done adding entry
> pciroute: dev 2 INTA on globint 17
> pciroute: done adding entry
> pciroute: dev 2 INTB on globint 18
> pciroute: done adding entry
> pciroute: dev 2 INTC on globint 19
> pciroute: done adding entry
> pciroute: dev 2 INTD on globint 16
> pciroute: done adding entry
> pciroute: dev 3 INTA on globint 18
> pciroute: done adding entry
> pciroute: dev 3 INTB on globint 19
> pciroute: done adding entry
> pciroute: dev 3 INTC on globint 16
> pciroute: done adding entry
> pciroute: dev 3 INTD on globint 17
> pciroute: done adding entry
> pciroute: dev 8 INTA on globint 19
> pciroute: done adding entry
> pciroute: dev 10 INTA on globint 18
> pciroute: done adding entry
> pciroute: dev 10 INTB on globint 21
> pciroute: done adding entry
> pciroute: dev 12 INTA on globint 23
> pciroute: done adding entry
> pciroute: dev 14 INTA on globint 18
> pciroute: done adding entry
> pciroute: dev 15 INTA on globint 21
> pciroute: done adding entry
> pciroute: done
> mpacpi: configuring PCI bus 1 int routing
> pciroute: dev 0 INTA on globint 16
> pciroute: done adding entry
> pciroute: dev 0 INTB on globint 17
> pciroute: done adding entry
> pciroute: dev 0 INTC on globint 18
> pciroute: done adding entry
> pciroute: dev 0 INTD on globint 19
> pciroute: done adding entry
> pciroute: done
> mpacpi: configuring PCI bus 2 int routing
> pciroute: dev 0 INTA on globint 17
> pciroute: done adding entry
> pciroute: dev 0 INTB on globint 18
> pciroute: done adding entry
> pciroute: dev 0 INTC on globint 19
> pciroute: done adding entry
> pciroute: dev 0 INTD on globint 16
> pciroute: done adding entry
> pciroute: done
> ioapic0: pin 2 attached to isa0 irq 0 (type 0<type=3D0> flags 0<pol=3D0,t=
rig=3D0>)
> ioapic0: pin 1 attached to isa0 irq 1 (type 0<type=3D0> flags 0<pol=3D0,t=
rig=3D0>)
> ioapic0: pin 3 attached to isa0 irq 3 (type 0<type=3D0> flags 0<pol=3D0,t=
rig=3D0>)
> ioapic0: pin 4 attached to isa0 irq 4 (type 0<type=3D0> flags 0<pol=3D0,t=
rig=3D0>)
> ioapic0: pin 5 attached to isa0 irq 5 (type 0<type=3D0> flags 0<pol=3D0,t=
rig=3D0>)
> ioapic0: pin 6 attached to isa0 irq 6 (type 0<type=3D0> flags 0<pol=3D0,t=
rig=3D0>)
> ioapic0: pin 7 attached to isa0 irq 7 (type 0<type=3D0> flags 0<pol=3D0,t=
rig=3D0>)
> ioapic0: pin 8 attached to isa0 irq 8 (type 0<type=3D0> flags 0<pol=3D0,t=
rig=3D0>)
> ioapic0: pin 9 attached to isa0 irq 9 (type 0<type=3D0> flags
> 0xd<pol=3D0x1=3DAct Hi,trig=3D0x3=3DLevel>)
> ioapic0: pin 10 attached to isa0 irq 10 (type 0<type=3D0> flags 0<pol=3D0=
,trig=3D0>)
> ioapic0: pin 11 attached to isa0 irq 11 (type 0<type=3D0> flags 0<pol=3D0=
,trig=3D0>)
> ioapic0: pin 12 attached to isa0 irq 12 (type 0<type=3D0> flags 0<pol=3D0=
,trig=3D0>)
> ioapic0: pin 13 attached to isa0 irq 13 (type 0<type=3D0> flags 0<pol=3D0=
,trig=3D0>)
> ioapic0: pin 14 attached to isa0 irq 14 (type 0<type=3D0> flags 0<pol=3D0=
,trig=3D0>)
> ioapic0: pin 15 attached to isa0 irq 15 (type 0<type=3D0> flags 0<pol=3D0=
,trig=3D0>)
> ioapic0: pin 22 attached to pci0 device 13 INT_A (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 16 attached to pci0 device 1 INT_A (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 17 attached to pci0 device 1 INT_B (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 18 attached to pci0 device 1 INT_C (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 19 attached to pci0 device 1 INT_D (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 17 attached to pci0 device 2 INT_A (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 18 attached to pci0 device 2 INT_B (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 19 attached to pci0 device 2 INT_C (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 16 attached to pci0 device 2 INT_D (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 18 attached to pci0 device 3 INT_A (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 19 attached to pci0 device 3 INT_B (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 16 attached to pci0 device 3 INT_C (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 17 attached to pci0 device 3 INT_D (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 19 attached to pci0 device 8 INT_A (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 18 attached to pci0 device 10 INT_A (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 21 attached to pci0 device 10 INT_B (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 23 attached to pci0 device 12 INT_A (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 18 attached to pci0 device 14 INT_A (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 21 attached to pci0 device 15 INT_A (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 16 attached to pci1 device 0 INT_A (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 17 attached to pci1 device 0 INT_B (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 18 attached to pci1 device 0 INT_C (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 19 attached to pci1 device 0 INT_D (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 17 attached to pci2 device 0 INT_A (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 18 attached to pci2 device 0 INT_B (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 19 attached to pci2 device 0 INT_C (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ioapic0: pin 16 attached to pci2 device 0 INT_D (type 0<type=3D0> flags
> 0xf<pol=3D0x3=3DAct Lo,trig=3D0x3=3DLevel>)
> ...
>
> Regards,
> Andrius V
I had some unexpected "breakthrough" on this bug, which probably won't
bring me closer to solution, but should help at least to narrow down
on the places to look at.
I managed to make network interface vte(4) work by forcing high
polarity on its pin 19 (instead of level low, it is level high).
Network started to work "properly" and was stable (could use ssh, scp,
connect to internet). The main caveat, it causes interrupt storm.
intrctl list would list huge number of interrupts.
So seemingly it issue only visible on level type interrupt with low
polarity. For some reason they are not registered or acknowledge
somewhere in the between device and ioapic.
--- a/sys/arch/x86/x86/ioapic.c
+++ b/sys/arch/x86/x86/ioapic.c
@@ -447,6 +447,8 @@ apic_set_redir(struct ioapic_softc *sc, int pin,
int idt_vec,
}
}
}
+ if (pin =3D=3D 19)
+ redlo &=3D ~IOAPIC_REDLO_ACTLO;
ioapic_write(sc, IOAPIC_REDHI(pin), redhi);
ioapic_write(sc, IOAPIC_REDLO(pin), redlo);
if (mp_verbose)
I tried to change ordering in the INTRSTUB_56()
(sys/arch/i386/i386/vector.S) for level interrupts in different ways,
but that didn't help. So likely it is not masking/unmasking/timeouts.
But it's hard to test this part, since it is an assembly code.
In general, initialization still looks good, but what happens with
those interrupts later I have no idea still.
Same trick didn't fully work for ohci/ehci, system freezes on boot. If
just one of them does the trick, interrupt storm is also visible, more
message on usb, but devices fail to attach.
Thus, USB reacts to polarity changes, just probably other factors
don't allow to work.
If switching polarity making it work gives you any idea on what can be
checked or what can be an issue, please share with me.
Regards,
Andrius V
Home |
Main Index |
Thread Index |
Old Index