Re: PCIVERBOSE causing kernel stack overflow during boot - why?

[Jaromír Doleček <jaromir.dolecek%gmail.com@localhost>]

Here is the output from lspci/pcictl.

I'll try that DDB_COMMANDONENTER also - the machine is remote though,
so I'll send it later when I get it.

Thanks.

Jaromir

2016-10-19 7:23 GMT+02:00 Paul Goyette <paul%whooppee.com@localhost>:
> On Tue, 18 Oct 2016, Paul Goyette wrote:
>
>> Just as an added experiment, can you try to boot the non-PCIVERBOSE
>> kernel, and at the boot prompt enter
>>
>>         load pciverbose
>>
>> before actually booting?
>>
>> As far as getting a back-trace, you could set DDB_COMMANDONENTER="bt" in
>> your config file ....
>>
>> The dmesg looks interesting, especially with that strange pci9 bus.  My
>> machine has a similar "management devices" pci bus.
>
>
> Also, if you have installed pkgsrc/sysutils/pciutils it would be useful to
> get the output from
>
>         lspci -tvnn
>
> Otherwise, please provide output from following two commands:
>
>         pcictl pci0 list -N
>         pcictl pci0 list -n
>
>
>
>>
>>
>>
>> On Mon, 17 Oct 2016, Jaromír DoleÄ~Mek wrote:
>>
>>> Hi,
>>>
>>> I've got an amd64 system which panics with 'stack overflow detected'
>>> on boot, somewhere halfway through probing pci9 bus, when booted with
>>> kernel with PCIVERBOSE. Same kernel config without PCIVERBOSE boots
>>> fine. dmesg without PCIVERBOSE is attached.
>>>
>>> Any idea what might be causing this?
>>>
>>> I've had cursory look at pci code, it doesn't seem as if anything
>>> would be allocating extra space there. Maybe some interaction with
>>> dev_verbose module code? Unfortunately can't get backtrace as this
>>> happens before the keyboard is probed and attached.
>>>
>>> Jaromir
>>>
>>>
>>> !DSPAM:58058f59188191671919093!
>>>
>>
>> +------------------+--------------------------+------------------------+
>> | Paul Goyette     | PGP Key fingerprint:     | E-mail addresses:      |
>> | (Retired)        | FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com   |
>> | Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd.org |
>> +------------------+--------------------------+------------------------+
>
>
> +------------------+--------------------------+------------------------+
> | Paul Goyette     | PGP Key fingerprint:     | E-mail addresses:      |
> | (Retired)        | FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com   |
> | Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd.org |
> +------------------+--------------------------+------------------------+

-[0000:00]-+-00.0  Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DMI2 [8086:6f00]
           +-01.0-[01]----00.0  LSI Logic / Symbios Logic MegaRAID SAS-3 3108 [Invader] [1000:005d]
           +-02.0-[02]--+-00.0  Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 [8086:1528]
           |            \-00.1  Intel Corporation Ethernet Controller 10-Gigabit X540-AT2 [8086:1528]
           +-02.2-[03]----00.0  Intel Corporation PCIe Data Center SSD [8086:0953]
           +-03.0-[04]--
           +-04.0  Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 0 [8086:6f20]
           +-04.1  Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 1 [8086:6f21]
           +-04.2  Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 2 [8086:6f22]
           +-04.3  Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 3 [8086:6f23]
           +-04.4  Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 4 [8086:6f24]
           +-04.5  Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 5 [8086:6f25]
           +-04.6  Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 6 [8086:6f26]
           +-04.7  Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Crystal Beach DMA Channel 7 [8086:6f27]
           +-05.0  Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Map/VTd_Misc/System Management [8086:6f28]
           +-05.1  Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D IIO Hot Plug [8086:6f29]
           +-05.2  Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D IIO RAS/Control Status/Global Errors [8086:6f2a]
           +-05.4  Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D I/O APIC [8086:6f2c]
           +-11.0  Intel Corporation C610/X99 series chipset SPSR [8086:8d7c]
           +-14.0  Intel Corporation C610/X99 series chipset USB xHCI Host Controller [8086:8d31]
           +-16.0  Intel Corporation C610/X99 series chipset MEI Controller #1 [8086:8d3a]
           +-16.1  Intel Corporation C610/X99 series chipset MEI Controller #2 [8086:8d3b]
           +-1a.0  Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #2 [8086:8d2d]
           +-1c.0-[05-06]----00.0-[06]----00.0  ASPEED Technology, Inc. ASPEED Graphics Family [1a03:2000]
           +-1c.2-[07]----00.0  Intel Corporation I210 Gigabit Network Connection [8086:1533]
           +-1c.3-[08]----00.0  Intel Corporation I210 Gigabit Network Connection [8086:1533]
           +-1d.0  Intel Corporation C610/X99 series chipset USB Enhanced Host Controller #1 [8086:8d26]
           +-1f.0  Intel Corporation C610/X99 series chipset LPC Controller [8086:8d44]
           +-1f.2  Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] [8086:8d02]
           \-1f.3  Intel Corporation C610/X99 series chipset SMBus Controller [8086:8d22]

000:00:0: Intel Core i7-6xxxK/Xeon-D Host Bridge (DMI2) (host bridge, revision 0x01) [pchb0]
000:01:0: Intel Xeon-D PCIe Root Port (x8 or x4 max) (PCI bridge, revision 0x01) [ppb0]
000:02:0: Intel Core i7-6xxxK/Xeon-D PCIe Root Port (PCI bridge, revision 0x01) [ppb1]
000:02:2: Intel Core i7-6xxxK/Xeon-D PCIe Root Port (PCI bridge, revision 0x01) [ppb2]
000:03:0: Intel Core i7-6xxxK/Xeon-D PCIe Root Port (x16, x8 or x4 max) (PCI bridge, revision 0x01) [ppb3]
000:04:0: Intel product 6f20 (miscellaneous system, revision 0x01)
000:04:1: Intel product 6f21 (miscellaneous system, revision 0x01)
000:04:2: Intel product 6f22 (miscellaneous system, revision 0x01)
000:04:3: Intel product 6f23 (miscellaneous system, revision 0x01)
000:04:4: Intel product 6f24 (miscellaneous system, revision 0x01)
000:04:5: Intel product 6f25 (miscellaneous system, revision 0x01)
000:04:6: Intel product 6f26 (miscellaneous system, revision 0x01)
000:04:7: Intel product 6f27 (miscellaneous system, revision 0x01)
000:05:0: Intel Core i7-6xxxK/Xeon-D IIO Address Map, VTD_Misc, System Manage (miscellaneous system, revision 0x01)
000:05:1: Intel Xeon-D IIO Hot Plug (miscellaneous system, revision 0x01)
000:05:2: Intel Core i7-6xxxK/Xeon-D IIO RAS, Control Status, Global Errors (miscellaneous system, revision 0x01)
000:05:4: Intel Core i7-6xxxK/Xeon-D I/O APIC (interrupt system, IO(x) APIC, revision 0x01)
000:17:0: Intel C61x/X99 SPSR (undefined, subclass 0x00, revision 0x05)
000:20:0: Intel C61x/X99 USB xHCI (USB serial bus, xHCI, revision 0x05) [xhci0]
000:22:0: Intel C61x/X99 MEI Controller (miscellaneous communications, revision 0x05)
000:22:1: Intel C61x/X99 MEI Controller (miscellaneous communications, revision 0x05)
000:26:0: Intel C61x/X99 USB EHCI (USB serial bus, EHCI, revision 0x05) [ehci0]
000:28:0: Intel C61x/X99 PCIE (PCI bridge, revision 0xd5) [ppb4]
000:28:2: Intel C61x/X99 PCIE (PCI bridge, revision 0xd5) [ppb6]
000:28:3: Intel C61x/X99 PCIE (PCI bridge, revision 0xd5) [ppb7]
000:29:0: Intel C61x/X99 USB EHCI (USB serial bus, EHCI, revision 0x05) [ehci1]
000:31:0: Intel X99 LPC (ISA bridge, revision 0x05) [ichlpcib0]
000:31:2: Intel C61x/X99 SATA Controller (AHCI) (SATA mass storage, AHCI 1.0, revision 0x05) [ahcisata0]
000:31:3: Intel C61x/X99 SMBus Controller (SMBus serial bus, revision 0x05) [ichsmb0]
001:00:0: Symbios Logic MegaRAID SAS3108 (RAID mass storage, revision 0x02)
002:00:0: Intel X540-AT2 10Gbase-T Ethernet (ethernet network, revision 0x01) [ixg0]
002:00:1: Intel X540-AT2 10Gbase-T Ethernet (ethernet network, revision 0x01) [ixg1]
003:00:0: Intel PCIe NVMe SSD (Flash mass storage, NVMe, revision 0x01) [nvme0]
005:00:0: ASPEED Technology AST1150 PCIe-to-PCI bridge (PCI bridge, revision 0x03) [ppb5]
006:00:0: ASPEED Technology ASPEED Graphics Family (VGA display, revision 0x30) [vga0]
007:00:0: Intel I210-T1 Ethernet Server Adapter (ethernet network, revision 0x03) [wm0]
008:00:0: Intel I210-T1 Ethernet Server Adapter (ethernet network, revision 0x03) [wm1]
255:11:0: Intel Xeon-D QPI (miscellaneous system, revision 0x01)
255:11:1: Intel Xeon-D QPI (performance counters DASP, revision 0x01)
255:11:2: Intel Xeon-D QPI (performance counters DASP, revision 0x01)
255:11:3: Intel Xeon-D QPI (miscellaneous system, revision 0x01)
255:12:0: Intel Xeon-D Caching Agent (Cbo Unicast) (miscellaneous system, revision 0x01)
255:12:1: Intel Xeon-D Caching Agent (Cbo Unicast) (miscellaneous system, revision 0x01)
255:12:2: Intel Xeon-D Caching Agent (Cbo Unicast) (miscellaneous system, revision 0x01)
255:12:3: Intel Xeon-D Caching Agent (Cbo Unicast) (miscellaneous system, revision 0x01)
255:12:4: Intel Xeon-D Caching Agent (Cbo Unicast) (miscellaneous system, revision 0x01)
255:12:5: Intel Xeon-D Caching Agent (Cbo Unicast) (miscellaneous system, revision 0x01)
255:12:6: Intel Xeon-D Caching Agent (Cbo Unicast) (miscellaneous system, revision 0x01)
255:12:7: Intel Xeon-D Caching Agent (Cbo Unicast) (miscellaneous system, revision 0x01)
255:13:0: Intel product 6fe8 (miscellaneous system, revision 0x01)
255:13:1: Intel product 6fe9 (miscellaneous system, revision 0x01)
255:13:2: Intel product 6fea (miscellaneous system, revision 0x01)
255:13:3: Intel product 6feb (miscellaneous system, revision 0x01)
255:13:4: Intel product 6fec (miscellaneous system, revision 0x01)
255:13:5: Intel product 6fed (miscellaneous system, revision 0x01)
255:15:0: Intel Xeon-D Caching Agent (miscellaneous system, revision 0x01)
255:15:1: Intel product 6ff9 (miscellaneous system, revision 0x01)
255:15:2: Intel product 6ffa (miscellaneous system, revision 0x01)
255:15:3: Intel product 6ffb (miscellaneous system, revision 0x01)
255:15:4: Intel Xeon-D Caching Agent (Cbo Broadcast) (miscellaneous system, revision 0x01)
255:15:5: Intel Xeon-D Caching Agent (Cbo Broadcast) (miscellaneous system, revision 0x01)
255:15:6: Intel Xeon-D Caching Agent (Cbo Broadcast) (miscellaneous system, revision 0x01)
255:16:0: Intel Xeon-D R2 PCIe Agent (miscellaneous system, revision 0x01)
255:16:1: Intel Xeon-D R2 PCIe Agent (performance counters DASP, revision 0x01)
255:16:5: Intel Core i7-6xxxK/Xeon-D Ubox (miscellaneous system, revision 0x01)
255:16:6: Intel Core i7-6xxxK/Xeon-D Ubox (performance counters DASP, revision 0x01)
255:16:7: Intel Core i7-6xxxK/Xeon-D Ubox (miscellaneous system, revision 0x01)
255:18:0: Intel Xeon-D Home Agent 0 (miscellaneous system, revision 0x01)
255:18:1: Intel Xeon-D Home Agent 0 (performance counters DASP, revision 0x01)
255:18:4: Intel product 6f60 (miscellaneous system, revision 0x01)
255:18:5: Intel product 6f38 (performance counters DASP, revision 0x01)
255:19:0: Intel Core i7-6xxxK/Xeon-D Memory Controller (Target Address, Therm (miscellaneous system, revision 0x01)
255:19:1: Intel Core i7-6xxxK/Xeon-D Memory Controller (Target Address, Therm (miscellaneous system, revision 0x01)
255:19:2: Intel Core i7-6xxxK/Xeon-D Memory Controller (Target Address Decode (miscellaneous system, revision 0x01)
255:19:3: Intel Core i7-6xxxK/Xeon-D Memory Controller (Target Address Decode (miscellaneous system, revision 0x01)
255:19:6: Intel Xeon-D DDR Ch0/1 Broadcast (miscellaneous system, revision 0x01)
255:19:7: Intel Xeon-D DDR Global Broadcast (miscellaneous system, revision 0x01)
255:20:0: Intel Core i7-6xxxK/Xeon-D Memory Controller (Thermal) (miscellaneous system, revision 0x01)
255:20:1: Intel Core i7-6xxxK/Xeon-D Memory Controller (Thermal) (miscellaneous system, revision 0x01)
255:20:2: Intel Core i7-6xxxK/Xeon-D Memory Controller (Error) (miscellaneous system, revision 0x01)
255:20:3: Intel Core i7-6xxxK/Xeon-D Memory Controller (Error) (miscellaneous system, revision 0x01)
255:20:4: Intel Xeon-D DDR Channel 0/1 Interface (miscellaneous system, revision 0x01)
255:20:5: Intel Xeon-D DDR Channel 0/1 Interface (miscellaneous system, revision 0x01)
255:20:6: Intel Xeon-D DDR Channel 0/1 Interface (miscellaneous system, revision 0x01)
255:20:7: Intel Xeon-D DDR Channel 0/1 Interface (miscellaneous system, revision 0x01)
255:22:0: Intel Core i7-6xxxK IMC (miscellaneous system, revision 0x01)
255:22:1: Intel product 6f79 (miscellaneous system, revision 0x01)
255:22:2: Intel product 6f6a (miscellaneous system, revision 0x01)
255:22:3: Intel product 6f6b (miscellaneous system, revision 0x01)
255:22:6: Intel product 6f6e (miscellaneous system, revision 0x01)
255:22:7: Intel product 6f6f (miscellaneous system, revision 0x01)
255:23:0: Intel Core i7-6xxxK IMC (miscellaneous system, revision 0x01)
255:23:1: Intel product 6fd1 (miscellaneous system, revision 0x01)
255:23:2: Intel product 6fd2 (miscellaneous system, revision 0x01)
255:23:3: Intel product 6fd3 (miscellaneous system, revision 0x01)
255:23:4: Intel product 6fb8 (miscellaneous system, revision 0x01)
255:23:5: Intel product 6fb9 (miscellaneous system, revision 0x01)
255:23:6: Intel product 6fba (miscellaneous system, revision 0x01)
255:23:7: Intel product 6fbb (miscellaneous system, revision 0x01)
255:30:0: Intel Core i7-6xxxK/Xeon-D PCU (miscellaneous system, revision 0x01)
255:30:1: Intel Core i7-6xxxK/Xeon-D PCU (miscellaneous system, revision 0x01)
255:30:2: Intel Core i7-6xxxK/Xeon-D PCU (miscellaneous system, revision 0x01)
255:30:3: Intel Core i7-6xxxK/Xeon-D PCU (miscellaneous system, revision 0x01)
255:30:4: Intel Core i7-6xxxK/Xeon-D PCU (miscellaneous system, revision 0x01)
255:31:0: Intel Xeon-D PCU (miscellaneous system, revision 0x01)
255:31:2: Intel Xeon-D PCU (miscellaneous system, revision 0x01)

000:00:0: 0x6f008086 (0x06000001)
000:01:0: 0x6f028086 (0x06040001)
000:02:0: 0x6f048086 (0x06040001)
000:02:2: 0x6f068086 (0x06040001)
000:03:0: 0x6f088086 (0x06040001)
000:04:0: 0x6f208086 (0x08800001)
000:04:1: 0x6f218086 (0x08800001)
000:04:2: 0x6f228086 (0x08800001)
000:04:3: 0x6f238086 (0x08800001)
000:04:4: 0x6f248086 (0x08800001)
000:04:5: 0x6f258086 (0x08800001)
000:04:6: 0x6f268086 (0x08800001)
000:04:7: 0x6f278086 (0x08800001)
000:05:0: 0x6f288086 (0x08800001)
000:05:1: 0x6f298086 (0x08800001)
000:05:2: 0x6f2a8086 (0x08800001)
000:05:4: 0x6f2c8086 (0x08002001)
000:17:0: 0x8d7c8086 (0xff000005)
000:20:0: 0x8d318086 (0x0c033005)
000:22:0: 0x8d3a8086 (0x07800005)
000:22:1: 0x8d3b8086 (0x07800005)
000:26:0: 0x8d2d8086 (0x0c032005)
000:28:0: 0x8d108086 (0x060400d5)
000:28:2: 0x8d148086 (0x060400d5)
000:28:3: 0x8d168086 (0x060400d5)
000:29:0: 0x8d268086 (0x0c032005)
000:31:0: 0x8d448086 (0x06010005)
000:31:2: 0x8d028086 (0x01060105)
000:31:3: 0x8d228086 (0x0c050005)
001:00:0: 0x005d1000 (0x01040002)
002:00:0: 0x15288086 (0x02000001)
002:00:1: 0x15288086 (0x02000001)
003:00:0: 0x09538086 (0x01080201)
005:00:0: 0x11501a03 (0x06040003)
006:00:0: 0x20001a03 (0x03000030)
007:00:0: 0x15338086 (0x02000003)
008:00:0: 0x15338086 (0x02000003)
255:11:0: 0x6f818086 (0x08800001)
255:11:1: 0x6f368086 (0x11010001)
255:11:2: 0x6f378086 (0x11010001)
255:11:3: 0x6f768086 (0x08800001)
255:12:0: 0x6fe08086 (0x08800001)
255:12:1: 0x6fe18086 (0x08800001)
255:12:2: 0x6fe28086 (0x08800001)
255:12:3: 0x6fe38086 (0x08800001)
255:12:4: 0x6fe48086 (0x08800001)
255:12:5: 0x6fe58086 (0x08800001)
255:12:6: 0x6fe68086 (0x08800001)
255:12:7: 0x6fe78086 (0x08800001)
255:13:0: 0x6fe88086 (0x08800001)
255:13:1: 0x6fe98086 (0x08800001)
255:13:2: 0x6fea8086 (0x08800001)
255:13:3: 0x6feb8086 (0x08800001)
255:13:4: 0x6fec8086 (0x08800001)
255:13:5: 0x6fed8086 (0x08800001)
255:15:0: 0x6ff88086 (0x08800001)
255:15:1: 0x6ff98086 (0x08800001)
255:15:2: 0x6ffa8086 (0x08800001)
255:15:3: 0x6ffb8086 (0x08800001)
255:15:4: 0x6ffc8086 (0x08800001)
255:15:5: 0x6ffd8086 (0x08800001)
255:15:6: 0x6ffe8086 (0x08800001)
255:16:0: 0x6f1d8086 (0x08800001)
255:16:1: 0x6f348086 (0x11010001)
255:16:5: 0x6f1e8086 (0x08800001)
255:16:6: 0x6f7d8086 (0x11010001)
255:16:7: 0x6f1f8086 (0x08800001)
255:18:0: 0x6fa08086 (0x08800001)
255:18:1: 0x6f308086 (0x11010001)
255:18:4: 0x6f608086 (0x08800001)
255:18:5: 0x6f388086 (0x11010001)
255:19:0: 0x6fa88086 (0x08800001)
255:19:1: 0x6f718086 (0x08800001)
255:19:2: 0x6faa8086 (0x08800001)
255:19:3: 0x6fab8086 (0x08800001)
255:19:6: 0x6fae8086 (0x08800001)
255:19:7: 0x6faf8086 (0x08800001)
255:20:0: 0x6fb08086 (0x08800001)
255:20:1: 0x6fb18086 (0x08800001)
255:20:2: 0x6fb28086 (0x08800001)
255:20:3: 0x6fb38086 (0x08800001)
255:20:4: 0x6fbc8086 (0x08800001)
255:20:5: 0x6fbd8086 (0x08800001)
255:20:6: 0x6fbe8086 (0x08800001)
255:20:7: 0x6fbf8086 (0x08800001)
255:22:0: 0x6f688086 (0x08800001)
255:22:1: 0x6f798086 (0x08800001)
255:22:2: 0x6f6a8086 (0x08800001)
255:22:3: 0x6f6b8086 (0x08800001)
255:22:6: 0x6f6e8086 (0x08800001)
255:22:7: 0x6f6f8086 (0x08800001)
255:23:0: 0x6fd08086 (0x08800001)
255:23:1: 0x6fd18086 (0x08800001)
255:23:2: 0x6fd28086 (0x08800001)
255:23:3: 0x6fd38086 (0x08800001)
255:23:4: 0x6fb88086 (0x08800001)
255:23:5: 0x6fb98086 (0x08800001)
255:23:6: 0x6fba8086 (0x08800001)
255:23:7: 0x6fbb8086 (0x08800001)
255:30:0: 0x6f988086 (0x08800001)
255:30:1: 0x6f998086 (0x08800001)
255:30:2: 0x6f9a8086 (0x08800001)
255:30:3: 0x6fc08086 (0x08800001)
255:30:4: 0x6f9c8086 (0x08800001)
255:31:0: 0x6f888086 (0x08800001)
255:31:2: 0x6f8a8086 (0x08800001)