Re: pci_intr_alloc() vs pci_intr_establish() - retry type?

[Masanobu SAITOH <msaitoh%execsw.org@localhost>]

On 2018/12/04 4:55, Jaromír Doleček wrote:
> Le lun. 3 déc. 2018 à 12:09, Masanobu SAITOH <msaitoh%execsw.org@localhost> a écrit :
> > C3000's AHCI has multi-vector MSI-X table and it doesn't work since
> > Nobember 20th...
>
> Can you try if by chance this code adapted nvme(4) changes anything on
> your system?
>
> http://www.netbsd.org/~jdolecek/ahcisata_msixoff.diff

It panics:
> [   1.0427481] panic: extent_alloc_region: bad size
> [   1.0427481] cpu0: Begin traceback...
> [   1.0427481] vpanic() at netbsd:vpanic+0x16f
> [   1.0427481] snprintf() at netbsd:snprintf
> [   1.0427481] extent_alloc_region() at netbsd:extent_alloc_region+0x29e
> [   1.0427481] bus_space_reserve() at netbsd:bus_space_reserve+0x95
> [   1.0427481] bus_space_map() at netbsd:bus_space_map+0x85
> [   1.0427481] ahci_pci_attach() at netbsd:ahci_pci_attach+0xde
> [   1.0427481] config_attach_loc() at netbsd:config_attach_loc+0x1a8
> [   1.0427481] config_found_sm_loc() at netbsd:config_found_sm_loc+0x48
> [   1.0427481] pci_probe_device() at netbsd:pci_probe_device+0x582
> [   1.0427481] pci_enumerate_bus() at netbsd:pci_enumerate_bus+0x197
> [   1.0427481] pcirescan() at netbsd:pcirescan+0x47
> [   1.0427481] pciattach() at netbsd:pciattach+0x193
> [   1.0427481] config_attach_loc() at netbsd:config_attach_loc+0x1a8
> [   1.0427481] config_found_sm_loc() at netbsd:config_found_sm_loc+0x48
> [   1.0427481] mp_pci_scan() at netbsd:mp_pci_scan+0xa3
> [   1.0427481] mainbus_attach() at netbsd:mainbus_attach+0x332
> [   1.0427481] config_attach_loc() at netbsd:config_attach_loc+0x1a8
> [   1.0427481] cpu_configure() at netbsd:cpu_configure+0x2b
> [   1.0427481] main() at netbsd:main+0x331
> [   1.0427481] cpu0: End traceback...
> [   1.0427481] fatal breakpoint trap in supervisor mode
> [   1.0427481] trap type 1 code 0 rip 0xffffffff8021de45 cs 0x8 rflags 0x202 cr2 0 ilevel 0x8 rsp 0xffffffff81d027f0
> [   1.0427481] curlwp 0xffffffff81858900 pid 0.1 lowest kstack 0xffffffff81cfe2c0
> Stopped in pid 0.1 (system) at  netbsd:breakpoint+0x5:  leave

pcictl dump shows:
> PCI configuration registers:
>   Common header:
>     0x00: 0x19c28086 0x02b00147 0x01060110 0x00000000
>
>     Vendor Name: Intel (0x8086)
>     Device Name: C3000 SATA Controller 1 (0x19c2)
>     Command register: 0x0147
>       I/O space accesses: on
>       Memory space accesses: on
>       Bus mastering: on
>       Special cycles: off
>       MWI transactions: off
>       Palette snooping: off
>       Parity error checking: on
>       Address/data stepping: off
>       System error (SERR): on
>       Fast back-to-back transactions: off
>       Interrupt disable: off
>     Status register: 0x02b0
>       Immediate Readiness: off
>       Interrupt status: inactive
>       Capability List support: on
>       66 MHz capable: on
>       User Definable Features (UDF) support: off
>       Fast back-to-back capable: on
>       Data parity error detected: off
>       DEVSEL timing: medium (0x1)
>       Slave signaled Target Abort: off
>       Master received Target Abort: off
>       Master received Master Abort: off
>       Asserted System Error (SERR): off
>       Parity error detected: off
>     Class Name: mass storage (0x01)
>     Subclass Name: SATA (0x06)
>     Interface Name: AHCI 1.0 (0x01)
>     Revision ID: 0x10
>     BIST: 0x00
>     Header Type: 0x00 (0x00)
>     Latency Timer: 0x00
>     Cache Line Size: 0bytes (0x00)
>
>   Type 0 ("normal" device) header:
>     0x10: 0xdffb4000 0xdffbd000 0x0000e071 0x0000e061
>     0x20: 0x0000e021 0xdffbc000 0x00000000 0x72708086
>     0x30: 0x00000000 0x00000080 0x00000000 0x0000010e
>
>     Base address register at 0x10
>       type: 32-bit nonprefetchable memory
>       base: 0xdffb4000
>     Base address register at 0x14
>       type: 32-bit nonprefetchable memory
>       base: 0xdffbd000
>     Base address register at 0x18
>       type: I/O
>       base: 0x0000e070
>     Base address register at 0x1c
>       type: I/O
>       base: 0x0000e060
>     Base address register at 0x20
>       type: I/O
>       base: 0x0000e020
>     Base address register at 0x24
>       type: 32-bit nonprefetchable memory
>       base: 0xdffbc000
>     Cardbus CIS Pointer: 0x00000000
>     Subsystem vendor ID: 0x8086
>     Subsystem ID: 0x7270
>     Expansion ROM Base Address Register: 0x00000000
>       base: 0x00000000
>       Expansion ROM Enable: off
>       Validation Status: Validation not supported
>       Validation Details: 0x0
>     Capability list pointer: 0x80
>     Reserved @ 0x38: 0x00000000
>     Maximum Latency: 0x00
>     Minimum Grant: 0x00
>     Interrupt pin: 0x01 (pin A)
>     Interrupt line: 0x0e
>
>   Capability register at 0x80
>     type: 0x05 (MSI)
>   Capability register at 0x70
>     type: 0x01 (Power Management)
>   Capability register at 0xa8
>     type: 0x12 (SATA)
>   Capability register at 0xd0
>     type: 0x11 (MSI-X)
>
>   PCI Power Management Capabilities Register
>     Capabilities register: 0x4003
>       Version: 1.2
>       PME# clock: off
>       Device specific initialization: off
>       3.3V auxiliary current: self-powered
>       D1 power management state support: off
>       D2 power management state support: off
>       PME# support D0: off
>       PME# support D1: off
>       PME# support D2: off
>       PME# support D3 hot: on
>       PME# support D3 cold: off
>     Control/status register: 0x00000008
>       Power state: D0
>       PCI Express reserved: off
>       No soft reset: on
>       PME# assertion: disabled
>       Data Select: 0
>       Data Scale: 0
>       PME# status: off
>     Bridge Support Extensions register: 0x00
>       B2/B3 support: off
>       Bus Power/Clock Control Enable: off
>     Data register: 0x00
>
>   PCI Message Signaled Interrupt
>     Message Control register: 0x0001
>       MSI Enabled: on
>       Multiple Message Capable: no (1 vector)
>       Multiple Message Enabled: off (1 vector)
>       64 Bit Address Capable: off
>       Per-Vector Masking Capable: off
>       Extended Message Data Capable: off
>       Extended Message Data Enable: off
>     Message Address register: 0xfee04000
>     Message Data register: 0x006f
>
>   MSI-X Capability Register
>     Message Control register: 0x0007
>       Table Size: 8
>       Function Mask: off
>       MSI-X Enable: off
>     Table offset register: 0x00000000
>       Table offset: 0x00000000
>       BIR: 0x0		<========================
>     Pending bit array register: 0x00000001
>       Pending bit array offset: 0x00000000
>       BIR: 0x1
>
>   Serial ATA Capability Register
>     Revision register: 0x0010
>       Revision: 1.0
>     BAR Register: 0x00000048
>       Register location: BAR 4  <======================
>       BAR offset: 0x00000010
>
>   Device-dependent header:
>     0x40: 0x00000000 0x00000000 0x00000000 0x00000000
>     0x50: 0x00000000 0x00000000 0x00000000 0x00000000
>     0x60: 0x00000000 0x00000000 0x00000000 0x00000000
>     0x70: 0x4003a801 0x00000008 0x00000000 0x00000000
>     0x80: 0x00017005 0xfee04000 0x0000006f 0x00000000
>     0x90: 0x00cf00cf 0x00200030 0x00000000 0x8000fa30
>     0xa0: 0x000000d4 0x2c1e1108 0x0010d012 0x00000048
>     0xb0: 0x00000000 0x00000000 0x00000000 0x00000000
>     0xc0: 0x00000000 0x00000000 0x00000000 0x00000000
>     0xd0: 0x00070011 0x00000000 0x00000001 0x00000000
>     0xe0: 0x00000000 0x00000000 0x00000000 0x00000000
>     0xf0: 0x00000000 0x00000000 0x01100f1c 0x00000000



> ahcisata(4) works for me on two different computers in MSI mode. I
> don't have any ahcisata device which supports MSI-X, but it is working
> on Cavium which has MSI-X. Perhaps some different BAR wierdness than
> Cavium, or simply the boards using MSI-X should use the "cavium" BAR?
>
> So another thing to try would be to make ahci_pci_abar() return
> AHCI_PCI_ABAR_CAVIUM always and see if it changes things.
>
> Also you can try rev 1.44 of ahcisata_pci.c with AHCISATA_DISABLE_MSIX
> to see if it works if forced to MSI only.

I've tested rev. 1.44 now and it worked on this machine.


> I wonder whether there is something we still need to adjust during
> general initialization, I see this on my hardware:
>
> re(4) - on my system works when attached either MSI or MSI-X
> nvme(4) - on my system works only when attached via MSI-X (regardless
> of the PCI_CAP_MSIX), doesn't work when forced to MSI
> ahcisata(4) works in MSI on my system, as MSI-X for skrll@, MSI-X
> doesn't work for you
>
> Jaromir


--
-----------------------------------------------
                SAITOH Masanobu (msaitoh%execsw.org@localhost
                                 msaitoh%netbsd.org@localhost)