MSI-X I/O space clash

[Tobias Nygren <tnn%NetBSD.org@localhost>]

Hi,

For some PCI-e devices the MSI-X vector table is embedded in BAR0.
If the driver maps the whole of BAR0 then the x86 MSI-X MD code can't
map the table because the bus space range is already reserved, and
we fail to establish interrupt handlers.
We already have some hacks in drivers to deal with this, for
example in xhci and nvme:

https://github.com/NetBSD/src/commit/c44e99c3172714cc4b9952bca60a00756fcf5123#diff-518473c01ea8c7a44eabd6c8b2c8c694

mcx(4) also has this problem but it's worse because the table
is not at the end of the BAR but in the middle of it, which makes
it convoluted to work around.

jmcneill@ suggested to make the following change to the MD code
instead:

--- sys/arch/x86/pci/msipic.c	26 Jun 2019 10:20:06 -0000	1.17
+++ sys/arch/x86/pci/msipic.c	28 Sep 2019 17:14:13 -0000
@@ -711,7 +711,7 @@ msipic_construct_msix_pic(const struct p
 		flags &= ~BUS_SPACE_MAP_PREFETCHABLE;
 	}
 	bssize = roundup(table_size, PAGE_SIZE);
-	err = bus_space_map(pa->pa_memt, memaddr + table_offset, bssize, flags,
+	err = _x86_memio_map(pa->pa_memt, memaddr + table_offset, bssize, flags,
 	    &bshandle);
 	bstag = pa->pa_memt;
 #endif
@@ -739,8 +739,8 @@ msipic_destruct_msix_pic(struct pic *msi
 	KASSERT(msix_pic->pic_type == PIC_MSIX);
 
 	msipic = msix_pic->pic_msipic;
-	bus_space_unmap(msipic->mp_bstag, msipic->mp_bshandle,
-	    msipic->mp_bssize);
+	_x86_memio_unmap(msipic->mp_bstag, msipic->mp_bshandle,
+	    msipic->mp_bssize, NULL);
 
 	msipic_destruct_common_msi_pic(msix_pic);
 }


I'd like to make this change and pull it up to netbsd-9 along with
mcx(4) attachment for amd64, unless there are objections.

Kind regards,
-Tobias