port-arm/57643: aarch64: smmu(4) seems mandatory for some SoC and/or memory configuration

[rokuyama.rk%gmail.com@localhost]

>Number:         57643
>Category:       port-arm
>Synopsis:       aarch64: smmu(4) seems mandatory for some SoC and/or memory configuration
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    port-arm-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Oct 04 06:55:00 +0000 2023
>Originator:     Rin Okuyama
>Release:        10.99.8
>Organization:
Internet Initiative Japan Inc.
>Environment:
NetBSD lx2k-igc-intel 10.99.8 NetBSD 10.99.8 (GENERIC64) #152: Wed Oct  4 12:28:25 JST 2023  rin@netbsd:/home/rin/src/sys/arch/evbarm/compile/GENERIC64 evbarm aarch64
>Description:
For HoneyComb LK2K with its latest (2021-08-10) UEFI firmware:

https://images.solid-run.com/LX2k/lx2160a_uefi

TX stalls indefinitely for upcoming igc(4) driver, if 64-bit DMA tag
is used; only a part of TX packet is marked processed, with
succeeding buffers up to EOP being untouched forever.

As far as I can see, this behavior has never been observed for
OpenBSD and Linux, that have SMMU driver.

It occurs LK2K with 64 and 32GB memory. On the other hand, I've
never observed similar errors on ROCKPro64 (U-Boot, 4GB memory)
and Quartz64 (UEFI, *8GB* memory).

Even on LK2K, this behavior has never been observed with 32-bit
DMA tag. Therefore, I *guess* that at least some SoC and/or
configuration require SMMU support for DMA from/to whole
physical space.
>How-To-Repeat:
On LX2K with will-be-committed igc(4) driver:

$ j=0; while iperf3 -c somewhere; do j=$((j + 1)); echo $j; done

Then, it stalls for j <~ 100.
>Fix:
(1) Test LK2K with <= 4GB memory, other boards with > 4GB memory?

(2) Import smmu(4) from OpenBSD. But, don't we need to have MI
    frameworks for IOMMUs?