Re: bus_dma without bouncing

[Yuri Honegger <yurix%NetBSD.org@localhost>]

> Am 27.10.2026 um 15:16 schrieb Taylor R Campbell <riastradh%NetBSD.org@localhost>:
> 
> NIC drivers allocate rx buffers for the device to read network data
> into through DMA.  In some devices the device's DMA has constraints,
> specified through the DMA tag and/or bus_dmamap_create parameters, and
> enforced in bus_dmamap_load by bouncing.
> 
> To avoid bouncing, or to satisfy alignment requirements imposed by the
> device[1], drivers will often allocate rx buffers themselves with
> bus_dmamem_alloc/map.

Another case that I currently don’t know how to represent is a device that support multisegment dma, but all segment sizes need to be a multiple of 64 bytes. This is the case for the DMA controller on some TI SoCs (am335x, am1808).

> 
> But, given constraints, bus_dmamap_create will _also_ allocate bounce
> buffers with bus_dmamem_alloc/map -- even if the driver will never
> bounce because it already allocated rx buffers satisfying the
> constraint.  This wastes physical pages and kva that will never be
> used -- except if there's a driver bug causing every rx buffer to be
> bounced unintentionally, which quietly gives bad performance!
> 
> Should we have a way to avoid this?

Yes, but that is treating the symptoms. I’ve been wondering what if we allow drivers to express more constraints for how their DMA memory looks like so they no longer need to manage bounce buffer themselves?

> 
> Here are a couple options that come to mind (with 30sec of thought put
> into each name, and alternate bike shed paint suggestions welcome):
> 
> 1. An option BUS_DMA_BOUNCELESS to bus_dmamap_create that causes
>   bus_dmamap_load to fail instead of bouncing if constraints are not
>   met.

BUS_DMA_NOBOUNCE would be a bit more in line with the other constants, but I don’t particularly care.

> 
>   => This simple API change generally obviates the need to allocate
>      bounce buffers in the scenario above.  However, some
>      architectural constraints such as confidential computing
>      architecture (AMD SEV, Intel TDX) may still require bouncing for
>      other reasons (must selectively expose page unencrypted to the
>      hypervisor).
> 
> 2. (a) An option BUS_DMA_ALLOCBUF to bus_dmamap_create that _forces_
>       it to allocate a bounce buffer with bus_dmamem_alloc/map
>       meeting the constraints.

How does this differ from BUS_DMA_ALLOCNOW?

>   (b) Another operation, such as bus_dmamap_bufmem, to get at the
>       bounce buffer so a driver can work directly with it instead of
>       calling bus_dmamem_alloc/map itself.
> 
>   => This would allow us to avoid the duplicate bounce buffer in the
>      confidential computing case.  But it's a more complicated change
>      to the API, and some bus_dma implementations -- such as sparc
>      and sparc64 -- appear not to bounce at all, ever, so they would
>      have to be adapted to add logic for allocating bounce bufers.

To me, the complexity doesn’t seem worth it compared to adding a BOUNCELESS flag.

> 
> 
> 
> [1] It seems there is no MI way with bus_dma tags or bus_dmamap_create
>    to make bus_dmamap_load enforce alignment by bouncing into an
>    aligned bounce buffer if the given address is misaligned.  But
>    there are plenty of devices that require aligned addresses for
>    DMA, such as Intel wireless cards which require 8- or 12-bit
>    alignment.  Am I missing something?  Is there just no way to
>    express this constraint in bus_dmamap_create so that
>    bus_dmamap_load will _either_ fail noisily or bounce on misaligned
>    inputs?