Re: wm(4) and the maximum buffer length for TSO

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

On Wed, Sep 19, 2012 at 06:34:15PM -0500, David Young wrote:
> wm(4) sets up its Tx DMA maps like this,
> 
>                 if ((error = bus_dmamap_create(sc->sc_dmat, WM_MAXTXDMA,
>                             WM_NTXSEGS, WTX_MAX_LEN, 0, 0,
>                             &sc->sc_txsoft[i].txs_dmamap)) != 0) {
> 
> WM_MAXTXDMA is round_page(IP_MAXPACKET) == round_page(65535) ==
> 65536.  Thus wm(4) will fail to map for Tx any mbuf whose m_pkthdr.len
> > 65536.  That's ok if tcp_output() produces a buffer no longer
> than 65536 bytes for the NIC to segment, but in practice it will
> produce a longer buffer because first it clamps the length to
> IP_MAXPACKET,
> 
>                 if (use_tso) {
>                         /*
>                          * Truncate TSO transfers to IP_MAXPACKET, and make
>                          * sure that we send equal size transfers down the
>                          * stack (rather than big-small-big-small-...).
>                          */
> #ifdef INET6
>                         CTASSERT(IPV6_MAXPACKET == IP_MAXPACKET);
> #endif
>                         len = (min(len, IP_MAXPACKET) / txsegsize) * 
> txsegsize;
> 
>       ...
> 
> and then it adds in the combined length of the IP and TCP headers:
> 
>         m->m_pkthdr.len = hdrlen + len;
> 
> In this way, wm(4) can see m->m_pkthdr.len greater than 65536 and fail
> to map m.  It will send no feedback to TCP to stop trying to send such
> long un-segmented buffers.  Also, it looks to me like it will retry
> forever to map the same mbuf for DMA---that matches the misbehavior that
> we're seeing at $DAYJOB, where the wm(4) ceases to transmit anything.

I's strange that I didn't run into this, I do use TSO with wm(4).
I guess that the problem is dependant on the value of txsegsize:
if its value is right, len will be rounded down and there is
enough space for the header.

> 
> It seems to me that drivers should advertise the maximum unsegmented
> buffer length that they support (at least some wm instances support
> 1MB),

Some are also limited to 64k

> and tcp_output() should be very careful not to send a buffer any
> longer than what is supported.  What do you think?

I think we need something like that.

-- 
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
     NetBSD: 26 ans d'experience feront toujours la difference
--