Re: kern/53562: bridge(4) breaks segmentation / TX checksum offloading

[Masanobu SAITOH <msaitoh%execsw.org@localhost>]

The following reply was made to PR kern/53562; it has been noted by GNATS.

From: Masanobu SAITOH <msaitoh%execsw.org@localhost>
To: gnats-bugs%NetBSD.org@localhost, kern-bug-people%netbsd.org@localhost,
 gnats-admin%netbsd.org@localhost, netbsd-bugs%netbsd.org@localhost, rokuyama%rk.phys.keio.ac.jp@localhost
Cc: msaitoh%execsw.org@localhost
Subject: Re: kern/53562: bridge(4) breaks segmentation / TX checksum
 offloading
Date: Mon, 10 Sep 2018 16:34:38 +0900

 Hi.
 
 On 2018/09/08 8:50, Rin Okuyama wrote:
 > The following reply was made to PR kern/53562; it has been noted by GNATS.
 > 
 > From: Rin Okuyama <rokuyama%rk.phys.keio.ac.jp@localhost>
 > To: Masanobu SAITOH <msaitoh%execsw.org@localhost>, Christoph Badura <bad%bsd.de@localhost>
 > Cc: Ryota Ozaki <ozaki-r%netbsd.org@localhost>,
 >   "gnats-bugs%NetBSD.org@localhost" <gnats-bugs%netbsd.org@localhost>, kern-bug-people%netbsd.org@localhost,
 >   gnats-admin%netbsd.org@localhost, netbsd-bugs%netbsd.org@localhost
 > Subject: Re: kern/53562: bridge(4) breaks segmentation / TX checksum
 >   offloading
 > Date: Sat, 8 Sep 2018 08:48:04 +0900
 > 
 >   On 2018/09/07 17:50, Masanobu SAITOH wrote:
 >   > On 2018/09/06 6:51, Christoph Badura wrote:
 >   >> On Wed, Sep 05, 2018 at 09:04:16PM +0900, Rin Okuyama wrote:
 >   >>> With both patches, TX works fine through an interface added to bridge,
 >   >>> even if TSO is enabled. However, RX from an host which shares the
 >   >>> interface via bridge does not work for larger packets (because of TSO?).
 >   >>> RX of smaller packets and TX work for that host.
 >   >>
 >   >> I would not expect any TSO flags to be set on a packet that is received
 >   >> from another host.  The packet should have been segmented on other host
 >   >> already.
 >   >>
 >   >>> I'm not sure if my setup should work in principle...
 >   >>
 >   >> Maybe you could describe your setup in more detail.  Looking at the PR I
 >   >> don't understand what exactly your setup is.
 >   >
 >   > I think so, too. I read if_bridge.c and I think ozaki-r's change should be
 >   > correct. There might be another bug. The detailed configuration is required
 >   > to reproduce the problem.
 >   
 >   Yeah, sorry for the poor explanation. I'm working on an userland
 >   application on Raspberry Pi, which emulates an ethernet adapter for
 >   a client connected via GPIO. Then, it sends and receives packets for
 >   the client using tap device bridged to a NIC. When TSO is enabled for
 >   that NIC, the client cannot retrieve file via ftp, for example.
 >   
 >   Well, my setup is hard to reproduce. The following would be a simplest
 >   example where your patch does not work:
 >   
 >   - NetBSD/amd64-currnet with wm0 connected to LAN
 >   
 >   - qemu-3.0.0nb2 installed from pkgsrc
 >   
 >   - setup tap and bridge as follows:
 >   
 >        # ifconfig tap0 create up
 >        # ifconfig bridge0 create
 >        # brconfig bridge0 add wm0 add tap0 up
 >   
 >   - run NetBSD/amd64 8.0 on QEMU with tap enabled:
 >   
 >        # qemu-system-x86_64 -m 128m -cdrom NetBSD-8.0-amd64.iso -boot d \
 >          -display curses -net tap,fd=3 -net nic 3<>/dev/tap0
 >   
 >   Then, the virtual host can send and receive small packets, e.g.,
 >   ping works well regardless of whether TSO is enabled or not for wm0.
 
   0)
   Is this "wm0" on the guest or the host?
   If my understanding is correct, writing packet via tap on host is not related
   to TSO...
 
   1)
   If you're using wm(4) on the guest, please try vioif(4) and see if the problem
 occurs or not.
 
   2)
   For checksum oofloading, the following kernel options will help for debugging:
 	INET_CSUM_COUNTERS
 	TCP_CSUM_COUNTERS
 	UDP_CSUM_COUNTERS
 
   These options add some event counters:
 > inet swcsum                                                  0    0 misc
 > inet hwcsum ok                                          419667   69 misc
 > inet hwcsum bad                                              0    0 misc
 > tcp6 swcsum                                                  0    0 misc
 > tcp6 hwcsum data                                             0    0 misc
 > tcp6 hwcsum ok                                               0    0 misc
 > tcp6 hwcsum bad                                              0    0 misc
 > tcp swcsum                                                   0    0 misc
 > tcp hwcsum data                                              0    0 misc
 > tcp hwcsum ok                                           419430   69 misc
 > tcp hwcsum bad                                               0    0 misc
 > udp swcsum                                                   0    0 misc
 > udp hwcsum data                                              0    0 misc
 > udp hwcsum ok                                              237    0 misc
 > udp hwcsum bad                                               0    0 misc
 > udp6 swcsum                                                  0    0 misc
 > udp6 hwcsum data                                             0    0 misc
 > udp6 hwcsum ok                                               0    0 misc
 > udp6 hwcsum bad                                              0    0 misc
 
   3)
   wm(4) has a lot of event counters, some of them are useful for debugging.
   To use them add "options WM_EVENT_COUNTERS" to your kernel config file.
 
   4)
   For TSO, current wm(4) is not perfect because it doesn't use m_defrag().
   I have not-yet-committed patch:
 
 	http://www.netbsd.org/~msaitoh/wm-defrag-20180906-0.dif
 
 > % vmstat -ev | grep toomany
 > wm0 txq00txtoomanyseg                                      0    0 misc
 
   If "wmX txqYYtxtoomanyseg" is increasing, the above diff would fix the
 problem.
 
 >   However, it cannot receive larger packets when TSO is enabled, e.g.,
 >   file cannot be retrieved through ftp. If TSO is turned off, everything
 >   works fine.
 >   
 >   >> It might be worth going over the original PR kern/27007 again.  Also note
 >   >> the second part of PR kern/21831 by yamt@.
 >   
 >   The second part of kern/21831 still affects at least for tap?
 >   
 >   >> Maybe we need ATF tests.  That certainly would be nice.  Also we now have
 >   >> more test cases to consider: npf and pf active on the bridge.
 >   >
 >   > I think adding offload functions into shmif(4) is a good idea to test offload
 >   > stuff.
 >   
 >   Well, I'm working on offload engine for shmif(4). The application
 >   mentioned above emulates TSO and TX/RX checksum offload by software.
 >   The algorithm, or at least ideas, can be easily applied to shmif(4).
 >   
 >   Thanks,
 >   rin
 >   
 > 
 
 
 -- 
 -----------------------------------------------
                  SAITOH Masanobu (msaitoh%execsw.org@localhost
                                   msaitoh%netbsd.org@localhost)