Re: kern/58124: lagg(4) fails to work with lacp protocol on mcx(4)

[s ymgch <s.ymgch228%gmail.com@localhost>]

Hi,

Could you check "ifconfig -m mcx0" and "ifconfig -m mcx1"?
LACP in lagg(4) needs "mediaopt full-duplex" to work, but mcx(4) may
be missing IFM_FDX.

On Mon, Apr 8, 2024 at 3:05 AM <vezhlys%gmail.com@localhost> wrote:
>
> >Number:         58124
> >Category:       kern
> >Synopsis:       lagg(4) fails to work with lacp protocol on mcx(4)
> >Confidential:   no
> >Severity:       serious
> >Priority:       medium
> >Responsible:    kern-bug-people
> >State:          open
> >Class:          sw-bug
> >Submitter-Id:   net
> >Arrival-Date:   Sun Apr 07 18:05:00 +0000 2024
> >Originator:     Andrius V
> >Release:        NetBSD 10
> >Organization:
> >Environment:
> >Description:
> Hi,
>
> I am trying to configure lagg(4) interface using lacp (802.3ad) protocol between
> my NAS server (amd64 machine) and Mikrotik CRS309-1G-8S+ router (RouterOS).
> My network adapter is QNAP QXG-10G2SF-CX4 (Mellanox ConnectX-4 Lx mcx(4) SFP+ DAC).
> Unfortunately I can't manage to make it work and communicate with my switch or
> receive IP address from my router's DHCP server.
>
> Switch should be configured correctly, I used it for Linux setup for years, also
> deprecated agr(4) interface "sometimes" works with certain manual actions applied.
> agr(4) is unstable though and looses connection soon or after reboot requiring
> manual interaction again, thus also unusable in general. It has "LACP Partner System ID"
> in the interface, but it stays empty for NetBSD (should be mac address of nas server).
>
> I am using dhcpcd DHCP client.
>
> Alternatively I tried loadbalance protocol, it works without issues in comparison.
>
> Besides these I have 2x igc(4) interfaces, but I am not using them for link aggregation.
>
> ifconfig.lagg0
> create
> laggproto lacp laggport mcx0 laggport mcx1
> up
>
> /etc/rc.conf
> ...
> net_interfaces="igc1 lagg0"
> dhcpcd=YES
> dhcpcd_flags="-qLM lagg0 igc1"
> ...
>
> ifconfig lagg0
> lagg0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         capabilities=0x3ff00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx>
>         capabilities=0x3ff00<UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx,TCP6CSUM_Tx>
>         capabilities=0x3ff00<UDP6CSUM_Rx,UDP6CSUM_Tx>
>         enabled=0x3ff00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx>
>         enabled=0x3ff00<UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx,TCP6CSUM_Tx>
>         enabled=0x3ff00<UDP6CSUM_Rx,UDP6CSUM_Tx>
>         ec_capabilities=0x7<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
>         ec_enabled=0x2<VLAN_HWTAGGING>
>         laggproto lacp
>         laggport:
>                 mcx0 pri=32768 flags=0
>                 mcx1 pri=32768 flags=0
>         address: xx:xx:xx:xx:xx:xx
>         status: no carrier
>
> mcx0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         capabilities=0x3ff00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx>
>         capabilities=0x3ff00<UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx,TCP6CSUM_Tx>
>         capabilities=0x3ff00<UDP6CSUM_Rx,UDP6CSUM_Tx>
>         enabled=0x3ff00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx>
>         enabled=0x3ff00<UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx,TCP6CSUM_Tx>
>         enabled=0x3ff00<UDP6CSUM_Rx,UDP6CSUM_Tx>
>         ec_capabilities=0x7<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
>         ec_enabled=0x2<VLAN_HWTAGGING>
>         address: xx:xx:xx:xx:xx:xx
>         media: Ethernet autoselect (10GBASE-CR1)
>         status: active
>
> mcx1: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         capabilities=0x3ff00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx>
>         capabilities=0x3ff00<UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx,TCP6CSUM_Tx>
>         capabilities=0x3ff00<UDP6CSUM_Rx,UDP6CSUM_Tx>
>         enabled=0x3ff00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx>
>         enabled=0x3ff00<UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx,TCP6CSUM_Tx>
>         enabled=0x3ff00<UDP6CSUM_Rx,UDP6CSUM_Tx>
>         ec_capabilities=0x7<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
>         ec_enabled=0x2<VLAN_HWTAGGING>
>         address: xx:xx:xx:xx:xx:xx
>         media: Ethernet autoselect (10GBASE-CR1)
>         status: active
>         link xx:xx:xx:xx:xx:xx
>
> dmesg for mcx:
> mcx0 at pci9 dev 0 function 0: Mellanox Technologies ConnectX-4 Lx (rev. 0x00)
> mcx0: FW 14.22.1002
> mcx0: Ethernet address xx:xx:xx:xx:xx:xx
> mcx1 at pci9 dev 0 function 1: Mellanox Technologies ConnectX-4 Lx (rev. 0x00)
> mcx1: FW 14.22.1002
> mcx1: Ethernet address xx:xx:xx:xx:xx:xx
>
> >How-To-Repeat:
> ifconfig lagg0 create
> ifconfig lagg0 laggproto lacp laggport mcx0 laggport mcx1
> ifconfig lagg0 up
>
> Start dhcpcd service or run dhcpcd lagg0 and observe that no IP is assigned, "status: no carrier" is displayed.
> >Fix:
>