wm workqueue and agr device

[BERTRAND Joël <joel.bertrand%systella.fr@localhost>]

	Hello,

	In a recent mail (if_wm.c workqueue), I have seen that it is possible
to activate workqueue on all ethernet devices contained in my server:

[     1.002997] wm0 at pci2 dev 0 function 0: I350 Gigabit Network
Connection (rev. 0x01)
[     1.002997] wm0: for TX and RX interrupting at msix0 vec 0 affinity to 0
[     1.002997] wm0: for TX and RX interrupting at msix0 vec 1 affinity to 1
[     1.002997] wm0: for TX and RX interrupting at msix0 vec 2 affinity to 2
[     1.002997] wm0: for TX and RX interrupting at msix0 vec 3 affinity to 3
[     1.002997] wm0: for TX and RX interrupting at msix0 vec 4 affinity to 4
[     1.002997] wm0: for TX and RX interrupting at msix0 vec 5 affinity to 5
[     1.002997] wm0: for TX and RX interrupting at msix0 vec 6 affinity to 6
[     1.002997] wm0: for TX and RX interrupting at msix0 vec 7 affinity to 7
[     1.002997] wm0: for LINK interrupting at msix0 vec 8
[     1.002997] wm0: PCI-Express bus
[     1.002997] wm0: 8192 words (16 address bits) SPI EEPROM, version
1.63, Image Unique ID 00000ffd
[     1.002997] wm0: Ethernet address b4:96:91:92:77:6e
[     1.002997] wm0: Copper
[     1.002997] wm0:
0x40674040<SPI,PCIE,NEWQUEUE,ASF_FIRM,ARC_SUBSYS,WOL,EEE,CRC_STRIP>
[     1.002997] ihphy0 at wm0 phy 1: I350 10/100/1000 media interface,
rev. 1
[     1.002997] wm1 at pci2 dev 0 function 1: I350 Gigabit Network
Connection (rev. 0x01)
[     1.002997] wm1: for TX and RX interrupting at msix1 vec 0 affinity to 0
[     1.002997] wm1: for TX and RX interrupting at msix1 vec 1 affinity to 1
[     1.002997] wm1: for TX and RX interrupting at msix1 vec 2 affinity to 2
[     1.002997] wm1: for TX and RX interrupting at msix1 vec 3 affinity to 3
[     1.002997] wm1: for TX and RX interrupting at msix1 vec 4 affinity to 4
[     1.002997] wm1: for TX and RX interrupting at msix1 vec 5 affinity to 5
[     1.002997] wm1: for TX and RX interrupting at msix1 vec 6 affinity to 6
[     1.002997] wm1: for TX and RX interrupting at msix1 vec 7 affinity to 7
[     1.002997] wm1: for LINK interrupting at msix1 vec 8
[     1.002997] wm1: PCI-Express bus
[     1.002997] wm1: 8192 words (16 address bits) SPI EEPROM, version
1.63, Image Unique ID 00000ffd
[     1.002997] wm1: Ethernet address b4:96:91:92:77:6f
[     1.002997] wm1: Copper
[     1.002997] wm1:
0x40474040<SPI,PCIE,NEWQUEUE,ASF_FIRM,ARC_SUBSYS,EEE,CRC_STRIP>
[     1.002997] ihphy1 at wm1 phy 1: I350 10/100/1000 media interface,
rev. 1
[     1.002997] wm2 at pci0 dev 25 function 0: I218 V Ethernet
Connection (rev. 0x00)
[     1.002997] wm2: interrupting at msi4 vec 0
[     1.002997] wm2: PCI-Express bus
[     1.002997] wm2: 2048 words FLASH, version 0.1.4
[     1.002997] wm2: Ethernet address 08:62:66:47:63:99
[     1.002997] wm2: 0x6a4080<FLASH,PCIE,ASF_FIRM,AMT,WOL,EEE>
[     1.002997] ihphy2 at wm2 phy 2: i217 10/100/1000 media interface,
rev. 5
[     1.002997] wm3 at pci6 dev 0 function 0: Intel i82574L (rev. 0x00)
[     1.002997] wm3: for TX and RX interrupting at msix6 vec 0 affinity to 1
[     1.002997] wm3: for TX and RX interrupting at msix6 vec 1 affinity to 2
[     1.002997] wm3: for LINK interrupting at msix6 vec 2
[     1.002997] wm3: PCI-Express bus
[     1.002997] wm3: 2048 words FLASH, version 1.8.0, Image Unique ID
0000ffff
[     1.002997] wm3: ASPM L0s and L1 are disabled to workaround the errata.
[     1.002997] wm3: Ethernet address 68:05:ca:02:b2:59
[     1.002997] wm3: 0x224080<FLASH,PCIE,ASF_FIRM,WOL>
[     1.002997] makphy0 at wm3 phy 1: Marvell 88E1149 Gigabit PHY, rev. 1
[     1.002997] wm4 at pci7 dev 0 function 0: Intel i82574L (rev. 0x00)
[     1.002997] wm4: for TX and RX interrupting at msix7 vec 0 affinity to 1
[     1.002997] wm4: for TX and RX interrupting at msix7 vec 1 affinity to 2
[     1.002997] wm4: for LINK interrupting at msix7 vec 2
[     1.002997] wm4: PCI-Express bus
[     1.002997] wm4: 2048 words FLASH, version 1.8.0, Image Unique ID
0000ffff
[     1.002997] wm4: ASPM L0s and L1 are disabled to workaround the errata.
[     1.002997] wm4: Ethernet address 68:05:ca:44:cf:30
[     1.002997] wm4: 0x224080<FLASH,PCIE,ASF_FIRM,WOL>
[     1.002997] makphy1 at wm4 phy 1: Marvell 88E1149 Gigabit PHY, rev. 1

wm0 and wm1 are bridged (bridge0) and connected to two NAS.
wm2 is connected to WAN
wm3 and wm4 are aggregated (agr0) and connected to LAN.

	I have tried to set in /etc/sysctl.conf:

hw.wm0.txrx_workqueue=1
hw.wm1.txrx_workqueue=1
hw.wm2.txrx_workqueue=1
hw.wm3.txrx_workqueue=1
hw.wm4.txrx_workqueue=1

and system quickly hangs (no panic, no internal debugger, no serial
console available, only a hard lock). Before lock, agr0 was very very slow.

	If I set:

hw.wm0.txrx_workqueue=1
hw.wm1.txrx_workqueue=1
hw.wm2.txrx_workqueue=1
hw.wm3.txrx_workqueue=0
hw.wm4.txrx_workqueue=0

system seems to run as expected. Thus I suppose that workqueue and agr
don't run together.

	My kernel is:
NetBSD legendre.systella.fr 9.2_STABLE NetBSD 9.2_STABLE (CUSTOM) #11:
Thu Mar 31 11:43:25 CEST 2022
root%legendre.systella.fr@localhost:/usr/src/netbsd-9/obj/sys/arch/amd64/compile/CUSTOM
amd64

CUSTOM as I have built ATQ support. I use NetBSD 9 (not 9.2) as rolling
release.

	Is it possible to use workqueue with agr ?

	Best regards,

	JB