10_beta and bridged network with NPF

To: NetBSD User Maillist <netbsd-users%NetBSD.org@localhost>, tech-kern%netbsd.org@localhost
Subject: 10_beta and bridged network with NPF
From: BERTRAND Joël <joel.bertrand%systella.fr@localhost>
Date: Fri, 25 Aug 2023 09:50:59 +0200
	Hello,

	This morning, after a power outage, one of my NAS was not mounted by
system. Kernel was built yesterday from officiel -10_beta source tree.

	With a kernel I have built last week, all NAS ran fine.

	Network configuration :
- w0 and w1 : 192.168.12.1/24. w0 is connected to first NAS
(192.168.12.2), w1 is connected to second one (192.168.12.3) ;
- w2 : WAN (188.231.xxx.yyy) ;
- w3 and w4 : lagg0 (LAN 192.168.10.128/24) ;
- re0 : connection to DMZ (192.168.1.1/24).

	First constatation :

legendre# nmap 192.168.12.2
Starting Nmap 7.94 ( https://nmap.org ) at 2023-08-25 09:24 CEST
Note: Host seems down. If it is really up, but blocking our ping probes,
try -Pn
Nmap done: 1 IP address (0 hosts up) scanned in 1.58 seconds

and I obtain in messages:
(sd0 bn 11514444578; cn 5622287 tn 25 sn 2)
[  6350.615844] sd0d: error writing fsbn 11514444633 of
11514444633-11514444696 (sd0 bn 11514444633; cn 5622287 tn 26 sn 25)
[  6350.615844] sd0d: error writing fsbn 11514444697 of
11514444697-11514444705 (sd0 bn 11514444697; cn 5622287 tn 28 sn 25)
[  6350.615844] sd0d: error writing fsbn 11514444706 of
11514444706-11514444728 (sd0 bn 11514444706; cn 5622287 tn 29 sn 2)
[  6350.615844] sd0d: error writing fsbn 11514444729 (sd0 bn
11514444729; cn 5622287 tn 29 sn 25)

Please not that 192.168.12.2 answer to ping:
legendre# ping 192.168.12.2
PING euclide.systella.fr (192.168.12.2): 56 data bytes
64 bytes from 192.168.12.2: icmp_seq=0 ttl=64 time=0.157942 ms
64 bytes from 192.168.12.2: icmp_seq=1 ttl=64 time=0.180560 ms
^C
----euclide.systella.fr PING Statistics----

and that I can open https console or do an ssh to 192.168.12.2 even if
nmap returns anything.

Second NAS (192.168.12.3):
legendre# nmap 192.168.12.3
Starting Nmap 7.94 ( https://nmap.org ) at 2023-08-25 09:27 CEST
Nmap scan report for leibnitz.systella.fr (192.168.12.3)
Host is up (0.000091s latency).
Not shown: 996 closed tcp ports (reset)
PORT     STATE SERVICE
22/tcp   open  ssh
443/tcp  open  https
3260/tcp open  iscsi
8080/tcp open  http-proxy
MAC Address: 24:5E:BE:14:44:57 (Qnap Systems)

Nmap done: 1 IP address (1 host up) scanned in 1.36 seconds
legendre#

	Of course, iSCSI run fine.

	If I swap wm0 and wm1, first NAS runs as expected, second one doesn't.

Ethernet configuration:
legendre# cat ifconfig.wm0
inet 192.168.12.1 netmask 255.255.255.0
ip4csum tcp4csum udp4csum tcp6csum udp6csum
mtu 9000
up
legendre# cat ifconfig.wm1
mtu 9000
ip4csum tcp4csum udp4csum tcp6csum udp6csum
up
legendre# cat ifconfig.bridge0
create
mtu 9000
#inet6 2001:7a8:a8ed:1::2 prefixlen 64 alias
!brconfig $int add wm0
!brconfig $int add wm1
!brconfig $int up

legendre# ifconfig
wm0:
flags=0x8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>
mtu 9000
        capabilities=0x7ff80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx>

capabilities=0x7ff80<TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx>
        capabilities=0x7ff80<TCP6CSUM_Tx,UDP6CSUM_Rx,UDP6CSUM_Tx,TSO6>
        enabled=0x3ff00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx>
        enabled=0x3ff00<UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx,TCP6CSUM_Tx>
        enabled=0x3ff00<UDP6CSUM_Rx,UDP6CSUM_Tx>
        ec_capabilities=0x17<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,EEE>
        ec_enabled=0x3<VLAN_MTU,VLAN_HWTAGGING>
        address: b4:96:91:92:77:6e
        media: Ethernet autoselect (1000baseT full-duplex)
        status: active
        inet6 fe80::b696:91ff:fe92:776e%wm0/64 flags 0 scopeid 0x1
        inet 192.168.12.1/24 broadcast 192.168.12.255 flags 0
wm1:
flags=0x8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>
mtu 9000
        capabilities=0x7ff80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx>

capabilities=0x7ff80<TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx>
        capabilities=0x7ff80<TCP6CSUM_Tx,UDP6CSUM_Rx,UDP6CSUM_Tx,TSO6>
        enabled=0x3ff00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx>
        enabled=0x3ff00<UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx,TCP6CSUM_Tx>
        enabled=0x3ff00<UDP6CSUM_Rx,UDP6CSUM_Tx>
        ec_capabilities=0x17<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,EEE>
        ec_enabled=0x3<VLAN_MTU,VLAN_HWTAGGING>
        address: b4:96:91:92:77:6f
        media: Ethernet autoselect (1000baseT full-duplex,master)
        status: active
        inet6 fe80::b696:91ff:fe92:776f%wm1/64 flags 0 scopeid 0x2
bridge0: flags=0x41<UP,RUNNING> mtu 9000
        capabilities=0x3ff00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx>

capabilities=0x3ff00<UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx,TCP6CSUM_Tx>
        capabilities=0x3ff00<UDP6CSUM_Rx,UDP6CSUM_Tx>
        enabled=0x3ff00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx>
        enabled=0x3ff00<UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx,TCP6CSUM_Tx>
        enabled=0x3ff00<UDP6CSUM_Rx,UDP6CSUM_Tx>
        status: active

	Ethernet adapter (dual Intel I350) seems to run as expected as I can
access to both NAS, but only one iscsi runs as expected.

	I have checked modified files between a running kernel and faulty one :

P sys/arch/amd64/conf/GENERIC
P sys/arch/x86/pci/amdsmn.c
P sys/arch/x86/pci/amdzentemp.c
P sys/arch/x86/pci/pci_machdep.c
P sys/dev/vnd.c
P sys/dev/pci/ahcisata_pci.c
P sys/dev/pci/ichsmb.c
P sys/dev/pci/ismt.c
P sys/dev/pci/pcidevs
P sys/dev/pci/pcidevs.h
P sys/dev/pci/pcidevs_data.h
P sys/dev/pci/sdhc_pci.c
P sys/external/bsd/drm2/dist/include/drm/drm_pciids.h
P sys/modules/Makefile
P sys/net/npf/npf_ruleset.c
P sys/net/npf/npf_tableset.c

	Nothing is direclty related to bridge, but npf support was patched. NPF
is configured and active on this server :

$lan_if = "lagg0"
$wan_if = "wm2"
$bacula_if = "wm0"
$video_if = "wm1"
#$wan_if = "ppp0"
$ext_v4 = inet4($wan_if)
$dmz_if = "re0"

set bpf.jit on;
alg "icmp"
...
group "bacula" on $bacula_if {
    pass final all
}

group "video" on $video_if {
    pass final all
}

group default {
    pass final on lo0 all
    block all
}

If I replace group default by:

group default {
    pass final on lo0 all
    pass all
}

both NAS run fine:

legendre# smartctl -a -d scsi /dev/rsd0d
smartctl 7.3 2022-02-28 r5338 [NetBSD 10.0_BETA amd64] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               QNAP
Product:              iSCSI Storage
Revision:             4.0
Compliance:           SPC-3
User Capacity:        11,790,624,751,616 bytes [11.7 TB]
Logical block size:   512 bytes
LU is fully provisioned
Logical Unit id:      0x6e843b646b168c9da55fd4257da884d2
Serial number:        46b168c9-a55f-4257-a884-2a8f2ccba65c
Device type:          disk
Transport protocol:   iSCSI
Local Time is:        Fri Aug 25 09:49:42 2023 CEST
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Disabled or Not Supported

=== START OF READ SMART DATA SECTION ===
SMART Health Status: OK
Current Drive Temperature:     0 C
Drive Trip Temperature:        0 C

Error Counter logging not supported


[GLTSD (Global Logging Target Save Disable) set. Enable Save with '-S on']
Device does not support Self Test logging
legendre# smartctl -a -d scsi /dev/rsd1d
smartctl 7.3 2022-02-28 r5338 [NetBSD 10.0_BETA amd64] (local build)
Copyright (C) 2002-22, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               QNAP
Product:              iSCSI Storage
Revision:             4.0
Compliance:           SPC-3
User Capacity:        11,798,543,597,568 bytes [11.7 TB]
Logical block size:   512 bytes
LU is fully provisioned
Logical Unit id:      0x6e843b63eced39fd0e2cd401ed92dfd5
Serial number:        3eced39f-0e2c-401e-92df-5e05394a420f
Device type:          disk
Transport protocol:   iSCSI
Local Time is:        Fri Aug 25 09:49:44 2023 CEST
SMART support is:     Available - device has SMART capability.
SMART support is:     Enabled
Temperature Warning:  Disabled or Not Supported

=== START OF READ SMART DATA SECTION ===
SMART Health Status: OK
Current Drive Temperature:     0 C
Drive Trip Temperature:        0 C

Error Counter logging not supported


[GLTSD (Global Logging Target Save Disable) set. Enable Save with '-S on']
Device does not support Self Test logging
legendre#

	I suspect a mistake in npf kernel support introduced by last patches.

	Best regards,

	JKB
Follow-Ups:
- Re: 10_beta and bridged network with NPF
  - From: Martin Husemann
Prev by Date: [PATCH] Fix handling of metadata_cksum
Next by Date: Re: 10_beta and bridged network with NPF
Previous by Thread: [PATCH] Fix handling of metadata_cksum
Next by Thread: Re: 10_beta and bridged network with NPF
Indexes:
Home | Main Index | Thread Index | Old Index