kern/56753: Certain VLAN IDs don't work for IPv6 across internal bridges.

[liman%cafax.se@localhost]

>Number:         56753
>Category:       kern
>Synopsis:       Certain VLAN IDs don't work for IPv6
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Mar 16 10:35:00 +0000 2022
>Originator:     Lars-Johan Liman
>Release:        NetBSD 9.2
>Organization:
#-------------------------------------------------------------------------
# Lars-Johan Liman, M.Sc.		 ! E-mail: liman%cafax.se@localhost
# Cafax AB				 ! HTTP  : //www.cafax.se/
# Computer Consultants, Sweden		 ! Voice : +46 8 - 564 702 30
#-------------------------------------------------------------------------
>Environment:
System: NetBSD trough.liman.net 9.2 NetBSD 9.2 (XEN3_DOM0) #0: Wed May 12 13:15:55 UTC 2021  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/xen/compile/XEN3_DOM0 amd64
Architecture: x86_64
Machine: amd64
>Description:
	This is a strange one ... :-)

	If you have Xen domU clients connected across a bridge(4) in the
	dom0, and want to use IPv6 to communicate between the domUs
	using different VLANs, that works well, EXCEPT if you are
	using VLAN ID 112 or VLAN ID 240 (maybe others too)?

	I note that the bit pattern for numbers 112 and 240 (decimal) are
	0111 0000 and 1111 0000 respectively, meaning that they have
	the sequence x111 0000 in common. We have tested on two
	separate dom0s, and VLAN IDs 100-111 and 113-126 work fine.
	112 does not work, and, after having looked at the bit pattern,
	I also tested 240 which also did not work. We have not tested
	other numbers.
									
	The problem manifests itself in the way that neighbor
	solicitations don't cross the bridge. Tcpdump on the sending
	host shows that they are sent, but no packet is received on
	the receiving host. I believe these are sent using IPv6
	multicast, but I'm not certain of the details. The exact same
	setup works fine with other VLAN IDs.
>How-To-Repeat:
	This is from my head:
	- Start with a dom0 running NetBSD/amd64 9.2-RELEASE
	  XEN3_DOM0 kernel and xen-4.11.4nb2 hypervisor. (The Xen
  	  version seems less relevant here, we have tested two.)
  	- Set up an internal bridge on the dom0.
	  - ifconfig bridge0 create ; ifconfig bridge0 up
  	- Create two amd64 virtual guests (domUs) that both have
	  vif = [ 'bridge=bridge0' ]
  	- Boot the domUs.
	- On each domU do (respectively):
	
	  domU-alpha:
	  - ifconfig xennet0 up
  	  - ifconfig vlan1 create
  	  - ifconfig vlan1 vlan 112 vlanif xennet0
  	  - ifconfig vlan1 10.1.1.1/24 up
    	  - ifconfig vlan1 3ffe:1:1:1::1/64

	  domU-bravo:
  	  - ifconfig xennet0 up
  	  - ifconfig vlan1 create
  	  - ifconfig vlan1 vlan 112 vlanif xennet0
  	  - ifconfig vlan1 10.1.1.2/24 up
    	  - ifconfig vlan1 3ffe:1:1:1::2/64

	- Use tcpdump to look at "icmp6" traffic on vlan1 on both
	  machines.
	  - alpha# tcpdump -n -i vlan1 icmp6 &
	  - bravo# tcpdump -n -i vlan1 icmp6 &

  	- Now try to ping the second from the first:
	  - alpha# ping6 -n 3ffe:1:1:1::2
	  For comparison try
  	  - alpha# ping -n 10.1.1.2
	  
  	- For more comparison, back out of the configuration above,
	  and repeat it with another VLAN ID (e.g., 111). The "ping6"
	  will then work.
	    
    	- Finnally utter the phrase "HUH?! WTF?!". :-)
>Fix:
	Workaround: avoid VLAN IDs 112 and 240 (and others?).
	Fix: no known.