kern/57690: XEN pvh guest GENERIC: xennet0: no rx clusters (possible mbuf/pool leak)

[kardel%netbsd.org@localhost]

>Number:         57690
>Category:       kern
>Synopsis:       XEN pvh guest GENERIC: xennet0: no rx clusters (possible mbuf/pool leak)
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Nov 10 12:30:00 +0000 2023
>Originator:     Frank Kardel
>Release:        NetBSD 10.0_BETA@20231005
>Organization:
	
>Environment:
	
	
System: NetBSD test2 10.0_BETA NetBSD 10.0_BETA (GENERIC) #0: Thu Oct 5 17:21:23 CEST 2023 kardel%gaia.acrys.com@localhost:/src/NetBSD/n10/src/obj.amd64/sys/arch/amd64/compile/GENERIC amd64
Architecture: x86_64
Machine: amd64
>Description:
	Running GENERIC as PVH guest on Xen shows after some time

[Wed Nov  8 10:51:42 UTC 2023] NetBSD 10.0_BETA (GENERIC) #0: Thu Oct  5 17:21:23 CEST 2023
[Wed Nov  8 10:51:42 UTC 2023] 	kardel%gaia.acrys.com@localhost:/src/NetBSD/n10/src/obj.amd64/sys/arch/amd64/compile/GENERIC
....
[Wed Nov  8 10:51:42 UTC 2023] boot device: dk0
[Wed Nov  8 10:51:42 UTC 2023] root on dk0
[Wed Nov  8 10:51:42 UTC 2023] root file system type: ffs
[Wed Nov  8 10:51:42 UTC 2023] kern.module.path=/stand/amd64/10.0/modules
[Wed Nov  8 10:51:42 UTC 2023] xenbus0: autoconfiguration error: can't get state for device/suspend/event-channel (2)
[Wed Nov  8 10:51:42 UTC 2023] xenbus0: autoconfiguration error: can't get state for device/suspend/event-channel (2)
[Wed Nov  8 10:51:42 UTC 2023] ZFS filesystem version: 5
[Wed Nov  8 22:27:16 UTC 2023] xennet0: rx no cluster
[Wed Nov  8 22:49:36 UTC 2023] xennet0: rx no cluster
...
[Fri Nov 10 05:00:27 UTC 2023] xennet0: rx no cluster
[Fri Nov 10 05:00:31 UTC 2023] xennet1: rx no cluster
[Fri Nov 10 05:00:46 UTC 2023] xennet0: rx no cluster
[Fri Nov 10 05:00:57 UTC 2023] xennet0: rx no mbuf
[Fri Nov 10 05:01:08 UTC 2023] xennet0: rx no mbuf
[Fri Nov 10 05:01:15 UTC 2023] xennet0: rx no mbuf
[Fri Nov 10 05:01:17 UTC 2023] xennet0: rx no cluster
...
[Fri Nov 10 05:48:37 UTC 2023] xennet0: rx no cluster
[Fri Nov 10 05:48:45 UTC 2023] xennet0: rx no cluster

	Mbuf resources are plenty at start.
netstat -m:
359 mbufs in use:
        334 mbufs allocated to data
        23 mbufs allocated to packet headers
        2 mbufs allocated to socket names and addresses
0 calls to protocol drain routines

sysctl kern.mbuf:
kern.mbuf.msize = 512
kern.mbuf.mclbytes = 2048
kern.mbuf.nmbclusters = 8192000
kern.mbuf.mblowat = 16
kern.mbuf.mcllowat = 8
kern.mbuf.nmbclusters_limit = 32768000

	According to ./arch/xen/xen/if_xennet_xenbus.c 
	storage for rx mbufs is allocated from the pool "xnfrx".
	After start I have yet to see the MBUFS are being freed.
Memory resource pool statistics
Name        Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
xnfrx       4096      779    0        0   779     0   779   779     0   inf    0

	So, needsless to say, the system moves perfectly usable to unusable.

>How-To-Repeat:
	Run a GENERIC PVH guest, do significant network i/o (e.g backups).
	Observe that allocations but no releases happen in the xnfrx pool.
	The MBUFs seem to be in the rx ring buffer, so it seems the rx ring
	grows possibly without bound.
>Fix:
	Find out why m_free*() is not being called or why the rx ring keeps growing as it seems.

>Unformatted: