Re: kern/57558: pgdaemon 100% busy - no scanning (ZFS case)

[Chuck Silvers <chuq%chuq.com@localhost>]

On Sun, May 05, 2024 at 08:33:03AM +0200, Frank Kardel wrote:
> The issue is not having enough KVA configured  The issue is that with
> large (not XEN specific) memory systems the page daemon attempts to keep
> always at least 10% KVA free. See uvm_km.c:uvm_km_va_starved_p(void) and
> uvm_pdaemon.c:uvm_pageout(void *arg).

ah yes, that is the problem.  this is really a mismatch between how much kmem space
is allocated vs. how much kmem space is allowed to be used before the pagedaemon
tries to reclaim some kmem space.  ZFS is just a victim in this because it happens
to use a lot of kmem space.

I think the right fix for this is to increase the amount of kmem space that we
allocate such that all of physical memory can be allocated as kmem without
the pagedaemon considering the system to be starved for kmem virtual space.
this means allocating an enough kmem space for 10/9 of physical memory,
so that even though only 9/10 of kmem virtual space can be used, there is still
enough kmem virtual space available for all of physical memory.

please try the attached patch.

-Chuck

Index: src/sys/uvm/uvm_km.c
===================================================================
RCS file: /home/chs/netbsd/cvs/src/sys/uvm/uvm_km.c,v
retrieving revision 1.165
diff -u -p -r1.165 uvm_km.c
--- src/sys/uvm/uvm_km.c	9 Apr 2023 09:00:56 -0000	1.165
+++ src/sys/uvm/uvm_km.c	5 May 2024 17:02:49 -0000
@@ -227,7 +227,14 @@ kmeminit_nkmempages(void)
 	}
 
 #if defined(NKMEMPAGES_MAX_UNLIMITED) && !defined(KMSAN)
-	npages = physmem;
+	/*
+	 * The extra 1/9 here is to account for uvm_km_va_starved_p()
+	 * wanting to keep 10% of kmem virtual space free.
+	 * The intent is that on "unlimited" platforms we should be able
+	 * to allocate all of physical memory as kmem without running short
+	 * of kmem virtual space.
+	 */
+	npages = (physmem * 10) / 9;
 #else
 
 #if defined(KMSAN)