Re: Growth in pool usage between netbsd-9 and -10?

[Greg Troxel <gdt%lexort.com@localhost>]

Paul Ripke <stix%stix.id.au@localhost> writes:

>> That seems a little excessive, to my eye? Is this "normal"? Having >1/4 of RAM
>> apparently used mostly for file metadata, if I'm reading this correctly? I
>> don't recall the usage under -9, unfortunately, but I also never had a reason
>> to look.

In general, kernel memory tends to fill with cache entries, and as long
as it is freed if needed for something else, that's fine.  It's only
memory without effective draining under pressure (hi zfs!) that is
problematic.

I think that vm.filemax is supposed to define the level above which file
cache is supposed to be freed with higher priority, and vm.filemin is
the level below which file cache is protected from freeing.   However I
am not sure if that is the block cache or also pool usage for metadata.

I use the program below "touchmem.c" to force memory to be used for
program data.  This causes a great deal of freeing usually.  It
obviously can lead to paging if too big, but you can also run it with
varying sizes and figure out when it starts being slow.

Actually I should enhance this to be a loop with timing to measure the
delay from the freeing activity by doing each amount twice, and to see
in the second runs the runtime vs size which should depart from linear
when it crosses into paging the program itself.

Running this in a xen domU with 2000 MB (not 2048), I pushed "file" as
shown by top to 383M.  This is with default (9) settings:

  vm.anonmin = 10
  vm.filemin = 10
  vm.execmin = 5
  vm.anonmax = 80
  vm.filemax = 50
  vm.execmax = 30
  vm.inactivepct = 33
  vm.bufcache = 15
  vm.bufmem = 170868736
  vm.bufmem_lowater = 58982400
  vm.bufmem_hiwater = 471859200

After running this with a high value:

$ time touchmem 500; time touchmem 1000; time touchmem 1500; time touchmem 2000

real    0m1.843s
user    0m0.169s
sys     0m1.676s

real    0m3.862s
user    0m0.580s
sys     0m3.281s

real    0m6.095s
user    0m0.648s
sys     0m5.446s

real    0m26.580s
user    0m1.332s
sys     0m8.701s

Which basically shows that a 1500 MB process is not really impaired and
a larger one (with data equal to the machine's entire RAM) is.  This
seems ok.  But it's 9.

----------------------------------------
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

int
main(int argc, char **argv)
{
	char *foo;
	size_t i, size;
	int pgsize;

	if (argc != 2) {
		printf("usage: touchmem size (in MB)\n");
		exit(EXIT_FAILURE);
	}
	  
	size = atoi(argv[1]) * 1024L * 1024L;
#if 0
	printf("size is %zd\n", size);
#endif

	pgsize = getpagesize();
	foo = malloc(size);
	if (foo == NULL) {
		printf("Can't malloc.\n");
		exit(EXIT_FAILURE);
	}

	for (i = 0; i < size; i += pgsize)
		foo[i] = 1;
	exit(EXIT_SUCCESS);
}