On 23/04/2020 08:18, Sad Clouds wrote:
I agree with all this but I'd add one more thing to the mix that might help your system ride out this better.On Wed, 22 Apr 2020 20:46:08 +0200 tlaronde%polynum.com@localhost wrote:(This can be also simply a memory leak in the program but since the RAM is not exhausted...).Without much detail about how much memory, swap, etc your system is using, it's anyone's guess really. You say "RAM is not exhausted" but that why would the system be swapping? Maybe you're not looking at it at the right time, it is possible for a process to malloc() large amount of memory and then quickly release it, by the time you look at it, some processes got swapped out, but the memory got released to the system.
If you have both memory intensive and filesystem intensive processes running on a NetBSD system the kernel filesystem cache can end up evicting programs running in the background that are inactive which then take a LONG time to recover. For a system with a reasonable amount of memory I found the vm.filemin and vm.filemax needed to be tweaked so that filesystem cache was more likely to be tweaked than program code.
I set: vm.filemax=10 vm.filemin=1 in /etc/sysctl.conf for a system with 16GB of RAM.Before I made those changes pkgsrc builds of things like rust and firefox would end up with over 60% of the RAM dedicated to FS cache forcing the swap out of the jenkins server process that was running the build! It would also do the same to live firefox/thunderbird processes running on the same system and pretty much the only way back was a reboot as these processes could never swap themselves back in in a sane amount of time :(
I've also seen the same sort of odd memory consumption behaviour in scripts that 'just' process text files. I once managed to write a perl script that parsed firewall logs that ended up consuming all ram on the system for a reasonable sized log. I never noticed it on FreeBSD and NetBSD (I just put the disk activity down to the fact it was crunching a large log file). However when I tried to run the same script on Linux it would never complete as Linux wasn't able to deal with its memory hog nature at the time (this was years ago on Linux kernel 2.0).
Mike Mike