Re: kern/38019: some kind of undetected deadlock slowly kills NetBSD-4.0_STABLE GENERIC.MP

["Greg A. Woods" <woods%planix.com@localhost>]

The following reply was made to PR kern/38019; it has been noted by GNATS.

From: "Greg A. Woods" <woods%planix.com@localhost>
To: NetBSD GNATS <gnats-bugs%NetBSD.org@localhost>
Cc: NetBSD Kernel Technical Discussion List <tech-kern%NetBSD.org@localhost>,
        NetBSD/i386 Discussion List <port-i386%NetBSD.org@localhost>,
        YAMAMOTO Takashi <yamt%NetBSD.org@localhost>
Subject: Re: kern/38019: some kind of undetected deadlock slowly kills 
NetBSD-4.0_STABLE GENERIC.MP
Date: Sat, 05 Apr 2008 12:56:13 -0400

 At Fri, 04 Apr 2008 15:16:31 -0400, Me-planix.com wrote:
 Subject: Re: kern/38019: some kind of undetected deadlock slowly kills 
NetBSD-4.0_STABLE GENERIC.MP
 > 
 > I was going though an old 3.0 kernel config for a client machine,
 > preparing to build it on the netbsd-4 and noted that I had applied the
 > 3.0 LAST_MINUTE hack to adjust KERNBASE_LOCORE as that machine has 4GB
 > of RAM.  Remembering this, noting that my PE2650 also has 4GB of RAM,
 > and given that the last few hangs always show processes stuck in
 > vmmapva, I've begun to wonder if maybe this hack might still be
 > necessary in netbsd-4 too.
 
 Is it possible that the following changes may correct the possible
 lockups caused by KVA exhaustion on large-RAM machines without requiring
 hacks to KERNBASE and/or manual manipulation of the various tunable
 limits such as vm.bufmem_hiwater?  If so I would like to request that
 these changes (and any others which are pre- or co-requisite) be pulled
 up to netbsd-4.  If desired I can pre-test the pull-up(s) on my machine.
 
 FYI, on my machine right now, after it has once again survived a near
 miraculous three days and nights without locking up while running the
 wrstuden-fixsa kernel (without any change to KERNBASE), I observe the
 following:
 
        12:01 [2051] # sysctl vm 
        vm.loadavg: 6.21 3.31 3.56
        vm.nkmempages = 32768
        vm.maxslp = 20
        vm.uspace = 12288
        vm.idlezero = 0
        vm.anonmin = 40
        vm.filemin = 5
        vm.execmin = 40
        vm.anonmax = 80
        vm.filemax = 50
        vm.execmax = 50
        vm.inactivepct = 33
        vm.bufcache = 15
        vm.bufmem = 333012992
        vm.bufmem_lowater = 75486720
        vm.bufmem_hiwater = 603893760
 
 
 ----------------------------
 revision 1.193
 date: 2008/03/25 23:21:43;  author: yamt;  state: Exp;  lines: +9 -3
 - for some ports, especially for ones without pmap_growkernel,
   buf_memcalc is used by bootstrap as well.  fix NULL dereference for them.
 - limit kva usage for each cache to 20% of vm_map.  XXX a bit arbitrary.
 - add a comment.
 ----------------------------
 revision 1.192
 date: 2008/03/23 10:39:52;  author: yamt;  state: Exp;  lines: +9 -9
 when calculating some cache sizes, consider the amount of available kva.
 PR/33185.
 ----------------------------
 
 Index: sys/kern/vfs_bio.c
 ===================================================================
 RCS file: /cvs/master/m-NetBSD/main/src/sys/kern/vfs_bio.c,v
 retrieving revision 1.191
 retrieving revision 1.193
 diff -u -r1.191 -r1.193
 --- sys/kern/vfs_bio.c 23 Mar 2008 10:33:15 -0000      1.191
 +++ sys/kern/vfs_bio.c 25 Mar 2008 23:21:43 -0000      1.193
 @@ -1,4 +1,4 @@
 -/*    $NetBSD: vfs_bio.c,v 1.191 2008/03/23 10:33:15 yamt Exp $       */
 +/*    $NetBSD: vfs_bio.c,v 1.193 2008/03/25 23:21:43 yamt Exp $       */
  
  /*-
   * Copyright (c) 2007 The NetBSD Foundation, Inc.
 @@ -114,7 +114,7 @@
   */
  
  #include <sys/cdefs.h>
 -__KERNEL_RCSID(0, "$NetBSD: vfs_bio.c,v 1.191 2008/03/23 10:33:15 yamt Exp 
$");
 +__KERNEL_RCSID(0, "$NetBSD: vfs_bio.c,v 1.193 2008/03/25 23:21:43 yamt Exp 
$");
  
  #include "fs_ffs.h"
  #include "opt_bufcache.h"
 @@ -388,6 +388,10 @@
        }
  }
  
 +/*
 + * note that for some ports this is used by pmap bootstrap code to
 + * determine kva size.
 + */
  u_long
  buf_memcalc(void)
  {
 @@ -413,7 +417,9 @@
                        printf("forcing bufcache %d -> 95", bufcache);
                        bufcache = 95;
                }
 -              n = physmem / 100 * bufcache;
 +              n = calc_cache_size(buf_map, bufcache,
 +                  (buf_map != kernel_map) ? 100 : BUFCACHE_VA_MAXPCT)
 +                  / PAGE_SIZE;
        }
  
        n <<= PAGE_SHIFT;
 @@ -437,12 +443,6 @@
        mutex_init(&buffer_lock, MUTEX_DEFAULT, IPL_NONE);
        cv_init(&needbuffer_cv, "needbuf");
  
 -      /*
 -       * Initialize buffer cache memory parameters.
 -       */
 -      bufmem = 0;
 -      buf_setwm();
 -
        if (bufmem_valimit != 0) {
                vaddr_t minaddr = 0, maxaddr;
                buf_map = uvm_km_suballoc(kernel_map, &minaddr, &maxaddr,
 @@ -452,6 +452,12 @@
        } else
                buf_map = kernel_map;
  
 +      /*
 +       * Initialize buffer cache memory parameters.
 +       */
 +      bufmem = 0;
 +      buf_setwm();
 +
        /* On "small" machines use small pool page sizes where possible */
        use_std = (physmem < atop(16*1024*1024));
  
 
 
 -- 
                                                Greg A. Woods
                                                Planix, Inc.
 
 <woods%planix.com@localhost>     +1 416 489-5852 x122     
http://www.planix.com/