Re: kern/38019: some kind of undetected deadlock slowly kills NetBSD-4.0_STABLE GENERIC.MP

["Greg A. Woods" <woods%planix.com@localhost>]

At Fri, 04 Apr 2008 15:16:31 -0400, Me-planix.com wrote:
Subject: Re: kern/38019: some kind of undetected deadlock slowly kills 
NetBSD-4.0_STABLE GENERIC.MP
> 
> I was going though an old 3.0 kernel config for a client machine,
> preparing to build it on the netbsd-4 and noted that I had applied the
> 3.0 LAST_MINUTE hack to adjust KERNBASE_LOCORE as that machine has 4GB
> of RAM.  Remembering this, noting that my PE2650 also has 4GB of RAM,
> and given that the last few hangs always show processes stuck in
> vmmapva, I've begun to wonder if maybe this hack might still be
> necessary in netbsd-4 too.

Is it possible that the following changes may correct the possible
lockups caused by KVA exhaustion on large-RAM machines without requiring
hacks to KERNBASE and/or manual manipulation of the various tunable
limits such as vm.bufmem_hiwater?  If so I would like to request that
these changes (and any others which are pre- or co-requisite) be pulled
up to netbsd-4.  If desired I can pre-test the pull-up(s) on my machine.

FYI, on my machine right now, after it has once again survived a near
miraculous three days and nights without locking up while running the
wrstuden-fixsa kernel (without any change to KERNBASE), I observe the
following:

        12:01 [2051] # sysctl vm 
        vm.loadavg: 6.21 3.31 3.56
        vm.nkmempages = 32768
        vm.maxslp = 20
        vm.uspace = 12288
        vm.idlezero = 0
        vm.anonmin = 40
        vm.filemin = 5
        vm.execmin = 40
        vm.anonmax = 80
        vm.filemax = 50
        vm.execmax = 50
        vm.inactivepct = 33
        vm.bufcache = 15
        vm.bufmem = 333012992
        vm.bufmem_lowater = 75486720
        vm.bufmem_hiwater = 603893760


----------------------------
revision 1.193
date: 2008/03/25 23:21:43;  author: yamt;  state: Exp;  lines: +9 -3
- for some ports, especially for ones without pmap_growkernel,
  buf_memcalc is used by bootstrap as well.  fix NULL dereference for them.
- limit kva usage for each cache to 20% of vm_map.  XXX a bit arbitrary.
- add a comment.
----------------------------
revision 1.192
date: 2008/03/23 10:39:52;  author: yamt;  state: Exp;  lines: +9 -9
when calculating some cache sizes, consider the amount of available kva.
PR/33185.
----------------------------

Index: sys/kern/vfs_bio.c
===================================================================
RCS file: /cvs/master/m-NetBSD/main/src/sys/kern/vfs_bio.c,v
retrieving revision 1.191
retrieving revision 1.193
diff -u -r1.191 -r1.193
--- sys/kern/vfs_bio.c  23 Mar 2008 10:33:15 -0000      1.191
+++ sys/kern/vfs_bio.c  25 Mar 2008 23:21:43 -0000      1.193
@@ -1,4 +1,4 @@
-/*     $NetBSD: vfs_bio.c,v 1.191 2008/03/23 10:33:15 yamt Exp $       */
+/*     $NetBSD: vfs_bio.c,v 1.193 2008/03/25 23:21:43 yamt Exp $       */
 
 /*-
  * Copyright (c) 2007 The NetBSD Foundation, Inc.
@@ -114,7 +114,7 @@
  */
 
 #include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: vfs_bio.c,v 1.191 2008/03/23 10:33:15 yamt Exp $");
+__KERNEL_RCSID(0, "$NetBSD: vfs_bio.c,v 1.193 2008/03/25 23:21:43 yamt Exp $");
 
 #include "fs_ffs.h"
 #include "opt_bufcache.h"
@@ -388,6 +388,10 @@
        }
 }
 
+/*
+ * note that for some ports this is used by pmap bootstrap code to
+ * determine kva size.
+ */
 u_long
 buf_memcalc(void)
 {
@@ -413,7 +417,9 @@
                        printf("forcing bufcache %d -> 95", bufcache);
                        bufcache = 95;
                }
-               n = physmem / 100 * bufcache;
+               n = calc_cache_size(buf_map, bufcache,
+                   (buf_map != kernel_map) ? 100 : BUFCACHE_VA_MAXPCT)
+                   / PAGE_SIZE;
        }
 
        n <<= PAGE_SHIFT;
@@ -437,12 +443,6 @@
        mutex_init(&buffer_lock, MUTEX_DEFAULT, IPL_NONE);
        cv_init(&needbuffer_cv, "needbuf");
 
-       /*
-        * Initialize buffer cache memory parameters.
-        */
-       bufmem = 0;
-       buf_setwm();
-
        if (bufmem_valimit != 0) {
                vaddr_t minaddr = 0, maxaddr;
                buf_map = uvm_km_suballoc(kernel_map, &minaddr, &maxaddr,
@@ -452,6 +452,12 @@
        } else
                buf_map = kernel_map;
 
+       /*
+        * Initialize buffer cache memory parameters.
+        */
+       bufmem = 0;
+       buf_setwm();
+
        /* On "small" machines use small pool page sizes where possible */
        use_std = (physmem < atop(16*1024*1024));
 


-- 
                                                Greg A. Woods
                                                Planix, Inc.

<woods%planix.com@localhost>     +1 416 489-5852 x122     http://www.planix.com/