Subject: kern/10892: Panic on NetbSD-1.5_ALPHA2 with NFS/IPv6/tlp
To: None <gnats-bugs@gnats.netbsd.org>
From: None <feico@pasta.cs.uit.no>
List: netbsd-bugs
Date: 08/24/2000 05:33:20
>Number:         10892
>Category:       kern
>Synopsis:       Repeatable panic by reading a large file over NFS
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Aug 24 05:34:00 PDT 2000
>Closed-Date:
>Last-Modified:
>Originator:     Feico Dillema
>Release:        NetBSD-1.5-ALPHA2 21-Aug-2000
>Organization:
University of Tromso
>Environment:
System: NetBSD drifter.dillema.net 1.5_ALPHA2 NetBSD 1.5_ALPHA2
>Description:

I get repeatable panics on my NetBSD-1.5_ALPHA2 system. Sources
updated as of yesterday. I have mounted /usr/src over NFS over IPv6
and try to do a make build. AFter some time my machine panics. I've
seen similar panics before I updated my kernel (it was the reason I
updated) simply by reading my large inbox into mutt (homedir also
mounted over NFS over IPv6). I've written down the trace from the
kernel debugger  after the last panic below (manually so there's 
the risk of typos). 

I'm running a make build now again (waiting for the panic?) and added
the output of vmstat -m fairly early after starting the make build
(after a reboot) to this mail. You might get some
useful info from that maybe? After a panic I also see warnings now and
then about mclpool limit reached (that's why I suspect a memory leak
somewhere).

The problem disappears when swithing to a different network card (from a
tlp at cardbus (100Mbs) to a ep0 at pcmcia (10Mbs)). Note that need to use
different kernels in both cases; the 3Com ep0 doesn't get detected on my
thinkpad 570 when cardbus is in the kernel).

Minor note: the kernel with the problem had also options TLP_DEBUG set in its
config. No idea whether that makes a difference.

===========

uvm_fault(0xc030a2c0, 0xdeadb000,0,3) -> 2
page fault trap code = 0

stopped in nfsio at frag6_deq+0xc: movl %edx, 0x8(%ecx)

db trace:

frag6_deq(c079d5c0) at - + 0xc
frag6_freef(c0830b80) at - +0x23
frag6_drain(f000ffe6,c0834800, e990dd30, c0119026) at - + 0x31
m_reclaim(1, c082f500, 37, c0646000, 20) at - + 0x39
tlp_add_rxbuf(c0646000,37) at - +0x11
tlp_rxintr(c0646000) at - +0x21e
tlp_intr(c0646000, e9547000,0,c05f5000,1) at - +0x15b
pccbbintr_function(c05f5000, c05ffc80,e9522442, de2caa1c,1) at - +0xce
pccbbintr(c05f5000) at - + 0x5a
Xintr11() at - + 0x5a
-- interrupt --
memcpy(e990decc,e990df34,2000,e990ded0,200) at memcpy + 0x1a
nfs_readrpc(e98306c0, e990df34,c06c9400,c2b6557c,c06a1000) at - + 0x78a
nfs_doio(c2b6557c,c06c9400,0) at - + 0x189
nfssvc_iod(e990e00c,0,c010032b,e990e00c,e9692e30) at - + 0x11b
start_nfsio(e990e00c) at - + 0xe

vmstat -m:

Memory statistics by bucket size
    Size   In Use   Free   Requests  HighWater  Couldfree
      16      645    123     147929    1280          0
      32      159    865      40449     640        525
      64     2541     83     231134     320          0
     128      219     37     137168     160          0
     256     1790     18     147756      80          0
     512      152     24      75716      40          0
    1024      151     13     501300      20          0
    2048       11     11      12312      10          0
    4096       13      9       7243       5          7
    8192       11     12       5921       5        876
   65536        6      0         13       5          0
  131072        3      0         10       5          0

Memory usage type by bucket size
    Size  Type(s)
      16  devbuf, pcb, routetbl, soname, proc, ip_moptions, in_multi, exec,
	  miscfs mount, VM swap, UVM amap, temp, USB, key mgmt, NDP
      32  devbuf, pcb, routetbl, ifaddr, soname, vnodes, LFS segment, MFS node,
	  ether_multi, VM swap, UVM amap, temp, USB
      64  devbuf, pcb, routetbl, fragtbl, ifaddr, session, NFS req, lockf,
	  LFS segment, in_multi, exec, NFS diroff, UVM amap, temp, DMA map, USB,
	  NDP
     128  devbuf, routetbl, fragtbl, ifaddr, cred, ttys, exec, VM swap,
	  UVM amap, temp, USB, USB device, NDP
     256  devbuf, pcb, routetbl, ifaddr, vnodes, VM map, file desc, proc,
	  NFS srvsock, NFS daemon, exec, NFS diroff, UVM amap, temp, DMA map,
	  USB, Pool, newblk
     512  devbuf, pcb, ifaddr, ioctlops, mount, namecache, UFS mount, file desc,
	  MSDOSFS mount, UVM amap, temp, USB device, key mgmt, NDP
    1024  devbuf, namei, NFS mount, UFS mount, proc, NQNFS Lease, Export Host,
	  ttys, exec, UVM amap, UVM aobj, temp, pagedep
    2048  devbuf, ifaddr, UFS mount, MSDOSFS mount, UVM amap, temp
    4096  devbuf, NFS node, namecache, UFS mount, ISOFS mount, UVM amap, temp,
	  inodedep
    8192  devbuf, UFS mount, UVM amap
   65536  devbuf, namecache, UVM amap
  131072  UVM amap

Memory statistics by type                        Type  Kern
         Type  InUse MemUse HighUse  Limit Requests Limit Limit Size(s)
        devbuf   447   280K    280K 19594K      463    0     0  16,32,64,128,256,512,1024,2048,4096,8192,65536
           pcb    34     8K      9K 19594K     3630    0     0  16,32,64,256,512
      routetbl   304    19K     19K 19594K      584    0     0  16,32,64,128,256
       fragtbl     0     0K      2K 19594K    15011    0     0  64,128
        ifaddr    29     8K      8K 19594K       31    0     0  32,64,128,256,512,2048
        soname     3     1K      1K 19594K       42    0     0  16,32
         namei     0     0K      9K 19594K   442071    0     0  1024
      ioctlops     0     0K      1K 19594K       34    0     0  512
          cred    11     2K      2K 19594K     5271    0     0  128
       session    20     2K      2K 19594K       28    0     0  64
         mount     7     4K      4K 19594K       10    0     0  512
       NFS req     0     0K      1K 19594K   118181    0     0  64
     NFS mount     2     2K      2K 19594K        2    0     0  1024
      NFS node     1     4K      4K 19594K        1    0     0  4096
        vnodes    42     6K     31K 19594K      855    0     0  32,256
     namecache     3    69K     69K 19594K        3    0     0  512,4096,65536
     UFS mount    14    31K     31K 19594K       15    0     0  512,1024,2048,4096,8192
        VM map     4     1K      1K 19594K        4    0     0  256
     file desc     6     3K      4K 19594K       65    0     0  256,512
         lockf     2     1K      1K 19594K       98    0     0  64
          proc     6     3K      3K 19594K        6    0     0  16,256,1024
   LFS segment     0     0K      1K 19594K      116    0     0  32,64
      MFS node     2     1K      1K 19594K        2    0     0  32
   NQNFS Lease     1     1K      1K 19594K        1    0     0  1024
   Export Host     9     9K      9K 19594K        9    0     0  1024
   NFS srvsock     3     1K      1K 19594K        3    0     0  256
    NFS daemon     1     1K      1K 19594K        1    0     0  256
   ip_moptions     1     1K      1K 19594K        1    0     0  16
      in_multi     9     1K      1K 19594K        9    0     0  16,64
   ether_multi     4     1K      1K 19594K        4    0     0  32
   ISOFS mount     1     4K      4K 19594K        1    0     0  4096
 MSDOSFS mount     1     2K      3K 19594K        2    0     0  512,2048
          ttys   210   137K    137K 19594K      210    0     0  128,1024
          exec     0     0K      2K 19594K    27067    0     0  16,64,128,256,1024
  miscfs mount     1     1K      1K 19594K        1    0     0  16
    NFS diroff  3425   530K    531K 19594K     4439    0     0  64,256
       VM swap     6     1K      1K 19594K        6    0     0  16,32,128
      UVM amap   843   515K    869K 19594K   650261    0     0  16,32,64,128,256,512,1024,2048,4096,8192,65536,131072
      UVM aobj     1     1K      1K 19594K        1    0     0  1024
          temp     2     5K      7K 19594K    37874    0     0  16,32,64,128,256,512,1024,2048,4096
       DMA map   140     9K      9K 19594K      140    0     0  64,256
           USB    14     2K      2K 19594K       14    0     0  16,32,64,128,256
    USB device     4     2K      2K 19594K        4    0     0  128,512
          Pool     4     1K      1K 19594K        4    0     0  256
      key mgmt    75    26K     30K 19594K      369    0     0  16,512
           NDP     6     1K      1K 19594K        8    0     0  16,64,128,512
       pagedep     1     1K      1K 19594K        1    0     0  1024
      inodedep     1     4K      4K 19594K        1    0     0  4096
        newblk     1     1K      1K 19594K        1    0     0  256

Memory Totals:  In Use    Free    Requests
                 1686K    223K     1306955
Memory resource pool statistics
Name        Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
phpool        40       91    0        0     1     0     1     1     0   inf    0
pmappl        68     8931    0     8895     1     0     1     1     0   inf    0
vmsppl       184     8931    0     8895     2     0     2     2     0   inf    0
vmmpepl       64   147552    0   147133     8     0     8     8     0   inf    1
uaoeltpl      84        0    0        0     0     0     0     0     0   inf    0
aobjpl        52        0    0        0     0     0     0     0     0   inf    0
amappl        40    73625    0    73383     3     0     3     3     0   inf    0
mbpl         256  1371633    0  1371567    10     0    10    10     1   inf    5
mclpl       2048   146992    0   146928    60     0    60    60     4   256   26
sockpl       164     3686    0     3652     2     0     2     2     0   inf    0
ttypl        284       42    0        0     3     0     3     3     0   inf    0
rndsample    528      276    0      273     1     0     1     1     0   inf    0
procpl       404     8942    0     8895     6     0     6     6     0   inf    1
pgrppl        24       62    0       38     1     0     1     1     0   inf    0
pcredpl       24     8942    0     8895     1     0     1     1     0   inf    0
plimitpl     156       16    0        6     1     0     1     1     0   inf    0
rusgepl       72     8895    0     8895     1     0     1     1     0   inf    1
filepl        48    65557    0    65486     2     0     2     2     0   inf    1
cwdipl        12     8931    0     8895     1     0     1     1     0   inf    0
fdescpl      124     8931    0     8895     2     0     2     2     0   inf    0
vnodepl      208     3950    0        0   208     0   208   208     0   inf    0
ncachepl      72     3950    0        0    71     0    71    71     0   inf    0
ffsinopl     212    12938    0    12659    64     0    64    64     0   inf   25
ext2fsinopl  212        0    0        0     0     0     0     0     0   inf    0
nfsnodepl    204    46100    0    42439   194     0   194   194     0   inf    0
nfsvapl      100    46100    0    42439    92     0    92    92     0   inf    0
cd9660nopl   108        0    0        0     0     0     0     0     0   inf    0
msdosnopl    100        0    0        0     0     0     0     0     0   inf    0
extent        20      159    0      143     1     0     1     1     0   inf    0
wdcspl        48    11589    0    11589     1     0     1     1     0   inf    1
bufpl        124     7835    0     7835     1     0     1     1     0   inf    1
rtentpl      128      236    0       12     8     0     8     8     0   inf    0
tcpcbpl      176       16    0        6     1     0     1     1     0   inf    0
synpl        168        2    0        2     1     0     1     1     0   inf    1
sigapl       840     8931    0     8895    11     0    11    11     0   inf    2
swp buf      152        0    0        0     0     0     0     0     0   inf    0
swp vnx       20        0    0        0     0     0     0     0     0   inf    0
swp vnd      128        0    0        0     0     0     0     0     0   inf    0

In use 2525K, total allocated 3036K; utilization 83.2%


>How-To-Repeat:
	On my machine simply by opening my large inbox over NFS into mutt or
	by starting a `make build' over NFS.
>Fix:
		unknown

>Release-Note:
>Audit-Trail:
>Unformatted:
 >(DRIFTER) #0: Thu Aug 21 i386