Subject: kern/10374: lockmgr: locking against myself during heavy disk IO
To: None <gnats-bugs@gnats.netbsd.org>
From: Aaron J. Grier <aaron@frye.com>
List: netbsd-bugs
Date: 06/15/2000 12:04:15
>Number:         10374
>Category:       kern
>Synopsis:       lockmgr: locking against myself during heavy disk IO
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Jun 15 12:05:00 PDT 2000
>Closed-Date:
>Last-Modified:
>Originator:     
>Release:        1.4.2
>Organization:
Frye Electronics, Tigard, OR
>Environment:
	
System: NetBSD kaben 1.4.2 NetBSD 1.4.2 (kaben) #4: Tue Jun 6 16:48:48 PDT 2000 aaron@kaben:/usr/src/sys/arch/i386/compile/kaben i386


>Description:
	during heavy disk IO (usually the daily cron jobs) the machine
will panic.  running a bunch of concurrent finds also triggers it.  This
appears to be the first lockmgr problem dealing with nfs_lookup according
to query-pr.

(gdb) target kcore netbsd.0.core
panic: lockmgr: locking against myself
#0  0xc012564d in lockstatus (lkp=0xc4ac0b30)
    at ../../../../kern/kern_lock.c:154
154     }
(gdb) where
#0  0xc012564d in lockstatus (lkp=0xc4ac0b30)
    at ../../../../kern/kern_lock.c:154
#1  0xc01f1bd3 in cpu_reboot (howto=260, bootstr=0x0)
    at ../../../../arch/i386/i386/machdep.c:1353
#2  0xc0132109 in panic (fmt=0xc012564d "lockmgr: locking against myself")
    at ../../../../kern/subr_prf.c:212
#3  0xc01259da in lockmgr (lkp=0xc4a60f0c, flags=65554, interlkp=0xc4a11e64)
    at ../../../../kern/kern_lock.c:328
#4  0xc01dc452 in ufs_lock (v=0xc4ac0b70)
    at ../../../../ufs/ufs/ufs_vnops.c:1585
#5  0xc014fc93 in vn_lock (vp=0xc4a11dd4, flags=65554)
    at ../../../../sys/vnode_if.h:724
#6  0xc014993a in vget (vp=0xc4a11dd4, flags=65554)
    at ../../../../kern/vfs_subr.c:872
#7  0xc01cb043 in ffs_sync (mp=0xc0487e00, waitfor=2, cred=0xc044e480, 
    p=0xc0297234) at ../../../../ufs/ffs/ffs_vfsops.c:825
#8  0xc014bd0c in sys_sync (p=0xc0297234, v=0x0, retval=0x0)
    at ../../../../kern/vfs_syscalls.c:524
#9  0xc014af6e in vfs_shutdown () at ../../../../kern/vfs_subr.c:2056
#10 0xc01f1bab in cpu_reboot (howto=256, bootstr=0x0)
    at ../../../../arch/i386/i386/machdep.c:1340
#11 0xc0132109 in panic (fmt=0xc012564d "lockmgr: locking against myself")
    at ../../../../kern/subr_prf.c:212
#12 0xc01259da in lockmgr (lkp=0xc4a60f0c, flags=65538, interlkp=0xc4a11e64)
    at ../../../../kern/kern_lock.c:328
#13 0xc01dc452 in ufs_lock (v=0xc4ac0c90)
    at ../../../../ufs/ufs/ufs_vnops.c:1585
#14 0xc014fc93 in vn_lock (vp=0xc4a11dd4, flags=65538)
    at ../../../../sys/vnode_if.h:724
#15 0xc0149ae2 in vrele (vp=0xc4a11dd4) at ../../../../kern/vfs_subr.c:948
#16 0xc01acd73 in nfs_lookup (v=0xc4ac0e04) at ../../../../nfs/nfs_vnops.c:822
#17 0xc01486c8 in lookup (ndp=0xc4ac0e94) at ../../../../sys/vnode_if.h:71
#18 0xc01483b7 in namei (ndp=0xc4ac0e94) at ../../../../kern/vfs_lookup.c:153
#19 0xc014d690 in sys___stat13 (p=0xc4aa8894, v=0xc4ac0f88, retval=0xc4ac0f80)
    at ../../../../kern/vfs_syscalls.c:1572
#20 0xc01f8f9a in syscall (frame={tf_es = -1078001633, tf_ds = 31, 
      tf_edi = -1077945036, tf_esi = -1077945028, tf_ebp = -1077945276, 
      tf_ebx = 53400, tf_edx = 819072, tf_ecx = 0, tf_eax = 278, 
      tf_trapno = 3, tf_err = 2, tf_eip = 1076397967, tf_cs = 23, 
      tf_eflags = 582, tf_esp = -1077946424, tf_ss = 31, tf_vm86_es = 0, 
      tf_vm86_ds = 0, tf_vm86_fs = 0, tf_vm86_gs = 0})
    at ../../../../arch/i386/i386/trap.c:782

NetBSD 1.4.2 (kaben) #4: Tue Jun  6 16:48:48 PDT 2000
    aaron@kaben:/usr/src/sys/arch/i386/compile/kaben
cpu0: family 6 model 1 step 9
cpu0: Intel Pentium Pro (686-class)
real mem  = 66715648
avail mem = 59785216
using 840 buffers containing 3440640 bytes of memory
biostramp installed @ 1000
mainbus0 (root)
pci0 at mainbus0 bus 0: configuration mode 1
pci0: i/o enabled, memory enabled
pchb0 at pci0 dev 0 function 0
pchb0: Intel 82441FX PCI and Memory Controller (PMC) (rev. 0x02)
pcib0 at pci0 dev 7 function 0
pcib0: Intel 82371SB PCI-to-ISA Bridge (PIIX3) (rev. 0x01)
pciide0 at pci0 dev 7 function 1: Intel 82371SB IDE Interface (PIIX3)
pciide0: bus-master DMA support present
pciide0: primary channel wired to compatibility mode
pciide0: disabling primary channel (no drives)
pciide0: secondary channel wired to compatibility mode
pciide0: disabling secondary channel (no drives)
uhci0 at pci0 dev 7 function 2: Intel 82371SB USB Host Controller (PIIX3) (rev. 0x01)
uhci0: interrupting at irq 10
uhci0: USB version 1.0
usb0 at uhci0
uhub0 at usb0
uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
vga0 at pci0 dev 11 function 0: S3 ViRGE/DX (rev. 0x01)
wsdisplay0 at vga0: console (80x25, vt100 emulation)
fxp0 at pci0 dev 15 function 0: Intel EtherExpress Pro 10+/100B Ethernet
fxp0: interrupting at irq 10
fxp0: Ethernet address 00:d0:b7:3f:c1:50
inphy0 at fxp0 phy 1: i82555 10/100 media interface, rev. 4
inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
Ensoniq AudioPCI 97 (audio multimedia, revision 0x08) at pci0 dev 17 function 0 not configured
ncr0 at pci0 dev 19 function 0: ncr 53c895 fast40 wide scsi
ncr0: interrupting at irq 11
ncr0: minsync=10, maxsync=137, maxoffs=31, 128 dwords burst, large dma fifo
ncr0: single-ended, open drain IRQ driver, using on-chip SRAM
ncr0: restart (scsi reset).
scsibus0 at ncr0: 16 targets, 8 luns per target
sd0 at scsibus0 targ 0 lun 0: <IBM, DDRS-34560D, DC1B> SCSI2 0/direct fixed
sd0(ncr0:0:0): WIDE SCSI (16 bit) enabled
sd0(ncr0:0:0): 40.0 MB/s (50 ns, offset 15)
sd0: 4357MB, 8387 cyl, 5 head, 212 sec, 512 bytes/sect x 8925000 sectors
isa0 at pcib0
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
lpt0 at isa0 port 0x378-0x37b irq 7
pckbc0 at isa0 port 0x60-0x64
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard
pmsi0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pmsi0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
spkr0 at pcppi0
sysbeep0 at pcppi0
npx0 at isa0 port 0xf0-0xff: using exception 16
fdc0 at isa0 port 0x3f0-0x3f7 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB, 80 cyl, 2 head, 18 sec
apm0 at mainbus0: Power Management spec V1.1 (BIOS mgmt disabled) (BIOS managing devices)
apm0: A/C state: on
apm0: battery charge state: no battery
biomask c40 netmask c40 ttymask 1cc2
wscons: wskbd0 glued to wsdisplay0 (console)
boot device: sd0
root on sd0a dumps on sd0b
mountroot: trying msdos...
mountroot: trying cd9660...
mountroot: trying nfs...
mountroot: trying lfs...
mountroot: trying ext2fs...
mountroot: trying ffs...
root file system type: ffs

coredump and debugging kernel available on request.

>How-To-Repeat:
	running a whole bunch of find jobs will do it.  (49 were run to
produce this crash, however, often as few as one or two will trigger it.)
>Fix:
	you guys are the kernel gurus.  :)
>Release-Note:
>Audit-Trail:
>Unformatted: