NetBSD-Bugs archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: kern/49065: ifconfig tun0 ... sequence locks up system



On 08/03/14 16:15, Frank Kardel wrote:
The following reply was made to PR kern/49065; it has been noted by GNATS.

From: Frank Kardel <kardel%netbsd.org@localhost>
To: gnats-bugs%NetBSD.org@localhost
Cc:
Subject: Re: kern/49065: ifconfig tun0 ... sequence locks up system
Date: Sun, 03 Aug 2014 17:14:32 +0200

  Hmmm, LOCKDEBUG gives me:
Starting named.
  panic: lockdebug_lookup: uninitialized lock (lock=0xc057e204, from=e8c9e670)
  cpu0: Begin traceback...
  0xdb3e3ba4: netbsd:db_panic+0xc
  0xdb3e3bd4: netbsd:vpanic+0x144
  0xdb3e3bec: netbsd:snprintf
  0xdb3e3c34: netbsd:lockdebug_wantlock+0x1e8
  0xdb3e3c7c: netbsd:mutex_enter+0x2b4
  0xdb3e3cb4: crypto:cryptoopen+0x84
  0xdb3e3ce4: netbsd:cdev_open+0xac
  0xdb3e3d3c: netbsd:spec_open+0x1e8
  0xdb3e3d5c: netbsd:VOP_OPEN+0x60
  0xdb3e3e34: netbsd:vn_open+0x1f8
  0xdb3e3eb4: netbsd:do_open+0xb0
  0xdb3e3ee4: netbsd:do_sys_openat+0x7c
  0xdb3e3f0c: netbsd:sys_open+0x34
  0xdb3e3f7c: netbsd:syscall+0x8c
  0xdb3e3fac: netbsd:swi_handler+0x98
  cpu0: End traceback...

This one I think should be fixed with the attached diff.


  That's strange.
disabling named gets me past that. Strangely on an sysinst upgraded RPI
  I cannot reproduce the issue, I thought...
But I bisected through the services running (namely named, mdnsd). Collected reproduceable facts:
       - It turns out that *running mdnsd*  will trigger the locking issue.
       - named triggers only a LOCKDEBUG (uninitialized lock) panic (see
  above).
So it looks like mdnsd's handling of the interfaces will leave a lock on
  the socket structure AFAICS.
So, I reproduced locally and can see this...

db> ps/w
PID   LID          COMMAND     EMUL  PRI WAIT-MSG    WAIT-CHANNEL
861  >   1              sed   netbsd   25              0
790      1         ifconfig   netbsd   41 tstile       cbfc5fc0
948      1               sh   netbsd   43 wait         cbd8fa58
567      1               sh   netbsd   38 wait         cbd8f198
954      1               sh   netbsd   41 wait         cbd8f358
923      1            login   netbsd   36 wait         cbd8fc18
753      1             cron   netbsd   35 nanoslp      ca1cb6a0
624      1            inetd   netbsd   37 kqueue       cbf5085c
956      1            mdnsd   netbsd   43 usbxfer      cbdddbcc
909      1             qmgr   netbsd   38 kqueue       cac9ffd4
606      1           pickup   netbsd   38 kqueue       cbf507bc
831      1           master   netbsd   38 kqueue       cac9f854
619      1             sshd   netbsd   39 select       cbf34008
632      1        makemandb   netbsd   24              0
437      1          syslogd   netbsd   43 kqueue       cbf500dc
435      1           dhcpcd   netbsd   43 wait         cbd8ea98
1        1             init   netbsd   36 wait         cbd8fdd8
0       48           system   netbsd   96 nfsiod       cbebceb4
0       47           system   netbsd   96 nfsiod       cbebc8cc
0       46           system   netbsd   96 nfsiod       cbebc85c
0       45           system   netbsd   96 nfsiod       cbebcd2c
0       44           system   netbsd  123 physiod      cb28d7dc
0       43           system   netbsd   96 lnxcmplt     cbd75720
0       42           system   netbsd  125 aiodoned     cbfb6334
0       41           system   netbsd  124 syncer       cbde85c0
0       40           system   netbsd  126 pgdaemon     c05b33c8
0       39           system   netbsd   96 vcdata       cbd7521c
0       38           system   netbsd   96 semacv       c05b4c68
0       37           system   netbsd   96 semacv       c05b4c50
0       36           system   netbsd   96 semacv       c05b4c38
0       32           system   netbsd   96 usbevt       cbe40570
0       31           system   netbsd   96 unpgc        c05b2aa8
0       30           system   netbsd  125 vmem_rehash  cbfb6814
0       29           system   netbsd   96 mmctaskq     cbef2834
0       20           system   netbsd   96 usbtsk       c05b2ba0
0       19           system   netbsd   96 usbtsk       c05b2b80
0       18           system   netbsd  123 dwc2         cbfb6d54
0       17           system   netbsd   96 iicintr      cbfb70f8
0       16           system   netbsd   96 iicintr      cbfb7278
0       15           system   netbsd   43 pmfsuspend   cbfb7a74
0       14           system   netbsd   43 pmfevent     cbfb7ad4
0       13           system   netbsd   96 sopendfr     c05b2a6c
0       12           system   netbsd   43 nfssilly     cbfb7d14
0       11           system   netbsd  125 cachegc      cbf442c0
0       10           system   netbsd  125 vrele        c0550f80
0        9           system   netbsd  125 vdrain       c0550f40
0        8           system   netbsd  125 mod_unld     c0596e20
0        7           system   netbsd  127 xcall        c0443ac8
0        6           system   netbsd  223              0
0        5           system   netbsd  220 tstile       cbfc5fc0
0        4           system   netbsd  221              0
0        3           system   netbsd  222 tstile       cbfc5fc0
0        2           system   netbsd    0              0
0        1           system   netbsd  125 uvm          c0546020
db> show lock cbfc5fc0
lock address : 0x00000000cbfc5fc0 type     :     sleep/adaptive
initialized  : 0x00000000c014e044
shared holds :                  0 exclusive:                  1
shares wanted:                  0 exclusive:                  3
current cpu  :                  0 last held:                  0
current lwp  : 0x00000000cb3c93a0 last held: 0x00000000ca1cbc20
last locked* : 0x00000000c02ab654 unlocked : 0x00000000c02ac82c
owner field  : 0x00000000ca1cbc20 wait/spin:                1/0

Turnstile chain at 0xc054fbc0.
=> Turnstile at 0xcbf43f48 (wrq=0xcbf43f58, rdq=0xcbf43f60).
=> 0 waiting readers:
=> 3 waiting writers: 0xcbf458c0 0xcbf45340 0xcad333c0
db> ps/l
PID    LID S CPU     FLAGS       STRUCT LWP *               NAME WAIT
861  >   1 7   0         0           cb3c93a0                sed
790      1 3   0         0           cad333c0           ifconfig tstile
948      1 3   0        80           cb3c9920                 sh wait
567      1 3   0        80           cb3c8b60                 sh wait
954      1 3   0        80           cb3c8e20                 sh wait
923      1 3   0        80           cb3c9be0              login wait
753      1 3   0        80           ca1cb6a0               cron nanoslp
624      1 3   0        80           cad32600              inetd kqueue
956      1 3   0         0           ca1cbc20              mdnsd usbxfer
909      1 3   0        80           cad32080               qmgr kqueue
606      1 3   0        80           cb3c88a0             pickup kqueue
831      1 3   0        80           ca1cb960             master kqueue
619      1 3   0        80           cad32b80               sshd select
632      1 2   0         0           cb3c8320          makemandb
437      1 3   0        80           cad33940            syslogd kqueue
435      1 3   0        80           cb3c8060             dhcpcd wait
1        1 3   0        80           cbde9640               init wait
0       48 3   0       200           cb3c85e0              nfsio nfsiod
0       47 3   0       200           cad33100              nfsio nfsiod
0       46 3   0       200           cad32e40              nfsio nfsiod
0       45 3   0       200           cad33680              nfsio nfsiod
0       44 3   0       200           cbece5a0            physiod physiod
0       43 3   0       280           cbde8040          VCHIQka-0 lnxcmplt
0       42 3   0       200           cbde8300           aiodoned aiodoned
0       41 3   0       200           cbde85c0            ioflush syncer
0       40 3   0       200           cbde9bc0           pgdaemon pgdaemon
0       39 3   0       280           cbde8880            vcaudio vcdata
0       38 3   0       280           cbde8b40           VCHIQs-0 semacv
0       37 3   0       280           cbece2e0           VCHIQr-0 semacv
0       36 3   0       280           cbece020            VCHIQ-0 semacv
0       32 3   0       200           cbde9900               usb0 usbevt
0       31 3   0       200           cbde9380              unpgc unpgc
0       30 3   0       200           cbde90c0        vmem_rehash vmem_rehash
0       29 3   0       200           cbde8e00             sdmmc0 mmctaskq
0       20 3   0       200           cbece860         usbtask-dr usbtsk
0       19 3   0       200           cbeceb20         usbtask-hc usbtsk
0       18 3   0       200           cbecede0               dwc2 dwc2
0       17 3   0       200           cbecf0a0               iic1 iicintr
0       16 3   0       200           cbecf360               iic0 iicintr
0       15 3   0       200           cbecf620         pmfsuspend pmfsuspend
0       14 3   0       200           cbecf8e0           pmfevent pmfevent
0       13 3   0       200           cbecfba0         sopendfree sopendfr
0       12 3   0       200           cbf44000           nfssilly nfssilly
0       11 3   0       200           cbf442c0            cachegc cachegc
0       10 3   0       200           cbf44580              vrele vrele
0        9 3   0       200           cbf44840             vdrain vdrain
0        8 3   0       200           cbf44b00          modunload mod_unld
0        7 3   0       200           cbf44dc0            xcall/0 xcall
0        6 1   0       200           cbf45080          softser/0
0        5 3   0       200           cbf45340          softclk/0 tstile
0        4 1   0       200           cbf45600          softbio/0
0        3 3   0       200           cbf458c0          softnet/0 tstile
0        2 1   0       201           cbf45b80             idle/0
0        1 3   0       200           c0546020            swapper uvm
db> bt/a cbf458c0
trace: pid 0 lid 3 at 0xcbf3fe7c
0xcbf3fe7c: netbsd:mi_switch+0xc
0xcbf3feac: netbsd:sleepq_block+0xa4
0xcbf3feec: netbsd:turnstile_block+0x2f8
0xcbf3ff34: netbsd:mutex_enter+0x1c4
0xcbf3ff64: netbsd:route_intr+0x30
0xcbf3ffac: netbsd:softint_dispatch+0xec
Bad frame pointer: 0xcac89a74
db> bt/a cbf45340
trace: pid 0 lid 5 at 0xcbf3be4c
0xcbf3be4c: netbsd:mi_switch+0xc
0xcbf3be7c: netbsd:sleepq_block+0xa4
0xcbf3bebc: netbsd:turnstile_block+0x2f8
0xcbf3bf04: netbsd:mutex_enter+0x1c4
0xcbf3bf1c: netbsd:frag6_fasttimo+0x1c
0xcbf3bf34: netbsd:pffasttimo+0x50
0xcbf3bf64: netbsd:callout_softclock+0x18c
0xcbf3bfac: netbsd:softint_dispatch+0xec
Bad frame pointer: 0xcb2abd1c
db> bt/a cad333c0
trace: pid 790 lid 1 at 0xcac89cec
0xcac89cec: netbsd:mi_switch+0xc
0xcac89d1c: netbsd:sleepq_block+0xa4
0xcac89d5c: netbsd:turnstile_block+0x2f8
0xcac89da4: netbsd:mutex_enter+0x1c4
0xcac89dcc: netbsd:soclose+0x20
0xcac89de4: netbsd:soo_close+0x20
0xcac89e24: netbsd:closef+0x6c
0xcac89e6c: netbsd:fd_free+0x174
0xcac89eec: netbsd:exit1+0x100
0xcac89f0c: netbsd:sys_exit+0x3c
0xcac89f7c: netbsd:syscall+0x8c
0xcac89fac: netbsd:swi_handler+0x98
 -- tf 0xbfffca44
db>
db> x/x softnet_lock
netbsd:softnet_lock:    cbfc5fc0


They're all waiting on softnet_lock

Nick
Index: sys/opencrypto/cryptodev.c
===================================================================
RCS file: /cvsroot/src/sys/opencrypto/cryptodev.c,v
retrieving revision 1.79
diff -u -p -r1.79 cryptodev.c
--- sys/opencrypto/cryptodev.c  25 Jul 2014 08:10:40 -0000      1.79
+++ sys/opencrypto/cryptodev.c  3 Aug 2014 16:14:07 -0000
@@ -2085,6 +2085,8 @@ void      cryptoattach(int);
 void
 cryptoattach(int num)
 {
+       crypto_init();
+
        pool_init(&fcrpl, sizeof(struct fcrypt), 0, 0, 0, "fcrpl",
            NULL, IPL_NET);     /* XXX IPL_NET ("splcrypto") */
        pool_init(&csepl, sizeof(struct csession), 0, 0, 0, "csepl",


Home | Main Index | Thread Index | Old Index