Re: kern/55933: diskless machine hangs at boot

[Martin Husemann <martin%duskware.de@localhost>]

The following reply was made to PR kern/55933; it has been noted by GNATS.

From: Martin Husemann <martin%duskware.de@localhost>
To: gnats-bugs%netbsd.org@localhost
Cc: 
Subject: Re: kern/55933: diskless machine hangs at boot
Date: Thu, 21 Jan 2021 20:37:39 +0100

 It seems to be looping in pmap of the shell process executing rc.d/network.
 On every third (or so) entry to ddb the "continue" kills the kernel due
 to reaching userret() with the pmap lock (of that shell process) still
 held.
 
 Below is the tail of a uvmhist log before the lock held panic.
 
 Martin
 
 1611256850.687880 prefetch_abort_handler#4094920@3: called!
 1611256850.687880 prefetch_abort_handler#4094920@3:  (pc=0x6d2fdffe, l=0x914fc88
 0, tf=0x9d431fb0)
 1611256850.687881 pmap_fault_fixup#4108685@3: called: pm=0x915c8070, va=0x6d2fd0
 00, ftype=0x5, user=1
 1611256850.687881 pmap_fault_fixup#4108685@3:  ti=0x8089c670 pai=0x915c849c asid
 =0x60
 1611256850.687887 prefetch_abort_handler#4094920@3:  <- emulated
 1611256850.687888 prefetch_abort_handler#4094921@3: called!
 1611256850.687888 prefetch_abort_handler#4094921@3:  (pc=0x6d2fdffe, l=0x914fc88
 0, tf=0x9d431fb0)
 1611256850.687889 pmap_fault_fixup#4108686@3: called: pm=0x915c8070, va=0x6d2fd0
 00, ftype=0x5, user=1
 1611256850.687889 pmap_fault_fixup#4108686@3:  ti=0x8089c670 pai=0x915c849c asid
 =0x60
 1611256850.687895 prefetch_abort_handler#4094921@3:  <- emulated
 1611256850.687896 prefetch_abort_handler#4094922@3: called!
 1611256850.687897 prefetch_abort_handler#4094922@3:  (pc=0x6d2fdffe, l=0x914fc88
 0, tf=0x9d431fb0)
 1611256850.687897 pmap_fault_fixup#4108687@3: called: pm=0x915c8070, va=0x6d2fd0
 00, ftype=0x5, user=1
 1611256850.687898 pmap_fault_fixup#4108687@3:  ti=0x8089c670 pai=0x915c849c asid
 =0x60
 1611256850.687903 prefetch_abort_handler#4094922@3:  <- emulated
 1611256850.687905 prefetch_abort_handler#4094923@3: called!
 1611256850.687905 prefetch_abort_handler#4094923@3:  (pc=0x6d2fdffe, l=0x914fc88
 0, tf=0x9d431fb0)
 1611256850.687905 pmap_fault_fixup#4108688@3: called: pm=0x915c8070, va=0x6d2fd0
 00, ftype=0x5, user=1
 1611256850.687906 pmap_fault_fixup#4108688@3:  ti=0x8089c670 pai=0x915c849c asid
 =0x60
 1611256850.687912 prefetch_abort_handler#4094923@3:  <- emulated
 1611256850.687913 prefetch_abort_handler#4094924@3: called!
 1611256850.687913 prefetch_abort_handler#4094924@3:  (pc=0x6d2fdffe, l=0x914fc88
 0, tf=0x9d431fb0)
 1611256850.687914 pmap_fault_fixup#4108689@3: called: pm=0x915c8070, va=0x6d2fd0
 00, ftype=0x5, user=1
 1611256850.687914 pmap_fault_fixup#4108689@3:  ti=0x8089c670 pai=0x915c849c asid
 =0x60
 1611256850.687920 prefetch_abort_handler#4094924@3:  <- emulated
 1611256850.687921 prefetch_abort_handler#4094925@3: called!
 1611256850.687922 prefetch_abort_handler#4094925@3:  (pc=0x6d2fdffe, l=0x914fc88
 0, tf=0x9d431fb0)
 1611256850.687923 pmap_fault_fixup#4108690@3: called: pm=0x915c8070, va=0x6d2fd0
 00, ftype=0x5, user=1
 1611256850.687923 pmap_fault_fixup#4108690@3:  ti=0x8089c670 pai=0x915c849c asid
 =0x60
 1611256850.687929 prefetch_abort_handler#4094925@3:  <- emulated
 1611256850.687930 prefetch_abort_handler#4094926@3: called!
 1611256850.687931 prefetch_abort_handler#4094926@3:  (pc=0x6d2fdffe, l=0x914fc88
 0, tf=0x9d431fb0)
 1611256850.687931 pmap_fault_fixup#4108691@3: called: pm=0x915c8070, va=0x6d2fd0
 00, ftype=0x5, user=1
 1611256850.687932 pmap_fault_fixup#4108691@3:  ti=0x8089c670 pai=0x915c849c asid
 =0x60
 1611256850.687936 prefetch_abort_handler#4094926@3:  <- emulated
 db{0}> c
 Mutex error: mi_userret,116: sleep lock held
 
 [  66.9145944] lock address : 0x00000000915c8070 type     :     sleep/adaptive
 [  66.9245970] initialized  : 0x00000000800a5e8c
 [  66.9245970] shared holds :                  0 exclusive:                  1
 [  66.9345900] shares wanted:                  0 exclusive:                  0
 [  66.9345900] relevant cpu :                  3 last held:                  3
 [  66.9445905] relevant lwp : 0x00000000914fc880 last held: 0x00000000914fc880
 [  66.9545904] last locked* : 0x00000000800a88b8 unlocked : 0x00000000800a8cac
 [  66.9645909] owner field  : 0x00000000914fc880 wait/spin:                0/0
 [  66.9645909] Turnstile: no active turnstile for this lock.
 
 [  66.9745909] panic: LOCKDEBUG: Mutex error: mi_userret,116: sleep lock held
 [  66.9745909] cpu3: Begin traceback...
 [  66.9845916] 0x9d431e04: netbsd:db_panic+0x14
 [  66.9845916] 0x9d431e1c: netbsd:vpanic+0xe4
 [  66.9945916] 0x9d431e34: netbsd:snprintf
 [  66.9945916] 0x9d431e74: netbsd:lockdebug_more
 [  67.0045918] 0x9d431eac: netbsd:lockdebug_barrier+0x14c
 [  67.0045918] 0x9d431ed4: netbsd:userret+0x110
 [  67.0145919] 0x9d431fac: netbsd:prefetch_abort_handler+0x444
 [  67.0145919] cpu3: End traceback...