Re: kern/56931: Panic in vfs_insmntque(): panic: TAILQ_* forw 0xffffec0928771d00 /usr/src/sys/kern/vfs_mount.c:487

[Leonardo Taccari <leot%NetBSD.org@localhost>]

The following reply was made to PR kern/56931; it has been noted by GNATS.

From: Leonardo Taccari <leot%NetBSD.org@localhost>
To: gnats-bugs%netbsd.org@localhost
Cc: 
Subject: Re: kern/56931: Panic in vfs_insmntque(): panic: TAILQ_* forw 0xffffec0928771d00 /usr/src/sys/kern/vfs_mount.c:487
Date: Mon, 18 Jul 2022 10:15:05 +0200

 Jason Thorpe writes:
 > 
 > > On Jul 17, 2022, at 2:55 PM, Leonardo Taccari <leot%NetBSD.org@localhost> wrote:
 > > 
 > > I was able to reproduce that again when building www/webkit-gtk and at
 > > least in that case I think it was while doing a `make clean' (at least
 > > that was the last line I was able to read from an SSH session after it
 > > get stuck and dropped to ddb(4)), the bt seems the same:
 > > 
 > >  panic: TAILQ_* forw 0xffff98a871b71d00 /usr/src/sys/kern/vfs_mount.c:487
 >
 > Itâ??s blowing up here:
 >
 >                 TAILQ_REMOVE(&vp->v_mount->mnt_vnodelist, vip, vi_mntvnodes);
 >
 > â?¦specifically in this sanity check:
 >
 > #define QUEUEDEBUG_TAILQ_OP(elm, field)                                 \
 >         if ((elm)->field.tqe_next &&                                    \
 >             (elm)->field.tqe_next->field.tqe_prev !=                    \
 >             &(elm)->field.tqe_next)                                     \
 >                 QUEUEDEBUG_ABORT("TAILQ_* forw %p %s:%d", (elm),        \
 >                     __FILE__, __LINE__);                                \
 >         if (*(elm)->field.tqe_prev != (elm))                            \
 >                 QUEUEDEBUG_ABORT("TAILQ_* back %p %s:%d", (elm),        \
 >                     __FILE__, __LINE__);
 >
 > From your core dump, can you print out the values of those pointers?
 > [...]
 
 Unfortunately... I don't know/figured out how.
 
 On crash(8) we can see:
 
  crash> bt
  sljit_modcmd() at 0
  kern_reboot() at sys_reboot
  db_sync_cmd() at db_sync_cmd+0x26
  db_command() at db_command+0xcc
  db_command_loop() at db_command_loop+0xa3
  db_trap() at db_trap+0xef
  kdb_trap() at kdb_trap+0x107
  trap() at trap+0x45c
  --- trap (number 1) ---
  breakpoint() at breakpoint+0x5
  vpanic() at vpanic+0x18c
  panic() at panic+0x3c
  vfs_insmntque() at vfs_insmntque+0x2be
  vcache_reclaim() at vcache_reclaim+0x280
  vrelel() at vrelel+0x2e2
  tmpfs_remove() at tmpfs_remove+0x1bb
  VOP_REMOVE() at VOP_REMOVE+0x5b
  do_sys_unlinkat() at do_sys_unlinkat+0x2a1
  syscall() at syscall+0x19e
  --- syscall (number 10) ---
  syscall+0x19e:
 
 While on gdb I could not see the interesting traces with
 `vfs_insmntque()' call (I was dropped to ddb(4) - I had ddb.onpanic=1 -
 and then typed `save'):
 
  $ gdb --symbols=/netbsd.gdb --eval-command="file /netbsd.gdb" --eval-command="target kvm netbsd.4.core"
  Reading symbols from /netbsd.gdb...
  Load new symbol table from "/netbsd.gdb"? (y or n) y
  Reading symbols from /netbsd.gdb...
  0xffffffff802265f1 in cpu_reboot (howto=howto@entry=256, bootstr=bootstr@entry=0x0) at /usr/src/sys/arch/amd64/amd64/machdep.c:720
  720                     dumpsys();
  (gdb) bt
  #0  0xffffffff802265f1 in cpu_reboot (howto=howto@entry=256, bootstr=bootstr@entry=0x0) at /usr/src/sys/arch/amd64/amd64/machdep.c:720
  #1  0xffffffff80e05530 in kern_reboot (howto=howto@entry=256, bootstr=bootstr@entry=0x0) at /usr/src/sys/kern/kern_reboot.c:73
  #2  0xffffffff80b82416 in db_sync_cmd (addr=<optimized out>, have_addr=<optimized out>, count=<optimized out>, modif=<optimized out>) at /usr/src/sys/ddb/db_command.c:1565
  #3  0xffffffff80b82c3c in db_command (last_cmdp=last_cmdp@entry=0xffffffff81a72660 <db_last_command>) at /usr/src/sys/ddb/db_command.c:960
  #4  0xffffffff80b831d3 in db_command_loop () at /usr/src/sys/ddb/db_command.c:619
  #5  0xffffffff80b8782f in db_trap (type=type@entry=1, code=code@entry=0) at /usr/src/sys/ddb/db_trap.c:91
  #6  0xffffffff802232c7 in kdb_trap (type=type@entry=1, code=code@entry=0, regs=regs@entry=0xffffaa0280374b50) at /usr/src/sys/arch/amd64/amd64/db_interface.c:250
  #7  0xffffffff80228c2c in trap (frame=0xffffaa0280374b50) at /usr/src/sys/arch/amd64/amd64/trap.c:315
  #8  0xffffffff80221103 in alltraps ()
  #9  0x0000000000000000 in ?? ()
 
 ...and if we look at threads there:
 
  (gdb) info thread
    Id   Target Id         Frame
  * 2.1  <kvm>             0xffffffff802265f1 in cpu_reboot (howto=howto@entry=256, bootstr=bootstr@entry=0x0) at /usr/src/sys/arch/amd64/amd64/machdep.c:720
 
 Any idea how to possibly adjust the frames that gdb is seeing or
 possible other way to get them?
 
 
 Thank you!