-current instability

[Thomas Klausner <tk%giga.or.at@localhost>]

Hi!

I've been getting more reboots (every couple of days) than usual
(which is none) lately on 7.99.59/amd64 from Feb 5 and kernels from
the week(s?) before.

The last two were this night a couple minutes apart.

I got similar backtraces in the core dumps for both of them:

(gdb) target kvm netbsd.104.core
0xffffffff8021a175 in cpu_reboot (howto=howto@entry=260, bootstr=bootstr@entry=0x0) at /usr/src/sys/arch/amd64/amd64/machdep.c:674
674     haltsys:
(gdb) bt
#0  0xffffffff8021a175 in cpu_reboot (howto=howto@entry=260, bootstr=bootstr@entry=0x0) at /usr/src/sys/arch/amd64/amd64/machdep.c:674
#1  0xffffffff809a3f4c in vpanic (fmt=fmt@entry=0xffffffff810d50fd "trap", ap=ap@entry=0xfffffe813b3dea08) at /usr/src/sys/kern/subr_prf.c:342
#2  0xffffffff809a4000 in panic (fmt=fmt@entry=0xffffffff810d50fd "trap") at /usr/src/sys/kern/subr_prf.c:258
#3  0xffffffff8021bf5e in trap (frame=0xfffffe813b3deb40) at /usr/src/sys/arch/amd64/amd64/trap.c:298
#4  0xffffffff8020115e in alltraps ()
#5  0xffffffff804f5175 in dev_findproduct (buf=0xfffffe8605e50028 "", len=384, words=0xffffffff81d17060 <error: Cannot access memory at address 0xffffffff81d17060>, nwords=<optimized out>,
    products=0xffffffff81d1bd20, nproducts=11062, vendor=1130, product=35) at /usr/src/sys/dev/dev_verbose.c:93
#6  0xffffffff81d23033 in ?? ()
#7  0x000000000000046a in ?? ()
#8  0xfffffe8500000023 in ?? ()
#9  0xfffffe813b3decc0 in ?? ()
#10 0xffffffff8044231f in usbd_fill_deviceinfo (dev=0xfffffe85a697ca88, di=0xfffffe8605e50020, usedev=0) at /usr/src/sys/dev/usb/usb_subr.c:1506
Backtrace stopped: frame did not save the PC
(gdb)

I was using USB at the time for my backups, and starting a bulk build.
First time syncthing was running, second time i hadn't started it yet.
Some NFS.



Getting info for all threads is not possible:

(gdb) thread apply all bt

Thread 2.1 (<kvm>):
/usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/gdbarch.c:4884: internal-error: gdbarch_addressable_memory_unit_size: Assertion `gdbarch != NULL' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Quit this debugging session? (y or n) y

This is a bug, please report it.  For instructions, see:
<http://www.gnu.org/software/gdb/bugs/>.

/usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/gdbarch.c:4884: internal-error: gdbarch_addressable_memory_unit_size: Assertion `gdbarch != NULL' failed.
A problem internal to GDB has been detected,
further debugging may prove unreliable.
Create a core file of GDB? (y or n) y
Abort (core dumped)
# gdb gdb gdb.core
...
[New process 1]
Core was generated by `gdb'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007c213651ca8a in _lwp_kill () from /usr/lib/libc.so.12
(gdb) bt
#0  0x00007c213651ca8a in _lwp_kill () from /usr/lib/libc.so.12
#1  0x00007c213651c145 in abort () at /usr/src/lib/libc/stdlib/abort.c:74
#2  0x0000000196d468f3 in dump_core () at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/utils.c:561
#3  0x0000000196d48b1d in internal_vproblem (problem=problem@entry=0x197432b00 <internal_error_problem>, file=<optimized out>, line=<optimized out>, fmt=<optimized out>,
    ap=ap@entry=0x7f7fffe4c238) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/utils.c:772
#4  0x0000000196d48bdd in internal_verror (file=<optimized out>, line=<optimized out>, fmt=<optimized out>, ap=ap@entry=0x7f7fffe4c238)
    at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/utils.c:798
#5  0x0000000196d1e5ae in internal_error (file=file@entry=0x197010828 "/usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/gdbarch.c", line=line@entry=4884,
    fmt=<optimized out>) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/common/errors.c:55
#6  0x0000000196de49a0 in gdbarch_addressable_memory_unit_size (gdbarch=<optimized out>) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/gdbarch.c:4884
#7  0x0000000196dc9c7f in target_read (ops=0x197454f00 <bsd_kvm_ops>, object=object@entry=TARGET_OBJECT_MEMORY, annex=annex@entry=0x0, buf=buf@entry=0x7f7fffe4c3bf "",
    offset=offset@entry=18446744071564271989, len=len@entry=1) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/target.c:1651
#8  0x0000000196dc9cd0 in target_read_memory (memaddr=memaddr@entry=18446744071564271989, myaddr=myaddr@entry=0x7f7fffe4c3bf "", len=len@entry=1)
    at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/target.c:1447
#9  0x0000000196e43f21 in amd64_stack_frame_destroyed_p (gdbarch=<optimized out>, pc=18446744071564271989)
    at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/amd64-tdep.c:2796
#10 amd64_epilogue_frame_sniffer (self=<optimized out>, this_frame=0x7c213070b0e0, this_prologue_cache=<optimized out>)
    at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/amd64-tdep.c:2811
#11 0x0000000196d4045f in frame_unwind_try_unwinder (this_frame=this_frame@entry=0x7c213070b0e0, this_cache=this_cache@entry=0x7c213070b0f8, unwinder=0x197357a00 <amd64_epilogue_frame_unwind>)
    at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/frame-unwind.c:107
#12 0x0000000196d407b5 in frame_unwind_find_by_frame (this_frame=this_frame@entry=0x7c213070b0e0, this_cache=this_cache@entry=0x7c213070b0f8)
    at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/frame-unwind.c:163
#13 0x0000000196d41c18 in compute_frame_id (fi=0x7c213070b0e0) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/frame.c:490
#14 get_prev_frame_if_no_cycle (this_frame=this_frame@entry=0x7c213070b010) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/frame.c:1822
#15 0x0000000196d43d2e in get_prev_frame_always_1 (this_frame=0x7c213070b010) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/frame.c:1996
#16 get_prev_frame_always (this_frame=this_frame@entry=0x7c213070b010) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/frame.c:2012
#17 0x0000000196d44437 in get_prev_frame (this_frame=this_frame@entry=0x7c213070b010) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/frame.c:2254
#18 0x0000000196d446b4 in unwind_to_current_frame (ui_out=<optimized out>, args=args@entry=0x7c213070b010) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/frame.c:1498
#19 0x0000000196d07c06 in catch_exceptions_with_msg (func_uiout=0x7c2138727b80, func=func@entry=0x196d446a8 <unwind_to_current_frame>, func_args=func_args@entry=0x7c213070b010,
    gdberrmsg=gdberrmsg@entry=0x0, mask=mask@entry=RETURN_MASK_ERROR) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/exceptions.c:183
#20 0x0000000196d07d6b in catch_exceptions (uiout=<optimized out>, func=func@entry=0x196d446a8 <unwind_to_current_frame>, func_args=func_args@entry=0x7c213070b010,
    mask=mask@entry=RETURN_MASK_ERROR) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/exceptions.c:163
#21 0x0000000196d41d2d in get_current_frame () at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/frame.c:1530
#22 0x0000000196c92ac3 in backtrace_command_1 (from_tty=1, no_filters=0, show_locals=0, count_exp=0x0) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/stack.c:1761
#23 backtrace_command (arg=0x0, from_tty=1) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/stack.c:1953
#24 0x0000000196d191ba in cmd_func (cmd=0x7c21387fe120, args=0x0, from_tty=1) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/cli/cli-decode.c:1913
#25 0x0000000196d4ac72 in execute_command (p=<optimized out>, from_tty=from_tty@entry=1) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/top.c:674
#26 0x0000000196df74ea in thread_apply_all_command (cmd=0x7c21307070b1 "bt", from_tty=1) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/thread.c:1810
#27 0x0000000196d191ba in cmd_func (cmd=0x7c21383113a0, args=0x7c21307070b1 "bt", from_tty=1) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/cli/cli-decode.c:1913
#28 0x0000000196d4ac72 in execute_command (p=<optimized out>, p@entry=0x7c21307070a0 "thread apply all bt", from_tty=1)
    at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/top.c:674
#29 0x0000000196de9793 in command_handler (command=0x7c21307070a0 "thread apply all bt") at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/event-top.c:628
#30 0x0000000196de9a3b in command_line_handler (rl=<optimized out>) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/event-top.c:820
#31 0x0000000196de8e3f in gdb_rl_callback_handler (rl=0x7c2130705060 '\377' <repeats 19 times>) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/event-top.c:200
#32 0x0000000196f5dfdc in rl_callback_read_char () at /usr/src/external/gpl3/gdb/dist/readline/callback.c:220
#33 0x0000000196de8d6e in gdb_rl_callback_read_char_wrapper (client_data=<optimized out>) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/event-top.c:173
#34 0x0000000196de9279 in stdin_event_handler (error=<optimized out>, client_data=0x7c2138723090) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/event-top.c:555
#35 0x0000000196d06f10 in gdb_wait_for_event (block=block@entry=1) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/event-loop.c:859
#36 0x0000000196d07042 in gdb_do_one_event () at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/event-loop.c:347
#37 0x0000000196d071ab in gdb_do_one_event () at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/event-loop.c:304
#38 start_event_loop () at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/event-loop.c:371
#39 0x0000000196c8b605 in captured_command_loop (data=data@entry=0x0) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/main.c:324
#40 0x0000000196d07de5 in catch_errors (func=func@entry=0x196c8b5da <captured_command_loop>, func_args=func_args@entry=0x0, errstring=errstring@entry=0x196fb902e "",
    mask=mask@entry=RETURN_MASK_ALL) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/exceptions.c:236
#41 0x0000000196c8c7ab in captured_main (data=0x7f7fffe4cc60) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/main.c:1149
---Type <return> to continue, or q <return> to quit---
#42 gdb_main (args=0x7f7fffe4cc60) at /usr/src/external/gpl3/gdb/lib/libgdb/../../dist/gdb/main.c:1159
#43 0x0000000196f974ca in main (argc=<optimized out>, argv=<optimized out>) at /usr/src/external/gpl3/gdb/bin/gdb/../../dist/gdb/gdb.c:32
(gdb)

Ideas?
 Thomas