At Thu, 10 Jan 2019 13:28:53 +0000, Patrick Welche <prlw1%cam.ac.uk@localhost> wrote:
Subject: Re: debugging symbols and gdb
>
>
> # gdb `which Xorg` X.core
> GNU gdb (GDB) 8.0.1
> ...
> Reading symbols from /usr/X11R7/bin/Xorg...Reading symbols from /usr/libdata/debug//usr/X11R7/bin/Xorg.debug...done.
> ...
> Program terminated with signal SIGBUS, Bus error.
> #0 0x00007f7fed6ad3e5 in ?? ()
> [Current thread is 1 (process 1)]
> (gdb) bt
> #0 0x00007f7fed6ad3e5 in ?? ()
> #1 0x00007f7ff3599900 in ?? ()
> #2 0x0000010100020037 in ?? ()
> #3 0x00007f7ff7bd13c0 in ?? ()
> #4 0x0000000000000100 in ?? ()
> #5 0x0300020001000000 in ?? ()
> #6 0x00007f7ff400759a in ?? ()
> #7 0x0b000a0009000800 in ?? ()
> #8 0x00007f7fffffdeb8 in ?? ()
> #9 0x00007f7fffffdeb0 in ?? ()
> #10 0x00007f7fffffdea6 in ?? ()
> #11 0x1b001a0019001800 in ?? ()
> #12 0x1f001e001d001c00 in ?? ()
> #13 0x2300220021002000 in ?? ()
> #14 0x2700260025002400 in ?? ()
> #15 0x2b002a0029002800 in ?? ()
> #16 0x00007f7ff355b140 in ?? ()
> #17 0x00007f7ff35090e0 in ?? ()
> #18 0x0000000d00000011 in ?? ()
> #19 0x20003a0039003800 in ?? ()
> #20 0x0000000000000000 in ?? ()
I'm guessing this is a newish GDB bug.
I've been seeing something very similar (on a system running a recent
build of sources from January 14), seemingly at random from one
invocation to another of gdb, when running a static-linked binary from
gdb, i.e. where I would set a backtrace (using the symbol name of a libc
function I wanted to stop on, so the symbol was available before the
run) and then run the program. If I remember correctly the only
difference is that I would see the symbol name for frame #0.
The first thing I noticed was that the problem would go away and I would
see all the symbol names if I reset debug-file-directory to point at
$DESTDIR/usr/libdata/debug (I don't currently have room on the system to
actually install the debug sets),
Then I noticed the second "Reading symbols from blah.debug...done", i.e.
reading the program's .debug file from the same directory, even without
resetting debug-file-directory. Further examination of that .debug file
showed it contained all of the missing symbols, so I can't see any
reason for gdb to look elsewhere, and indeed I didn't see it report any
attempts to look elsewhere. Somehow it was as if changing
debug-file-directory made it use the information it had already loaded.
Sometimes when gdb failed to report symbols I could reset
debug-file-directory and then get symbols, even though they were
presumably already all loaded from the .debug file. (I always set it to
a valid /usr/libdata/debug hierarchy, so I've no idea if any change
would suffice.)
Also somewhat mysteriously I didn't always see the same number of frames
in the symbol-less output -- and jumping to some middle frame and
backtracing from there would then show even more parent frames, but
never all of them.
I'm in the process of trying to fix a GCC build botch and then rebuild
this system again from most recent sources again, so unfortunately I
can't do a huge amount of further debugging of this issue in quite the
same environment. However it seems I can sort of demonstrate with the
"cat" and "cat.debug" files from my old OBJDIR, though this shows
something a little different than my first experiences.
Here I first move the cat.debug file aside, then try to load it from
within the session after a breakpoint. Before loading symbols the
backtrace looks as expected, but after loading them it is completely
broken. Then I put the cat.debug file back in place, and it is loaded
automatically and everything looks correct. I am unable to reproduce
the case where the initial backtrace after auto-loading the .debug file
is broken, at least not with this binary. (Note my attempt to run "cat"
from in the debugger is flawed, but none the less it does result in
hitting the breakpoint as desired for this demo.) (Note also that in
the last run the extra couple of frames are expected as they are from
inlined functions that are otherwise invisible without debugging
symbols, but in the second backtrace of the first run, after loading
symbols, the frame count is also completely wrong.)
$ file cat
cat: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, for NetBSD 8.99.27, not stripped
$ ls -l
total 2528
-rwxr-xr-x 1 woods wheel 255752 Dec 11 20:18 cat
-rw-r--r-- 1 woods wheel 4816 Dec 11 20:18 cat.d
-rwxr-xr-x 1 woods wheel 932992 Dec 11 20:18 cat.debug
-rw-r--r-- 1 woods wheel 42888 Dec 11 20:18 cat.o
$ mv cat.debug save.debug
$ gdb cat
GNU gdb (GDB) 8.0.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64--netbsd".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from cat...(no debugging symbols found)...done.
(gdb) break write
Breakpoint 1 at 0x400dc0
(gdb) run < /dev/null
Starting program: /build/woods/future/current-amd64-amd64-obj/building/work/woods/m-NetBSD-current/bin/cat/cat < /dev/null
[New LWP 1 of process 1339]
Thread 2 hit Breakpoint 1, 0x0000000000400dc0 in write ()
(gdb) bt
#0 0x0000000000400dc0 in write ()
#1 0x000000000041a649 in __sflush ()
#2 0x000000000041a6c6 in fflush ()
#3 0x0000000000409a12 in __vfprintf_unlocked_l ()
#4 0x000000000040d92c in vfprintf ()
#5 0x0000000000407d18 in fprintf ()
#6 0x0000000000406fc3 in vwarnc ()
#7 0x0000000000406f72 in warn ()
#8 0x0000000000400d9e in raw_args ()
#9 0x00000000004217e0 in main ()
(gdb) symbol-file save.debug
Reading symbols from save.debug...done.
(gdb) bt
#0 0x0000000000400dc0 in write ()
#1 0x000000000041a649 in __sflush (fp=0x7f7fffc2db70)
at /building/work/woods/m-NetBSD-current/lib/libc/stdio/fflush.c:107
#2 0x00007f7fffc2db70 in ?? ()
#3 0x0000000000421e28 in __func__.4643 ()
#4 0x0000000000000000 in ?? ()
(gdb) quit
A debugging session is active.
Inferior 1 [process 1339] will be killed.
Quit anyway? (y or n) y
$ mv save.debug cat.debug
$ gdb cat
GNU gdb (GDB) 8.0.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64--netbsd".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from cat...Reading symbols from /build/woods/future/current-amd64-amd64-obj/building/work/woods/m-NetBSD-current/bin/cat/cat.debug...done.
done.
(gdb) break write
Breakpoint 1 at 0x400dc0
(gdb) run < /dev/null
Starting program: /build/woods/future/current-amd64-amd64-obj/building/work/woods/m-NetBSD-current/bin/cat/cat < /dev/null
[New LWP 1 of process 18755]
Thread 2 hit Breakpoint 1, 0x0000000000400dc0 in write ()
(gdb) bt
#0 0x0000000000400dc0 in write ()
#1 0x000000000041a649 in __sflush (fp=fp@entry=0x7f7fff4f4730)
at /building/work/woods/m-NetBSD-current/lib/libc/stdio/fflush.c:107
#2 0x000000000041a6c6 in fflush (fp=fp@entry=0x7f7fff4f4730)
at /building/work/woods/m-NetBSD-current/lib/libc/stdio/fflush.c:73
#3 0x0000000000409a12 in __sbprintf (ap=<optimized out>, fmt=<optimized out>,
loc=<optimized out>, fp=0x6302d0 <__sF+304>)
at /building/work/woods/m-NetBSD-current/lib/libc/stdio/vfwprintf.c:187
#4 __vfprintf_unlocked_l (fp=fp@entry=0x6302d0 <__sF+304>,
loc=<optimized out>, fmt0=fmt0@entry=0x421e28 "%s: ",
ap=ap@entry=0x7f7fff4f4df8)
at /building/work/woods/m-NetBSD-current/lib/libc/stdio/vfwprintf.c:844
#5 0x000000000040d92c in vfprintf (fp=fp@entry=0x6302d0 <__sF+304>,
fmt0=fmt0@entry=0x421e28 "%s: ", ap=ap@entry=0x7f7fff4f4df8)
at /building/work/woods/m-NetBSD-current/lib/libc/stdio/vfwprintf.c:555
#6 0x0000000000407d18 in fprintf (fp=0x6302d0 <__sF+304>,
fmt=fmt@entry=0x421e28 "%s: ")
at /building/work/woods/m-NetBSD-current/lib/libc/stdio/fprintf.c:59
#7 0x0000000000406fc3 in _vwarnc (code=2, fmt=0x421a6a "%s",
ap=ap@entry=0x7f7fff4f4f08)
at /building/work/woods/m-NetBSD-current/lib/libc/gen/vwarnc.c:60
#8 0x0000000000406f9e in _vwarn (fmt=<optimized out>,
ap=ap@entry=0x7f7fff4f4f08)
at /building/work/woods/m-NetBSD-current/lib/libc/gen/vwarn.c:53
#9 0x0000000000406f72 in _warn (fmt=fmt@entry=0x421a6a "%s")
at /building/work/woods/m-NetBSD-current/lib/libc/gen/warn.c:60
#10 0x0000000000400d9e in raw_args (argv=argv@entry=0x7f7fff4f5118)
at /building/work/woods/m-NetBSD-current/bin/cat/cat.c:267
#11 0x00000000004217e0 in main (argc=3, argv=0x7f7fff4f5118)
at /building/work/woods/m-NetBSD-current/bin/cat/cat.c:136
(gdb) quit
A debugging session is active.
Inferior 1 [process 18755] will be killed.
Quit anyway? (y or n) y
--
Greg A. Woods
Planix, Inc.
<woods%planix.com@localhost> +1 250 762-7675 http://www.planix.com/
Attachment:
pgpTJtbNNw1dH.pgp
Description: OpenPGP Digital Signature