I am having trouble showing proper debug symbols in the backtrace in GDB in an ARM cross-compiled system, built using Yocto.
abc.c is a simple printf("Hello world\n"); program in C (nothing tricky). On the build machine:
> yocto-dir/build/tmp-angstrom-glibc/sysroots/x86_64-linux/usr/bin/arm-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-gcc abc --sysroot=yocto-dir/build/tmp-angstrom-glibc/sysroots/imx28scm -g -O0 -o abc > scp abc root@DEVICE-IP:~
On the ARM target:
> gdbserver :2345 abc
Start GDB on the build machine (from installed Yocto SDK):
> /usr/local/oecore-x86_64/sysroots/x86_64-angstromsdk-linux/usr/bin/arm-angstrom-linux-gnueabi/arm-angstrom-linux-gnueabi-gdb abc GNU gdb (Linaro GDB) 7.8-2014.09 Copyright (C) 2014 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "--host=x86_64-angstromsdk-linux --target=arm-angstrom-linux-gnueabi". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://bugs.linaro.org>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from abc...done. (gdb) target remote DEVICE-IP:2345 Remote debugging using DEVICE-IP:2345 warning: Unable to find dynamic linker breakpoint function. GDB will be unable to debug shared library initializers and track explicitly loaded dynamic code. Cannot access memory at address 0x0 0x4ae90a20 in ?? () (gdb) bt #0 0x4ae90a20 in ?? () #1 0x00000000 in ?? () Backtrace stopped: previous frame identical to this frame (corrupt stack?) (gdb) set sysroot yocto-dir/build/tmp-angstrom-glibc/sysroots/imx28scm Reading symbols from yocto-dir/build/tmp-angstrom-glibc/sysroots/imx28scm/lib/ld-linux.so.3...done. Loaded symbols for yocto-dir/build/tmp-angstrom-glibc/sysroots/imx28scm/lib/ld-linux.so.3 Cannot access memory at address 0x0
After setting the sysroot, it still does not give symbols.
(gdb) bt #0 0x4ae90a20 in ?? () #1 0x00000000 in ?? () Backtrace stopped: previous frame identical to this frame (corrupt stack?) (gdb) b main Breakpoint 1 at 0x84a8: file abc.c, line 5. (gdb) c Continuing. Breakpoint 1, main () at abc.c:5 5 printf("Hello world\n");
Okay, when it hits a breakpoint, it does display symbols.
(gdb) bt Cannot access memory at address 0x0 #0 main () at abc.c:5
However, it goes weird stepping beyond there.
(gdb) n Cannot access memory at address 0x1 0x4aea6ea0 in ?? () (gdb) bt #0 0x4aea6ea0 in ?? () #1 0x0000a014 in do_lookup_unique (Cannot access memory at address 0x1 undef_map=0x1, ref=0x0, strtab=0x56ebb27 <error: Cannot access memory at address 0x56ebb27>, sym=0x84a0 <main>, type_class=-1224757248, result=0x1, map=<optimized out>, new_hash=<optimized out>, undef_name=<optimized out>) at /usr/src/debug/glibc/2.24-r0/git/elf/dl-lookup.c:332 #2 do_lookup_x (undef_name=<optimized out>, new_hash=<optimized out>, old_hash=<optimized out>, ref=0x0, result=<optimized out>, scope=0x177ff8e, i=<optimized out>, version=<optimized out>, flags=-1224757248, skip=0x1, type_class=100, undef_map=0x1) at /usr/src/debug/glibc/2.24-r0/git/elf/dl-lookup.c:544 #3 0x4aec0b10 in ?? () Cannot access memory at address 0x1 Backtrace stopped: previous frame identical to this frame (corrupt stack?)
It can't find the proper version of libc.so.6.
(gdb) info sharedlibrary warning: .dynamic section for "yocto-dir/build/tmp-angstrom-glibc/sysroots/imx28scm/lib/libc.so.6" is not at the expected address (wrong library or version mismatch?) From To Syms Read Shared Object Library 0x000007d0 0x0001bee0 Yes yocto-dir/build/tmp-angstrom-glibc/sysroots/imx28scm/lib/ld-linux.so.3 0x4aee73c0 0x4afe2018 No yocto-dir/build/tmp-angstrom-glibc/sysroots/imx28scm/lib/libc.so.6 (gdb) n Cannot find bounds of current function
It does not give an ideal debugging experience.
There is a gcc inside yocto-dir sysroot (as used above), as well as in /usr/local/oecore-x86_64. They both behave the same. The /usr/local/oecore-x86_64 SDK is freshly built and installed.
Similarly, there is an imx28scm sysroot inside yocto-dir (as used above), as well as in /usr/local/oecore-x86_64, and they both behave the same. However, they clearly do have different versions of libc.so.6 - yocto-dir's is 14.8MB, and /usr/local/oecore-x86_64's is 1.3MB. This is a concern, however setting either of these locations as the sysroot does not fix the problem.
One workaround is to link with
-static. GDB does give symbols in this case:
(gdb) target remote DEVICE-IP:2345 Remote debugging using DEVICE-IP:2345 _start () at ../sysdeps/arm/start.S:79 79 ../sysdeps/arm/start.S: No such file or directory. (gdb) set sysroot yocto-dir/build/tmp-angstrom-glibc/sysroots/imx28scm (gdb) bt #0 _start () at ../sysdeps/arm/start.S:79 (gdb) b main Breakpoint 1 at 0x8480: file abc.c, line 5. (gdb) c Continuing. Breakpoint 1, main () at abc.c:5 5 printf("Hello world\n"); (gdb) n 6 return 0; (gdb) n 7 }
-Wl,--verbose seems to show it is linking with the library in the expected sysroot:
yocto-dir/build/tmp-angstrom-glibc/sysroots/x86_64-linux/usr/libexec/arm-angstrom-linux-gnueabi/gcc/arm-angstrom-linux-gnueabi/6.2.1/ld: Attempt to open yocto-dir/build/tmp-angstrom-glibc/sysroots/imx28scm/lib/libc.so.6 succeeded
The linker also finds this one, but it isn't referred to as libc.so.6, so presumably this is not interfering.
yocto-dir/build/tmp-angstrom-glibc/sysroots/x86_64-linux/usr/libexec/arm-angstrom-linux-gnueabi/gcc/arm-angstrom-linux-gnueabi/6.2.1/ld: Attempt to open yocto-dir/build/tmp-angstrom-glibc/sysroots/imx28scm/usr/lib/libc.so succeeded
Why is there a library version mismatch in this case? How can I get GDB to display symbols from the library which it expects? I do not wish to link statically.
Apparently GDB for ARM target has trouble with trying to load symbols before main() (Debugging shared libraries with gdbserver):
The problem I had was that gdbserver stops at the dynamic loader, before main, and the dynamic libraries are not yet loaded at that point, and so GDB does not know where the symbols will go in memory yet.
GDB appears to have some mechanisms to automatically load shared library symbols, and if I compile for host, and run gdbserver locally, running to main is not needed. But on the ARM target, that is the most reliable thing to do.
Therefore, set it to load shared symbols after main has been hit:
> b main > c <breakpoint hit> > set sysroot <sysroot>
Or reload the symbols after you hit main.
> set sysroot <sysroot> ... > b main > c <breakpoint hit> > nosharedlibrary > sharedlibrary
Or it might be useful in interfacing with IDE debuggers to set auto loading of symbols to be off on GDB startup:
> set auto-solib-add off
User contributions licensed under CC BY-SA 3.0