Difference between revisions of "EBC Exercise 14 gdb Debugging"

From eLinux.org
Jump to: navigation, search
(Initial page copied from Lab 09)
 
(Removed the host-side stuff)
Line 29: Line 29:
 
=== The Sample Program ===
 
=== The Sample Program ===
  
You can get the sample program from dfs.
 
 
<pre>
 
$ sftp dfs.rose-hulman.edu
 
Connecting to dfs.rose-hulman.edu...
 
yoder@dfs.rose-hulman.edu's password:
 
sftp> cd users/y/yoder/shared/beagleboard                                                                           
 
sftp> get gdbExample.tar.gz
 
sftp> exit
 
$ tar xvf gdbExample.tar.gz
 
gdbExample/
 
gdbExample/cfft.c
 
gdbExample/main_bench.c
 
gdbExample/changelog
 
gdbExample/distance.c
 
gdbExample/cfft.h
 
gdbExample/README
 
gdbExample/distance.h
 
gdbExample/common.h
 
gdbExample/Makefile
 
gdbExample/main_cfft.c
 
$ cd gdbExample
 
$ gedit Makefile
 
</pre>
 
Edit the '''Makefile''' and correct '''ARM_TOOLCHAIN_PATH''' for your machine.  Also search for '''install:''' and fix it for your beagle.
 
<pre>
 
$ make all install
 
</pre>
 
This will compile the code and '''scp''' to your Beagle. You've just compiled a program that performs a complex fft on random data.  It's main purpose is to see how fast it runs on the Beagle. (In case you are interested, I added to rule so you can compile it for your host computer.  Try '''make x86''' and compare the times on your host to those on the Beagle.)
 
 
The program takes several seconds to run on the Beagle, so you may want to edit the code so it doesn't run so many iterations.
 
 
=== Running gdb on the Beagle ===
 
 
On your Beagle try:
 
 
<pre>
 
$ gdb cfft_arm
 
gdb cfft_arm
 
GNU gdb (GDB) 7.1
 
Copyright (C) 2010 Free Software Foundation, Inc.
 
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
 
This is free software: you are free to change and redistribute it.
 
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
 
and "show warranty" for details.
 
This GDB was configured as "arm-angstrom-linux-gnueabi".
 
For bug reporting instructions, please see:
 
<http://www.gnu.org/software/gdb/bugs/>...
 
Reading symbols from /home/root/gdbExample/cfft_arm...done.
 
(gdb) b main
 
Breakpoint 1 at 0x8554
 
(gdb) r
 
Starting program: /home/root/gdbExample/cfft_arm
 
 
Breakpoint 1, 0x00008554 in main ()
 
</pre>
 
When first starting gdb notice the line '''This GDB was configured as "arm-angstrom-linux-gnueabi'''.  This is a good sign.  The first command '''b main''' sets a breakpoint at '''main'''.  The next command '''r'''uns to that break point. Now lets look at our code a this point.  Try the list command.
 
<pre>
 
(gdb) l
 
1 /home/yoder/BeagleBoard/oe/build/tmp-angstrom_2008_1/work/armv7a-angstrom-linux-gnueabi/glibc-2.9-r36.3/build-arm-angstrom-linux-gnueabi/csu/crtn.S: No such file or directory.
 
in /home/yoder/BeagleBoard/oe/build/tmp-angstrom_2008_1/work/armv7a-angstrom-linux-gnueabi/glibc-2.9-r36.3/build-arm-angstrom-linux-gnueabi/csu/crtn.S
 
(gdb) quit
 
</pre>
 
Hmmm....  We have a problem.  For gdb to be most useful we need to tell gcc to include debugging information.  Back to your host computer and do an ls on cfft_arm and note it's size.  Mine is about 18k.  Next edit the '''Makefile''' and find '''ARM_CFLAGS''' and add '''-g'''.  The line should look like:
 
<pre>
 
ARM_CFLAGS = $(CFLAGS) -g
 
</pre>
 
Now clean everything and remake.
 
<pre>
 
$ make clean all install
 
$ ls -l cfft_arm
 
</pre>
 
How has the size of cfft_arm changed?
 
 
Back to the Beagle. 
 
<pre>
 
$ gdp cfft_arm
 
(gdb) b main
 
(gdb) r
 
(gdb) l
 
42 }
 
43
 
44 static complex *new_complex_vector(int size);
 
45
 
46 int main ()
 
47 {
 
48   int i;
 
49   int N, n;
 
50   int nTimes;
 
51   float secs;
 
(gdb) n
 
56     complex *in = new_complex_vector(N);
 
(gdb) n
 
57     complex *out = new_complex_vector(N);
 
(gdb) n
 
59     fft_init (N);
 
(gdb) p N
 
$1 = 16
 
</pre>
 
Now '''l''' shows the code around the breakpoint.  If you aren't seeing code, be sure to '''scp''' your '''.c''' and '''.h''' files to the Beagle.
 
 
The '''n''' steps to the next line in the program, the '''p''' command prints the variable that is listed.  Use the '''s''' command to step into a function.
 
<pre>
 
 
(gdb) s
 
fft_init (N=16) at cfft.c:33
 
warning: Source file is more recent than executable.
 
33   tableW = malloc ((N / 2) * sizeof (complex));
 
(gdb) l
 
28
 
29 void fft_init (int N)
 
30 {
 
31   int i, j;
 
32
 
33   tableW = malloc ((N / 2) * sizeof (complex));
 
34   bndx = malloc (N * sizeof (int));
 
35   ndx = malloc ((N / 2) * sizeof (int));
 
36
 
37   ndx[0] = 0;
 
</pre>
 
That's enough to get you around a bit.  '''help''' will get you information about more commands.  '''c''' will continue from where you stopped.
 
 
Try this:
 
<pre>
 
gdb cfft_arm
 
gdb) b fft_exec
 
(gdb) r
 
(gdb) n 4
 
(gdb) l
 
(gdb) p n
 
</pre>
 
We're in the '''fft_exec''' routine and have '''n'''ext'ed a in a way and then printed the value of '''n'''.  The '''main''' routine also has an '''n''', how do we see it's value?  It's on another stack frame, so do a backtrace to see what's on the stack and then select the frame we want.
 
<pre>
 
(gdb) bt
 
#0  fft_exec (N=16, in=0x12090) at cfft.c:80
 
#1  0x0000865c in main () at main_cfft.c:62
 
(gdb) f 1
 
#1  0x0000865c in main () at main_cfft.c:62
 
62     fft_exec (N, out);
 
(gdb) p n
 
$7 = <value optimized out>
 
(gdb) f 0
 
#0  fft_exec (N=16, in=0x12090) at cfft.c:80
 
80     for (k = 0; k < i; k++)
 
(gdb) p n
 
$8 = 16
 
</pre>
 
The '''f''' command selects the frame we want to inspect.  Notice in this case the value of '''n''' in '''main''' has been ''optimized out'', that means the compiler found a way to compile the code that didn't need to put '''n''' on the stack.  In the '''Makefile''' you will find the '''-O3''' option will tells the compiler to optimize a lot.  Try removed the '''-O3''' and remaking everything and seeing if '''n''' appears in '''main'''.
 
 
=== gdb and core files ===
 
 
Sometimes you have a program that stops running expectedly. gdb can help find where it quit and why. Let's insert a bug in the cfft program and see how gdb finds it.  Edit cfft and replace this line near the bottom
 
<pre>
 
        a = j + r; // An error
 
</pre>
 
with
 
<pre>
 
        a = j + r/0; // An error
 
 
</pre>
 
so we will have a divide by zero error.  Remake and install
 
<pre>
 
$ make all install
 
</pre>
 
On the Beagle you should see:
 
<pre>
 
$ ./cfft_arm
 
Floating point exception
 
</pre>
 
Let the Beagle write a core file and see how gdb can use it
 
<pre>
 
$ ulimit -c unlimited
 
$ ./cfft_arm
 
Floating point exception (core dumped)
 
$ gdb ./cfft_arm core
 
GNU gdb (GDB) 7.1
 
Copyright (C) 2010 Free Software Foundation, Inc.
 
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
 
This is free software: you are free to change and redistribute it.
 
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
 
and "show warranty" for details.
 
This GDB was configured as "arm-angstrom-linux-gnueabi".
 
For bug reporting instructions, please see:
 
<http://www.gnu.org/software/gdb/bugs/>...
 
Reading symbols from /home/root/gdbExample/cfft_arm...done.
 
[New Thread 3525]
 
Reading symbols from /lib/libm.so.6...(no debugging symbols found)...done.
 
Loaded symbols for /lib/libm.so.6
 
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
 
Loaded symbols for /lib/libc.so.6
 
Reading symbols from /lib/ld-linux.so.3...(no debugging symbols found)...done.
 
Loaded symbols for /lib/ld-linux.so.3
 
Core was generated by `./cfft_arm'.
 
Program terminated with signal 8, Arithmetic exception.
 
#0  0x400c35fc in raise () from /lib/libc.so.6
 
(gdb) bt
 
#0  0x400c35fc in raise () from /lib/libc.so.6
 
#1  0x00009abc in __div0 ()
 
    at /home/yoder/BeagleBoard/oe/build/tmp-angstrom_2008_1/work/armv7a-angstrom-linux-gnueabi/gcc-cross-4.3.3-r16.1/gcc-4.3.3/libgcc/../gcc/config/arm/lib1funcs.asm:1079
 
#2  0x00008c80 in __udivsi3 ()
 
    at /home/yoder/BeagleBoard/oe/build/tmp-angstrom_2008_1/work/armv7a-angstrom-linux-gnueabi/gcc-cross-4.3.3-r16.1/gcc-4.3.3/libgcc/../gcc/config/arm/lib1funcs.asm:834
 
#3  0x00008888 in fft_exec (N=16, in=0x12090) at cfft.c:87
 
#4  0x0000868c in main () at main_cfft.c:62
 
</pre>
 
gdb tells you right where the error occurred.  You can even ask it to list the file.
 
<pre>
 
(gdb) list fft_exec
 
67   free (bndx);
 
68   free (tableW);
 
69 }
 
70
 
71 void fft_exec (int N, complex * in)
 
72 {
 
73   unsigned int n = N;
 
74   unsigned int a, b, i, j, k, r, s;
 
75   complex w, p;
 
76
 
(gdb) l 87
 
82       w = tableW[k];
 
83
 
84       r = 2 * n * k;
 
85       s = n * (1 + 2 * k);
 
86
 
87       for (j = 0; j < n; j++)
 
88       {
 
89         a = j + r/0; // An error
 
90         b = j + s;
 
91         cmult (p, w, in[b]);      //6 flop
 
</pre>
 
 
=== Remote gdb ===
 
 
Sometimes you can't run gdb on an embedded target since it doesn't have enough resources for all of gdb.  You can run gdb on your host and debug on the Beagle.  You must install a version of gdb on your host that is compiled for the target.
 
<pre>
 
$ source ~/.oe/environment-2008
 
$ bitbake gdb-cross
 
</pre>
 
The bitbake took about 4.5 minutes on a slow dsl line.
 
 
On your Beagle run:
 
<pre>
 
$ gdbserver localhost:2001 ./cfft_arm
 
</pre>
 
On your host run:
 
<pre>
 
$ source ~/BeagleBoard/oe/crossCompileEnv.sh
 
$ ${CROSS_COMPILE}gdb ./cfft_arm
 
GNU gdb (GDB) 7.1
 
Copyright (C) 2010 Free Software Foundation, Inc.
 
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
 
This is free software: you are free to change and redistribute it.
 
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
 
and "show warranty" for details.
 
This GDB was configured as "--host=i686-linux --target=arm-angstrom-linux-gnueabi".
 
For bug reporting instructions, please see:
 
<http://www.gnu.org/software/gdb/bugs/>...
 
Reading symbols from /home/yoder/BeagleBoard/gdbExample/cfft_arm...done.
 
</pre>
 
Notice the line that says <code>This GDB was configured as "--host=i686-linux --target=arm-angstrom-linux-gnueabi"</code>, this tells us we have the right version of gdb.
 
 
Now remotely connect to your Beagle:
 
<pre>
 
(gdb) target remote beagle2:2001
 
Remote debugging using beagle2:2001
 
warning: Unable to find dynamic linker breakpoint function.
 
GDB will be unable to debug shared library initializers
 
and track explicitly loaded dynamic code.
 
0x400007f0 in ?? ()
 
(gdb) b main
 
Cannot access memory at address 0x0
 
Breakpoint 1 at 0x8590: file main_cfft.c, line 47.
 
(gdb) c
 
Continuing.
 
warning: `/lib/libm.so.6': Shared library architecture unknown is not compatible with target architecture arm.
 
warning: .dynamic section for "/lib/libm.so.6" is not at the expected address (wrong library or version mismatch?)
 
warning: `/lib/libc.so.6': Shared library architecture unknown is not compatible with target architecture arm.
 
warning: .dynamic section for "/lib/libc.so.6" is not at the expected address (wrong library or version mismatch?)
 
Error while mapping shared library sections:
 
/lib/ld-linux.so.3: No such file or directory.
 
 
Breakpoint 1, main () at main_cfft.c:47
 
47 {
 
(gdb) l
 
42 }
 
43
 
44 static complex *new_complex_vector(int size);
 
45
 
46 int main ()
 
47 {
 
48   int i;
 
49   int N, n;
 
50   int nTimes;
 
51   float secs;
 
(gdb) n
 
56     complex *in = new_complex_vector(N);
 
(gdb) b fft_exec
 
Breakpoint 2 at 0x87d4: file cfft.c, line 77.
 
(gdb) c
 
Continuing.
 
 
Breakpoint 2, fft_exec (N=16, in=0x12090) at cfft.c:77
 
77   for (i = 1; i < N; i = i * 2)
 
(gdb) p n
 
$1 = 1
 
(gdb) n
 
72 {
 
(gdb) n 3
 
80     for (k = 0; k < i; k++)
 
(gdb) p n
 
$2 = 16
 
</pre>
 
And so on.  The nice thing is the remote binary, cfft_arm, doesn't have to have the debugging information in it.  On your Beagle try:
 
<pre>
 
$ ls -l cfft_arm
 
$ file cfft_arm
 
$ strip cfft_arm
 
$ ls -l cfft_arm
 
$ file cfft_arm
 
$ gdbserver locatl:2001 ./cfft_arm
 
</pre>
 
Mine went from about 26k to 8k when the symbols are removed, but it debugs just like before.
 
 
== cbrowser/cscope ==
 
 
When you changed the value of '''MAXPOW2''' how did you find where it was defined?
 
<pre>
 
$ grep MAXPOW *.c *.h
 
</pre>
 
works, but what if your files are spread over several directories? (Like the kernel).  '''cbrowser''' and '''cscope''' can help. On the host computer try:
 
<pre>
 
$ sudo apt-get install cbrowser
 
$ sudo apt-get install cscope
 
$ cscope -b
 
$ cbrowser
 
</pre>
 
You should see something like:
 
[[File:Cbrowser.png|200px‎]]
 
Try searching for other symbols.
 
  
 
== strace ==
 
== strace ==

Revision as of 15:02, 18 August 2011


As the code becomes more complex, more powerful debugging tools are needed. The GNU Project debugger (gdb) is the granddaddy of all debuggers. In this exercise you will learn how to install and use it on the Beagle. In a later exercise you will learn how to install it on your host and debug the Beagle remotely.

gdb

gdb, the GNU Project debugger, allows you to see what is going on inside another program while it executes -- or what another program was doing at the moment it crashed.

gdb can do four main kinds of things to help you catch bugs in the act:

  • Start your program, specifying anything that might affect its behavior.
  • Make your program stop on specified conditions.
  • Examine what has happened, when your program has stopped.
  • Change things in your program, so you can experiment with correcting the effects of one bug and go on to learn about another.

The program being debugged can be written in Ada, C, C++, Objective-C, Pascal (and many other languages). Those programs might be executing on the same machine as GDB (native) or on another machine (remote). GDB can run on most popular UNIX and Microsoft Windows variants.

For our lab we'll be using a C program and do local execution on the Beagle.

Installing gdb

On your Beagle run:

$ opkg update
$ opkg install gdb

The Sample Program

strace

strace is a nice debugging tool that shows all the system calls a program is making. The book gives some nice examples of using strace. One place I used it was to learn where httpd looked for configuration files.

$ strace httpd
...
open("/etc/httpd.conf", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
open("httpd.conf", O_RDONLY|O_LARGEFILE) = -1 ENOENT (No such file or directory)
rt_sigaction(SIGHUP, {0x1a39c, [], SA_RESTART|0x4000000}, NULL, 8) = 0
open("/dev/null", O_RDWR|O_LARGEFILE)   = 4
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x4001df98) = 3595
io_submit(0, 0, 0xfbad2088 <unfinished ... exit status 0>
Process 3594 detached