Debugging with GDB

As explained in the JTAG/SWD vs Bootloader tutorial, this won't work with the bootloader : you will need to use an adapter and compile your code with DEBUG=true and without BOOTLOADER=true. Remember to issue a make clean every time you modify the Makefile.

GDB (the GNU Project Debugger) is an extremely powerful tool to debug your code. It is originally available for desktop systems but a version ported for embedded systems is available in the toolchain. OpenOCD offers a GDB-compatible API; this basically means that you can fire up GDB and debug your program directly inside the microcontroller, while it is running (this is called on-microcontroller debugging and this is where the OCD in OpenOCD comes from). If you have never used a debugger before, you have no idea how powerful this is.

The main idea when debugging is to put some breakpoints (some kind of flags that tell the microcontroller to pause its execution and ask you what to do when it reaches a particular line of code or a particular function) before a piece of code that seems buggy, wait for it to reach it and break, then look around for something that doesn't look right. When the execution is paused, you have full control over the behaviour of the microcontroller : you can execute lines of code one by one, show the value of any variable at each step, dump the memory and even modify variables and call functions on the fly. If something isn't behaving as it should, you have everything you need to understand why.

GDB is a powerful and complex tool, but we'll see here how to use the most basic and useful features it offers. For more details, nothing beats the official documentation.

First, some code

For the purpose of this tutorial we'll assume you have an LED connected to PA00 and you want to make it blink using this code (I hope you're not already tired of blinking LEDs yet) :

#include <core.h>
#include <gpio.h>

int main() {
    // Init the microcontroller
    Core::init();

    // Define the pin on which the LED is connected
    GPIO::Pin led = GPIO::PB00;

    // Enable this pin in output to a HIGH state
    GPIO::enableOutput(led, GPIO::HIGH);

    while (1) {
        // Turn the LED on
        GPIO::setLow(led);

        // Wait for half a second
        Core::sleep(500);

        // Turn the LED off
        GPIO::setHigh(led);

        // Wait for half a second
        Core::sleep(500);
    }

}

As usual, fire up make and make flash. But after reboot, it doesn't blink. If you've looked closely to the code you might have already guessed what's wrong (the pin was defined to PB00 instead of PA00 where the LED is connected), but we'll see how to track the problem using GDB.

Starting a debug session

Starting a debug session is easy because once again, the Makefile takes care of everything. Simply type make debug (with an openocd instance started, as explained above) and you're good to go.

GDB assumes that your source code and the program running inside the microcontroller are in sync, so make sure you have saved everything, compiled, and flashed your program. Otherwise, some very weird things can happen. If you aren't sure, make clean; make; make flash doesn't cost much time.

You'll see a lot of info dropped on your screen at first, mostly non-interesting low-level and legal stuff. All you need to look at are the three last lines :

Reset_Handler () at ./libtungsten/startup_ARMCM4.S:149
149     ldr r1, =__etext
(gdb)
When you start GDB, it will connect to the adapter through OpenOCD, reset the microcontroller, tell you where its current execution point is and wait for instructions. The first two lines are the execution point and the (gdb) part is the prompt where you can enter debug commands. We'll see the most important commands as we go along, and a cheatsheet is available at the bottom of this page.

You might be surprised that after a reset, the execution point is not at the beginning of your main(), but in some strange .S file. This is because the main() is not technically where your code starts, it's more like a C/C++ standard. On ARM architectures, there is something called the vector table stored at the very beginning of the flash memory, that contains a list of function pointers (handlers) each dedicated to handle a specific exception. One of these exceptions is Reset, so the ResetHandler is where the execution begins. The ResetHandler is usually written in assembly language in a file called the startup code (here startup_ARMCM4.S) and its role is to do some low-level memory initialization and then to call your main().

This is already more than you need to know for the purpose of this tutorial but if you are curious, you can find more information in the Core module documentation and the chapter B1.5 ARMv7-M exception model in the ARMv7-M Architecture Reference Manual.

Setting breakpoints

Since we don't really care about the startup code, the first thing we'll do is advance the execution to the main(). The most simple way of achieving that is to put a breakpoint there and run until it breaks.

Setting a breakpoint is done with the break command, followed by a location. As we'll see later, there are many ways to indicate a location, but for now we'll only supply a function name, and GDB will be smart enough to understand that we want to break at the first line of this function :

(gdb) break main
Breakpoint 1 at 0x44: file test.cpp, line 4.

GDB proceeds by giving you more info about the breakpoint, such as its number and the location in your code and in memory.

If you have forgotten what breakpoints you have set up, use the info command :

(gdb) info breakpoints
Num     Type           Disp Enb Address    What
1       breakpoint     keep y   0x00000044 in main() at test.cpp:4

Execution control

Type continue to resume the execution :

(gdb) continue
Continuing.
Note: automatically using hardware breakpoints for read-only addresses.

Breakpoint 1, main () at test.cpp:4
4   int main() {

Ignore the "Note" for now. GDB tells you it is continuing the execution and if there was no breakpoint, it would have continued (you wouldn't have a prompt) indefinitely. In this case there is a breakpoint however, and it's hit almost instantly because the startup code is really short.

You might have recognized the last two lines before the prompt : they follow the same pattern as when you first started GDB. This is because when it stops execution, GDB always tells you where it stopped and (if available) it will show you the next line of code that will be executed. Earlier it was the reset handler, now it's the main, where you put your breakpoint.

You can use the list command to display a few lines of code surrounding the current one and getting a bit of context information :

(gdb) list
1   #include <core.h>
2   #include <gpio.h>
3    
4   int main() {
5       // Init the microcontroller
6       Core::init();
7    
8       // Define the pin on which the LED is connected
9       GPIO::Pin led = GPIO::PB00;
10

You could type continue again to resume operation, but that wouln't be very useful. Since the LED doesn't blink, it would be interesting to see if the loop and the setLow() and setHigh() functions get executed. For this, we will execute the code line by line and see how it goes, using the next command :

(gdb) next
6       Core::init();
(gdb) 
9       GPIO::Pin led = GPIO::PB00;
(gdb) 
12      GPIO::enableOutput(led, GPIO::HIGH);
(gdb) 
16          GPIO::setLow(led);
(gdb) 
19          Core::sleep(500);
(gdb) 
22          GPIO::setHigh(led);
(gdb) 
25          Core::sleep(500);
(gdb) 
16          GPIO::setLow(led);
(gdb) 
19          Core::sleep(500);
If you don't type any command and press return, GDB will execute again the last command you typed. That's handy to execute a bunch of lines by hitting the return key repeatedly.

You can see that everything looks normal and the loop gets executed, so the problem is not there. We need to dig a bit more.

Reading and writing variables

GDB allows you to print almost anything in your code, such as the values of variables, using the print command. Here, there is only one variable in our code, so let's look at it :

(gdb) print led
$1 = {
  port = GPIO::Port::B, 
  number = 0 '\000', 
  function = GPIO::Periph::A
}

"Wait, why is it Port B, it should be Port A doesn't it?"

Obviously, in this case, the mistake was already visible in the code. But print can display the value of any variable in the code at any time, such as counters, results of operation, or anything else.

We could recompile the code to test the Port A solution, but where is the fun in that? We can simply change the value on the fly and see if it works. Writing variables is done using the set command and following a C-like syntax :

(gdb) set led.port=GPIO::Port::A
(gdb) print led
$2 = {
  port = GPIO::Port::A, 
  number = 0 '\000', 
  function = GPIO::Periph::A
}

Looks good. Now, resume the execution :

(gdb) c
Continuing.

The prompt disappears because you can't issue commands when the microcontroller is running. But more importantly, the LED still doesn't blink. Did we do something wrong?

Look back at the code and think about it :

  1. led was set to PB00
  2. we executed until the loop and made a few turns
  3. we changed led to PA00
  4. we resumed the loop execution
... yep, the pin was not initialized, because enableOutput(led) was called when led was still PB00. We need to call enableOutput(led) on PA00, but is it possible to arbitrarily call a function inside GDB?

Calling functions on the fly

Spoiler : yes it is.

But first, we need to stop the microcontroller to get a prompt back. Simply press Ctrl+C (like the SIGINT signal on Unix/Linux), and voila.

Calling a function is achieved with the call command. Here, it will look like this :

(gdb) call GPIO::enableOutput(led, GPIO::HIGH)
Note: automatically using hardware breakpoints for read-only addresses.

Again, ignore the "Note". GDB will call the function, and when it returns, stop again at the same place and give you your prompt back.

Now, resume the execution again with continue. Hooray, it blinks!

Callstack

The callstack is the stack of function calls that lead you to the current point of execution. When you call a function B inside a function A, A is already on the stack and B is put over it. If you call a third function C inside B, the stack will look like A→B→C, and if you return from B, it will be removed from the stack. GDB can show you the current callstack, which is very useful to know how you got there in the first place (for exemple, when you reach a breakpoint).

Use the backtrace command inside GDB to print the callstack. It will show you the innermost function first and go back in the chain of function calls down to main. You can ask GDB to display the value of local variables inside each function at the same time using backtrace full. Finally, if you want to inspect the outer code that lead to this particular function call, you can move up and down the stack with the up and down commands.

See the next paragraph for examples of how to use the callstack.

Tracking errors

Using the callstack with breakpoints is very useful when an error happens and you want to know what triggered it.

For example, consider the following code :

USART::Port _usart = USART::Port::USART0;
ADC::Channel _adc = 1;

USART::setPin(_usart, USART::PinFunction::RX, {GPIO::Port::A,  5, GPIO::Periph::B});
USART::setPin(_usart, USART::PinFunction::TX, {GPIO::Port::A,  7, GPIO::Periph::B});
USART::enable(_usart, 9600);
ADC::enable(_adc);

Since the default pin mapping for the ADC channel number 1 is PA05, there is a pin conflict with the USART0 RX function. If you have an error handler configured (there is a default handler on Carbide which blinks the red LED rapidly) and you try this code, this error will be detected and the handler will be called.

Suppose that you are running a Carbide with the default handler, you try your code, and you see the red LED blinking. This means that an error happened, and you want to know what caused it. Fire up GDB and put a breakpoint on the handler :

(gdb) break Carbide::criticalHandler() 
Breakpoint 1 at 0x2678: file libtungsten/carbide.cpp, line 42.

Now, you will be able to inspect any CRITICAL error that happens. If you want to track a WARNING error, put the breakpoint on Carbide::warningHandler(). In any case, resume the execution with continue.

After a second or so, the error is triggered, the handler is called and the breakpoint is hit :

Breakpoint 1, Carbide::criticalHandler () at libtungsten/carbide.cpp:42
42      void criticalHandler() {

Display the current callstack with backtrace to see what caused the error :

(gdb) backtrace
#0  Carbide::criticalHandler () at libtungsten/carbide.cpp:42
#1  0x00001b44 in Error::happened (module=module@entry=Error::Module::GPIO, code=code@entry=1, severity=severity@entry=Error::Severity::CRITICAL) at libtungsten/sam4l/error.cpp:31
#2  0x00000ec8 in GPIO::enablePeripheral (pin=...) at libtungsten/sam4l/gpio.cpp:68
#3  0x00001bee in ADC::enable (channel=channel@entry=1 '\001') at libtungsten/sam4l/adc.cpp:56
#4  0x00000092 in main () at test.cpp:15

You have a summary of the problem almost at first glance :

  • your main tried to enable the ADC channel number 1 at line 15
  • in turn, the ADC module tried to put the default pin associated with this channel in peripheral function
  • the GPIO::enablePeripheral() function detected an error and called Error::happened() at line 68 with the arguments module=Module::GPIO and code=1

The error codes are defined inside each module header so you could take a look at gpio.h to see what the code 1 represents, but it's quicker to look directly inside GDB. Move up the stack twice to go to the Error::happened call :

(gdb) up
#1  0x00001b44 in Error::happened (module=module@entry=Error::Module::GPIO, code=code@entry=1, severity=severity@entry=Error::Severity::CRITICAL) at libtungsten/sam4l/error.cpp:31
31              handlers[static_cast(severity)]();
(gdb) 
#2  0x00000ec8 in GPIO::enablePeripheral (pin=...) at libtungsten/sam4l/gpio.cpp:68
68              Error::happened(Error::Module::GPIO, ERR_PIN_ALREADY_IN_USE, Error::Severity::CRITICAL);

There you go : the error is ERR_PIN_ALREADY_IN_USE, so you know that the ADC tried to used a pin that was already used by another peripheral and you need to check the pins_sam4l_XX.cpp file and your calls to the different setPin() of the other modules. The culprit shouldn't be hard to find in most cases.

Memory examination

Sometimes (although rarely enough, fortunately), displaying variables and callstacks is not enough : you need to go deeper and look at the raw memory. The command to use for this is x (they didn't even bother making an "examine" alias, probably because if you need it, you are already an advanced user anyway), followed by optional flags and a memory address. This can be useful for looking at registers in realtime (remember : registers are not physically located in the RAM block, but they are accessed through a memory addresse so it behaves exactly the same way).

For example, let's say you want to look at a register in the Cortex Core, for example the ICSR (Interrupt Control and State Register) register. The address of this register is given in the §B3.2.2 System control and ID registers in the ARMv7-M Architecture Reference Manual : 0xE000ED04 (see the Core module reference for more information). Use the following command to dump the value of this register :

(gdb) x /wx 0xE000ED04
0xe000ed04: 0x00000000

Before giving the address we want to dump, we gave two flags to the command : w and x, meaning we want to dump a word value (4 bytes) shown in hexadecimal format. You could also indicate a multiplier in order to dump several successive values at one : x /5wx. More info in the GDB documentation :

(gdb) help x
Examine memory: x/FMT ADDRESS.
ADDRESS is an expression for the memory address to examine.
FMT is a repeat count followed by a format letter and a size letter.
Format letters are o(octal), x(hex), d(decimal), u(unsigned decimal),
  t(binary), f(float), a(address), i(instruction), c(char), s(string)
  and z(hex, zero padded on the left).
Size letters are b(byte), h(halfword), w(word), g(giant, 8 bytes).
The specified number of objects of the specified size are printed
according to the format.

GDB tells us the register is empty, which tells you that in this case, no interrupt is currently active or pending.

Breakpoint commands

GDB is able to associate a list of commands to each breakpoint which will be executed whenever this breakpoint is hit. There are many cases in which this is useful, such as :

  • saving time by dumping application-specific context information everytime the execution is paused
  • avoiding race conditions by making sure a specific command is executed almost instantly after the breakpoint is reached (for example, dumping the value of a register that would have been reset by the time you type the x command)
  • quickly inspecting the execution of your program while minimizing the impact of the debugger, by closing the set of commands with continue (this will execute any previous command in the list and then resume the execution automatically, without giving you back the prompt)

This last example is a common use of breakpoint commands. Let's say you have written a readSensor() function and you want to check that it works properly and reacts when you physically act on the sensor. You can use a simple test loop such as this one :

int value=0;
while (true) {
    value = readSensor();
    Core::sleep(1000);
}

You can setup GDB to simply display value each time a new measurement is made :

(gdb) b test.cpp:12 # The "Core::sleep()" line
(gdb) commands
> silent # This will get rid of the breakpoint information that is displayed when a breakpoint is hit
> print value
> continue
> end # Close the list of commands with "end"
(gdb) set height 0 # Keep scrolling when the screen is full

Now, resume the execution and look at the screen while playing with the sensor, a new measurement will be displayed every second :

(gdb) continue
10
12
9
10
17
26
55
115
41
12
11
11

Let's save some time

Most of the most common GDB commands can be reduced to a few characters, in order to be typed more quickly :

  • break : b
  • info breakpoints : i b
  • list : l
  • continue : c
  • next : n
  • print : p
  • backtrace : bt