martes, 23 de julio de 2013

Debugging assembly in Pharo VM with GDB

Debugging with GDB can be a pain at first. Specially when you just start learning to use it, finding a bug with it seems to be almost impossible. Dealing with a big executable like PharoVM can be even worse because you feel lost in the code, and not to mention dealing with SIGALRM signaling all the time, which doesn't let you debug because the execution context changes all the time.

But GDB has some very powerful tricks that let you do amazing things, like back-in-time debugging (one of it's best kept secrets), so it's just a matter of acquiring experience. In the last few days I spent some time debugging assembly that I generated and run on-the-fly with Pharo and I want to share some of the most useful things I've learnt.

There are many things in GDB that can be configured. You can even generate commands for commonly used sequences of instructions. The simplest form of configuration is creating a .gdbinit file in your home directory. GDB executes the commands of this file when it starts. I based my .gdbinit file on one that Eliot sent to me some time ago, and added a few things.

I'm just attaching it at the end because it's mostly self explaining. It sets up some common options, improves the layout and defines some commands.

What I wanted to do was to debug the assembly I was generating and executing on-the-fly. To do that, I first start pharo as always. Then, I open a terminal and write

~$ sudo gdb --pid `pidof pharo`

this starts a gdb session and attaches to the running pharo process (here I'm assuming there's only one pharo process running, and that the executable name is pharo, which I'm not sure is always the case). Just because I put the .gdbinit file in my ~ directory, I can execute

(gdb) openwindows

to see this

which is much better. You can change focus with

focus next | prev | src | asm | regs | split

And do some other things, like these. But now I want to go back to pharo again so I enter continue. In pharo, I generate and run a bunch of assembly instructions, being the first of them an int3, which when executed triggers a breakpoint in gdb.

(gdb) c

Program received signal SIGTRAP, Trace/breakpoint trap.
0xaa312005 in ?? ()

if we try to step into the next instruction we get

(gdb) si
0x0809e300 in heartbeat_handler ()

this happens because in Pharo the SIGALRM is signaled periodically, and it will not let us debug our assembly. To avoid this we have to enter

(gdb) handle SIGALRM ignore

in .gdbinit I put begindebug command which does just that, and also enables back-in-time debuging. Now we can debug our code without problems. After we finish, we disable back-in-time debugging and re-enable SIGALRM, with this command I also added to .gdbinit

(gdb) enddebug

That's all for now, see you in the next post.

(gdb) q