gdb and django test server, debug a shared library

jason_m · 07-02-2013, 07:48 PM

Stated most simply, I'm unable to hit a breakpoint or step into code from a shared library that gets called from my django web application. Below is relevant information about what I'm trying to do.

- I wrote a shared library in c++
- Thought it would be nice to use its functionallity in a web site/app
- Created an 'extern "C"' interface to the library
- Created the python wrapper using ctypes
- Wrote a django web app, which makes use of the python wrapper to my library

I'm running the django test server (python manage.py runserver) and I get a segfault at the first place in the app where I'm passing a pointer to an object created by the shared library back through the shared library interface.

If I just fire up python (no django, no server), and try to mimick what the web app code does, I cannot reproduce the segfault. The library code works as expected.

I can do this:

Code:

$ gdb python
(gdb) break namespace::class::function
Function "namespace::class::function" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
(gdb) run
...

Where namespace::class::function is from my shared library, and gdb will hit the breakpoint and I can step through my c++ code. However, I also cannot reproduce the error like this.

However, I cannot get this to work with the test server (where the error exists).

Code:

$ gdb python
(gdb) break namespace::class::function
Function "namespace::class::function" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
(gdb) run manage.py runserver

The server segfaults, my breakpoint is never hit, and the backtrace does not show any of the calls inside my library.

Is it possible to debug this code while the server is running? I'm not sure what I'm looking for. I'm hoping that at the time an object is first created in the shared library and passed through the library interface to python, that I can inspect it and verify everything is in order, and find some place where things get out of order. So my immediate problem is getting the debugger working.

Thanks in advance.

jason_m · 07-03-2013, 08:12 AM

Interesting. I still can't say why the debgger isn't working as I would like when running the django test server, but it does look like my segmentation fault error lies in my use of ctypes. As a disclaimer, this is the first time I've used ctypes, more or less the first time I've used python, and the first time I've used django.

I've seen that ctypes has pointer() and byref() functions. I am not currently using those. I don't quite understand what gets passed to them. A python object? I'm passing around pointers to my c++ objects and my python objects just hold on to them as handles. When specifying the interface for passing/returning pointers, I've been using the type c_int. It looks like when the django test server is running, the address is being cut off. I wonder if I should be using c_long instead (or may if I really ought to look into pointer() and byref() more).

(I found the use of c_int on a SO post somewhere and ran with it, it didn't seem to break anything)

Anyway, I added a bunch of debug output in my library. Here's what I get when just playing with the library in python:

Code:

>>> import pen #this has the bindings to my library
>>> p = pen.ModelEnv(2013, "age", "svc")
libpen_wrapper: created new ModelEnv at address 0x2116c10
libpen_wrapper: ModelEnv age field is age
libpen_wrapper: attempting to get age field on model at address 0x2116c10
age
ModelEnv created
>>> p.ageField()
libpen_wrapper: attempting to get age field on model at address 0x2116c10
'age'
>>>

Now, watch what happens when running the django test server:

Code:

Django version 1.4.5, using settings 'pensite.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
libpen_wrapper: created new ModelEnv at address 0x7f107c100cd0
libpen_wrapper: ModelEnv age field is age
libpen_wrapper: attempting to get age field on model at address 0x7c100cd0
Segmentation fault (core dumped)

See how the address went from 0x7f107c100cd0 to 0x7c100cd0. Sure seems like the cause.

Got to go to work. This is going to be a long day... all day I'll be thinking solutions for this.

ntubski · 07-03-2013, 09:28 AM

Quote:

Originally Posted by jason_m

As a disclaimer, this is the first time I've used ctypes, more or less the first time I've used python, and the first time I've used django.

I haven't used ctypes either, but...

Quote:

When specifying the interface for passing/returning pointers, I've been using the type c_int.

Presumably you should be using c_void_p.

Quote:

I've seen that ctypes has pointer() and byref() functions. I am not currently using those. I don't quite understand what gets passed to them. A python object?

It looks like you pass a python ctype object to them to create pointers.

Quote:

It looks like when the django test server is running, the address is being cut off. I wonder if I should be using c_long instead (or may if I really ought to look into pointer() and byref() more).

Looks like the server is 64 bit, c_long would probably work, but c_void_p is more correct.

jason_m · 07-03-2013, 07:03 PM

Quote:

Originally Posted by ntubski

Presumably you should be using c_void_p.

Yes, this is the same conclusion I reached scanning over the ctypes documentation.

So this gets me past the segfault. But, for curiosity sake (and keeping with the spirit of the thread title), I still haven't figured out if there is a way to step through my library code as the test server is running. Any thoughts on this?