[PyKDE] Issues using SIP for small C++ classes

Giovanni Bajo rasky at develer.com
Thu Dec 7 17:11:17 GMT 2006


I'm trying to use SIP to wrap a simple C++ vector library. The simplest
example I'll use is a 2d vector:

struct P
   double x, y;

   // etc.

The whole wrapper works correctly, but I have issues with scalability
concerns. Let me explain.

It looks like that SIP only works with *pointers* to C++ objects: this is
crucial to its design. Basically, there is no way I can make it generate a
PyObject with the "struct P" embedded in it. sipWrapper always holds a
*pointer* to P. This contradicts the usual semantic of P in C++, which is a
lightweight aggregate of two doubles, meant to be used on the stack (and
even decomposed to its members by the C++ compiler), without ever going
through a heap allocation.

For reference, if I reimplement P in Pyrex, I obtain this:

struct __pyx_obj_6_geo2d_P {
  double x;
  double y;

while the sipWrapper looks like this:

typedef struct _sipWrapper {
        PyObject *user;   /* For the user to use. */
        union {
                void *cppPtr;  /* C/C++ object pointer. */
                void *(*afPtr)(); /* Access function. */
        } u;
        int flags;   /* Object flags. */
        PyObject *dict;   /* The instance dictionary. */
        struct _sipPySig *pySigList; /* Python signal list (complex). */
        struct _sipWrapper *next; /* Next object at this address. */
        struct _sipWrapper *first_child; /* First child object. */
        struct _sipWrapper *sibling_next; /* Next sibling. */
        struct _sipWrapper *sibling_prev; /* Previous sibling. */
        struct _sipWrapper *parent; /* Owning object. */
} sipWrapper;

This produces several bad pessimizations:

- SIP will always go through a heap allocation for every function that
returns a P ("new P"). It will also allocate the PyObject of course, but
that's necessary (and will go through the python obmalloc which I kind of
trust). It will *also* allocate a dict (see below). The overhead of these
heap allocations is dramatic.

- sizeof(sipWrapper) is large. Too large in this case: I create literally
millions of these objects... if there was infrastructure in-place to
generate sipWrapper-like types specific for each wrapped C++ type, I guess
that it could be modularized better, so not to waste memory for features
that are not wanted/needed/used (like the "user" pointer, or the signal

- Member accessors are properties in SIP, and they go through a function
call, even if I don't want to add any specific %MethodCode. The Pyrex
equivalent is able to go to the actual member like builtin types do (without
any custom code):

static struct PyMemberDef __pyx_members_6_geo2d_P[] = {
  {"x", T_DOUBLE, offsetof(struct __pyx_obj_6_geo2d_P, x), READONLY, 0},
  {"y", T_DOUBLE, offsetof(struct __pyx_obj_6_geo2d_P, y), READONLY, 0},
  {0, 0, 0, 0, 0}

- Every instance of sipWrapper has an instance dictionary, created in
sipWrapper_init(). There does not seem to be a way to tell SIP not to create
this dictionary (which is totally wasted memory in my case): P is an
immutable type, and I don't need/want other attributes added to it.

In the end, real-world benchmarks showed almost a 100% increase in runtime
by using SIP to wrap the C++ library (compared to an API-identical Pyrex
implementation of the library). I assume that memory occupation (and
fragmentation) also grew a lot, but I haven't produced numbers yet.

So, is there something that can be done? Is SIP the wrong tool for wrapping
this kind of lightweight C++ objects? Or would there be something that could
be planned to improve this situation? I'm happy to contribute something, if
there is something that can be done about it.

Giovanni Bajo

More information about the PyQt mailing list