[PyKDE] Re: Issues using SIP for small C++ classes

[Phil Thompson]

On Thursday 07 December 2006 7:55 pm, Giovanni Bajo wrote:
> Phil Thompson wrote:
> > The short answer is that there is nothing that can be done in the
> > short term.
>
> I see.
>
> > Note that Pyrex, as you describe, is not wrapping C++ classes - it is
> > creating Python equivalents of C++ classes (or do I mean C structs?).
> > I'm not sure how you would represent any class methods - and I don't
> > see how they could work anyway.
>
> I don't know what you mean here: I'm sure you know that you can fully
> define classes in Pyrex (with methods, etc.);

No, I've never looked at Pyrex. That's why I said "as you describe".

How are classes with method's "wrapped"?

> in fact, I have two 
> fully-equivalent implementations of the class P, with operators etc. (one
> in Pyrex, one in SIP/C++), to run benchmarks. In the part in which I was
> speaking of how the Pyrex code was accessing P's members (x & y), I was
> just noticing that it did not need any "getter" function, and was able to
> expose to the Python interpreter the actual member.

It wasn't the actual member because you didn't have an instance of the actual 
C++ class.

> > I think that the objects you could emulate would be so lightweight
> > that I wouldn't even bother with C++ in the first place - unless
> > space is the showstopper.
>
> In fact, in the same library, there are more important and complex
> classes/functions built *upon* P, which fully require C++ and SIP. But I
> was surprised of how large was the overhead of SIP for the leaf lightweight
> classes such as P.
>
> Pyrex has its problems too: it works better for these lightweight classes,
> but you can't really write Pyrex class (say, X) built upon another Pyrex
> class (say P): that is, you can, but the generated code of X will use P
> through the full Python API, so losing all speed. Basically, Pyrex is able
> to write optimized (=fast as C) classes which use C/C++ builtin types (int,
> float, ecc.), but these optimized classes can't be transitively used to
> build other optimized classes.
>
> I just reduced the SIP memory allocation overhead by redefining operator
> new for the classes and using PyMalloc (but this is only possible because I
> own the C++ library...). Some speed got back, but it's still somewhere like
> 50% slower than the Pyrex solution in real-world benchmarks. I'll see what
> happens when everything is finished, but looks like I'll have to leave with
> two different P implementations (one in Pyrex to be used from Python, and
> one in C++ to be used as basis for other classes).

It would certainly be possible to reduce the memory overhead because most of 
the extra fields in sipWrapper aren't needed for lightweight classes. It may 
also be possible to improve the interoperability between Pyrex objects and 
SIP objects. Is there anything that would make the two more easy to 
integrate?

Phil