[PyKDE] Threading, PyQt and SIP

[Phil Thompson]

On Monday 22 March 2004 02:19, Claus, Richard wrote:
>  I've written a library in C++ that takes a buffer of data and calls back
> application resolved virtual methods based on the contents of the buffer. 
> All the work is done in place, so there is no memory management other than
> temporaries created on the stack.  It would be very useful for us if this
> library were also available from Python, so I've used SIP 3.10.1 (snapshot
> 20040313-194) to expose its interface.  I haven't tried to do anything
> tricky with %MethodCode or anything like that.
>
> The results on Windows XP (Python 2.3.3, MSVS .Net 7.1) so far:
>
> - With a pure C++ application dynamically linked against this library, I
> have seen no problems and everything works as expected. - With single and
> multi-threaded, non-PyQt applications, I have seen no problems and
> everything works as expected. - With PyQt applications, I see no problems
> when I use the library from the Qt GUI thread (the one running
> QApplication.exec_loop()).

> - With PyQt applications, there are problems 
> when I use the library from a non-GUI thread.  Often Python crashes with
> "Fatal Python error: GC object already tracked".  With Python 2.2, the
> message was something like "...GC object already on linked list".  Other
> crashes are access violations in seemingly random locations.  Once, I saw a
> Python traceback involving libqtc.sipCallCtor(204,self,args) ->
> AttributeError: sipThis  before getting the GC error (sorry, I can't paste
> it in right now).  This appeared to happen when the GUI thread decided to
> update the screen.  The amount of time things appear to run correctly
> before crashing seems to be random.

How are the threads being created?

> There are no intentional interactions between the usage of my library and
> the rest of the PyQt application, e.g., the library's methods aren't trying
> to update the screen.

Problems like this are usually related to unintentional interactions :)

> Can someone please describe for me how to keep two 
> SIP applications running in two threads of the same process from tangling? 

The first thing I'd try is SIP v4. The thread handling is much more forgiving 
because it uses Python's new thread API.

> Or is the rule of the game that there can be only one thread using SIP
> generated code?  If so, can I solve this by going to one of the other C/C++
> to Python interface packages, e.g., SWIG or Boost?  Can these coexist with
> SIP?

I don't know about SWIG. Boost ignored threads and the GIL completely, I don't 
know if this has changed recently.

Phil