[PyKDE] PyThreadState_Get, PyEval_*Thread, NULL tstate.

[Patrick Stinson]

On Friday 13 February 2004 22:51, Phil Thompson wrote:
> On Saturday 14 February 2004 07:27, Patrick Stinson wrote:
> > On Friday 13 February 2004 10:53, Phil Thompson wrote:
> > > On Thursday 12 February 2004 20:48, Patrick Stinson wrote:
> > > > I posted a while ago about the 'PyEval_RestoreThread: NULL tstate:'
> > > > error, and now I'm getting the same problem with PyThreadState_Get
> > > > (if I remember correctly, its the call that happens before
> > > > PyEval_RestoreThread).
> > > >
> > > > urls:
> > > > http://www.mail-archive.com/pykde%40mats.imk.fraunhofer.de/msg01437.h
> > > >tm l
> > > > http://mats.imk.fraunhofer.de/pipermail/pykde/2003-December/006798.ht
> > > >ml and even earlier:
> > > > http://www.mail-archive.com/pykde%40mats.imk.fraunhofer.de/msg01437.h
> > > >tm l
> > > >
> > > > As Before, I'm writing an audio application that uses background
> > > > threads for buffering and audio streaming. The threads are entirely
> > > > contained in my c++ (sipped) lib and execution never enters python
> > > > code. What could cause a NULL thread state in the interpreter? This
> > > > occurs at different times, and with what seem slike varying levels of
> > > > complexity in pyqt Gui code. Does anyone know how I should be
> > > > interpreting this? What does this generally mean?
> > >
> > > I can't remember which version of SIP you are using.
> > >
> > > If you are using SIP v3 then you have to be *extremely* careful about
> > > managing the thread state.  Are you sure that "execution never enters
> > > python code"?
> > >
> > > Things are much easier with SIP v4 because it uses the new thread API
> > > in Python 2.3.
> > >
> > > Phil
> >
> > I will check harder for spots where it might. Currently I am using sip
> > and PyQt 3.8, because of the odd problem I was having with threads
> > appearing to 'lock up'. How is it that I can 'manage' the thread state at
> > the python level? Do you mean I should be careful how my py-objects are
> > used by different python threads, as the interpreter is not considered
> > thread-safe?
>
> You have to manage the GIL and the current thread state carefully to make
> sure that they are correct before you make any calls to Python from C/C++.
> The problem with the old API is that there is no way to safely determine
> the current state.
>
> SIP v3 deals with this by having a very clear sense of "Python-land" and
> "C++-land". The GIL is always released when going from Python to C++ and
> always acquired when going from C++ to Python. If you don't follow similar
> conventions then you will have problems.
>
> With virtual functions it can sometimes be difficult to see exactly when
> execution crosses the Python/C++ border. Sometimes I found calls to Python
> where I didn't expect them because deep in Qt a call to a virtual of a
> wrapped instance was made. This is why I added the -r flag to SIP to
> generate code to trace the execution.
>
> The new thread API used by SIP v4 is much easier to use because you can
> safely acquire the GIL without knowing if you already have it. The GIL is
> only released when going from Python to C++ if the call might take
> significant time to execute.
>
> Phil
>
> _______________________________________________
> PyKDE mailing list    PyKDE at mats.imk.fraunhofer.de
> http://mats.imk.fraunhofer.de/mailman/listinfo/pykde


wow, thank you very much for the in-depth response. Because my sip-wrapped lib 
only provides class-types, virtuals (which do exist there) are the only way 
'out' of c++ and into python. In fact, subclassing those types in python is a 
large part of my design. I'm getting the hint that if you are wrapping a 
c/c++ library that has no knowledge of python (like qt) and therefore cannot 
control attempts on the GIL, you really shouldn't subclass using virtuals in 
that fashion with sip3. But sip3-wrapped qt widgets are meant to be 
subclassed, hence my confusion.
I'll understand if I've carried this on too long.

Cheers, 
Pat
btw, this topic is spot on for my project, as I have many threads and many 
subclasses :)