[PyKDE] PyThreadState_Get, PyEval_*Thread, NULL tstate.

[Phil Thompson]

On Saturday 14 February 2004 10:44, Patrick Stinson wrote:
> On Friday 13 February 2004 22:51, Phil Thompson wrote:
> > On Saturday 14 February 2004 07:27, Patrick Stinson wrote:
> > > On Friday 13 February 2004 10:53, Phil Thompson wrote:
> > > > On Thursday 12 February 2004 20:48, Patrick Stinson wrote:
> > > > > I posted a while ago about the 'PyEval_RestoreThread: NULL tstate:'
> > > > > error, and now I'm getting the same problem with PyThreadState_Get
> > > > > (if I remember correctly, its the call that happens before
> > > > > PyEval_RestoreThread).
> > > > >
> > > > > urls:
> > > > > http://www.mail-archive.com/pykde%40mats.imk.fraunhofer.de/msg01437
> > > > >.h tm l
> > > > > http://mats.imk.fraunhofer.de/pipermail/pykde/2003-December/006798.
> > > > >ht ml and even earlier:
> > > > > http://www.mail-archive.com/pykde%40mats.imk.fraunhofer.de/msg01437
> > > > >.h tm l
> > > > >
> > > > > As Before, I'm writing an audio application that uses background
> > > > > threads for buffering and audio streaming. The threads are entirely
> > > > > contained in my c++ (sipped) lib and execution never enters python
> > > > > code. What could cause a NULL thread state in the interpreter? This
> > > > > occurs at different times, and with what seem slike varying levels
> > > > > of complexity in pyqt Gui code. Does anyone know how I should be
> > > > > interpreting this? What does this generally mean?
> > > >
> > > > I can't remember which version of SIP you are using.
> > > >
> > > > If you are using SIP v3 then you have to be *extremely* careful about
> > > > managing the thread state.  Are you sure that "execution never enters
> > > > python code"?
> > > >
> > > > Things are much easier with SIP v4 because it uses the new thread API
> > > > in Python 2.3.
> > > >
> > > > Phil
> > >
> > > I will check harder for spots where it might. Currently I am using sip
> > > and PyQt 3.8, because of the odd problem I was having with threads
> > > appearing to 'lock up'. How is it that I can 'manage' the thread state
> > > at the python level? Do you mean I should be careful how my py-objects
> > > are used by different python threads, as the interpreter is not
> > > considered thread-safe?
> >
> > You have to manage the GIL and the current thread state carefully to make
> > sure that they are correct before you make any calls to Python from
> > C/C++. The problem with the old API is that there is no way to safely
> > determine the current state.
> >
> > SIP v3 deals with this by having a very clear sense of "Python-land" and
> > "C++-land". The GIL is always released when going from Python to C++ and
> > always acquired when going from C++ to Python. If you don't follow
> > similar conventions then you will have problems.
> >
> > With virtual functions it can sometimes be difficult to see exactly when
> > execution crosses the Python/C++ border. Sometimes I found calls to
> > Python where I didn't expect them because deep in Qt a call to a virtual
> > of a wrapped instance was made. This is why I added the -r flag to SIP to
> > generate code to trace the execution.
> >
> > The new thread API used by SIP v4 is much easier to use because you can
> > safely acquire the GIL without knowing if you already have it. The GIL is
> > only released when going from Python to C++ if the call might take
> > significant time to execute.
> >
> > Phil
> >
> > _______________________________________________
> > PyKDE mailing list    PyKDE at mats.imk.fraunhofer.de
> > http://mats.imk.fraunhofer.de/mailman/listinfo/pykde
>
> wow, thank you very much for the in-depth response. Because my sip-wrapped
> lib only provides class-types, virtuals (which do exist there) are the only
> way 'out' of c++ and into python. In fact, subclassing those types in
> python is a large part of my design. I'm getting the hint that if you are
> wrapping a c/c++ library that has no knowledge of python (like qt) and
> therefore cannot control attempts on the GIL, you really shouldn't subclass
> using virtuals in that fashion with sip3.

There is no problem with doing this as SIP generates the code to handle it. 
Problems can arise if...

- your library does things with threads

- your wrappers have handwritten code that doesn't follow the border 
conventions

- you are interacting with wrappers created by another wrapper generator that 
doesn't handle the GIL (eg. Boost).

> But sip3-wrapped qt widgets are 
> meant to be subclassed, hence my confusion.
> I'll understand if I've carried this on too long.

No problem.

Phil