[PyQt] Re: Embedding w/ multi-threaded Python

[Marc Nations]

So are you basically running a client/server type network with one Python/Qt
install that multiple users are executing their scripts from? I'm trying to
understand where the problem is and why this is an issue.
If that's the case then just knowing the steps you have to take in making
sure that Python handles threads properly from one application can be tricky
could make this quite challenging. Python is not naturally thread-safe. It's
got some very good tools to make it so, but relying on the underlying native
functionality can lead to problems. Especially if you're trying to use one
interpreter to handle several instances.

So if you're running a client/server deployment of Python, you may also need
to deploy a thread pool using client/server functionality on the server
where Python resides. This would basically take one persistent running
instance of a main Python thread on the server that can connect to each
individual instance of a user which would be assigned a port where they can
communicate with the main thread. Python would be listening on these ports
for any commands that come through in the queue and be able to act on them.
That way there's only one instance of the GIL on the server which should be
able to control the individual requests from each user without starving out
the other threads.

However, that's all theoretical, and I have no idea how to set that up. This
is the way other server apps I've used control their threads, but I've never
used Python in a server environment.


On Tue, Mar 10, 2009 at 4:40 PM, McKay Davis <mckay.cpp at gmail.com> wrote:

> On Tue, Mar 10, 2009 at 11:08 AM, Phil Thompson
> <phil at riverbankcomputing.com> wrote:
> > On Tue, 10 Mar 2009 10:30:04 -1000, McKay Davis <mckay.cpp at gmail.com>
> > wrote:
> >> After no response, maybe more specifics with our Multi-Threaded Python
> >> w/ PyQt problem would help:
> >>
> >> In the main thread, we do:
> >>
> >> PyEval_InitThreads(); // acquires the lock
> >> PyInitialize();
> >> // other init stuff
> >> PyThread_release_lock()
> >>
> >> …and in each new Python thread we do:
> >> Py_NewInterpreter()
> >> …
> >> PyThreads_release_lock()
> >>
> >> We call release()/acquire() blocks whenever the python script does a
> >> blocking operation.  A specific example of a lockup with PyQT is
> >> running examples/tutorial/t8.pyw.  We basically:
> >>
> >> PyThreads_acquire_lock()
> >> <run example script>
> >> PyThreads_release_lock()
> >>
> >> When a slot is invoked as a result of the slider being moved we get a
> >> deadlock.  This happens when sip_ProtectVirt_languageChange calls
> >> PyThreads_release_lock() on an already released lock.  If we don’t
> >> release the lock after the main thread PyQT script code runs, then the
> >> script runs fine – but our other python script threads will be starved
> >> out.
> >>
> >> Anybody have any ideas on how to resolve this deadlock?
> >
> > PyQt uses the PyGILState_* functions. These aren't supported with
> multiple
> > interpreters - see the docs for Py_NewInterpreter().
>
> So its not possible to run PyQt with multiple python interpreters?
>
> This is too bad, because we require the ability to allow users to
> script our application using Python.  But, we would also like to
> internally use PyQt for GUI development.  If we only have one
> interpreter then we cannot use PyQt because we cannot allow end-users
> the ability to potentially develop PyQt under the commercial license.
>
> Is there a way to allow only our program to access PyQt and end-users
> access to the python interpreter too?
>
> -McKay
>
> _______________________________________________
> PyQt mailing list    PyQt at riverbankcomputing.com
> http://www.riverbankcomputing.com/mailman/listinfo/pyqt
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.riverbankcomputing.com/pipermail/pyqt/attachments/20090311/fc70c8e6/attachment.html