[PyQt] Multithreading, signals, reference counting and crash

[Ilya Kulakov]

Phil,

In other words PyQt does properly retain / release Python objects sent between threads via slot/signal connected with QueuedConnection?

> On 13 февр. 2016 г., at 15:00, Phil Thompson <phil at riverbankcomputing.com> wrote:
> 
> On 12 Feb 2016, at 10:39 pm, Jones, Bryan <bjones at ece.msstate.edu> wrote:
>> 
>> Phil,
>> 
>> Thanks for your response. I'm specifically concerned about the context of passing data between threads using the signal/slot mechanism. As you say, in the direct or blocking case, there's no concern with object lifetime.
>> 
>> To test this, I modified Ilya's code to make it easier to test the signal/slot mechanism on different types. A QObject fails, but standard Python types (dict, list, object, etc.) pass, showing that their reference count increases after an emit. I can even pass a list containing a QObject safely. This suggests to me that your code increases the reference count when emitting pure Python objects, but not when emitting Qt objects. Based on some digging (see below), I would expect Qt objects to arrive safely to the slot because they're copied, but this doesn't work in practice. What's safe and what's not safe when using the signal/slot mechanism when crossing thread boundaries?
> 
> PyQt tries to convert a Python object to something that the Qt meta-type system understands. If it can't then it wraps it in a PyQt_PyObject. This is registered with the meta-type system and part of its job is to manage the reference count of the object as it gets copied around the meta-type system.
> 
> If PyQt can convert the object then it's up to the meta-type system to manage the lifecycle of the converted C++ instance as you describe below.
> 
> It would be wrong for PyQt to use PyQt_PyObject for every type of Python object because that would mean you couldn't connect to slots implemented in C++ that do not understand PyQt_PyObject. However you can explicitly use PyQt_PyObject yourself by declaring the signal as...
> 
>    mysig = pyqtSignal('PyQt_PyObject')
> 
> ...and this will "protect" the object even if it was a QObject. I was about to include the link to the relevant bit of the documentation and realised that it's only in the current snapshot and I must have added it (the documentation) since the last release.
> 
>> Looking at the Qt source, from what I understand:
>> 
>> 1. In qobject.cpp, the queued_activate function is used to post an signal to another thread's event queue. (See qobject.cpp::activate for the generic mechanism used to emit a signal, which calls queued_activate for queued connections). Here's a helpful blog post on the topic.
>> 2. To do this, queued_activate makes a copy of each argument by invoking QMetaType::create (which invokes that type's copy constructor) in the signal being emitted.
>> 3. When all signals have been delivered, QMetaCallEvent::destroy frees memory used by this copy.
>> 
>> Based on this, I conclude that Qt allows the emission of signals with any type registered with the Qt meta-type system by copying the type, delivering it to all slots, then destroying the copy.
> 
> Your analysis is correct, but with one key omission. You cannot copy QObject instances. It is QObject* that is supported by the meta-type system and not QObject. You still have to manage the lifecycle of what the pointer points to (or use PyQt_PyObject).
> 
> Phil