[PyQt] Multithreading, signals, reference counting and crash
Elvis Stansvik
elvstone at gmail.com
Fri Feb 12 22:53:48 GMT 2016
2016-02-12 23:39 GMT+01:00 Jones, Bryan <bjones at ece.msstate.edu>:
> Phil,
>
> Thanks for your response. I'm specifically concerned about the context of
> passing data between threads using the signal/slot mechanism. As you say, in
> the direct or blocking case, there's no concern with object lifetime.
>
> To test this, I modified Ilya's code to make it easier to test the
> signal/slot mechanism on different types. A QObject fails, but standard
> Python types (dict, list, object, etc.) pass, showing that their reference
> count increases after an emit. I can even pass a list containing a QObject
> safely. This suggests to me that your code increases the reference count
> when emitting pure Python objects, but not when emitting Qt objects. Based
> on some digging (see below), I would expect Qt objects to arrive safely to
> the slot because they're copied, but this doesn't work in practice. What's
> safe and what's not safe when using the signal/slot mechanism when crossing
> thread boundaries?
>
> Looking at the Qt source, from what I understand:
>
> 1. In qobject.cpp, the queued_activate function is used to post an signal to
> another thread's event queue. (See qobject.cpp::activate for the generic
> mechanism used to emit a signal, which calls queued_activate for queued
> connections). Here's a helpful blog post on the topic.
> 2. To do this, queued_activate makes a copy of each argument by invoking
> QMetaType::create (which invokes that type's copy constructor) in the signal
> being emitted.
> 3. When all signals have been delivered, QMetaCallEvent::destroy frees
> memory used by this copy.
>
> Based on this, I conclude that Qt allows the emission of signals with any
> type registered with the Qt meta-type system by copying the type, delivering
> it to all slots, then destroying the copy.
If I understand you correctly, this is also what I've experienced from
a user perspective when passing objects across thread boundaries in Qt
from C++: that it's safe to do so (a copy will be made) when doing so
by const reference?
It's interesting to learn that you need to be more careful when
working from PyQt, or do I?
Thankful for any pointers or "best practices", as I haven't worked
with separate threads in PyQt so far, but might have to soon.
Elvis
>
> Thanks!
>
> Bryan
>
>
> import time
> import sys
> import gc
> import os
> import platform
>
> import sip
> from PyQt5.QtCore import QT_VERSION_STR
> from PyQt5.Qt import PYQT_VERSION_STR
> from PyQt5.QtCore import QObject, QThread, pyqtSignal, pyqtSlot
> from PyQt5.QtCore import qDebug, Qt
> from PyQt5.QtWidgets import QApplication, QWidget
>
> # Change to set the type of object passed between threads.
> objType = QObject
>
> def printObject(obj):
> qDebug('{} {:#X} {}'.format(type(obj), id(obj), obj))
>
> class SomeThread(QObject):
> signal = pyqtSignal(objType)
>
> def __init__(self):
> QObject.__init__(self)
> self.thread = QThread()
> self.wait = self.thread.wait
> self.moveToThread(self.thread)
> self.signal.connect(self.signalHandler, type = Qt.QueuedConnection)
> self.thread.started.connect(self.run)
> self.thread.start()
>
> @pyqtSlot(objType)
> def signalHandler(self, obj):
> qDebug('In signalHandler.')
> printObject(obj)
> qDebug("signalHandler obj ref counts: " + str(sys.getrefcount(obj)))
>
> @pyqtSlot()
> def run(self):
> qDebug("Thread sleeps for 0.1 sec")
> time.sleep(0.1)
> qDebug("Process events now")
> QApplication.processEvents()
> self.thread.exit()
>
> def someFunc(objType, th):
> obj = objType()
> printObject(obj)
> qDebug("obj ref counts before emit: " + str(sys.getrefcount(obj)))
> th.signal.emit(obj)
> qDebug("obj ref counts after emit: " + str(sys.getrefcount(obj)))
>
> app = None
> def main():
> print('Qt {}, SIP {}, PtQt {}, Python {}, OS {}, platform {}
> {}\n'.format(
> QT_VERSION_STR, sip.SIP_VERSION_STR, PYQT_VERSION_STR, sys.version,
> os.name, platform.system(), platform.release()))
>
> global app
> app = QApplication([])
> for x in range(100):
> print('\n\nTest {}'.format(x + 1))
> testType(objType)
>
> def testType(objType):
> th = SomeThread()
>
> someFunc(objType, th)
> gc.collect()
> th.thread.wait()
>
> if __name__ == '__main__':
> main()
>
> On Fri, Feb 12, 2016 at 1:57 PM, Jones, Bryan <bjones at ece.msstate.edu>
> wrote:
>>
>> I'd like to thank Ilya for bringing this to light. I thought I'd worked
>> around a similar problem, but I now see that I mistakenly assumed Python
>> semantics where C++ semantics apply. To make sure I understand correctly:
>>
>> 1. If a Python type can converted to a pass-by-value C++ type (int, long,
>> etc.) then it will be passed as the C++ value through the signal/slot
>> mechanism, meaning I don't need to retain a Python reference to it to keep
>> it valid.
>> 2. If a Python type cannot be converted to a pass-by-value C++ type
>> (string, big int, dict, object, etc.) then a pointer to it will be passed
>> through the signal/slot mechanism, meaning I must retain a Python reference
>> until the signal has been delivered to all slots.
>>
>> However, I do see Qt employing pointers (QFileSystemWatcher's signals emit
>> const QString&, QLabel::setMovie(QMovie*). Does this imply that the emitters
>> of these signals can somehow "know" when it's free to destroy the QString
>> they emitted? Or that developers which use QFileSystemWatcher should not
>> destroy it until all of its signals have been delivered? This seems hard to
>> do. How do C++ developers using Qt do this? Perhaps I can learn from their
>> approach.
>>
>> Bryan
>>
>> On Fri, Feb 12, 2016 at 3:43 AM, Phil Thompson
>> <phil at riverbankcomputing.com> wrote:
>>>
>>> On 11 Feb 2016, at 7:15 pm, Ilya Kulakov <kulakov.ilya at gmail.com> wrote:
>>> >
>>> > Phil,
>>> >
>>> > How does Qt's automatic unsubscribing in QObject's destructor works
>>> > then?
>>> >
>>> > Best Regards
>>> > Ilya Kulakov
>>>
>>> Unsubscribing to what?
>>>
>>> Qt maintains lots of internal data structures about connections, but
>>> these aren't exposed. As far as I know Qt does not track pointers to
>>> QObjects when they are sitting in a thread's event queue.
>>>
>>> Phil
>>>
>>> >> On 12 февр. 2016 г., at 0:30, Phil Thompson
>>> >> <phil at riverbankcomputing.com> wrote:
>>> >>
>>> >>
>>> >>> On 11 Feb 2016, at 6:16 pm, Ilya Kulakov <kulakov.ilya at gmail.com>
>>> >>> wrote:
>>> >>>
>>> >>> Phil,
>>> >>>
>>> >>> I said I don't know all the details :)
>>> >>>
>>> >>> PyQt "controls" both signals and slots. Doesn't it know how many
>>> >>> receivers are there before it sends?
>>> >>
>>> >> The short answer to the question is no. Also, with queued connections,
>>> >> a signal may be sitting in an unprocessed event queue for an indeterminate
>>> >> amount of time. In network terms, signals are UDP, not TCP.
>>> >>
>>> >> Phil
>>> >
>>>
>>> _______________________________________________
>>> PyQt mailing list PyQt at riverbankcomputing.com
>>> https://www.riverbankcomputing.com/mailman/listinfo/pyqt
>>
>>
>>
>>
>> --
>> Bryan A. Jones, Ph.D.
>> Associate Professor
>> Department of Electrical and Computer Engineering
>> 231 Simrall / PO Box 9571
>> Mississippi State University
>> Mississippi state, MS 39762
>> http://www.ece.msstate.edu/~bjones
>> bjones AT ece DOT msstate DOT edu
>> voice 662-325-3149
>> fax 662-325-2298
>>
>> Our Master, Jesus Christ, is on his way. He'll show up right on
>> time, his arrival guaranteed by the Blessed and Undisputed Ruler,
>> High King, High God.
>> - 1 Tim. 6:14b-15 (The Message)
>
>
>
>
> --
> Bryan A. Jones, Ph.D.
> Associate Professor
> Department of Electrical and Computer Engineering
> 231 Simrall / PO Box 9571
> Mississippi State University
> Mississippi state, MS 39762
> http://www.ece.msstate.edu/~bjones
> bjones AT ece DOT msstate DOT edu
> voice 662-325-3149
> fax 662-325-2298
>
> Our Master, Jesus Christ, is on his way. He'll show up right on
> time, his arrival guaranteed by the Blessed and Undisputed Ruler,
> High King, High God.
> - 1 Tim. 6:14b-15 (The Message)
>
> _______________________________________________
> PyQt mailing list PyQt at riverbankcomputing.com
> https://www.riverbankcomputing.com/mailman/listinfo/pyqt
More information about the PyQt
mailing list