[PyQt] Possible memory leak with signal connections

[Kevin Keating]

I've found what appears to be a memory leak caused by connecting signals to slots that lack a QtCore.pyqtSlot decorator.  I know that connections without the pyqtSlot decorator are expected to use more memory than connections with the decorator based on https://www.codeproject.com/Articles/1123088/PyQt-signal-slot-connection-performance.  In the script I've pasted below, though, connections without the pyqtSlot decorator continue to consume memory even after the signal has been disconnected and the QObjects have been discarded.  The script does the following:
  - Instantiates a bunch of SignalObjects, which are QObjects with a signal, and stores the SingalObjects in a list.
  - Instantiates a bunch of SlotObjects, which are QObjects with slots.  Each SlotObject slot is connected to the signals from all the SignalObjects.  The SlotObjects are then immediately discarded.
  - Discards all SignalObjects.
  - Runs gc.collect()
  - Runs the QApplication's event loop for a second in case there are any pending DeferredDelete events.

If the SlotObject class uses the QtCore.pyqtSlot decorator, then memory usage at the end of the script is the same as what it was at the beginning of the script, which makes sense since all the objects that get created should be completely destroyed before the script finishes.  Here's the output that I get with pyqtSlot decorators:
        Memory before signal_objects creation: 17.1MiB
        Memory before slot objects creation: 17.1MiB
                Difference: 0.0B
        Memory after slot objects creation: 17.1MiB
                Difference: 0.0B
        Memory after event loop runs: 17.1MiB
                Difference: 0.0B
If the SlotObject class doesn't use the QtCore.pyqtSlot decorator, though, then memory usage at the end of the script is substantially higher.  Here's the output that I get without pyqtSlot decorators:
        Memory before signal_objects creation: 17.0MiB
        Memory before slot objects creation: 17.0MiB
                Difference: 0.0B
        Memory after slot objects creation: 210.6MiB
                Difference: 193.5MiB
        Memory after event loop runs: 210.6MiB
                Difference: 193.5MiB
Is this expected behavior?  Without pyqtSlot decorators, is there anything I can do to recover the 193 MiB of memory other than terminating the process?  Thanks!

I've tested the script with Python 3.6.2, PyQt 5.12.2, and Qt 5.12.3 on Windows 10, Linux, and Mac OS.  I've also tested with Python 3.6.5, PyQt 5.13.1, and Qt 5.13.1 on Windows 10.  All of them give similar results.  The script requires the psutil package (https://pypi.org/project/psutil/) to monitor memory usage.

- Kevin


import gc
import os

from PyQt5 import QtCore, QtWidgets
import psutil


memory_start = None


class SignalObject(QtCore.QObject):

    mySignal = QtCore.pyqtSignal()


class SlotObject(QtCore.QObject):

    def __init__(self, signal_objects):
        super(SlotObject, self).__init__()

        for cur_signal_obj in signal_objects:
            cur_signal_obj.mySignal.connect(self.my_slot)
            cur_signal_obj.mySignal.connect(self.my_slot2)
            # Immediately disconnecting the signals allows some of the memory to
            # be recovered after the event loop runs, but not much
            # cur_signal_obj.mySignal.disconnect(self.my_slot)
            # cur_signal_obj.mySignal.disconnect(self.my_slot2)

    # @QtCore.pyqtSlot()
    def my_slot(self):
        print("Slot called!")

    # @QtCore.pyqtSlot()
    def my_slot2(self):
        print("Slot2 called!")


def memory_usage():
    process = psutil.Process(os.getpid())
    # with psutils 5.6, uss doesn't work on Windows without elevated permissions
    # return process.memory_full_info().uss
    return process.memory_info().vms


def format_bytes(x):
    for unit in ['', 'Ki', 'Mi', 'Gi', 'Ti', 'Pi', 'Ei', 'Zi']:
        if abs(x) < 1024.0:
            return "%2.1f%sB" % (x, unit)
        x /= 1024.0
    return "%.1fYiB" % x


def create_slot_objects(signal_objects):
    for _ in range(1000):
        SlotObject(signal_objects)


def report_memory(msg):
    mem_now = memory_usage()
    print(f"Memory {msg}: {format_bytes(mem_now)}")
    print(f"\tDifference: {format_bytes(mem_now - memory_start)}")


def report_memory_after_event_loop():
    report_memory("after event loop runs")


def main():
    global memory_start
    app = QtWidgets.QApplication([])
    QtCore.QTimer.singleShot(1000, report_memory_after_event_loop)
    QtCore.QTimer.singleShot(1100, app.quit)
    memory_start = memory_usage()
    print(f"Memory before signal_objects creation: {format_bytes(memory_start)}")
    signal_objects = [SignalObject() for _ in range(100)]
    report_memory("before slot objects creation")
    create_slot_objects(signal_objects)
    del signal_objects
    gc.collect()
    report_memory("after slot objects creation")
    app.exec_()


if __name__ == "__main__":
    main()
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.riverbankcomputing.com/pipermail/pyqt/attachments/20190919/0413763c/attachment.html>