[PyQt] Is PyQt really freeing memory?

[Phil Thompson]

On Tue, 11 Aug 2009 21:10:58 +0200, Albert Cervera i Areny
<albert at nan-tic.com> wrote:
> A Dimarts, 11 d'agost de 2009, Phil Thompson va escriure:
>> On Sun, 2 Aug 2009 02:12:52 +0200, Albert Cervera i Areny
>>
>> <albert at nan-tic.com> wrote:
>> > I've been having problems with my application consuming too much
memory
>> > after
>> > some time running and today decided to take a deeper look. I've ended
>> > up
>> > with
>> > the attach test.py script which either demonstrates PyQt is not
freeing
>> > memory
>> > appropiately when signals are involved or I simply don't understand
how
>> > this
>> > works.
>> >
>> > As you can see the script creates lists with 100.000 QObjects and
>> > prints
>> > the
>> > memory used. As python won't free memory but reuse what has already
>> > been
>> > freed
>> > I expect a call like:
>> >
>> > list = []
>> > fill in the list with lots of data
>> >
>> > to take as much memory as:
>> >
>> > list = []
>> > fill in the list with lots of data
>> > list = []
>> > fill in the list with the same lots of data
>> >
>> > If you give it a try, you'll realize that this is true if you create
>> > 100.000
>> > objects with no signal connections. But if you connect and disconnect 
>> > a
>> > signal for those objects, the memory used after the second fill is
>> > larger
>> > than
>> > after the first one.
>> >
>> > It seems to me that some data is being leaked in connect() and
>>
>> disconnect()
>>
>> > functions (which, by the way, take up a lot of memory).
>> >
>> > Here's the output of the script in my system:
>> >
>> > $ python test.py one no
>> > Executing 'one' without signals
>> > Memory:  21564
>> >
>> > $ python test.py one yes
>> > Executing 'one' with signals
>> > Memory:  64992
>> >
>> > $ python test.py two yes
>> > Executing 'two' with signals
>> > Memory:  125592
>> >
>> > $ python test.py two+remove yes
>> > Executing 'two+remove' with signals
>> > Memory:  93880
>> >
>> > $ python test.py three+remove yes
>> > Executing 'three+remove' with signals
>> > Memory:  122808
>> >
>> > So "two+remove yes" should be using 64992 Kb (just like "one yes") but
>> > it
>> > uses
>> > 50% more. The same happens with "three+remove yes", again more 30 Mb!
>>
>> The proxies that are created to allow Python callables to be used as Qt
>> slots are destroyed using QObject::deleteLater(). As your example
doesn't
>> have an event loop they never get destroyed.
>>
>> In tonight's snapshot I've changed the implementation so they get
>> destroyed
>> immediately as it does simplify the code a little.
> 
> Thank you very much. By the way, is there a reason why creating 100.000 
> objects uses only 21564 Kb and connecting them to a slot uses 64992 Kb.
It 
> seems a lot of memory for "just" connecting signals...

It's not to a slot, it's to a Python callable that needs a proxy to make it
look like a slot. The proxy is bigger than the original object (as it does
more).

You can prevent the creation of a proxy by making sure the "slot" is a real
Qt slot by using the @pyqtSlot decorator.

Phil