[PyQt] Is PyQt really freeing memory?

[Albert Cervera i Areny]

A Dimarts, 11 d'agost de 2009, Phil Thompson va escriure:
> On Tue, 11 Aug 2009 21:10:58 +0200, Albert Cervera i Areny
>
> <albert at nan-tic.com> wrote:
> > A Dimarts, 11 d'agost de 2009, Phil Thompson va escriure:
> >> On Sun, 2 Aug 2009 02:12:52 +0200, Albert Cervera i Areny
> >>
> >> <albert at nan-tic.com> wrote:
> >> > I've been having problems with my application consuming too much
>
> memory
>
> >> > after
> >> > some time running and today decided to take a deeper look. I've ended
> >> > up
> >> > with
> >> > the attach test.py script which either demonstrates PyQt is not
>
> freeing
>
> >> > memory
> >> > appropiately when signals are involved or I simply don't understand
>
> how
>
> >> > this
> >> > works.
> >> >
> >> > As you can see the script creates lists with 100.000 QObjects and
> >> > prints
> >> > the
> >> > memory used. As python won't free memory but reuse what has already
> >> > been
> >> > freed
> >> > I expect a call like:
> >> >
> >> > list = []
> >> > fill in the list with lots of data
> >> >
> >> > to take as much memory as:
> >> >
> >> > list = []
> >> > fill in the list with lots of data
> >> > list = []
> >> > fill in the list with the same lots of data
> >> >
> >> > If you give it a try, you'll realize that this is true if you create
> >> > 100.000
> >> > objects with no signal connections. But if you connect and disconnect
> >> > a
> >> > signal for those objects, the memory used after the second fill is
> >> > larger
> >> > than
> >> > after the first one.
> >> >
> >> > It seems to me that some data is being leaked in connect() and
> >>
> >> disconnect()
> >>
> >> > functions (which, by the way, take up a lot of memory).
> >> >
> >> > Here's the output of the script in my system:
> >> >
> >> > $ python test.py one no
> >> > Executing 'one' without signals
> >> > Memory:  21564
> >> >
> >> > $ python test.py one yes
> >> > Executing 'one' with signals
> >> > Memory:  64992
> >> >
> >> > $ python test.py two yes
> >> > Executing 'two' with signals
> >> > Memory:  125592
> >> >
> >> > $ python test.py two+remove yes
> >> > Executing 'two+remove' with signals
> >> > Memory:  93880
> >> >
> >> > $ python test.py three+remove yes
> >> > Executing 'three+remove' with signals
> >> > Memory:  122808
> >> >
> >> > So "two+remove yes" should be using 64992 Kb (just like "one yes") but
> >> > it
> >> > uses
> >> > 50% more. The same happens with "three+remove yes", again more 30 Mb!
> >>
> >> The proxies that are created to allow Python callables to be used as Qt
> >> slots are destroyed using QObject::deleteLater(). As your example
>
> doesn't
>
> >> have an event loop they never get destroyed.
> >>
> >> In tonight's snapshot I've changed the implementation so they get
> >> destroyed
> >> immediately as it does simplify the code a little.
> >
> > Thank you very much. By the way, is there a reason why creating 100.000
> > objects uses only 21564 Kb and connecting them to a slot uses 64992 Kb.
>
> It
>
> > seems a lot of memory for "just" connecting signals...
>
> It's not to a slot, it's to a Python callable that needs a proxy to make it
> look like a slot. The proxy is bigger than the original object (as it does
> more).
>
> You can prevent the creation of a proxy by making sure the "slot" is a real
> Qt slot by using the @pyqtSlot decorator.

Thanks again Phil. I modified the test application and run several tests to 
compare how the several options affect to memory usage and performance. Here 
are the results in my computer.

                    MEMORY     TIME
Without signals     21572Kb    0.96s 
With slots          32576Kb    4.07s
Python Callable     64988Kb    5.25s

Maybe it'd be good to add to the reference guide that using decorators is both 
faster and more memory efficient.

>
> Phil


-- 
Albert Cervera i Areny
http://www.NaN-tic.com
Mòbil: +34 669 40 40 18
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.py
Type: text/x-python
Size: 1468 bytes
Desc: not available
Url : http://www.riverbankcomputing.com/pipermail/pyqt/attachments/20090811/749872ff/test.py