[PyKDE] PyQT module size

[Rob Knapp]

On Thu, 2005-02-17 at 11:47 -0800, Brian Thomason wrote:
> We've been using PyQT/PyKDE here at Linspire for some time now to 
> develop a handful of applications.  One of these is Lsongs, and it has 
> grown quite large and has a very large memory footprint.  We're trying 
> to reduce this a bit in various ways (switching to gstreamer from 
> libxine for instance) and I was curious if there was a way to reduce the 
> footprint size of pyqt/pykde.  The shared object files themselves are 
> quite large - Larger than their C++ counterparts. 

I fought this battle while at a previous position (admittedly on
windows, not linux).  The only solution I could find was to compile QT
with the absolute minimum set of classes required for my application.
This got QT down to about 2Megs and PyQT down to about 3.5 or so. 

I can't remember the exact figures, but it doesn't matter since you are
using a different compiler.  I do recall that it was more than half for
me, but I wasn't using several large modules.  The other horrible hack
you can do to change this is to manually edit the main sip file and
remove (or comment out) the includes for the classes you don't need.
This gives you a fully functional QT, but a hobbled pyqt.  (And then I
would suggest changing the name of qt.pyd to linQt.pyd or some such so
that users can install their own full version of py qt....)

We also hit memory foot print issues, and the first thing I will say is
watch your python strings!  string A + string B  creates a new string,
so concatenating using the "+" operator uses a lot more memory than you
think.  Once you've done that, look on the web for options on optimizing
memory with python.  There are plenty of resources.  

If you are still having memory problems, the only thing I can suggest
(and this is what we did), is explictly delete as many variables as you
can.  This also can help.  

If you get _really_ desperate, you can force garbage collection in a
seperate (python) thread, but that lead to some stability issues for me.

Finally, if you really are at the end of the loop, if there is a call
like win32's SetWorkingSetSize that writes "unused" objects to virtual
memory, you can sprinkle that at strategic locations or even in a
background loop. (Probably need to expose whatever call it would be.)
Of course there is a pretty large performance hit here, but it can be
very effective in freeing up physical ram for other apps.  It also isn't
cross-platform, but considering that you work for a linux company, I
doubt that is the end of the world. :)

In my situation (a video conferencing suite), this was very useful as my
UI application's performance was virtually irrelevant as along as the
video and networking apps were running well.  (long story, everything
was in a seperate app so if one part failed, the conference could
continue).

None of this is probably what you want to hear.  AFAICT, there is no
magic bullet that shrinks PyQTs memory down.  What I can say if applying
everything except the SetWorkingSetSize, we went from about 50 Megs
while in conference to about 20 Megs while in conference.  We could have
saved more in a couple of areas specific to our application (the
skinning engine, and our interactions with zip files), but money ran out
before we could do that.

> I'm totally ignorant on python bindings so this may just be "the way it 
> is" and that's fine.  Just curious if there's a way to "shrink" these 
> down a bit.
> 
> Thanks,
> 
> -Brian
> --
> 
> This message contains information which may be confidential and privileged. Unless you are the 
> addressee (or authorized to receive for the addressee), you may not use, copy or disclose to anyone 
> the message or any information contained in the message. If you have received the message in error, 
> please advise the sender and delete the message.  Thank you.
> 
> _______________________________________________
> PyKDE mailing list    PyKDE at mats.imk.fraunhofer.de
> http://mats.imk.fraunhofer.de/mailman/listinfo/pykde
-- 
---
Rob "Myddrin" Knapp
myddrin at myddrin.com
http://www.myddrin.com