[review] [PyKDE] PyQT module size

[Jim Bublitz]

On Friday 18 February 2005 06:38, Brian Thomason wrote:
> We've been using PyQT/PyKDE here at Linspire for some time now to
> develop a handful of applications.  One of these is Lsongs, and it has
> grown quite large and has a very large memory footprint.  We're trying
> to reduce this a bit in various ways (switching to gstreamer from
> libxine for instance) and I was curious if there was a way to reduce the
> footprint size of pyqt/pykde.  The shared object files themselves are
> quite large - Larger than their C++ counterparts.

> I'm totally ignorant on python bindings so this may just be "the way it
> is" and that's fine.  Just curious if there's a way to "shrink" these
> down a bit.

Short answer: no.

Long answer: I agree that if you look at the module sizes as they sit on disk, 
it looks pretty horrendous. A simple KDE plugin program written using PyKDE 
looks like it will use something in the 15-20MB range if you add up the .so 
sizes. In reality though, the actual memory usage, especially since KDE is 
already running and its and Qt's libs are already loaded, isn't all that bad. 

I seem to recall the actual overhead for a fairly large program using several 
PyQt modules and 5 or 6 PyKDE modules  is somewhere around 6-8MB according to 
'free'. Another way to look at it is that the *first* app has a lot of 
overhead, but each successive PyQt/PyKDE app has very little - use them 
more :) Still another way to view it is that getting to QApplication or 
KApplication is very expensive, but everything after that is really cheap.

Most of the code is pretty much a fixed amount of overhead per method and 
that's written very efficiently by sip. The tradeoff then becomes size of 
code vs. completeness of code - ie, the only way to make PyQt or PyKDE 
smaller is by throwing out methods and classes. Throwing out methods is 
possible; throwing out classes can eventually become a dependency nightmare.

In either case, it's pretty hard to take a general purpose development system 
and predict what users will find useful or necessary down the road, so the 
philosophy (at least for PyKDE) has been to include everything that can 
possibly be included. Another solution would be to make more and smaller 
modules, but then it becomes a problem for users to figure out which module 
contains which classes and methods (and it's not an easy task for us either).

I've discussed this in the past, and the conclusion is usually "well, nobody 
ever complains about it" - this is literally (to my knowledge) the first 
complaint we've ever had. It's a question of how much you're willing to "pay" 
for the convenience of writing Qt/KDE code in Python, and it seems like the 
"market price" in KB is pretty high (and memory is pretty cheap).

There are ways that a custom set of bindings could be developed for your 
applications that only expose the methods/classes from PyQt/PyKDE that you 
really need. It would involve writing a custom set of C++ wrappers for the 
PyQt/PyKDE APIs and then binding those. Depending on what you need, that 
could be as small as a few hundred K for everything. For an application of 
any complexity (eg menus/toolbar/statusbar and a selection of various 
widgets, std file dialogs, etc) it might turn out to be quite a bit of work 
though, and the resulting names wouldn't "look" like Qt/KDE anymore.

You could also try taking the PyQt/PyKDE and throwing stuff out - that might 
save 30% or so, depending on what you need to leave in. However, when you do 
a menu using actions (esp standard actions) in KDE, for example, there's a 
lot of underlying code that you never see that PyKDE needs to bind (that's 
what a wrapper can get rid of).

Lastly, it seems to me there's probably a different approach that could access 
the Qt/KDE C++ libs without the per method overhead and be quite small, but 
it's likely to execute considerably slower and be pretty complex, esp when 
you get into things like virtual method overloading or mapping types between 
Python and C/C++. The latter might even destroy the initial size advantage. 
I'm also not sure if that would be possible if the symbols are stripped from 
the .so libs being accessed - don't know enough about .so structure to really 
say.

Jim