[PyKDE] New PyKDE Release (alpha6) and Killing PyKDE 3.8

[Jim Bublitz]

On Monday April 26 2004 23:22, Simon Edwards wrote:
> On Sunday 25 April 2004 00:01, Jim Bublitz wrote:
> > On Saturday April 24 2004 13:01, Simon Edwards wrote:
> > > On Friday 23 April 2004 18:46, Jim Bublitz wrote:
> > > > I have disowned libpythonize :) Who's going to be
> > > > responsible for that is kind of up in the air at the
> > > > moment.

> > > I really need it, so it looks like I just got the job.

> > Feel free to do anything you want with it - there isn't much
> > to it to begin with, and what little there is can probably
> > be organized better.

> I've been working with libpythonize the last few days and I
> reakon I've figured out its "mysteries". You may or may not
> recall that for things like the Python kcontrol modules there
> was a problem with 'import'ing some modules. Namely the *.so
> for a module would not load into python when everything is
> embedded. You would get an annoying "Import Error: unable to
> resolve symbol PyFooBar" etc. David didn't know why that
> happened, but he had a workaround, link any module *.so files
> you need with the kcontrol stub (like
> /usr/lib/python2.3/dyn-load/time.so if needed the 'time'
> module). That worked, and you could import modules, but David
> didn't know why it worked.

> Anyway, answer lies on the dlopen() man page:

>     Optionally, RTLD_GLOBAL may be or'ed with flag,  in  which
>  case the  external  symbols defined in the library will be
> made available to subsequently loaded libraries.

> When kcontrol loads a module, it uses dlopen() load in the our
> python stub. The stub loads and libpython gets pulled in too.
> Then later the python interpreter uses dlopen() too to load
> time.so (or some other module). Then it fails. The reason
> being that kcontrol does _not_ use RTLD_GLOBAL when opening
> our module stub, so libpython gets loaded but its symbols are
> _not_ made available to the next dlopen() call (the one for
> the module import). That is why an imported module can't
> resolve symbols from libpython.
>
> Now, why does David's workaround actually work? When you link
> the kcontrol module stub against the time.so python module,
> the linker specifies in the generated module stub that time.so
> should also be loaded when loading the stub (along with
> libpython). So when kcontrol loads the stub, libpython gets
> pulled in and also time.so _at the same time_. And everything
> resolves correctly. Later python uses dlopen() to load
> time.so. dlopen() sees that time.so is already loaded and just
> happily goes on its merry way and life is good...

> For bonus points: Is it possible to make a kcontrol module
> stub that loads libpythonize and libpython using dlopen(), but
> with the RTLD_GLOBAL flag set so that dlopen() in the python
> interpreter will work as expected? I guess I'll have to try
> that out... :-)

> (BTW, linking with -export-dynamic -Wl,-E doesn't seem to have
> much effect)

I'm not sure if this is the same situation (I think it is), but 
what I did when I originally played around with this is to write 
a module (.so) that KSpread would load and in turn that module 
loaded libpythonize. The second load was done using 
KLibLoader::globalLibrary (loading libpythonize linked with -Wl, 
E), and that solved the problem.

I believe it's the code *doing* the loading via dlopen that needs 
RTLD_GLOBAL set, and that's what KLibLoader::globalLibrary does. 
That would mean modifying the KDE loading code, not our libs, to 
load in a "single stage" (ie libpythonize linked in, not loaded 
in).

David originally started with *no* libpythonize - he just added 
the Python calls to his lib and linked in libpython.a. We 
reorganized it to use libpythonize, but linked it rather than 
explicitly loading it. Maybe trying to load it instead of 
linking to it would be worth a try.

The other possibility is to try using libtool, but I don't know 
enough about libtool to enjoy using it (or to know if it would 
solve anything), and there are version problems with it.

Jim