[PyQt] pyqtdeploy, Android and extensions... SOLVED!

[Robert Kent]

So, to finally answer my own question:

Android (unlike normal *nix) will only read symbols from the library 
that was specified during linking, i.e. I linked my numpy modules 
against libpython3.4 and then tried to use those exported by 
libpyqt5_android_test (my application). If I first build my application, 
then rebuild numpy linking against the output .so object then everything 
works fine, both in an android emulator and on a (unrooted) real device.

The next joy is packaging the extension .so modules into the apk. After 
trying every possible (wrong?) option I've finally done this by renaming 
them lib<module_name>.so and adding them to the ANDROID_EXTRA_LIBS 
variable in my .pro file. I've then installed an import hook and used 
importlib.machinary.ExtensionFileLoader to load them as required from 
the applications lib directory.

If anybody else is trying to do this and would like a more detailed 
explanation/examples then just get in touch, I will be happy to help you 
avoid this painful learning experience.

Thanks, Rob

On 12/04/2016 10:59, Robert Kent wrote:
> Sorry to bump my own threads again...
>
> Same issue I was experiencing yesterday persists, namely that the 
> extension module that I am trying to load is unable to find symbols 
> that should be available through my android application, thus:
>
> ImportError:
>   dlopen failed: cannot locate symbol "PyMem_Free" referenced by 
> numpy.core.multiarray.so
>
> I have checked that my application (libpyqt5_android_test.so) has the 
> required, unmangled symbol using nm. I have also checked (to the best 
> of my ability/knowledge) that this isn't some red herring caused by 
> the library that I am loading i.e a different error masked by python. 
> I have also tried loading the library by a) placing it next to the app 
> and letting pyqtdeploy load it, b) loading it via imp.load_module and 
> c) loading it using ExtensionFileLoader as well as changing the .so 
> files location in case this was a factor.
>
> My question is this: why can my library not find symbols that, in 
> theory, should be available from the object calling it?
>
> Any thoughts or suggestions on how to either fix or find out more 
> information would be really appreciated.
>
> Thanks very much, Rob
>
> On 11/04/2016 12:38, Robert Kent wrote:
>> If I run 'nm' on my applications output .so file then I can see that 
>> 'PyMem_Free' is exported as an unmangled symbol, so why can the numpy 
>> .so being imported not find it during dlopen()?
>>
>> Any one got any experience with this?
>>
>> All help and suggestions appreciated.
>>
>> Thanks, Rob
>>
>> On 11/04/2016 11:45, Robert Kent wrote:
>>> Ok, so a kind of success! If I add HAVE_DYNAMIC_LOADING to 
>>> pyconfig.h then it loads the .so file... finally!
>>>
>>> However, it now complains that it can't find a symbol...
>>>
>>> ImportError:
>>>   dlopen failed: cannot locate symbol "PyMem_Free" referenced by 
>>> numpy.core.multiarray.so
>>>
>>> Any thoughts anyone?
>>>
>>> On 11/04/2016 10:41, Phil Thompson wrote:
>>>> On 11 Apr 2016, at 10:31 am, Robert Kent <rob at gulon.co.uk> wrote:
>>>>> I definitely went back and re-compiled Python on Friday to ensure 
>>>>> I had used this switch... Which defines etc. should be set by this 
>>>>> flag to allow C extension loading?
>>>> It should define HAVE_DYNAMIC_LOADING and also cause 
>>>> dynload_shlib.c to be compiled and linked.
>>>>
>>>> Phil
>>>>
>>>>> On 11/04/2016 10:28, Phil Thompson wrote:
>>>>>> On 11 Apr 2016, at 10:22 am, Robert Kent <rob at gulon.co.uk> wrote:
>>>>>>> Hi Phil,
>>>>>>>
>>>>>>> Sorry to bump a thread, but anyone got any more ideas re. cause 
>>>>>>> or anything else I can attempt? Why would it be that python 
>>>>>>> (imp) doesn't know how to load C extensions as the traceback 
>>>>>>> implies? Are there any defines that need changing in pyconfig 
>>>>>>> for example to allow this?
>>>>>> Yes, but this should be taken care of when using 
>>>>>> --enable-dynamic-loading with pyqtdeploycli.
>>>>>>
>>>>>> Phil
>>>
>>> _______________________________________________
>>> PyQt mailing list    PyQt at riverbankcomputing.com
>>> https://www.riverbankcomputing.com/mailman/listinfo/pyqt
>>
>> _______________________________________________
>> PyQt mailing list    PyQt at riverbankcomputing.com
>> https://www.riverbankcomputing.com/mailman/listinfo/pyqt
>
> _______________________________________________
> PyQt mailing list    PyQt at riverbankcomputing.com
> https://www.riverbankcomputing.com/mailman/listinfo/pyqt