[PyKDE] Re: Using numpy/numarray in custom C++ code and exposing with sip

[Gerard Vermeulen]

PyQwt only copies data from numpy/numarray arrays to C++, not the other way
around. I explain what I would do (look at the code in CVS --
http://cvs.sourceforge.net/viewcvs.py/pyqwt/pyqwt/ -- or get the snapshot from
http://pyqwt.sourceforge.net/snapshot/PyQwt-20040118.tar.gz )

You have to import the C-API's in the main sip file. In sip/qwtmod.sip this
is done with:

%PostInitialisationCode
#ifdef HAS_NUMARRAY
    qwt_import_libnumarray();
#endif
#ifdef HAS_NUMERIC
    qwt_import_array();
#endif
%End

The qwt_import_* functions are defined in separate files in the numpy directory
for two reasons:
1. Name collisions in the header files that define the C-API of numpy and
   numarray (see below)
2. I do not like C++ code in sip files because it takes longer to rebuild
   after correct errors. 

The file sip/qwtcurve.sip shows how to copy data from a numpy array, numarray
array or normal Python sequence of doubles to a QwtArray:

    void setData(SIP_PYOBJECT, SIP_PYOBJECT);
%MethodCode
        QwtArray<double> xArray;
        if (-1 == try_PyObject_to_QwtArray(a0, xArray))
            return 0;

        QwtArray<double> yArray;
        if (-1 == try_PyObject_to_QwtArray(a1, yArray))
            return 0;

        Py_BEGIN_ALLOW_THREADS
        sipCpp->setData(xArray, yArray);
        Py_END_ALLOW_THREADS
%End

Again, try_PyObject_to_QwtArray delegates the C-API specific stuff to functions
in separate files. Note that at least the numpy conversion function makes it
unsafe to allow threading during the conversion (recent versions of numarray may
be thread safe).

I would try to write a getData() function like this:

    SIP_PYOBJECT getData()
%MethodCode
    Create a numerical Python extension array and assign it to a pointer
    Copy the data into it
    Return the pointer to the array 
%End

Maybe you have to replace SIP_PYOBJECT with void *.
The Numeric or numarray docs show to implement the pseudo code.

Gerard

On Fri, 06 Feb 2004 11:19:15 -0800
"Claus, Richard" <claus at SLAC.Stanford.EDU> wrote:

> Hi Ulrich & Gerard,
> 
>   Your discussion about numpy/numarray usage by pyQwt piqued my interest.  Could one of you please post a snippet of code showing how one exposes an array of, say, unsigneds, in custom C++ code as a numpy and/or numarray array in python via sip, or point me to where I can learn how to do this?  I tried to discover how to do this once but gave up due to time constraints.
> 
>   Thanks very much.
> 
> 	Ric
> 
> 
> > -----Original Message-----
> > From: Gerard Vermeulen [mailto:gvermeul at grenoble.cnrs.fr] 
> > Sent: Monday, February 02, 2004 12:00 AM
> > To: ulrich.berning at t-online.de
> > Cc: pykde at mats.imk.fraunhofer.de
> > Subject: Re: [PyKDE] How to add handwritten *.cpp files to a 
> > sip generated module with configure.py
> > 
> > 
> > On Mon, 02 Feb 2004 00:15:35 +0100
> > ulrich.berning at t-online.de (Ulrich Berning) wrote:
> > 
> > > Gerard Vermeulen schrieb:
> > > 
> > > >Hi Phil,
> > > >
> > > >Ulrich Berning kindly provided me with code to build PyQwt with 
> > > >configure.py.
> > > >
> > > >One of his changes was to put a few handwritten *.cpp files to 
> > > >interface the Numerical Python extension modules (numpy 
> > and numarray) 
> > > >in the qwtmod.sip file.
> > > >
> > > >The problem is that there are conflicts between the header 
> > files of 
> > > >numpy and numarray (numarray should become the successor 
> > of numpy, so 
> > > >there is an overlap in their API). Putting all the 
> > interface code in 
> > > >the qwtmod.sip file leads to compile errors because of the 
> > inclusion 
> > > >of the conflicting headers in the same .cpp file.
> > > >
> > > >Ulli did not see it, because he is only having numpy, I guess.
> > > >
> > > >  
> > > >
> > > Yes, this is true. I can't see why I should have numpy AND numarray.
> > > Because numarry will be the successor of numpy someday, but 
> > currently it 
> > > is not, I choosed to stay with numpy. The only reason to 
> > support numpy 
> > > AND numeric could be to provide a binary module that works 
> > either with 
> > > numpy or with numarray or with none of them. But then, the 
> > interface 
> > > code needs some modifactions. You have to find out in the 
> > init function 
> > > if numpy or numarray or none of both is available. Simply calling 
> > > import_array() and/or import_libnumarray() does not work. 
> > If you call 
> > > import_libnumarray() when nummarray is not installed you get a 
> > > Py_FatalError().
> > > 
> > > And at least, if I take a look at Numeric/arrayobject.h and
> > > numarray/arrayobject.h, I can see the following:
> > > 
> > > in Numeric/arrayobject.h:
> > > #ifndef Py_ARRAYOBJECT_H
> > > #define Py_ARRAYOBJECT_H
> > > 
> > > in numarray/arrayobject.h:
> > > #ifdef Py_ARRAYOBJECT_H
> > > #error "Can't use numarray Numeric compatability *and* 
> > Numeric in same
> > > module"
> > > #endif
> > > 
> > > My recommendation is, either numpy is used, if it is there 
> > OR numarray
> > > if it is there OR none of them. Using the numpy API AND the 
> > numarray API 
> > > in the same module may work, but was not intended by the Numerical 
> > > Python Group and seems to be dangerous.
> > > 
> > PyQwt is able to import both, since version 3.7 and since 
> > version 3.8 it includes numarray/libnumarray.h. It really 
> > works in the same Python program but it requires that calls 
> > to the Numeric API and numarray API are split into different 
> > source files (importing a C API from a C/C++ extension module 
> > is nothing else than setting up a table with pointers to 
> > functions into the module).
> > 
> > I provide the possibility to use both so that people can make 
> > the transition. It also facilitates support and testing. 
> > 
> > The purpose of my mail was to point out a facility that is 
> > lacking from Phil's new build system (but I can hack my way 
> > out).  It explains also why I am sticking so long with my own 
> > tools which are lacking in other respects.
> > 
> > Gerard
> > 
> > _______________________________________________
> > PyKDE mailing list    PyKDE at mats.imk.fraunhofer.de
> > http://mats.imk.fraunhofer.de/mailman/listinfo/pykde
> > 
>