[PyQt] PyQt 5.7, QSqlQueryModel.data() sub-classed override bug?

[Phil Thompson]

On 3 May 2018, at 5:05 pm, J Barchan <jnbarchan at gmail.com> wrote:
> 
> 
> 
> On 3 May 2018 at 16:17, Phil Thompson <phil at riverbankcomputing.com> wrote:
> On 3 May 2018, at 3:22 pm, J Barchan <jnbarchan at gmail.com> wrote:
> > Hey Phil,
> > 
> > Assuming I am not barking up the wrong tree, I see you yourself were discussing this issue in http://python.6.x6.nabble.com/PyQt5-NULL-QVariant-tp5188782p5190811.html.  It includes:
> > 
> > > Can you confirm that returning empty string and 0 for NULL is expected 
> > > behavior and not seen as a bug? 
> > 
> > Yes. If you want to distinguish null QVariants then use sip.enableautoconversion(). The problem is that you are not willing to do that. 
> > 
> > So I'm bumbling around trying to put a sip.enableautoconversion() in!  Trouble is, I don't understand the syntax, and I don't know where to put it (I'm not finding any examples, and you two obviously knew more than me about it :) )
> > 
> > 1. I'm assuming I want to suppress converting QVariants.  I'm trying 
> > ​​sip.enableautoconversion(QVariant, False)
> > but it doesn't know what QVariant is, and I don't know if it's supposed to or my syntax or imports or whatever?
> 
> ​​Import QVariant from QtCore.
> 
> > 2. I don't know where I'm supposed to disable & re-enable the autoconversion?  My method is an override called implicitly by Qt.
> > * Is this a "directive" that's supposed to be used e.g. as a Python method is read in, and so belongs around the whole function?
> > * Is this a "run-time* that I just need to put inside my method overload around where it calls the base class method?  Or by the time it has hit my overload is it too late because some conversion has already happened?
> > * Do I actually need to ​​put it around the whole of my top-level call to, say, QSqlQueryModel.rowCount(), which is what (I understand to be) invoking calls to the overridden QSqlQueryModel.data() method?
> 
> ​​I think you need to do the latter - at least as a workaround for the moment. I ​​need to add a directive to SIP to disable the auto-conversion during the code that handles the virtual and calls the Python re-implementation.
> 
> Phil
> 
> ​Hi Phil,
> 
> Right, I think we're in business, here goes...!​
> 
> First, I took your *original* response of "No and no." to indicate that my issue would have nothing to do with PyQt.  In the end I think we are agreeing it has turned out to be a PyQt issue after all.
> 
> Second, yes, I figured "​Import QVariant from QtCore" was what I was missing.  (Not helped by PyCharm warning me it really wanted a "sip wrappertype" for sip.enableautoconversion() instead of a "type", but it works OK at run-time.)  I actually interpreted http://pyqt.sourceforge.net/Docs/PyQt5/pyqt_qvariant.html:
> 
> v2 (the default for Python v3) does not expose the QVariant class to Python
> 
> ​as meaning that QVariant was not going to be defined/exported in any PyQt file​ at all.  But I see it still is.
> 
> Third, in view of your "​I think you need to do the latter [​put ​sip.enableautoconversion() around the whole of my top-level call] - at least as a workaround for the moment.", I started out doing that.  However, I found that the data() method is being called all over the place (e.g. throughout the QTableView displaying the model), which I guess makes sense, so that was soon a non-starter.  In any case it would be way-dangerous to have to change QVariant behaviour across a large area of code.  I would hugely prefer to only have to do so across the call to the base method from the derived method, and to my pleasure it seems to be working just fine like that.  So my override (to give you a flavor of why I need it) now actually reads:
> 
> def data(self, index: QtCore.QModelIndex, role=QtCore.Qt.DisplayRole) -> typing.Any:
>     DBModelCommon.dataIndexIsValid(index)    # do some checking that index is valid
> 
>     import sip
>     sip.enableautoconversion(QtCore.QVariant, False)
>     value = super().data(index, role)        # wrap call to QSqlQueryModel.data() inside no-convert-QVariant-to-Python
>     sip.enableautoconversion(QtCore.QVariant, True)
> 
>     return DBModelCommon.dataValue(value, role)    # do some (potential) post-processing on the value returned

A more resilient version would be...

    was_enabled = sip.enableautoconversion(QtCore.QVariant, False)
    ...
    sip.enableautoconversion(QtCore.QVariant, was_enabled)

> ​You will understand this better than I, but the implication is that the problem is not in my own Python definition of this override itself already doing a conversion and going wrong, but rather at the time it calls the base function and grabs its return value into Python?
> 
> So far all I can say is that from what i have seen it now handles my values --- both NULLs and non-NULLs --- ​correctly! :)

> Finally, I'm interested to understand what is going on here, and why you "​need to add a directive to SIP to disable the auto-conversion during the code that handles the virtual and calls the Python re-implementation". 

As you have shown it is not necessary in this case. However you would have problems for virtual re-implementations that are passed a QVariant as an argument rather than being passed back as a result.

> I'm assuming the override was returning a QVariant-converted-to-Python-type, then something about other (Qt) C++ code calling the override and not being happy about the return type, or something?  So how do you (or I) know whether there are going to be other cases (overrides) which, say, return or pass or receive a QVariant (which could be a NULL one), and will also fall foul of this issue?  Do you/I need to look through every Qt function?  Have you already done this and are confident that there can be only a few remaining which you have not spotted?  Is that how it works?

Very few contexts in Qt care about null QVariants - this may well be the only one. (An invalid QVariant - which mapped to None - is much more common.)

Phil