[PyQt] PyQt 5.7, QSqlQueryModel.data() sub-classed override bug?

Thu May 3 15:22:11 BST 2018

On 3 May 2018 at 14:54, J Barchan <jnbarchan at gmail.com> wrote:

> Ah ha!  I read in https://www.riverbankcomputing.com/
> pipermail/pyqt/2016-April/037326.html:
>
> >>>>
> * I am currently updating QGIS to PyQt5 (and Qt5 and Python3)>>>> Since this update, NULL QVariant strings are converted to empty strings '' and all numbers to 0 when converted from C++ to python objects.*
>
> And that is exactly what I am experiencing: the string-NULL is giving me
> empty string and the int-NULL is giving me 0.  So isn't this a PyQt issue I
> am seeing after all??
>
> On 3 May 2018 at 14:50, J Barchan <jnbarchan at gmail.com> wrote:
>
>> Hang on.  This is an area I do not understand, doubtless you do.
>>
>> The C++ QSqlQueryModel::data() method is supposed to return a QVariant.
>> Am I maybe losing the QVariant-ness when I write my own PyQt overload
>> which returns the base method's result, because Python-esque conversion is
>> going on?
>>
>> I'm trying to understand http://pyqt.sourceforge.net/Do
>> cs/PyQt5/pyqt_qvariant.html, but I don't really.  I do note:
>> > There is no obvious way to represent a null QVariant
>> <http://pyqt.sourceforge.net/Docs/PyQt5/api/QtCore/qvariant.html#PyQt5-QtCore-QVariant>
>> as a standard Python object.
>>
>> Is there a connection between this and the fact that it goes wrong when
>> the value it should be returning is the NULL returned from the SQL query?
>>
>> On 3 May 2018 at 13:55, J Barchan <jnbarchan at gmail.com> wrote:
>>
>>>
>>>
>>> On 3 May 2018 at 13:05, Phil Thompson <phil at riverbankcomputing.com>
>>> wrote:
>>>
>>>> On 3 May 2018, at 12:25 pm, J Barchan <jnbarchan at gmail.com> wrote:
>>>> >
>>>> > 
>>>> > I am finding (in PyQt 5.7 at least) that sub-classing QSqlQueryModel
>>>> and overriding its data() method produces an incorrect result when the
>>>> value retrieved from a MySQL database is NULL.
>>>> >
>>>> > Full details are in https://forum.qt.io/topic/9036
>>>> 3/inexplicable-qsqlquerymodel-handling-of-null-value, and particularly
>>>> post # https://forum.qt.io/topic/90363/inexplicable-qsqlquerymodel-
>>>> handling-of-null-value.  Nobody has tried it in C++ for me to date to
>>>> verify, but I'm suspecting this might be a PyQt bug?
>>>> >
>>>> > Briefly:
>>>> > My SELECT query returns a column which is NULLable, and has NULL as
>>>> its value.  Where I expect "blank" as the end value, I actually get, for
>>>> example, 0 if the column type is int or '' if the type is string, etc.
>>>> >
>>>> > This is when I sub-class QSqlQueryModel.  If all I have is:
>>>> > class DBQueryModel(QtSql.QSqlQueryModel):
>>>> >     def __init__(self, parent=None):
>>>> >         super().__init__(parent)
>>>> > I get the "NULL"/"blank".  However, as soon as I add just:
>>>> > def data(self, index: QtCore.QModelIndex, role=QtCore.Qt.DisplayRole)
>>>> -> typing.Any:
>>>> >     return super().data(index, role)
>>>> > I get those values instead of NULL.
>>>> >
>>>> > Note that my override is based on the Qt definition of the method at
>>>> http://doc.qt.io/qt-5/qsqlquerymodel.html#data:
>>>> > QVariant QSqlQueryModel::data(const QModelIndex &item, int role =
>>>> Qt::DisplayRole) const
>>>> >
>>>> > Note that the default for role is Qt::DisplayRole.  However, in
>>>> QtSql.py I see:
>>>> > def data(self, QModelIndex, role=None): # real signature unknown;
>>>> restored from __doc__
>>>> >     """ data(self, QModelIndex, role: int = Qt.DisplayRole) -> Any """
>>>> >     pass
>>>> >  You will notice that the comment shows the default should be
>>>> Qt.DisplayRole, but the declaration defaults it to None instead.
>>>>
>>>> What is QtSql.py?
>>>>
>>>> If you want to know the signature of a method pss it to help().
>>>>
>>>> > I don't know enough to be sure, but would that be the underlying
>>>> cause of the unexpected behaviour?
>>>> >
>>>> > FWIW, I have tried making my override be:
>>>> > def data(self, index: QtCore.QModelIndex, role=None)
>>>> > instead, but same bad behaviour.
>>>> >
>>>> > 1. Is this indeed a bug in PyQt, and the cause of my issue?
>>>>
>>>> No and no.
>>>>
>>>> > 2. If so, I presume you (Phil!) will be kind enough to fix.  However,
>>>> for my part I am stuck with PyQt 5.7 for the foreseeable future.  If the
>>>> fix is indeed to change code in the latest/next release, is there anything
>>>> I can do in existing code (my override) to make it work in 5.7, as a
>>>> workaround?  (in real code I need the override, as I do other processing)
>>>> >
>>>> > My coding has come to halt as I cannot proceed without a fix.  So I
>>>> should be obliged for any early response as to whether this is the cause of
>>>> my woes.  I do realise PyQt support/fixes are quite voluntary, and so thank
>>>> whoever in advance!
>>>>
>>>> Phil
>>>
>>>
>>> For QtSql.py:
>>>
>>> Hmm, I had not realised.  I use PyCharm as my IDE.  From there, while
>>> I am coding, I can click on anything PyQt and ask for "Go to
>>> definition/declaration".  The editor then 9in this case) opens me up into a
>>> file named QtSql.py, showing me in this case [extract]:
>>>
>>> class QSqlQueryModel(__PyQt5_QtCore.QAbstractTableModel):
>>>>
>>>> ...
>>>>
>>>> def columnCount(self, parent=None, *args, **kwargs): # real signature unknown; NOTE: unreliably restored from __doc__
>>>>     """ columnCount(self, parent: QModelIndex = QModelIndex()) -> int """
>>>>     pass
>>>>
>>>> def data(self, QModelIndex, role=None): # real signature unknown; restored from __doc__
>>>>     """ data(self, QModelIndex, role: int = Qt.DisplayRole) -> Any """
>>>>     pass
>>>>
>>>> def endInsertColumns(self): # real signature unknown; restored from __doc__
>>>>     """ endInsertColumns(self) """
>>>>     pass
>>>>
>>>>
>>> etc.  I had *assumed* this was a file supplied with PyQt.  I guess now
>>> it's "generated on the fly" by PyCharm (I see its path is in a PyCharm
>>> temporary directory).  At other times, it might open, say,
>>> /usr/lib/python3/dist-packages/PyQt5/QtCore.pyi, which I think is a
>>> file you supply.  Oh, at the head of this QtSql.py I see:
>>>
>>> # encoding: utf-8
>>> # module PyQt5.QtSql
>>> # from /usr/lib/python3/dist-packages/PyQt5/QtSql.cpython-35m-x86_64-linux-gnu.so
>>> # by generator 1.145
>>> # no doc
>>>
>>>
>>> You'll probably understand all this better than I!
>>>
>>>
>>>
>>> For my problem:
>>>
>>> I think I now understand better why it's not a PyQt method definition
>>> issue.
>>>
>>> However, from the linked Qt forum discussion, I'm stuck between a rock &
>>> a hard place, because the only help I'm getting is that it might be a PyQt
>>> issue.  I do not have C++ to try that out.  So, I wonder if I might ask you
>>> if you can make any suggestion as to the cause, even if it is not a PyQt
>>> issue, given that you are familiar with Qt at least?
>>>
>>> To summarise my problem as briefly as possible:
>>>
>>> 1.
>>> I start with:
>>>
>>> model = QtSql.QSqlQueryModel(self)
>>> model.setQuery("SELECT LandlordNo, SMTPAccountId FROM landlords WHERE SMTPAccountId IS NULL")
>>>
>>> # or plain "SELECT NULL AS SMTPAccountId", to eliminate anything about the column definition being an issue
>>>
>>> rowCount = model.rowCount()
>>> if rowCount > 0:
>>>     rec = model.record(0)
>>>     field = rec.field("SMTPAccountId")
>>>     isn = field.isNull()
>>>
>>> SMTPAccointId returns NULL from MySQL.  *At this point field.isNull()
>>> correctly returns True.*
>>>
>>> 2.
>>> I sub-class QSqlQueryModel, and use that, with quite simply, exactly:
>>>
>>> class DBQueryModel(QtSql.QSqlQueryModel):
>>>     def __init__(self, parent=None):
>>>         super().__init__(parent)
>>>
>>> and use that sub-class in place of QSqlQueryModel:   model =
>>> DBQueryModel(self)
>>> *And it this point point field.isNull() *still* correctly returns
>>> True.*
>>>
>>> 3.
>>> Then I add *just exactly this* to my sub-class:
>>>
>>> def data(self, index: QtCore.QModelIndex, role=QtCore.Qt.DisplayRole) -> typing.Any:
>>>     return super().data(index, role)
>>>
>>> (I've also tried role=None) You can see that simply calls the base
>>> class method.  *But now field.isNull() returns False!!*  The
>>> application sees 0 instead of NULL for the value (SMTPAccountId is
>>> declared INT NULL), or '' if I use SELECT NULL AS SMTPAccountId so it
>>> counts as string.
>>>
>>>
>>> This leaves me completely stumped.  I have no idea where the problem is
>>> (there shouldn't be a problem!).  I have to sub-class the data() method
>>> for other purposes, but then it handles NULL (only) incorrectly.
>>>
>>> Would you have any idea what is going on here?  My thanks in advance.
>>>
>>>
>>> --
>>> Kindest,
>>> Jonathan
>>>
>>
>>
>>
>> --
>> Kindest,
>> Jonathan
>>
>
>
>
> --
> Kindest,
> Jonathan
>

Hey Phil,

Assuming I am not barking up the wrong tree, I see you yourself were
discussing this issue in
http://python.6.x6.nabble.com/PyQt5-NULL-QVariant-tp5188782p5190811.html.
It includes:

> Can you confirm that returning empty string and 0 for NULL is expected
> > behavior and not seen as a bug?
>
> Yes. If you want to distinguish null QVariants then use
> sip.enableautoconversion(). The problem is that you are not willing to do
> that.
>

So I'm bumbling around trying to put a sip.enableautoconversion() in!
Trouble is, I don't understand the syntax, and I don't know where to put it
(I'm not finding any examples, and you two obviously knew more than me
about it :) )

1. I'm *assuming* I want to suppress converting QVariants.  I'm trying

sip.enableautoconversion(QVariant, False)

but it doesn't know what QVariant is, and I don't know if it's supposed to
or my syntax or imports or whatever?

2. I don't know *where* I'm supposed to disable & re-enable the
autoconversion?  My method is an override called implicitly by Qt.
* Is this a "directive" that's supposed to be used e.g. as a Python method
is read in, and so belongs around the whole function?
* Is this a "run-time* that I just need to put inside my method overload
around where it calls the base class method?  Or by the time it has hit my
overload is it too late because some conversion has already happened?
* Do I actually need to put it around the whole of my top-level call to,
say, QSqlQueryModel.rowCount(), which is what (I understand to be) invoking
calls to the overridden QSqlQueryModel.data() method?

Many, many thanks for your time & patience.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.riverbankcomputing.com/pipermail/pyqt/attachments/20180503/5926683f/attachment-0001.html>