[PyQt] QSortFilterProxyModel::sort() not sorting on column types

[J Barchan]

On Thu, 7 Feb 2019 at 23:22, Maurizio Berti <maurizio.berti at gmail.com>
wrote:

> Any kind of object can be assigned to a (any) DataRole to a valid
>>> QModelIndex.
>>>
>>
>> I suspect this is the key to my misunderstanding.  When I pass a Python
>> datetime to some PyQt function which expects a QDate (I can't think of
>> one), or a Python decimal.Decimal where PyQt/Qt function expects a float,
>> they get auto-converted, right?
>>
>
> I think that "basic" standard data types are used transparently between
> Python and Qt. I don't know how this exactly works behind the scenes
> (expecially for "advanced" numeric types, like qint8 or qlonglong), I think
> that standard types like int, float and strings are just used as they are
> represented internally by the Python interpreter, while PyQt automatically
> offers automatic conversion for more advanced classes whenever it makes
> sense.
>
> In the old PyQt4 docs, it explicitly says:
>
>> A Python date object may be used whenever a QDate is expected.
>
> The same appears for QDateTime and QTime, but for other "compatible" types
> also, like when using string objects for QByteArrays.
>
> This means that you can use a Python date object as an argument for any
> method that expects a QDate (for example, the QDateTimeEdit.setDate()),
> including the QDate __init__ itself (that's your case), since new QDate
> objects can be also created using the QDate(QDate) initialization method.
> So, a Python date/time is "accepted" as an argument and automatically
> converted to a Qt date/time object for the meanings of the method called.
> The original object is unchanged and there's no reference to it from the Qt
> side of things.
>
>
>> But when I pass those to, say, QStandardItemModel.setData(), you are
>> saying there is no conversion and the Python object is stored as-is,
>> right?  And when QStandardItemModel.sort()/lessThan() comes along and
>> looks at the data which is a Python datetime or Decimal type, this does
>> not count as the same situation (e.g. explicit argument to function) where
>> PyQt would auto-convert as necessary --- quite possibly because we are down
>> in C++ Qt code by then which knows nothing about Python --- so it fails.
>> Is that about right?
>>
>
> Quite correct. Remember that an item model doesn't care about the type of
> data stored (and it shouldn't), it's only an abstract categorization of a
> collection of objects.
> Think of it as an advanced Python list, where you could put any kind of
> object in it. It wouldn't make sense to "change" the object type when you
> add it to the list, right?
> The only scenarios in which the data type becomes important is when the
> data is represented (in an item view) or some level of interaction is
> required, such as finding data or comparing. For visual representation it
> behaves like printing the Python list: if the object has __str__
> implemented it will show the returned string, even if the object is not a
> string, so it will display the content if the object is "known" as
> printable for Qt; but if you try to sort a non builtin Python type, you
> will likely get back the original sorting, since it doesn't know how to
> handle the data.
>
> I wanted to do a couple of test to better demonstrate all this, but it
> seems that I'm not able to automatically display a Python date object in a
> QTableView (I thought I did it while testing yesterday, but maybe I was
> wrong), requiring an item delegate to keep the original object and actually
> display the date.
> Just out of curiosity, did you actually see the dates in the fields for
> which you used Python date objects as setData argument (without the QDate
> conversion)?
>
> Finally, about your last sentence, it's not that "Qt knows nothing about
> Python". The model can "see" the stored object, but it's an unknown data
> type (remember, it has not been "converted", when storing data to a model
> it should not change its contents), thus the it has no way to know how to
> sort it.
>
>>
> When sorting, models usually call the private method isVariantLessThan()
> you can see here:
>
> https://code.woboq.org/qt5/qtbase/src/corelib/itemmodels/qabstractitemmodel.cpp.html#_ZN25QAbstractItemModelPrivate17isVariantLessThanERK8QVariantS2_N2Qt15CaseSensitivityEb
> You'll see that the default case switch behaves as the QVariant string
> type, trying to transform the QVariant to a string. Since PyQt5 hides some
> features due to the "transparency" of data types (there are no QStrings nor
> QVariants by default), I'll show this with PyQt4, which probably better
> explains what happens under the hood. Remember that with SIP v1 (the
> default for PyQt4) data models usually returned QVariants, requiring the
> transformation to toPyObject to get the original form.
>
> >>> from PyQt4 import QtCore
> >>> intVariant = QtCore.QVariant(float(5))
> >>> intVariant.typeName()
> 'double'
> >>> intVariant.toString()
> PyQt4.QtCore.QString(u'5')
> >>> from datetime import date
> >>> dateVariant = QtCore.QVariant(date.today())
> >>> dateVariant.typeName()
> 'PyQt_PyObject'
> >>> dateVariant.toString()
> PyQt4.QtCore.QString(u'')
>
> As you can see, the intVariant is correctly "transformed" to an internal
> double type and, while isVariantLessThan will obviously use the default
> numeric comparison, its toString method returns the string representation.
> In the second case, the QVariant type is indeed a "special" Qt object type,
> and its string is null. Some might object that, if the object has a __str__
> method implemented, it should return that, but that's open to (another)
> debate.
> At this point, lessThan will always receive False for unknown data types,
> leaving the whole sorting unchanged.
>
> Well, that was educational for me too :-)
> Hope this helps you as well!
>
> Regards,
> Maurizio
>
> --
> È difficile avere una convinzione precisa quando si parla delle ragioni
> del cuore. - "Sostiene Pereira", Antonio Tabucchi
> http://www.jidesk.net
>

Hi Maurizio,

Yet again, thank you for your interest.  Though my head is beginning to
ache over all this now! :)

Again, I *think* you may have hit the nail on the head when you write:

but it seems that I'm not able to automatically display a Python date
> object in a QTableView (I thought I did it while testing yesterday, but
> maybe I was wrong), requiring an item delegate to keep the original object
> and actually display the date.
> Just out of curiosity, did you actually see the dates in the fields for
> which you used Python date objects as setData argument (without the QDate
> conversion)?


Until yesterday, I would have claimed that, yes, the QTableView does show
these python datetimes from the model natively.  Then I noticed in
something new I was doing that they were coming out blank.  I am beginning
to believe that you are quite right and it does *not* auto-handle/convert
them....

Please understand: until a year or so ago I had never used Python or Qt.
Now I develop & maintain (on my own, so no one I can ask) an existing PyQt
project with tens of thousands of lines of existing code.  Written by
different people over years with not a single comment and impenetrable
logic.  This means I don't always know what's happening where.

Now, if my understanding that PyQt was auto-converting datetime <-> QDate,
at least in the context of model data/table view, turns out to be baseless,
then this all begins to make a lot  more sense.  If the table view won't
display a Python datetime, because it knows nothing about it, then I can
appreciate it will have similar problem not being able to sort by it.

Since you are enjoying yourself(!), I also claimed that a column with
Python type decimal.Decimal in it (that's what we use for all our monetary
numbers, we need two decimal places and no loss of precision so float is no
good) fails to sort in just the same way as dateime.date.  Again, there I
*thought* some auto-conversion was going to float which the sort does know
about.  You *might* like to see how that behaves for you?  If that too does
not sort and does not display correctly (i.e. it's not handled
intrinsically by model/view code), then again I wonder whether we don't
have our own explicit code in, say, .data(row, col, Qt.DisplayRole) which
converts to a fixed-2-decimals string output.  So again if it can't be
displayed natively I would understand why it can't be sorted on.

I would do this myself today if I could.  However as of yesterday our whole
web site no longer works, following a move by our ISP from PHP 5.6 to 7.x.
Reading up, I discover that for years until PHP 5.6 it used a "mysql"
extension library for all database interaction, and now that has been
removed and replaced with a new "mysqli" extension library, which is
different, so our whole web site cannot access its database (inc.
WordPress).  So today, having never done any PHP and knowing nothing about
how it all works, I must urgently learn and fix the whole site... trust you
understand!

Ciao, e grazie :)

-- 
Kindest,
Jonathan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.riverbankcomputing.com/pipermail/pyqt/attachments/20190208/c9c62a44/attachment-0001.html>