<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">Hi Matt,</div><div class="gmail_quote"><br></div><div class="gmail_quote">I'm in no position to comment on your wider point, but...</div>
<div class="gmail_quote"><br></div><div class="gmail_quote">On 9 May 2013 19:59, Matt Newell <span dir="ltr"><<a href="mailto:newellm@blur.com" target="_blank">newellm@blur.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
On Monday, May 06, 2013 07:49:25 AM Phil Thompson wrote:<br>
> The first PyQt5 snapshots are now available. You will need the current SIP<br>
> snapshot. PyQt5 can be installed alongside PyQt4.<br>
><br>
> I welcome any suggestions for additional changes - as PyQt5 is not<br>
> intended to be compatible with PyQt4 it is an opportunity to fix and<br>
> improve things.<br>
><br>
> Current changes from PyQt4:<br>
><br>
> - Versions of Python earlier than v2.6 are not supported.<br>
><br>
> - PyQt4 supported a number of different API versions (QString, QVariant<br>
> etc.). PyQt5 only implements v2 of those APIs for all versions of Python.<br>
><br>
<br>
I haven't looked into this deeper but I am a bit worried about the possible<br>
performance impacts of QString always being converted to a python str/unicode.<br>
(Not to mention the added porting work when going c++ <-> python).<br>
<br>
The vast majority of the PyQt code that we use loads data from libraries that<br>
deal with Qt types, and either directly loads that data into widgets, or does<br>
some processing then loads the data into widgets. I suspect that this kind of<br>
usage is very common.<br>
<br>
As an example a user of QtSql with the qsqlpsql driver that loads data and<br>
displays it in a list view is going to see the following data<br>
transformations/copies:<br>
<br>
PyQt4 with v1 QString api:<br>
<br>
libpq data comes from socket<br>
-> QString (probable utf8->utf16)<br>
-> PyQt wrapper of QString (actual data not copied or converted)<br>
-> QString (pointer dereference to get Qt type)<br>
<br>
PyQt5, PyQt4 with v2 QString api:<br>
<br>
libpq data comes from socket<br>
-> QString (probable utf8->utf16)<br>
-> unicode (deep copy of data)<br>
-> QString (deep copy of data)<br>
<br>
So instead of one conversion we now have one conversion and two deep copies.<br>
Another very probable side-effect is that in many cases either the original<br>
QString and/or the unicode object will be held in memory, resulting in two or<br>
possibly even three copies of the data. Even if all but the last stage is<br>
freed, there will still be 2 or 3 copies in memory during processing depending<br>
on how the code is written, which can reduce performance quite a bit depending<br>
on data size because of cpu cache flushing.<br>
<br>
So far this is completely theoretical, and I'm sure in a large portion of<br>
applications will have no noticeable effect, however I don't like the idea that<br>
things may get permanently less efficient for apps that do process and display<br>
larger data sets.<br>
<br>
The one thing that stands out to me as possibly being a saving grace is the<br>
fact that (at least in my understanding) both Qt and python use utf16 as their<br>
internal string format, which means fast copies instead of slower conversions,<br>
and that it may be possible with some future Qt/python changes to actually<br>
allow QString -> unicode -> QString without any data copies.<br></blockquote><div><br></div><div>FWIW, the Python-uses-roughly-utf16 meme is a common oversimplification. First, as I'm sure most people know, there are significant changes between Python2 str/unicode and Python3 str. That cannot but be reflected in differences between the CPython usage across the 2/3 boundary.</div>
<div><br></div><div>What is less well known is that there is a significant change to CPython between 3.2 and 3.3 where the latter can store a str as either an array of 8, 16 or 32 bit values with automatic run-time conversions between them (and API changes to match). So whatever else happens within PyQt, I don't think the aspiration to the old 1-copy model can be relied on. In the event, this is what I came up with for the QString to str direction (corrections/optimisations welcome!):</div>
<div><pre style="vertical-align:top;line-height:14.390625px;font-size:12px;margin-bottom:0px;margin-top:0px;padding:0px;background-image:none"><span style="color:rgb(0,119,136)"><br></span></pre><pre style="vertical-align:top;line-height:14.390625px;font-size:12px;margin-bottom:0px;margin-top:0px;padding:0px;background-image:none">
<span style="color:rgb(0,119,136)"></span></pre></div><div><pre class="" style="font-size:12px;line-height:14.390625px;margin-top:0px;margin-bottom:0px;padding:0px;background-image:none;vertical-align:top;color:rgb(0,0,0)">
PyObject <span class="" style="color:rgb(0,0,64)">*</span>Python<span class="" style="color:rgb(0,128,128)">::</span><span class="" style="color:rgb(0,119,136)">unicode</span><span class="" style="color:rgb(0,128,0)">(</span><span class="" style="color:rgb(0,0,255)">const</span> QString <span class="" style="color:rgb(0,0,64)">&</span>string<span class="" style="color:rgb(0,128,0)">)</span>
<span class="" style="color:rgb(0,128,0)">{</span>
<span class="" style="color:rgb(51,153,0)">#if PY_MAJOR_VERSION < 3</span>
<span class="" style="color:rgb(255,0,0);font-style:italic">/* Python 2.x. <a href="http://docs.python.org/2/c-api/unicode.html">http://docs.python.org/2/c-api/unicode.html</a> */</span>
PyObject <span class="" style="color:rgb(0,0,64)">*</span>s <span class="" style="color:rgb(0,0,128)">=</span> PyString_FromString<span class="" style="color:rgb(0,128,0)">(</span>PQ<span class="" style="color:rgb(0,128,0)">(</span>string<span class="" style="color:rgb(0,128,0)">)</span><span class="" style="color:rgb(0,128,0)">)</span><span class="" style="color:rgb(0,128,128)">;</span>
PyObject <span class="" style="color:rgb(0,0,64)">*</span>u <span class="" style="color:rgb(0,0,128)">=</span> PyUnicode_FromEncodedObject<span class="" style="color:rgb(0,128,0)">(</span>s, <span class="" style="color:rgb(255,0,0)">"utf-8"</span>, <span class="" style="color:rgb(255,0,0)">"strict"</span><span class="" style="color:rgb(0,128,0)">)</span><span class="" style="color:rgb(0,128,128)">;</span>
Py_DECREF<span class="" style="color:rgb(0,128,0)">(</span>s<span class="" style="color:rgb(0,128,0)">)</span><span class="" style="color:rgb(0,128,128)">;</span>
<span class="" style="color:rgb(0,0,255)">return</span> u<span class="" style="color:rgb(0,128,128)">;</span>
<span class="" style="color:rgb(51,153,0)">#elif PY_MINOR_VERSION < 3</span>
<span class="" style="color:rgb(255,0,0);font-style:italic">/* Python 3.2 or less. <a href="http://docs.python.org/3.2/c-api/unicode.html#unicode-objects">http://docs.python.org/3.2/c-api/unicode.html#unicode-objects</a> */</span>
<span class="" style="color:rgb(51,153,0)">#ifdef Py_UNICODE_WIDE</span>
<span class="" style="color:rgb(0,0,255)">return</span> PyUnicode_DecodeUTF16<span class="" style="color:rgb(0,128,0)">(</span><span class="" style="color:rgb(0,128,0)">(</span><span class="" style="color:rgb(0,0,255)">const</span> <span class="" style="color:rgb(0,0,255)">char</span> <span class="" style="color:rgb(0,0,64)">*</span><span class="" style="color:rgb(0,128,0)">)</span>string.<span class="" style="color:rgb(0,119,136)">constData</span><span class="" style="color:rgb(0,128,0)">(</span><span class="" style="color:rgb(0,128,0)">)</span>, string.<span class="" style="color:rgb(0,119,136)">length</span><span class="" style="color:rgb(0,128,0)">(</span><span class="" style="color:rgb(0,128,0)">)</span> <span class="" style="color:rgb(0,0,64)">*</span> <span class="" style="color:rgb(0,0,221)">2</span>, <span class="" style="color:rgb(0,0,221)">0</span>, <span class="" style="color:rgb(0,0,221)">0</span><span class="" style="color:rgb(0,128,0)">)</span><span class="" style="color:rgb(0,128,128)">;</span>
<span class="" style="color:rgb(51,153,0)">#else</span>
<span class="" style="color:rgb(0,0,255)">return</span> PyUnicode_FromUnicode<span class="" style="color:rgb(0,128,0)">(</span>string.<span class="" style="color:rgb(0,119,136)">constData</span><span class="" style="color:rgb(0,128,0)">(</span><span class="" style="color:rgb(0,128,0)">)</span>, string.<span class="" style="color:rgb(0,119,136)">length</span><span class="" style="color:rgb(0,128,0)">(</span><span class="" style="color:rgb(0,128,0)">)</span><span class="" style="color:rgb(0,128,0)">)</span><span class="" style="color:rgb(0,128,128)">;</span>
<span class="" style="color:rgb(51,153,0)">#endif</span>
<span class="" style="color:rgb(51,153,0)">#else /* Python 3.3 or greater. <a href="http://docs.python.org/3.3/c-api/unicode.html#unicode-objects">http://docs.python.org/3.3/c-api/unicode.html#unicode-objects</a> */</span>
<span class="" style="color:rgb(0,0,255)">return</span> PyUnicode_FromKindAndData<span class="" style="color:rgb(0,128,0)">(</span>PyUnicode_2BYTE_KIND, string.<span class="" style="color:rgb(0,119,136)">constData</span><span class="" style="color:rgb(0,128,0)">(</span><span class="" style="color:rgb(0,128,0)">)</span>, string.<span class="" style="color:rgb(0,119,136)">length</span><span class="" style="color:rgb(0,128,0)">(</span><span class="" style="color:rgb(0,128,0)">)</span><span class="" style="color:rgb(0,128,0)">)</span><span class="" style="color:rgb(0,128,128)">;</span>
<span class="" style="color:rgb(51,153,0)">#endif</span>
<span class="" style="color:rgb(0,128,0)">}</span></pre></div><div><br></div><div style>The referenced URLs contain more material.</div><div style><br></div><div style>Hth, Shaheed</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
At some point I will try to do some benchmarks and look into the actual code<br>
to see if there is an elegant solution to this potential problem.<br>
<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><br></blockquote>kMatt<br>
_______________________________________________<br>
PyQt mailing list <a href="mailto:PyQt@riverbankcomputing.com" target="_blank">PyQt@riverbankcomputing.com</a><br>
<a href="http://www.riverbankcomputing.com/mailman/listinfo/pyqt" target="_blank">http://www.riverbankcomputing.com/mailman/listinfo/pyqt</a><br>
</blockquote></div><br></div></div>