[PyQt] PyQt4 and Python 3.0

[Daniel Miller]

I should clarify a few things before I embark on a rebuttal:

- I have deployed my code to my end users many times since I started  
to build my PyQt4 compatibility layer (it is not finished yet, and I  
can't afford to stop pushing out bug fixes and new features for a few  
months while I do a rewrite).

- I am moving to PyQt4 because I need to use some of the new features  
(the Graphics View Framework, for one). If I did not need these new  
features in PyQt4 I'd be content to stay with PyQt3.


On Oct 6, 2008, at 2:32 PM, Phil Thompson wrote:
> On Mon, 6 Oct 2008 12:27:42 -0400, Daniel Miller  
> <daniel at keystonewood.com>
> wrote:
>>>
>>>> I hope Phil can stick to the really needed stuff. Porting stuff  
>>>> from
>>>> Qt/KDE 3 to 4 has proven to be draining work which I hope to avoid
>>>> having to do again in the near future.
>>>
>>> Which is why I say don't port, rewrite.
>
> Before I start I should point out that I say "rewrite", not  
> "rewrite from
> scratch". My main point is that, when presented with a major change in
> technology then don't try to get away with making as few changes as
> possible - which is what "port" means to me.

Why should I rewrite an entire code base that is working fine and  
does not need to be touched? That kind of disruption only introduces  
new bugs. It seems with your logic that whenever I need even one new  
feature from PyQt4 I am forced to rewrite my entire application (or  
at least everything that touches PyQt). It's insane to think that  
everyone who uses PyQt has the time and resources to do that.

>> If you don't believe me go read Joel Spolsky's article
>> "Things You Should Never Do, Part I" <http://www.joelonsoftware.com/
>> articles/fog0000000069.html>
>
> ...everybody knows that, in real life, software *does* rust.

I think you're being stubborn... Of course software ages, but if it  
(or at least most parts of it) meets the requirements then I should  
not be forced to rewrite it.

>> For one thing, when an investor gives the OK to spend many hours to
>> rewrite an application s/he normally expects to see new features and
>> or functionality, none of which will exist when the only new feature
>> is a new version of a library.
>
> They expect to see whatever you've agreed to provide. If they see
> insufficient benefit in moving to a new library, then don't do it.

Again, I don't think you're being entirely honest here. I need to add  
new features, which are required by my employer. To add those new  
features, I need the latest version of PyQt. However, I don't want  
(nor can I afford) to rewrite my application every time I decide to  
upgrade to a new version of PyQt. I'm starting to see this as a  
significant disadvantage to using PyQt in the first place--the devs  
are not concerned with maintaining backward compatibility or making  
the upgrade path smooth for existing users. It seems to be a library  
for toy projects that can be torn down and implemented from scratch  
on a whim.

>> I'm in the process of maintaining and
>> porting an application from PyQt3 to PyQt4. ...
>
> So you spend a lot of time refactoring the code so that when you do  
> the
> fork it is as late as possible and the amount of code being forked  
> is a
> little as possible.

It's not a fork because there are never two separate branches of code  
that need to be maintained at the same time. There just comes a point  
in time when the application no longer uses PyQt3 and at that point  
it uses PyQt4--this is analogous to switching to a new version of any  
module or package.

>> Unfortunately, the PyQt team doesn't ship the Q3Support classes,
>
> It is available.

Actually, I'm using the parts of it that I can, although it's a royal  
pain because PyQt needs to be patched, and sometimes the patch needs  
to be updated by hand since it's not always up to date with the  
latest version of PyQt4. Not to mention, I still need to write a  
compatibility layer because the names of all the support classes  
start with Q3. The compatibility layer consists of 'qt' and 'qttable'  
modules that contain classes that look and act exactly like they did  
in PyQt3.

>
>> which are critical for anyone who can't simply throw everything away
>> and start over new. The Python 3 team, in contrast to the PyQt team,
>> is making the switch from Python2 to Python3 as painless as possible.
>> The py2to3 tool should minimize the porting pain, and they have very
>> well-though-out advice for those who need to support both Python2 and
>> Python3: develop for 2.6 and use py2to3 to "compile" a Python3
>> version. IMPORTANT: DO NOT TOUCH the py2to3 generated code until
>> you're ready to throw away (forever) the old 2.6 code base. I.E. if
>> you need to make changes, make them in the 2.6 version of the code
>> and "recompile" with py2to3 for as long as both versions of Python
>> must be supported.
>
> If the Python approach requires you to specifically target 2.6 then  
> move to
> 3.0 then I think that approach is flawed as it involves two  
> transitions. I
> would recommend treating 2.6 as an incremental upgrade from 2.5 and  
> do a
> single transition from 2.x to 3.0.

It is certainly not flawed if you need a version of your code that  
works on both Python 2 and Python 3. Or if you want to do two small  
transitions in a controlled manner with a few minor code changes  
rather than one big transition that results in a major rewrite and  
possibly lots of downtime. If there is no single version of PyQt that  
runs on both versions then the major rewrite will be the only option.  
That means that every application that uses PyQt will be doing  
another major rewrite when they switch to Python 3 and  
PyQtX...hopefully they will all be done with the migration to PyQt4  
by then...

>
>> ...
>> The new version is exactly like the old one except it uses PyQt4
>> instead of PyQt3; no learning curve, no new file formats, nothing
>> different except PyQt4.
>
> That's the problem. You then end up with a design that's based on  
> the needs
> of PyQt3 and trying to bend PyQt4 to fit it. You also find that  
> it's more
> difficult than it should be to exploit new features of PyQt4.

Not at all. You talk about PyQt4 like it's some entirely new beast  
sharing next to nothing with the old PyQt3. Maybe that's how the  
underlying implementation looks, but the end-user functionality is  
95% the same as it was in PyQt3--it still shows windows with widgets  
on the screen, and users still interact with them in the same way  
they did before. Adapting PyQt4 to the old PyQt3 API is not that  
hard, it's just very tedious.

However, it's much less tedious to implement a single new version of  
qt.QSettings based on PyQt4.QtGui.QSettings than to rewrite every  
place in my application that happens to touch the QSettings API (for  
example). Newer parts of may application can import  
PyQt.QtGui.QSettings and use that API if they need the new  
functionality. Incidentally, there's not much new in QSettings, so I  
don't really see why there was a need to change the API so  
drastically in a backward-incompatible way--it just causes a lot of  
pain. If the API really needed to be changed, then the old methods  
should have been deprecated in a transitional version, giving time to  
migrate to the new API's without completely breaking backward  
compatibility. Then new releases could be pushed out while the  
migration was taking place rather than requiring a rewrite through  
the duration of which no releases can be made because critical things  
are still broken.

> If all you end up with is the same application, doing the same  
> thing but
> using PyQt4 instead of PyQt3, then I don't understand why you  
> bothered.

I need new features in PyQt4...if I didn't I certainly wouldn't have  
switched.

But I want to change (and debug) one thing at a time. I don't want to  
be debugging PyQt migration bugs at the same time I'm debugging new  
features in my code. That's why I want a seamless transition. When I  
switch from PyQt3 to PyQt4 I want that to be the only change that I'm  
making--then I'll know where the problem originated. If I'm  
introducing PyQt4 at the same time I'm introducing new features I'll  
have no idea if the problem is related to my implementation of the  
new feature or if it's a bug in PyQt4.

> I would reverse it. Write a compatibility layer that makes PyQt3  
> look like
> PyQt4. Then, when you finally throw away PyQt3, you throw away the
> compatibility layer as well.

Umm, that would be fine if my application was coded against the PyQt4  
API, but then I wouldn't need to have this discussion, would I? The  
point is to be able to release updates while I'm working on the  
transition to PyQt4, without maintaining two completely independent  
code branches.

> Note that I wouldn't call the Q3Support classes a compatibility layer.

I agree. In a C++ app it might be possible (even simple?) to do  
global find/replace and use the compile step to make sure you've  
renamed everything properly, but with Python this is not so simple.  
It would have been nice if PyQt4 would have provided a 'qt' module  
(namespace) that had the exact same contents as PyQt3--that's what  
I'm writing for my application. I can migrate away from using this  
layer as I refactor and develop new features in my application. The  
key is that it's done incrementally in a stable and predictable  
manner, and it's done on my timeframe, and I can release as often as  
I need to during the transition.

>> While it is unquestionably worthwhile for me to write a compatibility
>> layer for my project, if several teams out there decide to port their
>> projects this way then the combined cost of each team developing
>> their own compatibility layer far exceeds the cost of doing it once
>> for everyone.
>
> There's nothing stopping you organising this, or publishing your
> compatibility layer.

Unfortunately my compatibility layer probably would not be very  
useful to the general public since I'm only implementing what is  
absolutely necessary for my application.

> Perhaps if I gave a fuller definition of "rewrite"...
>
> "To refactor your application over a period of time so that the  
> code is
> exactly the same as it would have been if you had been able to  
> start with a
> blank piece of paper."

Why should I start with a blank piece of paper when I already have  
99% of what I need and it's all working correctly? You seem to think  
there's value in rewriting code that is working perfectly. I see this  
as a waste of my valuable time, which I could be spending  
implementing new features.

> Note that you will not achieve this [starting with a blank piece of  
> paper] with the way you are doing your
> compatibility layer.

Nor do I want to achieve that. I want to move forward; I do not want  
to spend lots of time rewriting code that already works.

> To me a much more important element of the process than all these  
> layers is
> automated, comprehensive testing (the holy grail of GUI  
> development). With
> that in place you can refactor with real confidence without needing to
> fork.

And with that in place switching to PyQt4 would be just as painful.  
Introducing something as wildly different as PyQt4 would break so  
many tests that there would be no way to tell if you're getting  
anything done as you start to try to fix the damage.

fix 1... run tests... 4023 tests failed
fix 2... run tests... 4022 tests failed
fix 3... run tests... 4056 tests failed
...

The point of testing is to make small changes and fix one thing at a  
time. When something breaks you know exactly what it is because  
you've got a test telling you what it is, and when you fix it you  
know it's fixed because all your tests pass again. However, this  
development strategy will not work in a migration to PyQt4 because so  
many tests will break initially that you'd have no idea where to  
start--the number of broken tests would be huge, and any given fix  
may or may not reduce the number of failing tests. In fact, the  
number of broken tests could quite plausibly increase if you happen  
to fix one thing but at the same time break something else. The  
problem is that there is no way to introduce PyQt4 gradually.

The damage is done in PyQt4, and we can't turn back the clock  
(although a good compatibility layer could still be written). Just  
don't introduce this type of major breakage again, please. Especially  
not in the transition to Python 3, as it would require us to test two  
new things at the same time: Python 3 and PyQtX.

~ Daniel