[PyQt] A Heads Up for PyQt Packagers

[Phil Thompson]

On 25 Jun 2018, at 12:42 pm, Hans-Peter Jansen <hpj at urpla.net> wrote:
> 
> Dear Phil,
> 
> On Sonntag, 17. Juni 2018 22:27:54 Phil Thompson wrote:
>> This will mainly be of interest to people who create sip, PyQt4, PyQt5 etc.
>> packages for Linux but also anybody who builds from source.
>> 
>> Starting with v5.11 (ie. the next release) PyQt5 will use a private copy of
>> the sip module. Packagers have the choice of including it with PyQt5 or
>> creating a new package (PyQt5_sip?). The wheels will use the second
>> approach (because the module is Python version specific, whereas the PyQt5
>> modules themselves are not).
>> 
>> There will be one last release of PyQt4 which will also use a private copy
>> of the sip module.
>> 
>> As a matter of interest, wxPython also uses a private copy of the sip
>> module.
>> 
>> The change fixes a 20 year old design mistake and makes it much easier to
>> move on with SIP v5 without worrying about the baggage of PyQt4.
> 
> While this might ease further sip development, it will create headaches for 
> sip users for what I can imagine, since this will lead to a sip version 
> inflation over time on certain systems.
> 
> Attending sip and pyqt development for about 20 years now, I can see, that  
> external backflow to sip development is pretty low unfortunately. On the other 
> hand, it has *always* been a pleasure to package your stuff and to work with.

> From a packager POV, ending with sip4_qt4, sip4_qt5, sip4_kde4, sip4_kde5, 
> sip4_mypackage isn't exactly funny. Adding sip5 to the mix will further add 
> jags to the picture. 
> 
> If a developer want to use the sip technology, first step would be choosing 
> the relevant sip version, clone it, and stick with it. Further improvements to 
> sip will most probably pass by since there's no pressure to get it right for 
> all sip users.
> 
> Given the scale of the generated bindings, I fear, that issues may pass 
> unnoticed with the new model much longer than now. This problem could be 
> combated with automatic generation of (python) test code, but this is a 
> mammoth project, that isn't going to be realized with the available manpower 
> any time soon. With the KDE bindings in mind, I bet, that there are a lot of 
> code paths, that were never exercised, nowhere. With my personal rule of thump 
> "code paths, that aren't exercised, are broken", this leaves behind an 
> uncomfortable feeling in this regard.
> 
> Hence, I'm not sure, if sip diversity will pay off in the end.
> 
> Please don't get me wrong, but branching and unitizing might be viable 
> solutions to the problem, that also reduce code diversity.

Whether or not you create new packages for the private copy of the sip module is entirely down to you as a packager. If it were me I would do it so that the number and names of the packages would be exactly the same as they are today. Only the dependencies between them would change. Specifically I would include the private copy of the sip module in my PyQt4 and PyQt5 packages. The dependency of your PyQt4/PyQt5 packages on your SIP package would be removed. Whether the private copy was created by SIP4 or SIP5 is irrelevant. Any kde4 or kde5 would not include a private copy of the sip module - they would automatically use PyQt's copy.

When it comes to wheels I take a different approach and package the private copy of the sip module separately, ie. it has its own wheel. The reason for this is that the sip module is dependent on a specific version of Python (3.5, 3.6, 3.7 etc) but presents a version-independent API to PyQt etc. Therefore to add support for (say) Python v3.8 I just need to release a new sip module wheel for that version and the existing PyQt wheels will automatically work.

I think one cause of confusion is my use of the word "private". PyQt and PyQt3D and QScintilla etc, etc form a hierachy of modules rooted at the QtCore module. All of those modules share the same "private" copy of the sip module. It is perfectly reasonable for me (as the author of PyQt) to place restrictions on the version of the sip module shared by all those modules (including any written by other people). However it's not reasonable for those restrictions to be forced onto authors of other modules that have nothing to do with PyQt.

Phil