High sip-build memory usage when configuring PyQt

[Phil Thompson]

On 24/12/2022 17:35, Dmitry Shachnev wrote:
> Hi Phil, and merry Christmas!
> 
> When sip-build is generating bindings for PyQt5, the memory usage 
> constantly
> grows, and by the end of generation reaches about 3.5 GB.
> 
> I am using --no-make option, so this is only about generating code, not
> building it. The memory grows during the phase when SIP prints messages 
> like
> these:
> 
>   Generating the QtCore bindings...
>   Generating the QtCore .pyi file...
>   Generating the QtNetwork bindings...
>   Generating the QtNetwork .pyi file...
> 
> This is a big problem for us in Debian, because in the Debian MIPS 
> port, a
> process can allocate a maximum of 2 GB of memory [1]. Because of this, 
> pyqt5
> build keeps failing with MemoryError [2].
> 
> I use the latest versions of everything:
> - SIP 6.7.5
> - PyQt-builder 1.14.0
> - PyQt5 5.15.7 (I believe it will happen with PyQt6 too)
> 
> This is a regression between SIP 6.6 and 6.7. With SIP 6.6 there were 
> no such
> issues.
> 
> Is it possible to free some memory after generating every binding? Or 
> maybe
> you can suggest some workaround to reduce memory consumption?
> 
> [1]: https://wiki.debian.org/MIPSPort#Generic_issues
> [2]:
> https://buildd.debian.org/status/fetch.php?pkg=pyqt5&arch=mipsel&ver=5.15.7%2Bdfsg-2%2Bb1&stamp=1671885622&raw=0

The porting to Python from C is bound to cause potential memory and 
speed issues. The next stage (v6.8) will remove the remaining C code and 
this will certainly require less memory (but I have no idea how much). 
v6.9 will deprecate all features to be removed in v7 and v6.9.x will be 
speed and memory optimisation releases.

Phil