sip: feature: make build faster by running configure and codegen in parallel

[Phil Thompson]

Thanks but I’m not looking at optimisations until all the C code has 
been replaced by Python code.

Phil

On 17/06/2022 19:43, Milan Hauth wrote:
> done parallel configure in
> https://github.com/milahu/sip/tree/parallel-configure
> 
> used in
> https://github.com/NixOS/nixpkgs/pull/177530
> 
> with parallel configure on 32 cores
> the pyqt6 build takes 400 instead of 500 seconds
> 
> parallel codegen is harder, because, for example:
> QtCore codegen must run before QtOpenGL codegen
> because QtCore defines %Feature PyQt_OpenGL_ES2
> 
> WIP for parallel codegen in
> https://github.com/milahu/sip/tree/parallel-codegen
> 
> please feel free to use my code
> 
> 
> 2022-06-11 22:29 GMT+02:00, Milan Hauth <milahu at gmail.com>:
>> currently there are two bottlenecks in sip: configure and codegen
>> 
>> these run on a single core, which takes some time
>> 
>> code:
>> 
>> sipbuild/project.py
>> 
>> ```py
>>     def update_buildable_bindings(self):
>>        # ...
>> 
>>         # TODO perf: run this in parallel
>>         for b in list(self.bindings.values()):
>>             if not b.is_buildable():
>>                 del self.bindings[b.name]
>> ```
>> 
>> is_buildable is implemented for example by pyqt-builder in
>> pyqtbuild/bindings.py
>> 
>> ```py
>>     def is_buildable(self):
>> ```
>> 
>> challenge:
>> capture stdout and stderr of the configure's
>> and send them to logfiles, to allow later inspection
>> 
>> bottleneck 2 is codegen:
>> 
>> sipbuild/builder.py
>> 
>> ```py
>>         # TODO perf: run this in parallel
>>         for bindings in project.bindings.values():
>>             project.progress(
>>                     "Generating the {0} 
>> bindings".format(bindings.name))
>> ```
>> 
>> the build phase of pyqt-builder
>> already runs parallel with the --jobs argument
>> 
>> pyqtbuild/builder.py
>> 
>> ```py
>>         options.append(
>>                 Option('jobs', option_type=int,
>>                         help="run N make jobs in parallel", 
>> metavar='N'))
>> ```
>>