sip: feature: make build faster by running configure and codegen in parallel

[Milan Hauth]

done parallel configure in
https://github.com/milahu/sip/tree/parallel-configure

used in
https://github.com/NixOS/nixpkgs/pull/177530

with parallel configure on 32 cores
the pyqt6 build takes 400 instead of 500 seconds

parallel codegen is harder, because, for example:
QtCore codegen must run before QtOpenGL codegen
because QtCore defines %Feature PyQt_OpenGL_ES2

WIP for parallel codegen in
https://github.com/milahu/sip/tree/parallel-codegen

please feel free to use my code


2022-06-11 22:29 GMT+02:00, Milan Hauth <milahu at gmail.com>:
> currently there are two bottlenecks in sip: configure and codegen
>
> these run on a single core, which takes some time
>
> code:
>
> sipbuild/project.py
>
> ```py
>     def update_buildable_bindings(self):
>        # ...
>
>         # TODO perf: run this in parallel
>         for b in list(self.bindings.values()):
>             if not b.is_buildable():
>                 del self.bindings[b.name]
> ```
>
> is_buildable is implemented for example by pyqt-builder in
> pyqtbuild/bindings.py
>
> ```py
>     def is_buildable(self):
> ```
>
> challenge:
> capture stdout and stderr of the configure's
> and send them to logfiles, to allow later inspection
>
> bottleneck 2 is codegen:
>
> sipbuild/builder.py
>
> ```py
>         # TODO perf: run this in parallel
>         for bindings in project.bindings.values():
>             project.progress(
>                     "Generating the {0} bindings".format(bindings.name))
> ```
>
> the build phase of pyqt-builder
> already runs parallel with the --jobs argument
>
> pyqtbuild/builder.py
>
> ```py
>         options.append(
>                 Option('jobs', option_type=int,
>                         help="run N make jobs in parallel", metavar='N'))
> ```
>