Python 3.12 segfault with mixin classes

[Phil Thompson]

Personally I've never a problem with the C API. If a call isn't 
documented then don't use it. The only improvements for me would be to 
slightly expand the scope of the limited API so that it could be used to 
write the sip module.

Phil

On 09/06/2023 08:40, Damon Lynch wrote:
> Of interest to those who have not seen it — a discussion of the many
> problems around Python's C API, and what folks are doing to ameliorate 
> them:
> 
> https://pyfound.blogspot.com/2023/05/the-python-language-summit-2023-three.html
> 
> On Fri, 9 Jun 2023 at 09:56, Thomas Caswell <tcaswell at gmail.com> wrote:
> 
>> I am also seeing segfaults with the cpython main branch + pyqt6.  I 
>> had a
>> suspicion it was that change, but had not had time to verify or make a
>> minimal example (making a Matplotlib figure segfaulted).
>> 
>> numpy and cython also had issues, see
>> https://github.com/python/cpython/issues/104614 , however even with 
>> those
>> PRs landed I'm still seeing segfaults with PyQt6.
>> 
>> Tom
>> 
>> On Thu, Jun 8, 2023 at 5:22 PM Florian Bruhin <me at the-compiler.org> 
>> wrote:
>> 
>>> Hey,
>>> 
>>> When trying PyQt 6.5.1 with Python 3.12 Beta 2, something like:
>>> 
>>>         import sys
>>>         from PyQt6.QtWidgets import QWidget, QApplication
>>> 
>>>         class Mixin: pass
>>>         class Command(Mixin, QWidget): pass
>>> 
>>>         app = QApplication(sys.argv)
>>>         cmd = Command()
>>>         cmd.show()
>>>         app.exec()
>>> 
>>> segfaults here on the "cmd.Command()" line:
>>> 
>>>         _PyDict_Next (op=0x0, ppos=0x7fffffffbd50, 
>>> pkey=0x7fffffffbd58,
>>> pvalue=0x7fffffffbd60, phash=phash at entry=0x0) at
>>> Objects/dictobject.c:2114
>>>         2114       if (!PyDict_Check(op))
>>>         (gdb) bt
>>>         #0  _PyDict_Next (op=0x0, ppos=0x7fffffffbd50,
>>> pkey=0x7fffffffbd58,
>>> pvalue=0x7fffffffbd60, phash=phash at entry=0x0) at
>>> Objects/dictobject.c:2114
>>>         #1  0x000055555572a37e in PyDict_Next (op=<optimized out>,
>>> ppos=<optimized out>, pkey=<optimized out>, pvalue=<optimized out>) 
>>> at
>>> Objects/dictobject.c:2189
>>>         #2  0x00007ffff2436499 in trawl_hierarchy(_typeobject*,
>>> qpycore_metaobject*, QMetaObjectBuilder&, QList<EnumFlag>&,
>>> QList<_qpycore_pyqtSignal const*>&, QMap<unsigned int, 
>>> std::pair<_object*,
>>> _object*> >&) ()
>>>            from .../lib/python3.12/site-packages/PyQt6/QtCore.abi3.so
>>>         #3  0x00007ffff2436cc7 in trawl_hierarchy(_typeobject*,
>>> qpycore_metaobject*, QMetaObjectBuilder&, QList<EnumFlag>&,
>>> QList<_qpycore_pyqtSignal const*>&, QMap<unsigned int, 
>>> std::pair<_object*,
>>> _object*> >&) ()
>>>            from .../lib/python3.12/site-packages/PyQt6/QtCore.abi3.so
>>>         #4  0x00007ffff2436cc7 in trawl_hierarchy(_typeobject*,
>>> qpycore_metaobject*, QMetaObjectBuilder&, QList<EnumFlag>&,
>>> QList<_qpycore_pyqtSignal const*>&, QMap<unsigned int, 
>>> std::pair<_object*,
>>> _object*> >&) ()
>>>            from .../lib/python3.12/site-packages/PyQt6/QtCore.abi3.so
>>>         #5  0x00007ffff24357ba in
>>> qpycore_get_dynamic_metaobject(_sipWrapperType*) [clone .part.0] () 
>>> from
>>> .../lib/python3.12/site-packages/PyQt6/QtCore.abi3.so
>>>         #6  0x00007ffff24363d8 in
>>> qpycore_get_qmetaobject(_sipWrapperType*,
>>> _sipTypeDef const*) () from .../lib/python3.12/site-
>>> packages/PyQt6/QtCore.abi3.so
>>>         #7  0x00007ffff242a4e5 in 
>>> qpycore_qobject_finalisation(_object*,
>>> QObject*, _object*, _object**) () from .../lib/python3.12/site-
>>> packages/PyQt6/QtCore.abi3.so
>>>         #8  0x00007ffff74efbbc in sipSimpleWrapper_init
>>> (self=0x7ffff765ebb0,
>>> args=(), kwds=0x0) at sip_core.c:8950
>>>         #9  0x0000555555758f0d in type_call (type=<optimized out>,
>>> type at entry=0x555555e53e60, args=args at entry=(), kwds=kwds at entry=0x0) 
>>> at
>>> Objects/typeobject.c:1671
>>>         [...]
>>> 
>>> (Note the op=0x0).
>>> 
>>> I tried to build a debug PyQt, but it couldn't find Python.h of my
>>> (non-systemwide) debug Python, and I couldn't find a way to make
>>> sip-install
>>> pass a custom -I to gcc.
>>> 
>>> I was able to bisect it in CPython to this commit:
>>> 
>>> https://github.com/python/cpython/commit/de64e7561680fdc5358001e9488091e75d417
>>> 4a3
>>> <https://github.com/python/cpython/commit/de64e7561680fdc5358001e9488091e75d4174a3>
>>> which moves tp_dict/tp_bases/tp_mro to PyInterpreterState.
>>> 
>>> I lack the context to tell more about what's happening here.
>>> Is this a CPython bug? If so, is there any chance of reproducing it
>>> without
>>> PyQt involved?
>>> 
>>> Thanks,
>>> 
>>> Florian
>>> 
>> 
>> 
>> --
>> Thomas Caswell
>> tcaswell at gmail.com
>>