Python 3.12: "Python memory allocator called without holding the GIL" on shutdown

[Florian Bruhin]

Hey,

After avoiding multiple inheritance, I was able to run almost the entire
qutebrowser testsuite successfully on Python 3.12.

There seems to be one remaining issue, where something like:

    python3.12 -c "from PyQt6.QtWidgets import QApplication" 

results in:

    Fatal Python error: _PyMem_DebugFree: Python memory allocator called without holding the GIL
    Python runtime state: finalizing (tstate=0x0000555555c75118)

With the following stacktrace:

	[...]
	#4  0x000055555584790a in fatal_error_exit (status=<optimized out>) at Python/pylifecycle.c:2693
	#5  0x00005555558491c4 in fatal_error (fd=2, header=header at entry=1, prefix=prefix at entry=0x5555559206d0 <__func__.3> "_PyMem_DebugFree", msg=msg at entry=0x55555591ff70 "Python memory allocator called without holding the GIL", 
		status=status at entry=-1) at Python/pylifecycle.c:2874
	#6  0x000055555584922e in _Py_FatalErrorFunc (func=func at entry=0x5555559206d0 <__func__.3> "_PyMem_DebugFree", msg=msg at entry=0x55555591ff70 "Python memory allocator called without holding the GIL") at Python/pylifecycle.c:2890
	#7  0x000055555573f0ba in _PyMem_DebugCheckGIL (func=func at entry=0x5555559206d0 <__func__.3> "_PyMem_DebugFree") at Objects/obmalloc.c:2146
	#8  0x000055555573f137 in _PyMem_DebugFree (ctx=0x555555c01168 <_PyRuntime+200>, ptr=0x555555df1cc0) at Objects/obmalloc.c:2170
	#9  0x00005555557406a5 in PyMem_Free (ptr=<optimized out>) at Objects/obmalloc.c:616
	#10 0x00007ffff74e5296 in sip_api_free (mem=<optimized out>) at sip_core.c:1905
	#11 0x00007ffff74f529a in sipOMFinalise (om=om at entry=0x7ffff74ff820 <cppPyMap>) at sip_object_map.c:69
	#12 0x00007ffff74e5351 in finalise () at sip_core.c:1807
	#13 0x0000555555847634 in call_ll_exitfuncs (runtime=runtime at entry=0x555555c010a0 <_PyRuntime>) at Python/pylifecycle.c:2993
	#14 0x0000555555848d9b in Py_FinalizeEx () at Python/pylifecycle.c:1966
	#15 0x00005555558755d9 in Py_RunMain () at Modules/main.c:691
	[...]

I was able to bisect this to the following Python change:

	https://github.com/python/cpython/commit/6036c3e856f033bf13e929536e7bf127fdd921c9
	"Clarify GILState-related Code"
    https://github.com/python/cpython/pull/101161

As before: Is this a CPython bug, or just some problem in sip/PyQt uncovered by
that change?

Thanks,
Florian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <https://www.riverbankcomputing.com/pipermail/pyqt/attachments/20230609/58504033/attachment.sig>