[PyQt] [BUG] PyQt5 default behaviour related to sip.setdestroyonexit()

Tue Dec 15 13:48:46 GMT 2015

On Tue, Dec 15, 2015 at 2:46 PM, Phil Thompson
<phil at riverbankcomputing.com> wrote:
>
>> On 14 Dec 2015, at 5:08 p.m., Vladimir Rutsky <rutsky.vladimir at gmail.com> wrote:
>>
>> Attached example crashes under debug Python.
>>
>> On Ubuntu 14.04 with debug Python 3.4.3, Qt 5.5.1, PyQt 5.5.1:
>>
>> $ QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1 python pyqt5_destroy_on_exit_test.py
>> Weak ref <weakref at 0x7f8dc000f358; dead> is dead
>> Segmentation fault (core dumped)
>> $
>
> I can't reproduce the problem.

In which environment you tried to reproduce this issue?
Did you use debug Python interpreter and did you enable Qt Accessibility?
This issue is not reproduced on release version of Python.

Just in case here are detailed instructions to reproduce this bug:

1. Obtain Python interpreter with enabled debug.
2. Run example in debug Python interpreter with enabled Qt
Accessibility support.

In Linux I can force accessibility support using
QT_LINUX_ACCESSIBILITY_ALWAYS_ON
environment variable:

$ QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1 python pyqt5_destroy_on_exit_test.py

In Windows 7 Qt accessibility enabled by default, I think.

3. Window with single button appears. Click "Push me" button once.
4. Context menu with "test" appears. Click on "test" menu item.
5. Close window with "Push me" button.

Observed behaviour: program crashes on exit.

>
>> On Windows 7 with debug Python 3.4.3, Qt 5.5.1, PyQt 5.5.1 fails
>> without additional accessibility configuration.
>>
>> As far as I understand crash is caused by destructor of
>> QAccessibleCache (QtGui module private singleton) which tries to
>> destroy connection to QMenu (destroy[QObject] signal) which has been
>> already freed by then *but not C++-desctructed*. QMenu object is freed
>> by gc after interpreter exit because it's trapped in reference cycle.
>> Sip dealloc doesn't call C++-destructor because interpreter is nulled
>> and destroy_on_exit is False by default
>
> snip
>
>> If I change default destroy_on_exit behaviour with
>> sip.setdestroyonexit(True) everything works correctly. It's not clear
>> why objects are not destroyed on exit by default.
>
> "Objects" here means instances of C++ classes.
Yes.

> It's because the order in which the dtors are called cannot be predicted and can cause crashes on exit if Qt doesn't like the order.
Can you provide example of Qt objects for which destruction order matters?

For most QObject hierarchies order of C++ objects destruction
shouldn't matter AFAIK.

>
>> Anyway, if objects are not destroyed on exit their memory shouldn't be
>> freed too, otherwise code similar to that of the attached example will
>> fail.
>
> I don't understand this sentence. There are two "objects" - the C++ class instance and the Python object that wraps it. setdestroyonexit() applies to the C++ class instance. I don't know what you mean by "their memory".

I thought that sip wrapper (Python object) and C++ object were
allocated in a single malloc() call
(with placement new to call C++ constructor), so Python and C++
objects share same memory chunk.
And I thought that with setdestroyonexit(False) that memory chunk is
being deallocated using free(),
but without calling C++ destructor.
Now I see that my assumptions were incorrect: C++ object is allocated
with a simple "new" inside Python object,
that wraps that C++ object (e.g. C++ allocation is done
init_type_QMenu, and deallocation in release_QMenu/dealloc_QMenu).

>
> Can you give me a stack trace of the crash?
>
> Phil

Here is stack trace on Ubuntu 14.04 (Qt without debug symbols):

$ QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1 gdb --args python
pyqt5_destroy_on_exit_test.py
GNU gdb (Ubuntu 7.9-1ubuntu1) 7.9
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python...done.
(gdb) r
Starting program:
/home/bob/work/modernization/protoacs/env_dbg/bin/python
pyqt5_destroy_on_exit_test.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Weak ref <weakref at 0x7fffdf17c358; dead> is dead
[New Thread 0x7fffdf9b2700 (LWP 13728)]
[New Thread 0x7fffe03f6700 (LWP 13727)]
[New Thread 0x7fffea824700 (LWP 13726)]

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff5c282b8 in qpycore_qobject_metaobject
(pySelf=0x7fffdf185898, base=0x7fffed84a080
<sipTypeDef_QtWidgets_QMenu>) at
../qpy/QtCore/qpycore_qobject_helpers.cpp:52
warning: Source file is more recent than executable.
52          if (pySelf && ((pyqtWrapperType *)Py_TYPE(pySelf))->metaobject)
(gdb) bt
#0  0x00007ffff5c282b8 in qpycore_qobject_metaobject
(pySelf=0x7fffdf185898, base=0x7fffed84a080
<sipTypeDef_QtWidgets_QMenu>) at
../qpy/QtCore/qpycore_qobject_helpers.cpp:52
#1  0x00007fffed334e64 in sipQMenu::metaObject (this=0xeef160) at
sipQtWidgetspart0.cpp:132467
#2  0x00007ffff5564a63 in QObject::~QObject() () from
/home/bob/Qt/5.5/gcc_64/lib/libQt5Core.so.5
#3  0x00007ffff1698058 in ?? () from /home/bob/Qt/5.5/gcc_64/lib/libQt5Gui.so.5
#4  0x00007ffff6adb259 in __run_exit_handlers (status=0,
listp=0x7ffff6e5d6c8 <__exit_funcs>,
run_list_atexit=run_list_atexit at entry=true) at exit.c:82
#5  0x00007ffff6adb2a5 in __GI_exit (status=<optimized out>) at exit.c:104
#6  0x00007ffff6ac0ecc in __libc_start_main (main=0x41e4d6 <main>,
argc=2, argv=0x7fffffffdc58, init=<optimized out>, fini=<optimized
out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdc48)
    at libc-start.c:321
#7  0x000000000041e409 in _start ()
(gdb) list
47      // This is the helper for all implementations of QObject::metaObject().
48      const QMetaObject *qpycore_qobject_metaobject(sipSimpleWrapper *pySelf,
49              sipTypeDef *base)
50      {
51          // Return the dynamic meta-object if there is one.
52          if (pySelf && ((pyqtWrapperType *)Py_TYPE(pySelf))->metaobject)
53              return ((pyqtWrapperType *)Py_TYPE(pySelf))->metaobject->mo;
54
55          // Fall back to the static Qt meta-object.
56          return reinterpret_cast<const QMetaObject
*>(((pyqt5ClassTypeDef *)base)->static_metaobject);
(gdb)

On Windows I have Qt with debug symbols:

  QtCore_d.pyd!qpycore_qobject_metaobject(_sipSimpleWrapper *
pySelf=0x044b6928, _sipTypeDef * base=0x0435fdc8) Line 52 C++
  QtWidgets_d.pyd!sipQMenu::metaObject() Line 197613 C++
  Qt5Cored.dll!QObject::~QObject() Line 987 C++
  Qt5Guid.dll!QAccessibleCache::~QAccessibleCache() C++
  Qt5Guid.dll!QAccessibleCache::`vector deleting destructor'(unsigned int) C++
  Qt5Guid.dll!``anonymous
namespace'::Q_QGS_qAccessibleCache::innerFunction'::`8'::Cleanup::~Cleanup()
Line 46 C++
  Qt5Guid.dll!``anonymous
namespace'::Q_QGS_qAccessibleCache::innerFunction'::`9'::`dynamic
atexit destructor for 'cleanup''() C++
  Qt5Guid.dll!_CRT_INIT(void * hDllHandle=0x03370000, unsigned long
dwReason=0, void * lpreserved=0x00000001) Line 416 C
  Qt5Guid.dll!__DllMainCRTStartup(void * hDllHandle=0x03370000,
unsigned long dwReason=0, void * lpreserved=0x00000001) Line 522 C
  Qt5Guid.dll!_DllMainCRTStartup(void * hDllHandle=0x03370000,
unsigned long dwReason=0, void * lpreserved=0x00000001) Line 472 C
  ntdll.dll!_LdrpCallInitRoutine at 16 () Unknown
  ntdll.dll!_LdrShutdownProcess at 0 () Unknown
  ntdll.dll!_RtlExitUserProcess at 4 () Unknown
  kernel32.dll!_ExitProcessStub at 4 () Unknown
  msvcr110d.dll!__crtExitProcess(int status=0) Line 726 C
  msvcr110d.dll!doexit(int code=0, int quick=0, int retcaller=0) Line 639 C
  msvcr110d.dll!exit(int code=0) Line 395 C
  python_d.exe!__tmainCRTStartup() Line 549 C
  python_d.exe!wmainCRTStartup() Line 377 C
  kernel32.dll!@BaseThreadInitThunk at 12 () Unknown
  ntdll.dll!___RtlUserThreadStart at 8 () Unknown
  ntdll.dll!__RtlUserThreadStart at 8 () Unknown

Regards,

Vladimir Rutsky