[PyQt] Deadlock in 5.9.1

Milorad Pop-Tosic pop at hiri.com
Thu Oct 26 17:15:43 BST 2017 

Hi,


We're still experiencing this deadlock in PyQt 5.9. It manifests itself as the application being frozen. We haven't been able to produce a test case which faithfully reproduces the problem since it looks like a timing issue. We got the stack traces below when we attached to the frozen process with GDB on Linux.


The stack trace indicates that the main thread got stuck in QObject.connect in PyQtMonitor::monitor which was apparently added in 5.9. At the same time, the worker thread is in a QObject destructor. Any ideas?

So far we know that it happens on Windows, Mac and Linux and on Python 3.5.2.

[ MAINTHREAD ]

#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1  0x00007fcaee602055 in QBasicMutex::lockInternal() () from target:/snap/hiri/x1/libQt5Core.so.5
#2  0x00007fcaee6020a6 in QMutex::lock() () from target:/snap/hiri/x1/libQt5Core.so.5
#3  0x00007fcaee80635a in ?? () from target:/snap/hiri/x1/libQt5Core.so.5
#4  0x00007fcaee809acb in QObject::connect(QObject const*, char const*, QObject const*, char const*, Qt::ConnectionType) () from target:/snap/hiri/x1/libQt5Core.so.5
#5  0x00007fcaeee6b0fd in PyQtMonitor::monitor(QObject*) () from target:/snap/hiri/x1/PyQt5.QtCore.so
#6  0x00007fcaeb99ce55 in sipSimpleWrapper_init () from target:/snap/hiri/x1/sip.so
#7  0x00007fcaf185bfe6 in type_call.lto_priv.175 (type=<optimized out>, args=0x7fcaf27b0048, kwds=0x0) at ../Objects/typeobject.c:905
#8  0x00007fcaf195254e in PyObject_Call (func=0x2071a88, arg=<optimized out>, kw=<optimized out>) at ../Objects/abstract.c:2165
#9  0x00007fcaeb99e8c6 in sipWrapInstance () from target:/snap/hiri/x1/sip.so
#10 0x00007fcaeb99249b in sip_api_convert_from_type () from target:/snap/hiri/x1/sip.so
#11 0x00007fcaeb99b088 in buildObject () from target:/snap/hiri/x1/sip.so
#12 0x00007fcaeb99b809 in call_method () from target:/snap/hiri/x1/sip.so
#13 0x00007fcaeb99ba0c in sip_api_call_method () from target:/snap/hiri/x1/sip.so
#14 0x00007fcaeee25da6 in sipVH_QtCore_1(PyGILState_STATE, void (*)(_sipSimpleWrapper*, PyGILState_STATE), _sipSimpleWrapper*, _object*, QObject*, QEvent*) () from target:/snap/hiri/x1/PyQt5.QtCore.so
#15 0x00007fcaeee27830 in sipQObject::eventFilter(QObject*, QEvent*) () from target:/snap/hiri/x1/PyQt5.QtCore.so
#16 0x00007fcaee7d8b44 in QCoreApplicationPrivate::sendThroughApplicationEventFilters(QObject*, QEvent*) () from target:/snap/hiri/x1/libQt5Core.so.5
#17 0x00007fcae66b24f8 in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from target:/snap/hiri/x1/libQt5Widgets.so.5
#18 0x00007fcae66b98e7 in QApplication::notify(QObject*, QEvent*) () from target:/snap/hiri/x1/libQt5Widgets.so.5
#19 0x00007fcae70fa2ae in sipQApplication::notify(QObject*, QEvent*) () from target:/snap/hiri/x1/PyQt5.QtWidgets.so
#20 0x00007fcaee7d8eb8 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from target:/snap/hiri/x1/libQt5Core.so.5
#21 0x00007fcaeadbea90 in QWindow::setIcon(QIcon const&) () from target:/snap/hiri/x1/libQt5Gui.so.5
#22 0x00007fcae66da9b2 in QWidgetPrivate::setWindowIcon_sys() () from target:/snap/hiri/x1/libQt5Widgets.so.5
#23 0x00007fcae66f08f5 in QWidget::event(QEvent*) () from target:/snap/hiri/x1/libQt5Widgets.so.5
#24 0x00007fcae68746e2 in QSplashScreen::event(QEvent*) () from target:/snap/hiri/x1/libQt5Widgets.so.5
#25 0x00007fcae707c993 in sipQSplashScreen::event(QEvent*) () from target:/snap/hiri/x1/PyQt5.QtWidgets.so
#26 0x00007fcae66b24bc in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from target:/snap/hiri/x1/libQt5Widgets.so.5
#27 0x00007fcae66b98e7 in QApplication::notify(QObject*, QEvent*) () from target:/snap/hiri/x1/libQt5Widgets.so.5
#28 0x00007fcae70fa2ae in sipQApplication::notify(QObject*, QEvent*) () from target:/snap/hiri/x1/PyQt5.QtWidgets.so
#29 0x00007fcaee7d8eb8 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from target:/snap/hiri/x1/libQt5Core.so.5
#30 0x00007fcae66b62d8 in QApplicationPrivate::notifyWindowIconChanged() () from target:/snap/hiri/x1/libQt5Widgets.so.5
#31 0x00007fcae6f8499b in meth_QApplication_setWindowIcon () from target:/snap/hiri/x1/PyQt5.QtWidgets.so
#32 0x00007fcaf1879039 in PyCFunction_Call (func=0x7fcad0055360, args=0x7fcac01c5780, kwds=<optimized out>) at ../Objects/methodobject.c:109
#33 0x00007fcaf19851b5 in call_function (oparg=<optimized out>, pp_stack=0x7ffe72df4b60) at ../Python/ceval.c:4705
#34 PyEval_EvalFrameEx (f=<optimized out>, throwflag=<optimized out>) at ../Python/ceval.c:3236
#35 0x00007fcaf1a15c6c in _PyEval_EvalCodeWithName.lto_priv.1742 (_co=0x7fcaef9e1810, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=0, kws=0x0, kwcount=0, defs=0x0,
    defcount=0, kwdefs=0x0, closure=0x0, name=0x0, qualname=0x0) at ../Python/ceval.c:4018
#36 0x00007fcaf1a15d43 in PyEval_EvalCodeEx (_co=<optimized out>, globals=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kws=<optimized out>, kwcount=0,
    defs=0x0, defcount=0, kwdefs=0x0, closure=0x0) at ../Python/ceval.c:4039
#37 0x00007fcaf197d9cb in PyEval_EvalCode (co=<optimized out>, globals=<optimized out>, locals=<optimized out>) at ../Python/ceval.c:777
#38 0x00000000004029e0 in ?? ()
#39 0x0000000000402db5 in ?? ()
#40 0x00007fcaf1e72830 in __libc_start_main (main=0x401930, argc=1, argv=0x7ffe72df8e68, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffe72df8e58)
    at ../csu/libc-start.c:291
#41 0x0000000000401969 in ?? ()

[ WORKER ]

Thread 5 (Thread 0x7fcac1bfd700 (LWP 21348)):
#0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:225
#1  0x00007fcaf197d47f in PyCOND_TIMEDWAIT (cond=0x7fcaf1e42360 <gil_cond>, mut=0x7fcaf1e423e0 <gil_mutex>, us=5000) at ../Python/condvar.h:103
#2  take_gil (tstate=tstate at entry=0x7fcab401cb40) at ../Python/ceval_gil.h:224
#3  0x00007fcaf197d969 in PyEval_RestoreThread (tstate=0x7fcab401cb40) at ../Python/ceval.c:450
#4  0x00007fcaf199d67b in PyGILState_Ensure () at ../Python/pystate.c:821
#5  0x00007fcaeb994c94 in sip_api_is_py_method () from target:/snap/hiri/x1/sip.so
#6  0x00007fcaeee21369 in sipQObject::disconnectNotify(QMetaMethod const&) () from target:/snap/hiri/x1/PyQt5.QtCore.so
#7  0x00007fcaee80af73 in QObject::~QObject() () from target:/snap/hiri/x1/libQt5Core.so.5
#8  0x00007fcaeede45b9 in sipQObject::~sipQObject() () from target:/snap/hiri/x1/PyQt5.QtCore.so
#9  0x00007fcaee8043f8 in QObject::event(QEvent*) () from target:/snap/hiri/x1/libQt5Core.so.5
#10 0x00007fcaeee2874b in sipQObject::event(QEvent*) () from target:/snap/hiri/x1/PyQt5.QtCore.so
#11 0x00007fcae66b24bc in QApplicationPrivate::notify_helper(QObject*, QEvent*) () from target:/snap/hiri/x1/libQt5Widgets.so.5
#12 0x00007fcae66b98e7 in QApplication::notify(QObject*, QEvent*) () from target:/snap/hiri/x1/libQt5Widgets.so.5
#13 0x00007fcae70fa2ae in sipQApplication::notify(QObject*, QEvent*) () from target:/snap/hiri/x1/PyQt5.QtWidgets.so
#14 0x00007fcaee7d8eb8 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from target:/snap/hiri/x1/libQt5Core.so.5
#15 0x00007fcaee7db54b in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from target:/snap/hiri/x1/libQt5Core.so.5
#16 0x00007fcaee82bad3 in ?? () from target:/snap/hiri/x1/libQt5Core.so.5
#17 0x00007fcaebe63197 in g_main_context_dispatch () from target:/snap/hiri/x1/lib/x86_64-linux-gnu/libglib-2.0.so.0
#18 0x00007fcaebe633f0 in ?? () from target:/snap/hiri/x1/lib/x86_64-linux-gnu/libglib-2.0.so.0
#19 0x00007fcaebe6349c in g_main_context_iteration () from target:/snap/hiri/x1/lib/x86_64-linux-gnu/libglib-2.0.so.0
#20 0x00007fcaee82b11f in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from target:/snap/hiri/x1/libQt5Core.so.5
#21 0x00007fcaee7d74aa in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from target:/snap/hiri/x1/libQt5Core.so.5
#22 0x00007fcaee605dfc in QThread::exec() () from target:/snap/hiri/x1/libQt5Core.so.5
#23 0x00007fcaeee3bc00 in sipQThread::run() () from target:/snap/hiri/x1/PyQt5.QtCore.so
#24 0x00007fcaee60a84a in ?? () from target:/snap/hiri/x1/libQt5Core.so.5
#25 0x00007fcaf15b56ba in start_thread (arg=0x7fcac1bfd700) at pthread_create.c:333
#26 0x00007fcaf1f593dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

Thanks

Milorad

________________________________
From: Milorad Pop-Tosic
Sent: Wednesday, August 30, 2017 2:29:04 PM
To: pyqt at riverbankcomputing.com
Subject: Deadlock in 5.9.1


We're seeing a deadlock after upgrading from PyQt5.8.2 to 5.9.1. Since our application is complex, it's difficult to pinpoint to what's causing this. The problem is intermittent, but it can be reproduced by repeatedly clicking on an action. I managed to get this stacktrace with VisualStudio:


  ntdll.dll!_NtWaitForSingleObject at 12
() Unknown
  KernelBase.dll!WaitForSingleObjectEx() Unknown
  python35.dll!take_gil(_ts * tstate=0x1f194160) Line 224 C
  python35.dll!PyEval_RestoreThread(_ts * tstate=0x1f194160) Line 452 C
  python35.dll!PyGILState_Ensure() Line 821 C
  sip.pyd!689b3355() Unknown
  [Frames below may be incorrect and/or missing, no symbols loaded for sip.pyd]
  QtCore.pyd!654b68a4() Unknown
> Qt5Core.dll!QObject::~QObject() Line 986 C++
  QtCore.pyd!654b4744() Unknown
  Qt5Core.dll!QObject::event(QEvent * e=0x1e632358) Line 1273 C++
  QtCore.pyd!654b73a7() Unknown
  Qt5Widgets.dll!QApplicationPrivate::notify_helper(QObject * receiver=0x1e632358, QEvent * e) Line 3717 C++
  Qt5Widgets.dll!QApplication::notify(QObject * receiver=0x1e632438, QEvent * e=0x19a6a948) Line 3680 C++
  QtWidgets.pyd!648bd38b() Unknown
  Qt5Core.dll!QCoreApplication::notifyInternal2(QObject * receiver, QEvent * event) Line 1018 C++
  Qt5Core.dll!QCoreApplicationPrivate::sendPostedEvents(QObject * receiver=0x00000000, int event_type, QThreadData * data=0x1e632438) Line 1678 C++
  Qt5Core.dll!QEventDispatcherWin32::sendPostedEvents() Line 1062 C++
  Qt5Core.dll!qt_internal_proc(HWND__ * hwnd=0x00460908, unsigned int message=0x00000401, unsigned int wp=0x00000000, long lp=0x00000000) Line 236 C++
  user32.dll!__InternalCallWinProc at 20
() Unknown
  user32.dll!_UserCallWinProcCheckWow at 36
() Unknown
  user32.dll!_DispatchMessageWorker at 8
() Unknown
  user32.dll!_DispatchMessageW at 4
() Unknown
  Qt5Core.dll!QEventDispatcherWin32::processEvents(QFlags<enum QEventLoop::ProcessEventsFlag> flags={...}) Line 634 C++
  Qt5Core.dll!QEventLoop::exec(QFlags<enum QEventLoop::ProcessEventsFlag> flags={...}) Line 211 C++
  Qt5Core.dll!QThread::exec() Line 515 C++
  QtCore.pyd!6543a023() Unknown
  kernel32.dll!@BaseThreadInitThunk at 12
() Unknown
  ntdll.dll!__RtlUserThreadStart() Unknown
  ntdll.dll!__RtlUserThreadStart at 8
() Unknown

As far as I understand this stacktrace, something is trying to get GIL at a QObject destruction, so I'm guessing that something that was supposed to release the GIL didn't.

We're going to keep trying to isolate the problem into a standalone test case, but any pointers would be much appreciated.


Thanks

Milorad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.riverbankcomputing.com/pipermail/pyqt/attachments/20171026/d76120aa/attachment-0001.html>

More information about the PyQt mailing list