Deadlock in QThreadPool destructor
Ales Erjavec
ales.erjavec324 at gmail.com
Mon Jan 31 11:49:11 GMT 2022
Hi,
A deadlock can happen in QThreadPool destructor if any QRunnables are
still running/pending and attempt to acquire the GIL.
```
from PyQt5.QtCore import QThreadPool, QRunnable
class Runnable(QRunnable):
def run(self):
print("in run")
def test_one():
pool = QThreadPool(maxThreadCount=1)
pool.start(Runnable())
pool.start(Runnable())
pool.start(Runnable())
pool.start(Runnable())
del pool # should invoke ~QThreadPool here
i = 1
while True:
print("Run ", i, flush=True)
test_one() # this mostly just executes once and deadlocks.
i += 1
```
Relevant extract from program sample
```
Call graph:
2721 Thread_2615382 DispatchQueue_1: com.apple.main-thread (serial)
+ 2721 start (in libdyld.dylib) + 1 [0x7fff6c6d83d5]
+ 2721 main (in Python) + 53 [0x10d7b6f65]
+ 2721 Py_Main (in Python) + 3554 [0x10d8dcb12]
+ 2721 PyRun_SimpleFileExFlags (in Python) + 882 [0x10d8c2bf2]
+ 2721 PyRun_FileExFlags (in Python) + 209 [0x10d8c3381]
+ 2721 PyEval_EvalCode (in Python) + 100 [0x10d88d244]
+ 2721 _PyEval_EvalCodeWithName (in Python) + 2447
[0x10d8981af]
+ 2721 _PyEval_EvalFrameDefault (in Python) + 27559
[0x10d893eb7]
+ 2721 call_function (in Python) + 401 [0x10d897721]
+ 2721 fast_function (in Python) + 381 [0x10d898a3d]
+ 2721 _PyEval_EvalFrameDefault (in Python) +
15504 [0x10d890fa0]
+ 2721 subtype_dealloc (in Python) + 1049
[0x10d827b99]
+ 2721 sipWrapper_dealloc (in
sip.cpython-36m-darwin.so) + 37 [0x10e7ba765]
+ 2721 forgetObject (in
sip.cpython-36m-darwin.so) + 182 [0x10e7bab96]
+ 2721
sipQThreadPool::~sipQThreadPool() (in QtCore.abi3.so) + 44
[0x10de40d4c]
+ 2721 QThreadPool::~QThreadPool()
(in QtCore) + 34 [0x10e12e4b2]
+ 2721
QThreadPoolPrivate::waitForDone(int) (in QtCore) + 179 [0x10e12df73]
+ 2721
QWaitCondition::wait(QMutex*, QDeadlineTimer) (in QtCore) + 93
[0x10e13080d]
+ 2721
QWaitConditionPrivate::wait(QDeadlineTimer) (in QtCore) + 59
[0x10e1308cb]
+ 2721 _pthread_cond_wait
(in libsystem_pthread.dylib) + 722 [0x7fff6c8cf56e]
+ 2721 __psynch_cvwait
(in libsystem_kernel.dylib) + 10 [0x7fff6c810866]
2721 Thread_2615392: Thread (pooled)
2721 thread_start (in libsystem_pthread.dylib) + 13 [0x7fff6c8cb40d]
2721 _pthread_start (in libsystem_pthread.dylib) + 66 [0x7fff6c8cf249]
2721 _pthread_body (in libsystem_pthread.dylib) + 126
[0x7fff6c8cc2eb]
2721 QThreadPrivate::start(void*) (in QtCore) + 329 [0x10e128a79]
2721 QThreadPoolThread::run() (in QtCore) + 124 [0x10e12cf4c]
2721 sipQRunnable::run() (in QtCore.abi3.so) + 51
[0x10de549a3]
2721 sip_api_is_py_method_12_8 (in
sip.cpython-36m-darwin.so) + 68 [0x10e7bfa74]
2721 PyGILState_Ensure (in Python) + 109 [0x10d8c257d]
2721 PyEval_RestoreThread (in Python) + 41 [0x10d88ce99]
2721 take_gil (in Python) + 235 [0x10d88c8db]
2716 _pthread_cond_wait (in
libsystem_pthread.dylib) + 722 [0x7fff6c8cf56e]
! 2656 __psynch_cvwait (in
libsystem_kernel.dylib) + 10,12 [0x7fff6c810866,0x7fff6c810868]
! 60 cerror_nocancel (in
libsystem_kernel.dylib) + 20,0,...
[0x7fff6c80d467,0x7fff6c80d453,...]
2 _pthread_cond_wait (in
libsystem_pthread.dylib) + 453 [0x7fff6c8cf461]
! 2 __gettimeofday (in
libsystem_kernel.dylib) + 17 [0x7fff6c810e32]
2 _pthread_cond_wait (in
libsystem_pthread.dylib) + 726,731 [0x7fff6c8cf572,0x7fff6c8cf577]
1 _pthread_cond_wait (in
libsystem_pthread.dylib) + 381 [0x7fff6c8cf419]
1 _pthread_mutex_droplock (in
libsystem_pthread.dylib) + 494 [0x7fff6c8cd500]
```
The same seems to happen in PyQt5 and PyQt6
Granted that this is a bad code (it should explicitly wait on the pool
and/or all the runnable tasks), but the ~QThreadPool does document
that it will wait for all tasks to complete and should therefore
release the GIL.
Thanks
More information about the PyQt
mailing list