Deadlock in QThreadPool destructor

Ales Erjavec ales.erjavec324 at gmail.com
Mon Jan 31 11:49:11 GMT 2022


Hi,

A deadlock can happen in QThreadPool destructor if any QRunnables are
still running/pending and attempt to acquire the GIL.

```
from PyQt5.QtCore import QThreadPool, QRunnable


class Runnable(QRunnable):
    def run(self):
        print("in run")


def test_one():
    pool = QThreadPool(maxThreadCount=1)
    pool.start(Runnable())
    pool.start(Runnable())
    pool.start(Runnable())
    pool.start(Runnable())
    del pool # should invoke ~QThreadPool here

i = 1
while True:
    print("Run ", i, flush=True)
    test_one()  # this mostly just executes once and deadlocks.
    i += 1
```

Relevant extract from program sample
```
Call graph:
    2721 Thread_2615382   DispatchQueue_1: com.apple.main-thread  (serial)
    + 2721 start  (in libdyld.dylib) + 1  [0x7fff6c6d83d5]
    +   2721 main  (in Python) + 53  [0x10d7b6f65]
    +     2721 Py_Main  (in Python) + 3554  [0x10d8dcb12]
    +       2721 PyRun_SimpleFileExFlags  (in Python) + 882  [0x10d8c2bf2]
    +         2721 PyRun_FileExFlags  (in Python) + 209  [0x10d8c3381]
    +           2721 PyEval_EvalCode  (in Python) + 100  [0x10d88d244]
    +             2721 _PyEval_EvalCodeWithName  (in Python) + 2447
[0x10d8981af]
    +               2721 _PyEval_EvalFrameDefault  (in Python) + 27559
 [0x10d893eb7]
    +                 2721 call_function  (in Python) + 401  [0x10d897721]
    +                   2721 fast_function  (in Python) + 381  [0x10d898a3d]
    +                     2721 _PyEval_EvalFrameDefault  (in Python) +
15504  [0x10d890fa0]
    +                       2721 subtype_dealloc  (in Python) + 1049
[0x10d827b99]
    +                         2721 sipWrapper_dealloc  (in
sip.cpython-36m-darwin.so) + 37  [0x10e7ba765]
    +                           2721 forgetObject  (in
sip.cpython-36m-darwin.so) + 182  [0x10e7bab96]
    +                             2721
sipQThreadPool::~sipQThreadPool()  (in QtCore.abi3.so) + 44
[0x10de40d4c]
    +                               2721 QThreadPool::~QThreadPool()
(in QtCore) + 34  [0x10e12e4b2]
    +                                 2721
QThreadPoolPrivate::waitForDone(int)  (in QtCore) + 179  [0x10e12df73]
    +                                   2721
QWaitCondition::wait(QMutex*, QDeadlineTimer)  (in QtCore) + 93
[0x10e13080d]
    +                                     2721
QWaitConditionPrivate::wait(QDeadlineTimer)  (in QtCore) + 59
[0x10e1308cb]
    +                                       2721 _pthread_cond_wait
(in libsystem_pthread.dylib) + 722  [0x7fff6c8cf56e]
    +                                         2721 __psynch_cvwait
(in libsystem_kernel.dylib) + 10  [0x7fff6c810866]
    2721 Thread_2615392: Thread (pooled)
      2721 thread_start  (in libsystem_pthread.dylib) + 13  [0x7fff6c8cb40d]
        2721 _pthread_start  (in libsystem_pthread.dylib) + 66  [0x7fff6c8cf249]
          2721 _pthread_body  (in libsystem_pthread.dylib) + 126
[0x7fff6c8cc2eb]
            2721 QThreadPrivate::start(void*)  (in QtCore) + 329  [0x10e128a79]
              2721 QThreadPoolThread::run()  (in QtCore) + 124  [0x10e12cf4c]
                2721 sipQRunnable::run()  (in QtCore.abi3.so) + 51
[0x10de549a3]
                  2721 sip_api_is_py_method_12_8  (in
sip.cpython-36m-darwin.so) + 68  [0x10e7bfa74]
                    2721 PyGILState_Ensure  (in Python) + 109  [0x10d8c257d]
                      2721 PyEval_RestoreThread  (in Python) + 41  [0x10d88ce99]
                        2721 take_gil  (in Python) + 235  [0x10d88c8db]
                          2716 _pthread_cond_wait  (in
libsystem_pthread.dylib) + 722  [0x7fff6c8cf56e]
                          ! 2656 __psynch_cvwait  (in
libsystem_kernel.dylib) + 10,12  [0x7fff6c810866,0x7fff6c810868]
                          ! 60 cerror_nocancel  (in
libsystem_kernel.dylib) + 20,0,...
[0x7fff6c80d467,0x7fff6c80d453,...]
                          2 _pthread_cond_wait  (in
libsystem_pthread.dylib) + 453  [0x7fff6c8cf461]
                          ! 2 __gettimeofday  (in
libsystem_kernel.dylib) + 17  [0x7fff6c810e32]
                          2 _pthread_cond_wait  (in
libsystem_pthread.dylib) + 726,731  [0x7fff6c8cf572,0x7fff6c8cf577]
                          1 _pthread_cond_wait  (in
libsystem_pthread.dylib) + 381  [0x7fff6c8cf419]
                            1 _pthread_mutex_droplock  (in
libsystem_pthread.dylib) + 494  [0x7fff6c8cd500]
```

The same seems to happen in PyQt5 and PyQt6

Granted that this is a bad code (it should explicitly wait on the pool
and/or all the runnable tasks), but the ~QThreadPool does document
that it will wait for all tasks to complete and should therefore
release the GIL.

Thanks


More information about the PyQt mailing list