Details
P3: Somewhat important Description
This is from original discussion on the mailing list. Unfortunately I didn't manage to create a minimal example reproducing the issue. However, from the stack traces below it seems to be clear that the signal.disconnect call results in trying to lock a mutex (BasicMutex::lockInternal) while holding the GIL (which can be seen when looking at the stack trace of the other thread which hangs in PyGILState_Ensure.
To be honest I don't know the internals of PySide/shiboken good enough to create a minimal reproducing example, but I assume that the disconnect call is supposed to release the GIL before actually trying to lock the mutex.
Here is my message from the mailing list describing the issue:
The problem occurs very rarely (maybe in one of 100 tries), but it is reproducable in a larger application. Please forgive to not have a stripped down test, maybe someone can already see what is happening from the information given. If not, I'd probably have to reproduce the issue in a smaller test case.
It seems that the signal.disconnect deadlocks sometimes.
There are two classes involved:
class MyFilter(ClassDerivingFromQObject): # snip def onOpen(self): logger.info("onOpen start") pb = Services.getService("PlaybackControl") pb.sequenceOpened.connect(self.seqOpened) logger.info("onOpen stop") def onClose(self): logger.info("onClose start") pb = Services.getService("PlaybackControl") # pb is a QSharedPointer to a python class deriving from QObject (the project uses shiboken) s = pb.sequenceOpened s.disconnect(self.seqOpened) # this call deadlocks logger.info("onClose stop") def seqOpened(self, *args): logger.info("seqOpened")
And the other involved class
class GenericReader(AnotherClassDerivingFromQObject): # snip def onClose(self): pb = Services.getService("PlaybackControl") # pb is a QSharedPointer to a python class deriving from QObject (the project uses shiboken) pb.removeConnections(self) # this call deadlocks
I managed to get the python tracebacks using the very cool project py-spy:
Thread 197717 (idle): "Dummy-1" onClose (myfilter.py:23) _stateTransition (nexxT/core/FilterEnvironment.py:310) close (nexxT/core/FilterEnvironment.py:366) performOperation (nexxT/core/Thread.py:197) run (nexxT/core/Thread.py:52) Thread 197716 (idle) onClose (nexxT/filters/GenericReader.py:361) _stateTransition (nexxT/core/FilterEnvironment.py:310) close (nexxT/core/FilterEnvironment.py:366) performOperation (nexxT/core/Thread.py:197) run (nexxT/core/Thread.py:52)
And the tracebacks from gdb of these threads are here:
Thread 10 (Thread 0x7f5aa4a8a6c0 (LWP 197717) "filter"): #0 0x00007f5ac1dcf4f9 in syscall () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007f5aab4cae95 in QBasicMutex::lockInternal() () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #2 0x00007f5aab38f2d5 in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #3 0x00007f5aab390307 in QMetaObject::disconnectOne(QObject const*, int, QObject const*, int) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #4 0x00007f5aac6af048 in PySide::qobjectDisconnectCallback(QObject*, char const*, _object*) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/libpyside6.abi3.so.6.4 #5 0x00007f5aabbd985b in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/QtCore.abi3.so #6 0x0000000000547ac6 in ?? () #7 0x00000000005f7f50 in PyObject_CallObject () #8 0x00007f5aac6a489d in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/libpyside6.abi3.so.6.4 #9 0x00000000005230d0 in ?? () #10 0x000000000053ac2c in PyObject_Vectorcall () #11 0x000000000052b940 in _PyEval_EvalFrameDefault () #12 0x00000000005855a4 in ?? () #13 0x000000000058510e in ?? () #14 0x00007f5aac69d2f1 in PySide::SignalManager::callPythonMetaMethod(QMetaMethod const&, void**, _object*, bool) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/libpyside6.abi3.so.6.4 #15 0x00007f5aac69d5d6 in PySide::SignalManager::SignalManagerPrivate::qtMethodMetacall(QObject*, int, void**) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/libpyside6.abi3.so.6.4 #16 0x00007f5aab38937c in QObject::event(QEvent*) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #17 0x00007f5aab33929a in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #18 0x00007f5aab33fdfd in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #19 0x00007f5aab5e11b3 in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #20 0x00007f5aab91c7a9 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0 #21 0x00007f5aab91ca38 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0 #22 0x00007f5aab91cacc in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0 #23 0x00007f5aab5e0b3a in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #24 0x00007f5aab344d5b in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #25 0x00007f5aab450eac in QThread::exec() () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #26 0x00007f5aabc729c0 in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/QtCore.abi3.so #27 0x000000000051f567 in ?? () #28 0x000000000053ac2c in PyObject_Vectorcall () #29 0x000000000052b940 in _PyEval_EvalFrameDefault () #30 0x00000000005855a4 in ?? () #31 0x0000000000585148 in ?? () #32 0x00007f5aabc72915 in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/QtCore.abi3.so #33 0x00007f5aab4cac7f in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #34 0x00007f5ac1d56fd4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #35 0x00007f5ac1dd75bc in ?? () from /lib/x86_64-linux-gnu/libc.so.6 Thread 9 (Thread 0x7f5a9c2896c0 (LWP 197716) "reader"): #0 0x00007f5ac1d53d36 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #1 0x00007f5ac1d566dc in pthread_cond_timedwait () from /lib/x86_64-linux-gnu/libc.so.6 #2 0x00000000004f91da in ?? () #3 0x0000000000528bf2 in PyEval_RestoreThread () #4 0x000000000064dacc in PyGILState_Ensure () #5 0x00007f5ac141e16d in Shiboken::GilState::GilState() () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/shiboken6/libshiboken6.abi3.so.6.4 #6 0x00007f5ac08cae13 in FilterWrapper::disconnectNotify(QMetaMethod const&) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/nexxT/binary/linux_x86_64/release/cnexxT.abi3.so #7 0x00007f5aab39546a in QObject::~QObject() () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #8 0x00007f5aabbd6ae6 in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/QtCore.abi3.so #9 0x00007f5ac141b93a in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/shiboken6/libshiboken6.abi3.so.6.4 #10 0x000000000058cedd in ?? () #11 0x0000000000515d7f in ?? () #12 0x000000000053fb3f in ?? () #13 0x000000000052b338 in _PyEval_EvalFrameDefault () #14 0x00000000005855a4 in ?? () #15 0x000000000058510e in ?? () #16 0x00007f5aac69d2f1 in PySide::SignalManager::callPythonMetaMethod(QMetaMethod const&, void**, _object*, bool) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/libpyside6.abi3.so.6.4 #17 0x00007f5aac69d5d6 in PySide::SignalManager::SignalManagerPrivate::qtMethodMetacall(QObject*, int, void**) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/libpyside6.abi3.so.6.4 #18 0x00007f5aab38937c in QObject::event(QEvent*) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #19 0x00007f5aab33929a in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #20 0x00007f5aab33fdfd in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #21 0x00007f5aab5e11b3 in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #22 0x00007f5aab91c7a9 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0 #23 0x00007f5aab91ca38 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0 #24 0x00007f5aab91cacc in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0 #25 0x00007f5aab5e0b3a in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #26 0x00007f5aab344d5b in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #27 0x00007f5aab450eac in QThread::exec() () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #28 0x00007f5aabc729c0 in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/QtCore.abi3.so #29 0x000000000051f567 in ?? () #30 0x000000000053ac2c in PyObject_Vectorcall () #31 0x000000000052b940 in _PyEval_EvalFrameDefault () #32 0x00000000005855a4 in ?? () #33 0x0000000000585148 in ?? () #34 0x00007f5aabc72915 in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/QtCore.abi3.so #35 0x00007f5aab4cac7f in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6 #36 0x00007f5ac1d56fd4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6 #37 0x00007f5ac1dd75bc in ?? () from /lib/x86_64-linux-gnu/libc.so.6