Uploaded image for project: 'Qt for Python'
  1. Qt for Python
  2. PYSIDE-2367

QSignal.disconnect deadlocks when overwriting disconnectNotify() in multiple thread setups

    XMLWordPrintable

Details

    • Bug
    • Resolution: Fixed
    • P3: Somewhat important
    • 6.6.0
    • 6.4.1, 6.5.3
    • PySide
    • None
    • Linux/X11, Windows
    • 310e20676 (dev), 5d6f08b82 (6.6), 5f5d7e0fc (6.5)

    Description

      This is from original discussion on the mailing list. Unfortunately I didn't manage to create a minimal example reproducing the issue. However, from the stack traces below it seems to be clear that the signal.disconnect call results in trying to lock a mutex (BasicMutex::lockInternal) while holding the GIL (which can be seen when looking at the stack trace of the other thread which hangs in PyGILState_Ensure.

      To be honest I don't know the internals of PySide/shiboken good enough to create a minimal reproducing example, but I assume that the disconnect call is supposed to release the GIL before actually trying to lock the mutex.

      Here is my message from the mailing list describing the issue:

      The problem occurs very rarely (maybe in one of 100 tries), but it is reproducable in a larger application. Please forgive to not have a stripped down test, maybe someone can already see what is happening from the information given. If not, I'd probably have to reproduce the issue in a smaller test case.

      It seems that the signal.disconnect deadlocks sometimes.

      There are two classes involved:

      class MyFilter(ClassDerivingFromQObject):
          # snip
          def onOpen(self):
              logger.info("onOpen start")
              pb = Services.getService("PlaybackControl")
              pb.sequenceOpened.connect(self.seqOpened)
              logger.info("onOpen stop")
      
          def onClose(self):
              logger.info("onClose start")
              pb = Services.getService("PlaybackControl") # pb is a QSharedPointer to a python class deriving from QObject (the project uses shiboken)
              s = pb.sequenceOpened
              s.disconnect(self.seqOpened) # this call deadlocks
              logger.info("onClose stop")
      
          def seqOpened(self, *args):
              logger.info("seqOpened")
      

      And the other involved class

      class GenericReader(AnotherClassDerivingFromQObject):
          # snip
         def onClose(self):
             pb = Services.getService("PlaybackControl") # pb is a QSharedPointer to a python class deriving from QObject (the project uses shiboken)
             pb.removeConnections(self) # this call deadlocks
      

      I managed to get the python tracebacks using the very cool project py-spy:

      Thread 197717 (idle): "Dummy-1"
         onClose (myfilter.py:23)
         _stateTransition (nexxT/core/FilterEnvironment.py:310)
         close (nexxT/core/FilterEnvironment.py:366)
         performOperation (nexxT/core/Thread.py:197)
         run (nexxT/core/Thread.py:52)
      Thread 197716 (idle)
         onClose (nexxT/filters/GenericReader.py:361)
         _stateTransition (nexxT/core/FilterEnvironment.py:310)
         close (nexxT/core/FilterEnvironment.py:366)
         performOperation (nexxT/core/Thread.py:197)
         run (nexxT/core/Thread.py:52)
      

      And the tracebacks from gdb of these threads are here:

      Thread 10 (Thread 0x7f5aa4a8a6c0 (LWP 197717) "filter"):
      #0  0x00007f5ac1dcf4f9 in syscall () from /lib/x86_64-linux-gnu/libc.so.6
      #1  0x00007f5aab4cae95 in QBasicMutex::lockInternal() () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #2  0x00007f5aab38f2d5 in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #3  0x00007f5aab390307 in QMetaObject::disconnectOne(QObject const*, int, QObject const*, int) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #4  0x00007f5aac6af048 in PySide::qobjectDisconnectCallback(QObject*, char const*, _object*) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/libpyside6.abi3.so.6.4
      #5  0x00007f5aabbd985b in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/QtCore.abi3.so
      #6  0x0000000000547ac6 in ?? ()
      #7  0x00000000005f7f50 in PyObject_CallObject ()
      #8  0x00007f5aac6a489d in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/libpyside6.abi3.so.6.4
      #9  0x00000000005230d0 in ?? ()
      #10 0x000000000053ac2c in PyObject_Vectorcall ()
      #11 0x000000000052b940 in _PyEval_EvalFrameDefault ()
      #12 0x00000000005855a4 in ?? ()
      #13 0x000000000058510e in ?? ()
      #14 0x00007f5aac69d2f1 in PySide::SignalManager::callPythonMetaMethod(QMetaMethod const&, void**, _object*, bool) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/libpyside6.abi3.so.6.4
      #15 0x00007f5aac69d5d6 in PySide::SignalManager::SignalManagerPrivate::qtMethodMetacall(QObject*, int, void**) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/libpyside6.abi3.so.6.4
      #16 0x00007f5aab38937c in QObject::event(QEvent*) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #17 0x00007f5aab33929a in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #18 0x00007f5aab33fdfd in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #19 0x00007f5aab5e11b3 in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #20 0x00007f5aab91c7a9 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
      #21 0x00007f5aab91ca38 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
      #22 0x00007f5aab91cacc in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
      #23 0x00007f5aab5e0b3a in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #24 0x00007f5aab344d5b in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #25 0x00007f5aab450eac in QThread::exec() () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #26 0x00007f5aabc729c0 in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/QtCore.abi3.so
      #27 0x000000000051f567 in ?? ()
      #28 0x000000000053ac2c in PyObject_Vectorcall ()
      #29 0x000000000052b940 in _PyEval_EvalFrameDefault ()
      #30 0x00000000005855a4 in ?? ()
      #31 0x0000000000585148 in ?? ()
      #32 0x00007f5aabc72915 in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/QtCore.abi3.so
      #33 0x00007f5aab4cac7f in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #34 0x00007f5ac1d56fd4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
      #35 0x00007f5ac1dd75bc in ?? () from /lib/x86_64-linux-gnu/libc.so.6
      
      Thread 9 (Thread 0x7f5a9c2896c0 (LWP 197716) "reader"):
      #0  0x00007f5ac1d53d36 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
      #1  0x00007f5ac1d566dc in pthread_cond_timedwait () from /lib/x86_64-linux-gnu/libc.so.6
      #2  0x00000000004f91da in ?? ()
      #3  0x0000000000528bf2 in PyEval_RestoreThread ()
      #4  0x000000000064dacc in PyGILState_Ensure ()
      #5  0x00007f5ac141e16d in Shiboken::GilState::GilState() () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/shiboken6/libshiboken6.abi3.so.6.4
      #6  0x00007f5ac08cae13 in FilterWrapper::disconnectNotify(QMetaMethod const&) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/nexxT/binary/linux_x86_64/release/cnexxT.abi3.so
      #7  0x00007f5aab39546a in QObject::~QObject() () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #8  0x00007f5aabbd6ae6 in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/QtCore.abi3.so
      #9  0x00007f5ac141b93a in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/shiboken6/libshiboken6.abi3.so.6.4
      #10 0x000000000058cedd in ?? ()
      #11 0x0000000000515d7f in ?? ()
      #12 0x000000000053fb3f in ?? ()
      #13 0x000000000052b338 in _PyEval_EvalFrameDefault ()
      #14 0x00000000005855a4 in ?? ()
      #15 0x000000000058510e in ?? ()
      #16 0x00007f5aac69d2f1 in PySide::SignalManager::callPythonMetaMethod(QMetaMethod const&, void**, _object*, bool) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/libpyside6.abi3.so.6.4
      #17 0x00007f5aac69d5d6 in PySide::SignalManager::SignalManagerPrivate::qtMethodMetacall(QObject*, int, void**) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/libpyside6.abi3.so.6.4
      #18 0x00007f5aab38937c in QObject::event(QEvent*) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #19 0x00007f5aab33929a in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #20 0x00007f5aab33fdfd in QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #21 0x00007f5aab5e11b3 in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #22 0x00007f5aab91c7a9 in g_main_context_dispatch () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
      #23 0x00007f5aab91ca38 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
      #24 0x00007f5aab91cacc in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
      #25 0x00007f5aab5e0b3a in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #26 0x00007f5aab344d5b in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #27 0x00007f5aab450eac in QThread::exec() () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #28 0x00007f5aabc729c0 in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/QtCore.abi3.so
      #29 0x000000000051f567 in ?? ()
      #30 0x000000000053ac2c in PyObject_Vectorcall ()
      #31 0x000000000052b940 in _PyEval_EvalFrameDefault ()
      #32 0x00000000005855a4 in ?? ()
      #33 0x0000000000585148 in ?? ()
      #34 0x00007f5aabc72915 in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/QtCore.abi3.so
      #35 0x00007f5aab4cac7f in ?? () from /home/wiedeman/develop/nexxT-deadlock/venv/lib/python3.11/site-packages/PySide6/Qt/lib/libQt6Core.so.6
      #36 0x00007f5ac1d56fd4 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
      #37 0x00007f5ac1dd75bc in ?? () from /lib/x86_64-linux-gnu/libc.so.6
      

      Attachments

        1. minimal_test.py
          3 kB
        2. pyside2367_thread_fixed.py
          3 kB
        3. pyside2367.py
          3 kB
        4. pyside2367_stack.txt
          246 kB
        5. pyside2367_log.txt
          7 kB
        6. pyside2367_diag.diff
          2 kB
        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            kleint Friedemann Kleint
            bdev a dev
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews