Details
-
Suggestion
-
Resolution: Unresolved
-
P4: Low
-
None
-
None
-
None
Description
As is well known, one of the few reliable ways of preventing deadlocks in multithreaded applications that use locks is to enforce a (partial) locking order. The question is where the Python GIL sits in this order and how that relates both to QT and to other native libraries that might use locks. Right now there doesn't seem to be any documented guidance related to QT, but there are hints e.g. https://www.qt.io/blog/qt-for-python-5.15.0-is-out that PySide does not always drop the GIL when calling C/C++ methods.
The obvious choices for how the GIL would be included in the locking order for a hybrid multithreaded C/C+/Python application that also uses native C/C+ locking are (a) First, and (b) Last. Option (a) is somewhat problematic because it disallows a thread from executing Python code while holding a native lock - because Python could drop and reaquire the GIL at any time, violating the locking order. That leaves us with option (b), putting the GIL last in the locking order.
Under option (b), we don't ever want to allow our C/C++ code to be called while holding the GIL because if our C/C++ code needs to acquire a lock for some reason it would violate the locking order and could deadlock. This is where QT comes in. Even if all of my own Python wrappers of my C++ objects drop the GIL immediately (I am using SWIG where this is a trivial configuration; but the same would apply for other wrappers including Shiboken) a Python method call via PySide that fails to drop the GIL but that ends up triggering (via callback, virtual method, etc.) a call to one of my C++ QObjects that then acquires another native lock with the GIL still held will violate the locking order, creating a potential deadlock.
I am sure that such scenarios were considered by QT engineering in the development of the threading fixes in QT for Python 5.14.2 and 5.15. But with no published guidance developers are at a loss to understand the proper protocol:
Does QT for Python guarantee that the GIL will be dropped prior to calling user code? Otherwise do methods that might perform locking need to call PyGILState_Check() and explicitly drop the GIL? Is some other locking protocol recommended?
Attachments
Issue Links
- relates to
-
PYSIDE-2221 Enable Multiple Cores by Supporting Removal of the GIL
- Open