Consider the following short program. demo() is a trivial async function that creates a QObject instance, connects a Python signal, and then exits. For current purposes, "async function" is basically just the same as a generator. Down below I'll explain why I suspect the underlying bug affects lots of different situations beyond just async functions, but this is the reproducer I have at hand.
Anyway, when we call send(None) on this object, we expect to get a StopIteration exception.
So there are two weird things here: the StopIteration exception is being printed on the console for some reason, and then the actual send method is raising SystemError instead of StopIteration, with a strange error message complaining about what looks like a bug in the Python interpreter.
Here's what's happening:
1. In CPython, the gen_send_ex function implements the send method on coroutines and generators. When a generator/coroutine completes, it uses the C API to raise StopIteration, and then it DECREFs the generator/coroutine frame object.
2. In our code, the Py_DECREF call causes the reference count on myqobject to drop to 0, so it's garbage collected, and its tp_dealloc hook runs.
3. From there, we eventually end up in PySide2::GlobalReceiverV2::qt_metacall, which contains the following code:
This is the bug: this code can be run from arbitrary contexts, including contexts where an exception has already been raised. gen_send_ex → Py_DECREF is one of them, but there are probably lots of others. So you can't just blindly call into Python, and you can't assume that PyErr_Occurred() indicates that an exception was raised by your code.
I think this is currently a showstopper for using PySide2 and async/await together, and might be the cause of other strange behaviors as well.
For an example of the correct way to call Python when you might be in some weird context like a GC destructor, see the slot_tp_finalize code in CPython, which is used to invoke user-supplied _del_ methods:
- You need to save/restore the exception state before invoking the Python code, using PyErr_Fetch/PyErr_Restore
- Instead of that tricky loop, you can probably just call PyErr_WriteUnraisable. It'll also produce a much clearer message, so I wouldn't have been left staring at the plain text StopIteration: 1 wondering where the heck it was coming from .
- I'm not sure if this would be a functional change or just a stylistic one, but it's definitely weird to be calling PyErr_Occurred() directly. Usually you check the return value from invoking Python code to tell you whether it failed or not, and then if it failed you know that there's a live exception.
Also, I noticed that PyErr_Occurred occurs more than 40 times in the PySide2 codebase. I'm nervous that there might be more places that use it incorrectly like this.
Full stack trace for the point where the StopIteration gets discarded:
See also: https://bugs.python.org/issue40789