* Merge gen and frame state variables into one.
* Replace stack pointer with depth in PyFrameObject. Makes code easier to read and saves a word of memory.
* Add failing test.
* bpo-29590: fix stack trace for gen.throw() with yield from (GH-NNNN)
When gen.throw() is called on a generator after a "yield from", the
intermediate stack trace entries are lost. This commit fixes that.
This updates _PyErr_ChainStackItem() to use _PyErr_SetObject()
instead of _PyErr_ChainExceptions(). This prevents a hang in
certain circumstances because _PyErr_SetObject() performs checks
to prevent cycles in the exception context chain while
_PyErr_ChainExceptions() doesn't.
The previous commits on bpo-29587 got exception chaining working
with gen.throw() in the `yield` case. This patch also gets the
`yield from` case working.
As a consequence, implicit exception chaining now also works in
the asyncio scenario of awaiting on a task when an exception is
already active.
Tests are included for both the asyncio case and the pure
generator-only case.
_PyErr_ChainExceptions() now ensures that the first parameter is an
exception type, as done by _PyErr_SetObject().
* The following function now check PyExceptionInstance_Check() in an
assertion using a new _PyBaseExceptionObject_cast() helper
function:
* PyException_GetTraceback(), PyException_SetTraceback()
* PyException_GetCause(), PyException_SetCause()
* PyException_GetContext(), PyException_SetContext()
* PyExceptionClass_Name() now checks PyExceptionClass_Check() with an
assertion.
* Remove XXX comment and add gi_exc_state variable to _gen_throw().
* Remove comment from test_generators
This is a follow-up to GH-19823 that removes the check that the
exception value isn't NULL, prior to calling _PyErr_ChainExceptions().
This enables implicit exception chaining for gen.throw() in more
circumstances.
The commit also adds a test that a particular code snippet involving
gen.throw() doesn't crash. The test shows why the new
`gi_exc_state.exc_type != Py_None` check that was added is necessary.
Without the new check, the code snippet (as well as a number of other
tests) crashes on certain platforms (e.g. Fedora but not Mac).
Before this commit, if an exception was active inside a generator
when calling gen.throw(), that exception was lost (i.e. there was
no implicit exception chaining). This commit fixes that by
setting exc.__context__ when calling gen.throw(exc).
Before this commit, if an exception was active inside a generator
when calling gen.throw(), then that exception was lost (i.e. there
was no implicit exception chaining). This commit fixes that.
Modify test_coroutines, test_cprofile, test_generators, test_raise,
test_ssl and test_yield_from to use
support.catch_unraisable_exception() rather than
support.captured_stderr().
test_thread: remove test_save_exception_state_on_error() which is now
updated. test_unraisable_exception() checks that sys.unraisablehook()
is called to handle _thread.start_new_thread() exception.
test_cprofile now rely on unittest for test discovery: replace
support.run_unittest() with unittest.main().
* Copy test_exceptions.test_unraisable() to
test_sys.UnraisableHookTest().
* Use catch_unraisable_exception() in test_coroutines,
test_exceptions, test_generators.
* Fix multiple typos in code comments
* Add spacing in comments (test_logging.py, test_math.py)
* Fix spaces at the beginning of comments in test_logging.py
The current behaviour of yield expressions inside comprehensions and
generator expressions is essentially an accident of implementation - it
arises implicitly from the way the compiler handles yield expressions inside
nested functions and generators.
Since the current behaviour wasn't deliberately designed, and is inherently
confusing, we're deprecating it, with no current plans to reintroduce it.
Instead, our advice will be to use a named nested generator definition
for cases where this behaviour is desired.
If we have a chain of generators/coroutines that are 'yield from'ing
each other, then resuming the stack works like:
- call send() on the outermost generator
- this enters _PyEval_EvalFrameDefault, which re-executes the
YIELD_FROM opcode
- which calls send() on the next generator
- which enters _PyEval_EvalFrameDefault, which re-executes the
YIELD_FROM opcode
- ...etc.
However, every time we enter _PyEval_EvalFrameDefault, the first thing
we do is to check for pending signals, and if there are any then we
run the signal handler. And if it raises an exception, then we
immediately propagate that exception *instead* of starting to execute
bytecode. This means that e.g. a SIGINT at the wrong moment can "break
the chain" – it can be raised in the middle of our yield from chain,
with the bottom part of the stack abandoned for the garbage collector.
The fix is pretty simple: there's already a special case in
_PyEval_EvalFrameEx where it skips running signal handlers if the next
opcode is SETUP_FINALLY. (I don't see how this accomplishes anything
useful, but that's another story.) If we extend this check to also
skip running signal handlers when the next opcode is YIELD_FROM, then
that closes the hole – now the exception can only be raised at the
innermost stack frame.
This shouldn't have any performance implications, because the opcode
check happens inside the "slow path" after we've already determined
that there's a pending signal or something similar for us to process;
the vast majority of the time this isn't true and the new check
doesn't run at all.
Raise PendingDeprecationWarning when generator raises StopIteration
and no __future__ import is used. Fix offenders in the stdlib
and tests.
See also issue 22906.
Thanks to Nick Coghlan and Berker Peksag for reviews.
At entry, save or swap the exception state even if PyEval_EvalFrameEx() is
called with throwflag=0. At exit, the exception state is now always restored or
swapped, not only if why is WHY_YIELD or WHY_RETURN. Patch co-written with
Antoine Pitrou.
name, and use it in the representation of a generator (``repr(gen)``). The
default name of the generator (``__name__`` attribute) is now get from the
function instead of the code. Use ``gen.gi_code.co_name`` to get the name of
the code.
annotate text signatures in docstrings, resulting in fewer false
positives. "self" parameters are also explicitly marked, allowing
inspect.Signature() to authoritatively detect (and skip) said parameters.
Issue #20326: Argument Clinic now generates separate checksums for the
input and output sections of the block, allowing external tools to verify
that the input has not changed (and thus the output is not out-of-date).
PyMethodDescr_Type, _PyMethodWrapper_Type, and PyWrapperDescr_Type)
have been modified to provide introspection information for builtins.
Also: many additional Lib, test suite, and Argument Clinic fixes.
Mark Shannon
Chris Jerdonek
Pablo Galindo
Victor Stinner
Victor Stinner
Serhiy Storchaka
Isaiah Peng
Leo Arias
Yury Selivanov
Mike
Nathaniel J. Smith
Victor Stinner
Benjamin Peterson
Kushal Das
Martin Panter
Yury Selivanov
Larry Hastings
Antoine Pitrou
Andrew Svetlov
Nick Coghlan