traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Implement PEP 380 - 'yield from' (closes #11682)

This commit is contained in:
Nick Coghlan 2012-01-13 21:43:40 +10:00
parent e51757f6de
commit 1f7ce62bd6
33 changed files with 872 additions and 421 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
Doc/library/dis.rst
View File

@ -431,6 +431,13 @@ the stack so that it is available for further iterations of the loop.
Pops ``TOS`` and yields it from a :term:`generator`.
.. opcode:: YIELD_FROM
Pops ``TOS`` and delegates to it as a subiterator from a :term:`generator`.
.. versionadded:: 3.3
.. opcode:: IMPORT_STAR
Loads all symbols not starting with ``'_'`` directly from the module TOS to the

11
Doc/library/exceptions.rst
View File

@ -250,7 +250,16 @@ The following exceptions are the exceptions that are usually raised.
.. exception:: StopIteration
Raised by built-in function :func:`next` and an :term:`iterator`\'s
:meth:`__next__` method to signal that there are no further values.
:meth:`__next__` method to signal that there are no further items to be
produced by the iterator.
The exception object has a single attribute :attr:`value`, which is
given as an argument when constructing the exception, and defaults
to :const:`None`.
When a generator function returns, a new :exc:`StopIteration` instance is
raised, and the value returned by the function is used as the
:attr:`value` parameter to the constructor of the exception.
.. exception:: SyntaxError

30
Doc/reference/expressions.rst
View File

@ -318,7 +318,7 @@ Yield expressions
.. productionlist::
yield_atom: "(" `yield_expression` ")"
yield_expression: "yield" [`expression_list`]
yield_expression: "yield" [`expression_list` | "from" `expression`]
The :keyword:`yield` expression is only used when defining a generator function,
and can only be used in the body of a function definition. Using a
@ -336,7 +336,10 @@ the internal evaluation stack. When the execution is resumed by calling one of
the generator's methods, the function can proceed exactly as if the
:keyword:`yield` expression was just another external call. The value of the
:keyword:`yield` expression after resuming depends on the method which resumed
the execution.
the execution. If :meth:`__next__` is used (typically via either a
:keyword:`for` or the :func:`next` builtin) then the result is :const:`None`,
otherwise, if :meth:`send` is used, then the result will be the value passed
in to that method.
.. index:: single: coroutine
@ -346,12 +349,29 @@ suspended. The only difference is that a generator function cannot control
where should the execution continue after it yields; the control is always
transferred to the generator's caller.
The :keyword:`yield` statement is allowed in the :keyword:`try` clause of a
:keyword:`yield` expressions are allowed in the :keyword:`try` clause of a
:keyword:`try` ... :keyword:`finally` construct. If the generator is not
resumed before it is finalized (by reaching a zero reference count or by being
garbage collected), the generator-iterator's :meth:`close` method will be
called, allowing any pending :keyword:`finally` clauses to execute.
When ``yield from expression`` is used, it treats the supplied expression as
a subiterator. All values produced by that subiterator are passed directly
to the caller of the current generator's methods. Any values passed in with
:meth:`send` and any exceptions passed in with :meth:`throw` are passed to
the underlying iterator if it has the appropriate methods. If this is not the
case, then :meth:`send` will raise :exc:`AttributeError` or :exc:`TypeError`,
while :meth:`throw` will just raise the passed in exception immediately.
When the underlying iterator is complete, the :attr:`~StopIteration.value`
attribute of the raised :exc:`StopIteration` instance becomes the value of
the yield expression. It can be either set explicitly when raising
:exc:`StopIteration`, or automatically when the sub-iterator is a generator
(by returning a value from the sub-generator).
The parentheses can be omitted when the :keyword:`yield` expression is the
sole expression on the right hand side of an assignment statement.
.. index:: object: generator
The following generator's methods can be used to control the execution of a
@ -444,6 +464,10 @@ generator functions::
The proposal to enhance the API and syntax of generators, making them
usable as simple coroutines.
:pep:`0380` - Syntax for Delegating to a Subgenerator
The proposal to introduce the :token:`yield_from` syntax, making delegation
to sub-generators easy.
.. _primaries:

24
Doc/reference/simple_stmts.rst
View File

@ -425,10 +425,10 @@ When :keyword:`return` passes control out of a :keyword:`try` statement with a
:keyword:`finally` clause, that :keyword:`finally` clause is executed before
really leaving the function.
In a generator function, the :keyword:`return` statement is not allowed to
include an :token:`expression_list`. In that context, a bare :keyword:`return`
indicates that the generator is done and will cause :exc:`StopIteration` to be
raised.
In a generator function, the :keyword:`return` statement indicates that the
generator is done and will cause :exc:`StopIteration` to be raised. The returned
value (if any) is used as an argument to construct :exc:`StopIteration` and
becomes the :attr:`StopIteration.value` attribute.
.. _yield:
@ -450,6 +450,7 @@ The :keyword:`yield` statement is only used when defining a generator function,
and is only used in the body of the generator function. Using a :keyword:`yield`
statement in a function definition is sufficient to cause that definition to
create a generator function instead of a normal function.
When a generator function is called, it returns an iterator known as a generator
iterator, or more commonly, a generator. The body of the generator function is
executed by calling the :func:`next` function on the generator repeatedly until
@ -469,14 +470,25 @@ resumed before it is finalized (by reaching a zero reference count or by being
garbage collected), the generator-iterator's :meth:`close` method will be
called, allowing any pending :keyword:`finally` clauses to execute.
When ``yield from expression`` is used, it treats the supplied expression as
a subiterator, producing values from it until the underlying iterator is
exhausted.
For full details of :keyword:`yield` semantics, refer to the :ref:`yieldexpr`
section.
.. seealso::
:pep:`0255` - Simple Generators
The proposal for adding generators and the :keyword:`yield` statement to Python.
:pep:`0342` - Coroutines via Enhanced Generators
The proposal that, among other generator enhancements, proposed allowing
:keyword:`yield` to appear inside a :keyword:`try` ... :keyword:`finally` block.
The proposal to enhance the API and syntax of generators, making them
usable as simple coroutines.
:pep:`0380` - Syntax for Delegating to a Subgenerator
The proposal to introduce the :token:`yield_from` syntax, making delegation
to sub-generators easy.
.. _raise:

17
Doc/whatsnew/3.3.rst
View File

@ -195,6 +195,22 @@ inspection of exception attributes::
print("You are not allowed to read document.txt")
PEP 380: Syntax for Delegating to a Subgenerator
================================================
PEP 380 adds the ``yield from`` expression, allowing a generator to delegate
part of its operations to another generator. This allows a section of code
containing 'yield' to be factored out and placed in another generator.
Additionally, the subgenerator is allowed to return with a value, and the
value is made available to the delegating generator.
While designed primarily for use in delegating to a subgenerator, the ``yield
from`` expression actually allows delegation to arbitrary subiterators.
(Implementation by Greg Ewing, integrated into 3.3 by Renaud Blanch, Ryan
Kelly and Nick Coghlan, documentation by Zbigniew Jędrzejewski-Szmek and
Nick Coghlan)
PEP 3155: Qualified name for classes and functions
==================================================
@ -208,7 +224,6 @@ it provides better information about where they were actually defined, and
how they might be accessible from the global scope.
Example with (non-bound) methods::
>>> class C:
... def meth(self):
... pass

5
Grammar/Grammar
View File

@ -121,7 +121,7 @@ arglist: (argument ',')* (argument [',']
|'**' test)
# The reason that keywords are test nodes instead of NAME is that using NAME
# results in an ambiguity. ast.c makes sure it's a NAME.
argument: test [comp_for] | test '=' test # Really [keyword '='] test
argument: (test) [comp_for] | test '=' test # Really [keyword '='] test
comp_iter: comp_for | comp_if
comp_for: 'for' exprlist 'in' or_test [comp_iter]
comp_if: 'if' test_nocond [comp_iter]
@ -129,4 +129,5 @@ comp_if: 'if' test_nocond [comp_iter]
# not used in grammar, but may appear in "node" passed from Parser to Compiler
encoding_decl: NAME
yield_expr: 'yield' [testlist]
yield_expr: 'yield' [yield_arg]
yield_arg: 'from' test | testlist

6
Include/Python-ast.h
View File

@ -245,6 +245,7 @@ struct _expr {
} GeneratorExp;
struct {
int is_from;
expr_ty value;
} Yield;
@ -487,8 +488,9 @@ expr_ty _Py_DictComp(expr_ty key, expr_ty value, asdl_seq * generators, int
#define GeneratorExp(a0, a1, a2, a3, a4) _Py_GeneratorExp(a0, a1, a2, a3, a4)
expr_ty _Py_GeneratorExp(expr_ty elt, asdl_seq * generators, int lineno, int
col_offset, PyArena *arena);
#define Yield(a0, a1, a2, a3) _Py_Yield(a0, a1, a2, a3)
expr_ty _Py_Yield(expr_ty value, int lineno, int col_offset, PyArena *arena);
#define Yield(a0, a1, a2, a3, a4) _Py_Yield(a0, a1, a2, a3, a4)
expr_ty _Py_Yield(int is_from, expr_ty value, int lineno, int col_offset,
PyArena *arena);
#define Compare(a0, a1, a2, a3, a4, a5) _Py_Compare(a0, a1, a2, a3, a4, a5)
expr_ty _Py_Compare(expr_ty left, asdl_int_seq * ops, asdl_seq * comparators,
int lineno, int col_offset, PyArena *arena);

1
Include/frameobject.h
View File

@ -27,6 +27,7 @@ typedef struct _frame {
to the current stack top. */
PyObject **f_stacktop;
PyObject *f_trace; /* Trace function */
PyObject *f_yieldfrom; /* Iterator being delegated to by yield from */
/* In a generator, we need to be able to swap between the exception
state inside the generator and the exception state of the calling

1
Include/genobject.h
View File

@ -34,6 +34,7 @@ PyAPI_DATA(PyTypeObject) PyGen_Type;
PyAPI_FUNC(PyObject *) PyGen_New(struct _frame *);
PyAPI_FUNC(int) PyGen_NeedsFinalizing(PyGenObject *);
PyAPI_FUNC(int) PyGen_FetchStopIterationValue(PyObject **);
#ifdef __cplusplus
}

1
Include/graminit.h
View File

@ -81,3 +81,4 @@
#define comp_if 334
#define encoding_decl 335
#define yield_expr 336
#define yield_arg 337

1
Include/opcode.h
View File

@ -52,6 +52,7 @@ extern "C" {
#define STORE_LOCALS 69
#define PRINT_EXPR 70
#define LOAD_BUILD_CLASS 71
#define YIELD_FROM 72
#define INPLACE_LSHIFT 75
#define INPLACE_RSHIFT 76

7
Include/pyerrors.h
View File

@ -51,6 +51,11 @@ typedef struct {
Py_ssize_t written; /* only for BlockingIOError, -1 otherwise */
} PyOSErrorObject;
typedef struct {
PyException_HEAD
PyObject *value;
} PyStopIterationObject;
/* Compatibility typedefs */
typedef PyOSErrorObject PyEnvironmentErrorObject;
#ifdef MS_WINDOWS
@ -380,6 +385,8 @@ PyAPI_FUNC(int) PyUnicodeTranslateError_SetReason(
const char *reason /* UTF-8 encoded string */
);
/* create a StopIteration exception with the given value */
PyAPI_FUNC(PyObject *) PyStopIteration_Create(PyObject *);
/* These APIs aren't really part of the error implementation, but
often needed to format error messages; the native C lib APIs are

1
Lib/opcode.py
View File

@ -87,6 +87,7 @@ def_op('STORE_LOCALS', 69)
def_op('PRINT_EXPR', 70)
def_op('LOAD_BUILD_CLASS', 71)
def_op('YIELD_FROM', 72)
def_op('INPLACE_LSHIFT', 75)
def_op('INPLACE_RSHIFT', 76)

3
Lib/test/test_ast.py
View File

@ -813,7 +813,8 @@ class ASTValidatorTests(unittest.TestCase):
self._check_comprehension(factory)
def test_yield(self):
self.expr(ast.Yield(ast.Name("x", ast.Store())), "must have Load")
self.expr(ast.Yield(0, ast.Name("x", ast.Store())), "must have Load")
self.expr(ast.Yield(1, ast.Name("x", ast.Store())), "must have Load")
def test_compare(self):
left = ast.Name("x", ast.Load())

42
Lib/test/test_generators.py
View File

@ -728,29 +728,6 @@ Ye olde Fibonacci generator, tee style.
syntax_tests = """
>>> def f():
... return 22
... yield 1
Traceback (most recent call last):
..
SyntaxError: 'return' with argument inside generator
>>> def f():
... yield 1
... return 22
Traceback (most recent call last):
..
SyntaxError: 'return' with argument inside generator
"return None" is not the same as "return" in a generator:
>>> def f():
... yield 1
... return None
Traceback (most recent call last):
..
SyntaxError: 'return' with argument inside generator
These are fine:
>>> def f():
@ -866,20 +843,6 @@ These are fine:
>>> type(f())
<class 'generator'>
>>> def f():
... if 0:
... lambda x: x # shouldn't trigger here
... return # or here
... def f(i):
... return 2*i # or here
... if 0:
... return 3 # but *this* sucks (line 8)
... if 0:
... yield 2 # because it's a generator (line 10)
Traceback (most recent call last):
SyntaxError: 'return' with argument inside generator
This one caused a crash (see SF bug 567538):
>>> def f():
@ -1566,11 +1529,6 @@ Traceback (most recent call last):
...
SyntaxError: 'yield' outside function
>>> def f(): return lambda x=(yield): 1
Traceback (most recent call last):
...
SyntaxError: 'return' with argument inside generator
>>> def f(): x = yield = y
Traceback (most recent call last):
...

32
Lib/test/test_grammar.py
View File

@ -458,7 +458,39 @@ class GrammarTests(unittest.TestCase):
check_syntax_error(self, "class foo:return 1")
def test_yield(self):
# Allowed as standalone statement
def g(): yield 1
def g(): yield from ()
# Allowed as RHS of assignment
def g(): x = yield 1
def g(): x = yield from ()
# Ordinary yield accepts implicit tuples
def g(): yield 1, 1
def g(): x = yield 1, 1
# 'yield from' does not
check_syntax_error(self, "def g(): yield from (), 1")
check_syntax_error(self, "def g(): x = yield from (), 1")
# Requires parentheses as subexpression
def g(): 1, (yield 1)
def g(): 1, (yield from ())
check_syntax_error(self, "def g(): 1, yield 1")
check_syntax_error(self, "def g(): 1, yield from ()")
# Requires parentheses as call argument
def g(): f((yield 1))
def g(): f((yield 1), 1)
def g(): f((yield from ()))
def g(): f((yield from ()), 1)
check_syntax_error(self, "def g(): f(yield 1)")
check_syntax_error(self, "def g(): f(yield 1, 1)")
check_syntax_error(self, "def g(): f(yield from ())")
check_syntax_error(self, "def g(): f(yield from (), 1)")
# Not allowed at top level
check_syntax_error(self, "yield")
check_syntax_error(self, "yield from")
# Not allowed at class scope
check_syntax_error(self, "class foo:yield 1")
check_syntax_error(self, "class foo:yield from ()")
def test_raise(self):
# 'raise' test [',' test]

5
Lib/test/test_parser.py
View File

@ -50,6 +50,10 @@ class RoundtripLegalSyntaxTestCase(unittest.TestCase):
self.check_suite("def f(): (yield 1)*2")
self.check_suite("def f(): return; yield 1")
self.check_suite("def f(): yield 1; return")
self.check_suite("def f(): yield from 1")
self.check_suite("def f(): x = yield from 1")
self.check_suite("def f(): f((yield from 1))")
self.check_suite("def f(): yield 1; return 1")
self.check_suite("def f():\n"
" for x in range(30):\n"
" yield x\n")
@ -621,7 +625,6 @@ class OtherParserCase(unittest.TestCase):
with self.assertRaises(TypeError):
parser.expr("a", "b")
def test_main():
support.run_unittest(
RoundtripLegalSyntaxTestCase,

2
Lib/test/test_sys.py
View File

@ -727,7 +727,7 @@ class SizeofTest(unittest.TestCase):
nfrees = len(x.f_code.co_freevars)
extras = x.f_code.co_stacksize + x.f_code.co_nlocals +\
ncells + nfrees - 1
check(x, size(vh + '12P3i' + CO_MAXBLOCKS*'3i' + 'P' + extras*'P'))
check(x, size(vh + '13P3i' + CO_MAXBLOCKS*'3i' + 'P' + extras*'P'))
# function
def func(): pass
check(func, size(h + '12P'))

2
Misc/ACKS
View File

@ -96,6 +96,7 @@ Dominic Binks
Philippe Biondi
Stuart Bishop
Roy Bixler
Renaud Blanch
Mike Bland
Martin Bless
Pablo Bleyer
@ -482,6 +483,7 @@ Geert Jansen
Jack Jansen
Bill Janssen
Thomas Jarosch
Zbigniew Jędrzejewski-Szmek
Julien Jehannet
Drew Jenkins
Flemming Kjær Jensen

4
Misc/NEWS
View File

@ -10,6 +10,10 @@ What's New in Python 3.3 Alpha 1?
Core and Builtins
-----------------
- PEP 380, Issue #11682: Add "yield from <x>" to support easy delegation to
subgenerators (initial patch by Greg Ewing, integration into 3.3 by
Renaud Blanch, Ryan Kelly, Zbigniew Jędrzejewski-Szmek and Nick Coghlan)
- Issue #13748: Raw bytes literals can now be written with the ``rb`` prefix
as well as ``br``.

52
Modules/parsermodule.c
View File

@ -976,6 +976,7 @@ VALIDATER(comp_iter); VALIDATER(comp_if);
VALIDATER(testlist_comp); VALIDATER(yield_expr);
VALIDATER(or_test);
VALIDATER(test_nocond); VALIDATER(lambdef_nocond);
VALIDATER(yield_arg);
#undef VALIDATER
@ -1636,22 +1637,49 @@ validate_raise_stmt(node *tree)
}
/* yield_expr: 'yield' [testlist]
/* yield_expr: 'yield' [yield_arg]
*/
static int
validate_yield_expr(node *tree)
{
int nch = NCH(tree);
int res = (validate_ntype(tree, yield_expr)
&& ((nch == 1) || (nch == 2))
&& validate_name(CHILD(tree, 0), "yield"));
if (res && (nch == 2))
res = validate_testlist(CHILD(tree, 1));
return (res);
if (nch < 1 || nch > 2)
return 0;
if (!validate_ntype(tree, yield_expr))
return 0;
if (!validate_name(CHILD(tree, 0), "yield"))
return 0;
if (nch == 2) {
if (!validate_yield_arg(CHILD(tree, 1)))
return 0;
}
return 1;
}
/* yield_arg: 'from' test | testlist
*/
static int
validate_yield_arg(node *tree)
{
int nch = NCH(tree);
if (!validate_ntype(tree, yield_arg))
return 0;
switch (nch) {
case 1:
if (!validate_testlist(CHILD(tree, nch - 1)))
return 0;
break;
case 2:
if (!validate_name(CHILD(tree, 0), "from"))
return 0;
if (!validate_test(CHILD(tree, 1)))
return 0;
break;
default:
return 0;
}
return 1;
}
/* yield_stmt: yield_expr
*/
@ -2665,9 +2693,9 @@ validate_argument(node *tree)
{
int nch = NCH(tree);
int res = (validate_ntype(tree, argument)
&& ((nch == 1) || (nch == 2) || (nch == 3))
&& validate_test(CHILD(tree, 0)));
&& ((nch == 1) || (nch == 2) || (nch == 3)));
if (res)
res = validate_test(CHILD(tree, 0));
if (res && (nch == 2))
res = validate_comp_for(CHILD(tree, 1));
else if (res && (nch == 3))

2
Objects/abstract.c
View File

@ -2267,7 +2267,6 @@ PyObject_CallMethod(PyObject *o, char *name, char *format, ...)
func = PyObject_GetAttrString(o, name);
if (func == NULL) {
PyErr_SetString(PyExc_AttributeError, name);
return 0;
}
@ -2311,7 +2310,6 @@ _PyObject_CallMethod_SizeT(PyObject *o, char *name, char *format, ...)
func = PyObject_GetAttrString(o, name);
if (func == NULL) {
PyErr_SetString(PyExc_AttributeError, name);
return 0;
}
va_start(va, format);

66
Objects/exceptions.c
View File

@ -487,8 +487,70 @@ SimpleExtendsException(PyExc_Exception, TypeError,
/*
* StopIteration extends Exception
*/
SimpleExtendsException(PyExc_Exception, StopIteration,
"Signal the end from iterator.__next__().");
static PyMemberDef StopIteration_members[] = {
{"value", T_OBJECT, offsetof(PyStopIterationObject, value), 0,
PyDoc_STR("generator return value")},
{NULL} /* Sentinel */
};
static int
StopIteration_init(PyStopIterationObject *self, PyObject *args, PyObject *kwds)
{
Py_ssize_t size = PyTuple_GET_SIZE(args);
PyObject *value;
if (BaseException_init((PyBaseExceptionObject *)self, args, kwds) == -1)
return -1;
Py_CLEAR(self->value);
if (size > 0)
value = PyTuple_GET_ITEM(args, 0);
else
value = Py_None;
Py_INCREF(value);
self->value = value;
return 0;
}
static int
StopIteration_clear(PyStopIterationObject *self)
{
Py_CLEAR(self->value);
return BaseException_clear((PyBaseExceptionObject *)self);
}
static void
StopIteration_dealloc(PyStopIterationObject *self)
{
_PyObject_GC_UNTRACK(self);
StopIteration_clear(self);
Py_TYPE(self)->tp_free((PyObject *)self);
}
static int
StopIteration_traverse(PyStopIterationObject *self, visitproc visit, void *arg)
{
Py_VISIT(self->value);
return BaseException_traverse((PyBaseExceptionObject *)self, visit, arg);
}
PyObject *
PyStopIteration_Create(PyObject *value)
{
return PyObject_CallFunctionObjArgs(PyExc_StopIteration, value, NULL);
}
ComplexExtendsException(
PyExc_Exception, /* base */
StopIteration, /* name */
StopIteration, /* prefix for *_init, etc */
0, /* new */
0, /* methods */
StopIteration_members, /* members */
0, /* getset */
0, /* str */
"Signal the end from iterator.__next__()."
);
/*

5
Objects/frameobject.c
View File

@ -20,6 +20,7 @@ static PyMemberDef frame_memberlist[] = {
{"f_builtins", T_OBJECT, OFF(f_builtins), READONLY},
{"f_globals", T_OBJECT, OFF(f_globals), READONLY},
{"f_lasti", T_INT, OFF(f_lasti), READONLY},
{"f_yieldfrom", T_OBJECT, OFF(f_yieldfrom), READONLY},
{NULL} /* Sentinel */
};
@ -444,6 +445,7 @@ frame_dealloc(PyFrameObject *f)
Py_CLEAR(f->f_exc_type);
Py_CLEAR(f->f_exc_value);
Py_CLEAR(f->f_exc_traceback);
Py_CLEAR(f->f_yieldfrom);
co = f->f_code;
if (co->co_zombieframe == NULL)
@ -475,6 +477,7 @@ frame_traverse(PyFrameObject *f, visitproc visit, void *arg)
Py_VISIT(f->f_exc_type);
Py_VISIT(f->f_exc_value);
Py_VISIT(f->f_exc_traceback);
Py_VISIT(f->f_yieldfrom);
/* locals */
slots = f->f_code->co_nlocals + PyTuple_GET_SIZE(f->f_code->co_cellvars) + PyTuple_GET_SIZE(f->f_code->co_freevars);
@ -508,6 +511,7 @@ frame_clear(PyFrameObject *f)
Py_CLEAR(f->f_exc_value);
Py_CLEAR(f->f_exc_traceback);
Py_CLEAR(f->f_trace);
Py_CLEAR(f->f_yieldfrom);
/* locals */
slots = f->f_code->co_nlocals + PyTuple_GET_SIZE(f->f_code->co_cellvars) + PyTuple_GET_SIZE(f->f_code->co_freevars);
@ -711,6 +715,7 @@ PyFrame_New(PyThreadState *tstate, PyCodeObject *code, PyObject *globals,
f->f_lasti = -1;
f->f_lineno = code->co_firstlineno;
f->f_iblock = 0;
f->f_yieldfrom = NULL;
_PyObject_GC_TRACK(f);
return f;

217
Objects/genobject.c
View File

@ -5,6 +5,9 @@
#include "structmember.h"
#include "opcode.h"
static PyObject *gen_close(PyGenObject *gen, PyObject *args);
static void gen_undelegate(PyGenObject *gen);
static int
gen_traverse(PyGenObject *gen, visitproc visit, void *arg)
{
@ -90,12 +93,18 @@ gen_send_ex(PyGenObject *gen, PyObject *arg, int exc)
/* If the generator just returned (as opposed to yielding), signal
* that the generator is exhausted. */
if (result == Py_None && f->f_stacktop == NULL) {
Py_DECREF(result);
result = NULL;
/* Set exception if not called by gen_iternext() */
if (arg)
if (result && f->f_stacktop == NULL) {
if (result == Py_None) {
/* Delay exception instantiation if we can */
PyErr_SetNone(PyExc_StopIteration);
} else {
PyObject *e = PyStopIteration_Create(result);
if (e != NULL) {
PyErr_SetObject(PyExc_StopIteration, e);
Py_DECREF(e);
}
}
Py_CLEAR(result);
}
if (!result || f->f_stacktop == NULL) {
@ -111,8 +120,8 @@ gen_send_ex(PyGenObject *gen, PyObject *arg, int exc)
Py_XDECREF(t);
Py_XDECREF(v);
Py_XDECREF(tb);
Py_DECREF(f);
gen->gi_frame = NULL;
Py_DECREF(f);
}
return result;
@ -125,16 +134,90 @@ return next yielded value or raise StopIteration.");
static PyObject *
gen_send(PyGenObject *gen, PyObject *arg)
{
return gen_send_ex(gen, arg, 0);
int exc = 0;
PyObject *ret;
PyObject *yf = gen->gi_frame ? gen->gi_frame->f_yieldfrom : NULL;
/* XXX (ncoghlan): Are the incref/decref on arg and yf strictly needed?
* Or would it be valid to rely on borrowed references?
*/
Py_INCREF(arg);
if (yf) {
Py_INCREF(yf);
if (PyGen_CheckExact(yf)) {
ret = gen_send((PyGenObject *)yf, arg);
} else {
if (arg == Py_None)
ret = PyIter_Next(yf);
else
ret = PyObject_CallMethod(yf, "send", "O", arg);
}
if (ret) {
Py_DECREF(yf);
goto done;
}
gen_undelegate(gen);
Py_CLEAR(arg);
if (PyGen_FetchStopIterationValue(&arg) < 0) {
exc = 1;
}
Py_DECREF(yf);
}
ret = gen_send_ex(gen, arg, exc);
done:
Py_XDECREF(arg);
return ret;
}
PyDoc_STRVAR(close_doc,
"close(arg) -> raise GeneratorExit inside generator.");
/*
* This helper function is used by gen_close and gen_throw to
* close a subiterator being delegated to by yield-from.
*/
static int
gen_close_iter(PyObject *yf)
{
PyObject *retval = NULL;
if (PyGen_CheckExact(yf)) {
retval = gen_close((PyGenObject *)yf, NULL);
if (retval == NULL) {
return -1;
}
} else {
PyObject *meth = PyObject_GetAttrString(yf, "close");
if (meth == NULL) {
if (!PyErr_ExceptionMatches(PyExc_AttributeError)) {
PyErr_WriteUnraisable(yf);
}
PyErr_Clear();
} else {
retval = PyObject_CallFunction(meth, "");
Py_DECREF(meth);
if (!retval)
return -1;
}
}
Py_XDECREF(retval);
return 0;
}
static PyObject *
gen_close(PyGenObject *gen, PyObject *args)
{
PyObject *retval;
PyObject *yf = gen->gi_frame ? gen->gi_frame->f_yieldfrom : NULL;
int err = 0;
if (yf) {
Py_INCREF(yf);
err = gen_close_iter(yf);
gen_undelegate(gen);
Py_DECREF(yf);
}
if (err == 0)
PyErr_SetNone(PyExc_GeneratorExit);
retval = gen_send_ex(gen, Py_None, 1);
if (retval) {
@ -196,7 +279,7 @@ gen_del(PyObject *self)
_Py_NewReference(self);
self->ob_refcnt = refcnt;
}
assert(PyType_IS_GC(self->ob_type) &&
assert(PyType_IS_GC(Py_TYPE(self)) &&
_Py_AS_GC(self)->gc.gc_refs != _PyGC_REFS_UNTRACKED);
/* If Py_REF_DEBUG, _Py_NewReference bumped _Py_RefTotal, so
@ -209,8 +292,8 @@ gen_del(PyObject *self)
* undone.
*/
#ifdef COUNT_ALLOCS
--self->ob_type->tp_frees;
--self->ob_type->tp_allocs;
--(Py_TYPE(self)->tp_frees);
--(Py_TYPE(self)->tp_allocs);
#endif
}
@ -226,10 +309,55 @@ gen_throw(PyGenObject *gen, PyObject *args)
PyObject *typ;
PyObject *tb = NULL;
PyObject *val = NULL;
PyObject *yf = gen->gi_frame ? gen->gi_frame->f_yieldfrom : NULL;
if (!PyArg_UnpackTuple(args, "throw", 1, 3, &typ, &val, &tb))
return NULL;
if (yf) {
PyObject *ret;
int err;
Py_INCREF(yf);
if (PyErr_GivenExceptionMatches(typ, PyExc_GeneratorExit)) {
err = gen_close_iter(yf);
Py_DECREF(yf);
gen_undelegate(gen);
if (err < 0)
return gen_send_ex(gen, Py_None, 1);
goto throw_here;
}
if (PyGen_CheckExact(yf)) {
ret = gen_throw((PyGenObject *)yf, args);
} else {
PyObject *meth = PyObject_GetAttrString(yf, "throw");
if (meth == NULL) {
if (!PyErr_ExceptionMatches(PyExc_AttributeError)) {
Py_DECREF(yf);
return NULL;
}
PyErr_Clear();
Py_DECREF(yf);
gen_undelegate(gen);
goto throw_here;
}
ret = PyObject_CallObject(meth, args);
Py_DECREF(meth);
}
Py_DECREF(yf);
if (!ret) {
PyObject *val;
gen_undelegate(gen);
if (PyGen_FetchStopIterationValue(&val) == 0) {
ret = gen_send_ex(gen, val, 0);
Py_DECREF(val);
} else {
ret = gen_send_ex(gen, Py_None, 1);
}
}
return ret;
}
throw_here:
/* First, check the traceback argument, replacing None with
NULL. */
if (tb == Py_None) {
@ -272,7 +400,7 @@ gen_throw(PyGenObject *gen, PyObject *args)
PyErr_Format(PyExc_TypeError,
"exceptions must be classes or instances "
"deriving from BaseException, not %s",
typ->ob_type->tp_name);
Py_TYPE(typ)->tp_name);
goto failed_throw;
}
@ -291,9 +419,74 @@ failed_throw:
static PyObject *
gen_iternext(PyGenObject *gen)
{
return gen_send_ex(gen, NULL, 0);
PyObject *val = NULL;
PyObject *ret;
int exc = 0;
PyObject *yf = gen->gi_frame ? gen->gi_frame->f_yieldfrom : NULL;
if (yf) {
Py_INCREF(yf);
/* ceval.c ensures that yf is an iterator */
ret = Py_TYPE(yf)->tp_iternext(yf);
if (ret) {
Py_DECREF(yf);
return ret;
}
gen_undelegate(gen);
if (PyGen_FetchStopIterationValue(&val) < 0)
exc = 1;
Py_DECREF(yf);
}
ret = gen_send_ex(gen, val, exc);
Py_XDECREF(val);
return ret;
}
/*
* In certain recursive situations, a generator may lose its frame
* before we get a chance to clear f_yieldfrom, so we use this
* helper function.
*/
static void
gen_undelegate(PyGenObject *gen) {
if (gen->gi_frame) {
Py_XDECREF(gen->gi_frame->f_yieldfrom);
gen->gi_frame->f_yieldfrom = NULL;
}
}
/*
* If StopIteration exception is set, fetches its 'value'
* attribute if any, otherwise sets pvalue to None.
*
* Returns 0 if no exception or StopIteration is set.
* If any other exception is set, returns -1 and leaves
* pvalue unchanged.
*/
int
PyGen_FetchStopIterationValue(PyObject **pvalue) {
PyObject *et, *ev, *tb;
PyObject *value = NULL;
if (PyErr_ExceptionMatches(PyExc_StopIteration)) {
PyErr_Fetch(&et, &ev, &tb);
Py_XDECREF(et);
Py_XDECREF(tb);
if (ev) {
value = ((PyStopIterationObject *)ev)->value;
Py_DECREF(ev);
}
} else if (PyErr_Occurred()) {
return -1;
}
if (value == NULL) {
value = Py_None;
}
Py_INCREF(value);
*pvalue = value;
return 0;
}
static PyObject *
gen_repr(PyGenObject *gen)

2
Parser/Python.asdl
View File

@ -59,7 +59,7 @@ module Python
| DictComp(expr key, expr value, comprehension* generators)
| GeneratorExp(expr elt, comprehension* generators)
-- the grammar constrains where yield expressions can occur
| Yield(expr? value)
| Yield(int is_from, expr? value)
-- need sequences for compare to distinguish between
-- x < 4 < 3 and (x < 4) < 3
| Compare(expr left, cmpop* ops, expr* comparators)

27
Python/Python-ast.c
View File

@ -231,7 +231,9 @@ static char *GeneratorExp_fields[]={
"generators",
};
static PyTypeObject *Yield_type;
_Py_IDENTIFIER(is_from);
static char *Yield_fields[]={
"is_from",
"value",
};
static PyTypeObject *Compare_type;
@ -810,7 +812,7 @@ static int init_types(void)
GeneratorExp_type = make_type("GeneratorExp", expr_type,
GeneratorExp_fields, 2);
if (!GeneratorExp_type) return 0;
Yield_type = make_type("Yield", expr_type, Yield_fields, 1);
Yield_type = make_type("Yield", expr_type, Yield_fields, 2);
if (!Yield_type) return 0;
Compare_type = make_type("Compare", expr_type, Compare_fields, 3);
if (!Compare_type) return 0;
@ -1747,13 +1749,14 @@ GeneratorExp(expr_ty elt, asdl_seq * generators, int lineno, int col_offset,
}
expr_ty
Yield(expr_ty value, int lineno, int col_offset, PyArena *arena)
Yield(int is_from, expr_ty value, int lineno, int col_offset, PyArena *arena)
{
expr_ty p;
p = (expr_ty)PyArena_Malloc(arena, sizeof(*p));
if (!p)
return NULL;
p->kind = Yield_kind;
p->v.Yield.is_from = is_from;
p->v.Yield.value = value;
p->lineno = lineno;
p->col_offset = col_offset;
@ -2795,6 +2798,11 @@ ast2obj_expr(void* _o)
case Yield_kind:
result = PyType_GenericNew(Yield_type, NULL, NULL);
if (!result) goto failed;
value = ast2obj_int(o->v.Yield.is_from);
if (!value) goto failed;
if (PyObject_SetAttrString(result, "is_from", value) == -1)
goto failed;
Py_DECREF(value);
value = ast2obj_expr(o->v.Yield.value);
if (!value) goto failed;
if (_PyObject_SetAttrId(result, &PyId_value, value) == -1)
@ -5337,8 +5345,21 @@ obj2ast_expr(PyObject* obj, expr_ty* out, PyArena* arena)
return 1;
}
if (isinstance) {
int is_from;
expr_ty value;
if (_PyObject_HasAttrId(obj, &PyId_is_from)) {
int res;
tmp = _PyObject_GetAttrId(obj, &PyId_is_from);
if (tmp == NULL) goto failed;
res = obj2ast_int(tmp, &is_from, arena);
if (res != 0) goto failed;
Py_XDECREF(tmp);
tmp = NULL;
} else {
PyErr_SetString(PyExc_TypeError, "required field \"is_from\" missing from Yield");
return 1;
}
if (_PyObject_HasAttrId(obj, &PyId_value)) {
int res;
tmp = _PyObject_GetAttrId(obj, &PyId_value);
@ -5350,7 +5371,7 @@ obj2ast_expr(PyObject* obj, expr_ty* out, PyArena* arena)
} else {
value = NULL;
}
*out = Yield(value, lineno, col_offset, arena);
*out = Yield(is_from, value, lineno, col_offset, arena);
if (*out == NULL) goto failed;
return 0;
}

21
Python/ast.c
View File

@ -2369,13 +2369,24 @@ ast_for_expr(struct compiling *c, const node *n)
}
return ast_for_binop(c, n);
case yield_expr: {
node *an = NULL;
node *en = NULL;
int is_from = 0;
expr_ty exp = NULL;
if (NCH(n) == 2) {
exp = ast_for_testlist(c, CHILD(n, 1));
if (NCH(n) > 1)
an = CHILD(n, 1); /* yield_arg */
if (an) {
en = CHILD(an, NCH(an) - 1);
if (NCH(an) == 2) {
is_from = 1;
exp = ast_for_expr(c, en);
}
else
exp = ast_for_testlist(c, en);
if (!exp)
return NULL;
}
return Yield(exp, LINENO(n), n->n_col_offset, c->c_arena);
return Yield(is_from, exp, LINENO(n), n->n_col_offset, c->c_arena);
}
case factor:
if (NCH(n) == 1) {
@ -2399,7 +2410,7 @@ ast_for_call(struct compiling *c, const node *n, expr_ty func)
/*
arglist: (argument ',')* (argument [',']| '*' test [',' '**' test]
| '**' test)
argument: [test '='] test [comp_for] # Really [keyword '='] test
argument: [test '='] (test) [comp_for] # Really [keyword '='] test
*/
int i, nargs, nkeywords, ngens;
@ -2693,7 +2704,7 @@ ast_for_flow_stmt(struct compiling *c, const node *n)
continue_stmt: 'continue'
return_stmt: 'return' [testlist]
yield_stmt: yield_expr
yield_expr: 'yield' testlist
yield_expr: 'yield' testlist | 'yield' 'from' test
raise_stmt: 'raise' [test [',' test [',' test]]]
*/
node *ch;

46
Python/ceval.c
View File

@ -1828,6 +1828,52 @@ PyEval_EvalFrameEx(PyFrameObject *f, int throwflag)
why = WHY_RETURN;
goto fast_block_end;
TARGET(YIELD_FROM)
u = POP();
x = PyObject_GetIter(u);
Py_DECREF(u);
if (x == NULL)
break;
/* x is now the iterator, make the first next() call */
retval = (*Py_TYPE(x)->tp_iternext)(x);
if (!retval) {
/* iter may be exhausted */
Py_CLEAR(x);
if (!PyErr_ExceptionMatches(PyExc_StopIteration)) {
/* some other exception */
break;
}
/* try to get return value from exception */
PyObject *et, *ev, *tb;
PyErr_Fetch(&et, &ev, &tb);
Py_XDECREF(et);
Py_XDECREF(tb);
/* u is return value */
u = NULL;
if (ev) {
u = PyObject_GetAttrString(ev, "value");
Py_DECREF(ev);
if (u == NULL) {
if (!PyErr_ExceptionMatches(PyExc_AttributeError)) {
/* some other exception */
break;
}
PyErr_Clear();
}
}
if (u == NULL) {
u = Py_None;
Py_INCREF(u);
}
PUSH(u);
continue;
}
/* x is iterator, retval is value to be yielded */
f->f_yieldfrom = x;
f->f_stacktop = stack_pointer;
why = WHY_YIELD;
goto fast_yield;
TARGET(YIELD_VALUE)
retval = POP();
f->f_stacktop = stack_pointer;

6
Python/compile.c
View File

@ -840,6 +840,7 @@ opcode_stack_effect(int opcode, int oparg)
case IMPORT_STAR:
return -1;
case YIELD_VALUE:
case YIELD_FROM:
return 0;
case POP_BLOCK:
@ -3318,7 +3319,12 @@ compiler_visit_expr(struct compiler *c, expr_ty e)
else {
ADDOP_O(c, LOAD_CONST, Py_None, consts);
}
if (e->v.Yield.is_from) {
ADDOP(c, YIELD_FROM);
}
else {
ADDOP(c, YIELD_VALUE);
}
break;
case Compare_kind:
return compiler_compare(c, e);

352
Python/graminit.c
View File

@ -1791,7 +1791,7 @@ static arc arcs_80_0[1] = {
{167, 1},
};
static arc arcs_80_1[2] = {
{9, 2},
{168, 2},
{0, 1},
};
static arc arcs_80_2[1] = {
@ -1802,171 +1802,188 @@ static state states_80[3] = {
{2, arcs_80_1},
{1, arcs_80_2},
};
static dfa dfas[81] = {
{256, "single_input", 0, 3, states_0,
"\004\050\060\200\000\000\000\240\340\223\160\220\045\200\020\000\000\206\120\076\204"},
{257, "file_input", 0, 2, states_1,
"\204\050\060\200\000\000\000\240\340\223\160\220\045\200\020\000\000\206\120\076\204"},
{258, "eval_input", 0, 3, states_2,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
{259, "decorator", 0, 7, states_3,
"\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{260, "decorators", 0, 2, states_4,
"\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{261, "decorated", 0, 3, states_5,
"\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{262, "funcdef", 0, 8, states_6,
"\000\000\020\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{263, "parameters", 0, 4, states_7,
"\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{264, "typedargslist", 0, 18, states_8,
"\000\000\040\200\001\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{265, "tfpdef", 0, 4, states_9,
"\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{266, "varargslist", 0, 18, states_10,
"\000\000\040\200\001\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{267, "vfpdef", 0, 2, states_11,
"\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{268, "stmt", 0, 2, states_12,
"\000\050\060\200\000\000\000\240\340\223\160\220\045\200\020\000\000\206\120\076\204"},
{269, "simple_stmt", 0, 4, states_13,
"\000\040\040\200\000\000\000\240\340\223\160\000\000\200\020\000\000\206\120\076\200"},
{270, "small_stmt", 0, 2, states_14,
"\000\040\040\200\000\000\000\240\340\223\160\000\000\200\020\000\000\206\120\076\200"},
{271, "expr_stmt", 0, 6, states_15,
"\000\040\040\200\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
{272, "testlist_star_expr", 0, 3, states_16,
"\000\040\040\200\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
{273, "augassign", 0, 2, states_17,
"\000\000\000\000\000\000\376\037\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{274, "del_stmt", 0, 3, states_18,
"\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{275, "pass_stmt", 0, 2, states_19,
"\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{276, "flow_stmt", 0, 2, states_20,
"\000\000\000\000\000\000\000\000\340\001\000\000\000\000\000\000\000\000\000\000\200"},
{277, "break_stmt", 0, 2, states_21,
"\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000"},
{278, "continue_stmt", 0, 2, states_22,
"\000\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\000\000\000\000"},
{279, "return_stmt", 0, 3, states_23,
"\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000"},
{280, "yield_stmt", 0, 2, states_24,
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\200"},
{281, "raise_stmt", 0, 5, states_25,
"\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000"},
{282, "import_stmt", 0, 2, states_26,
"\000\000\000\000\000\000\000\000\000\022\000\000\000\000\000\000\000\000\000\000\000"},
{283, "import_name", 0, 3, states_27,
"\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000\000"},
{284, "import_from", 0, 8, states_28,
"\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000\000\000\000"},
{285, "import_as_name", 0, 4, states_29,
"\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{286, "dotted_as_name", 0, 4, states_30,
"\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{287, "import_as_names", 0, 3, states_31,
"\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{288, "dotted_as_names", 0, 2, states_32,
"\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{289, "dotted_name", 0, 2, states_33,
"\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{290, "global_stmt", 0, 3, states_34,
"\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000"},
{291, "nonlocal_stmt", 0, 3, states_35,
"\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000"},
{292, "assert_stmt", 0, 5, states_36,
"\000\000\000\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\000\000"},
{293, "compound_stmt", 0, 2, states_37,
"\000\010\020\000\000\000\000\000\000\000\000\220\045\000\000\000\000\000\000\000\004"},
{294, "if_stmt", 0, 8, states_38,
"\000\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000"},
{295, "while_stmt", 0, 8, states_39,
"\000\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000"},
{296, "for_stmt", 0, 10, states_40,
"\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000"},
{297, "try_stmt", 0, 13, states_41,
"\000\000\000\000\000\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\000"},
{298, "with_stmt", 0, 5, states_42,
"\000\000\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000"},
{299, "with_item", 0, 4, states_43,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
{300, "except_clause", 0, 5, states_44,
"\000\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000"},
{301, "suite", 0, 5, states_45,
"\004\040\040\200\000\000\000\240\340\223\160\000\000\200\020\000\000\206\120\076\200"},
{302, "test", 0, 6, states_46,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
{303, "test_nocond", 0, 2, states_47,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
{304, "lambdef", 0, 5, states_48,
"\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000"},
{305, "lambdef_nocond", 0, 5, states_49,
"\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000"},
{306, "or_test", 0, 2, states_50,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\020\000\000\206\120\076\000"},
{307, "and_test", 0, 2, states_51,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\020\000\000\206\120\076\000"},
{308, "not_test", 0, 3, states_52,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\020\000\000\206\120\076\000"},
{309, "comparison", 0, 2, states_53,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
{310, "comp_op", 0, 4, states_54,
"\000\000\000\000\000\000\000\000\000\000\000\000\002\000\220\177\000\000\000\000\000"},
{311, "star_expr", 0, 3, states_55,
"\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{312, "expr", 0, 2, states_56,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
{313, "xor_expr", 0, 2, states_57,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
{314, "and_expr", 0, 2, states_58,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
{315, "shift_expr", 0, 2, states_59,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
{316, "arith_expr", 0, 2, states_60,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
{317, "term", 0, 2, states_61,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
{318, "factor", 0, 3, states_62,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
{319, "power", 0, 4, states_63,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\120\076\000"},
{320, "atom", 0, 9, states_64,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\120\076\000"},
{321, "testlist_comp", 0, 5, states_65,
"\000\040\040\200\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
{322, "trailer", 0, 7, states_66,
"\000\040\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\020\000\000"},
{323, "subscriptlist", 0, 3, states_67,
"\000\040\040\002\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
{324, "subscript", 0, 5, states_68,
"\000\040\040\002\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
{325, "sliceop", 0, 3, states_69,
"\000\000\000\002\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{326, "exprlist", 0, 3, states_70,
"\000\040\040\200\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000"},
{327, "testlist", 0, 3, states_71,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
{328, "dictorsetmaker", 0, 11, states_72,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
{329, "classdef", 0, 8, states_73,
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\004"},
{330, "arglist", 0, 8, states_74,
"\000\040\040\200\001\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
{331, "argument", 0, 4, states_75,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000"},
{332, "comp_iter", 0, 2, states_76,
"\000\000\000\000\000\000\000\000\000\000\000\020\001\000\000\000\000\000\000\000\000"},
{333, "comp_for", 0, 6, states_77,
"\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000"},
{334, "comp_if", 0, 4, states_78,
"\000\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000"},
{335, "encoding_decl", 0, 2, states_79,
"\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{336, "yield_expr", 0, 3, states_80,
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\200"},
static arc arcs_81_0[2] = {
{73, 1},
{9, 2},
};
static label labels[168] = {
static arc arcs_81_1[1] = {
{24, 2},
};
static arc arcs_81_2[1] = {
{0, 2},
};
static state states_81[3] = {
{2, arcs_81_0},
{1, arcs_81_1},
{1, arcs_81_2},
};
static dfa dfas[82] = {
{256, "single_input", 0, 3, states_0,
"\004\050\060\200\000\000\000\240\340\223\160\220\045\200\020\000\000\206\120\076\204\000"},
{257, "file_input", 0, 2, states_1,
"\204\050\060\200\000\000\000\240\340\223\160\220\045\200\020\000\000\206\120\076\204\000"},
{258, "eval_input", 0, 3, states_2,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{259, "decorator", 0, 7, states_3,
"\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{260, "decorators", 0, 2, states_4,
"\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{261, "decorated", 0, 3, states_5,
"\000\010\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{262, "funcdef", 0, 8, states_6,
"\000\000\020\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{263, "parameters", 0, 4, states_7,
"\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{264, "typedargslist", 0, 18, states_8,
"\000\000\040\200\001\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{265, "tfpdef", 0, 4, states_9,
"\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{266, "varargslist", 0, 18, states_10,
"\000\000\040\200\001\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{267, "vfpdef", 0, 2, states_11,
"\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{268, "stmt", 0, 2, states_12,
"\000\050\060\200\000\000\000\240\340\223\160\220\045\200\020\000\000\206\120\076\204\000"},
{269, "simple_stmt", 0, 4, states_13,
"\000\040\040\200\000\000\000\240\340\223\160\000\000\200\020\000\000\206\120\076\200\000"},
{270, "small_stmt", 0, 2, states_14,
"\000\040\040\200\000\000\000\240\340\223\160\000\000\200\020\000\000\206\120\076\200\000"},
{271, "expr_stmt", 0, 6, states_15,
"\000\040\040\200\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{272, "testlist_star_expr", 0, 3, states_16,
"\000\040\040\200\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{273, "augassign", 0, 2, states_17,
"\000\000\000\000\000\000\376\037\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{274, "del_stmt", 0, 3, states_18,
"\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{275, "pass_stmt", 0, 2, states_19,
"\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{276, "flow_stmt", 0, 2, states_20,
"\000\000\000\000\000\000\000\000\340\001\000\000\000\000\000\000\000\000\000\000\200\000"},
{277, "break_stmt", 0, 2, states_21,
"\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{278, "continue_stmt", 0, 2, states_22,
"\000\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{279, "return_stmt", 0, 3, states_23,
"\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{280, "yield_stmt", 0, 2, states_24,
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000"},
{281, "raise_stmt", 0, 5, states_25,
"\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000\000"},
{282, "import_stmt", 0, 2, states_26,
"\000\000\000\000\000\000\000\000\000\022\000\000\000\000\000\000\000\000\000\000\000\000"},
{283, "import_name", 0, 3, states_27,
"\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000\000\000"},
{284, "import_from", 0, 8, states_28,
"\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000\000\000\000\000\000\000\000"},
{285, "import_as_name", 0, 4, states_29,
"\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{286, "dotted_as_name", 0, 4, states_30,
"\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{287, "import_as_names", 0, 3, states_31,
"\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{288, "dotted_as_names", 0, 2, states_32,
"\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{289, "dotted_name", 0, 2, states_33,
"\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{290, "global_stmt", 0, 3, states_34,
"\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000\000"},
{291, "nonlocal_stmt", 0, 3, states_35,
"\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000\000\000"},
{292, "assert_stmt", 0, 5, states_36,
"\000\000\000\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\000\000\000"},
{293, "compound_stmt", 0, 2, states_37,
"\000\010\020\000\000\000\000\000\000\000\000\220\045\000\000\000\000\000\000\000\004\000"},
{294, "if_stmt", 0, 8, states_38,
"\000\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000"},
{295, "while_stmt", 0, 8, states_39,
"\000\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\000\000"},
{296, "for_stmt", 0, 10, states_40,
"\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000"},
{297, "try_stmt", 0, 13, states_41,
"\000\000\000\000\000\000\000\000\000\000\000\000\004\000\000\000\000\000\000\000\000\000"},
{298, "with_stmt", 0, 5, states_42,
"\000\000\000\000\000\000\000\000\000\000\000\000\040\000\000\000\000\000\000\000\000\000"},
{299, "with_item", 0, 4, states_43,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{300, "except_clause", 0, 5, states_44,
"\000\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000"},
{301, "suite", 0, 5, states_45,
"\004\040\040\200\000\000\000\240\340\223\160\000\000\200\020\000\000\206\120\076\200\000"},
{302, "test", 0, 6, states_46,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{303, "test_nocond", 0, 2, states_47,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{304, "lambdef", 0, 5, states_48,
"\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000"},
{305, "lambdef_nocond", 0, 5, states_49,
"\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000"},
{306, "or_test", 0, 2, states_50,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\020\000\000\206\120\076\000\000"},
{307, "and_test", 0, 2, states_51,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\020\000\000\206\120\076\000\000"},
{308, "not_test", 0, 3, states_52,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\020\000\000\206\120\076\000\000"},
{309, "comparison", 0, 2, states_53,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{310, "comp_op", 0, 4, states_54,
"\000\000\000\000\000\000\000\000\000\000\000\000\002\000\220\177\000\000\000\000\000\000"},
{311, "star_expr", 0, 3, states_55,
"\000\000\000\200\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{312, "expr", 0, 2, states_56,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{313, "xor_expr", 0, 2, states_57,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{314, "and_expr", 0, 2, states_58,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{315, "shift_expr", 0, 2, states_59,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{316, "arith_expr", 0, 2, states_60,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{317, "term", 0, 2, states_61,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{318, "factor", 0, 3, states_62,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{319, "power", 0, 4, states_63,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\120\076\000\000"},
{320, "atom", 0, 9, states_64,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\000\000\000\000\000\120\076\000\000"},
{321, "testlist_comp", 0, 5, states_65,
"\000\040\040\200\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{322, "trailer", 0, 7, states_66,
"\000\040\000\000\000\000\000\000\000\100\000\000\000\000\000\000\000\000\020\000\000\000"},
{323, "subscriptlist", 0, 3, states_67,
"\000\040\040\002\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{324, "subscript", 0, 5, states_68,
"\000\040\040\002\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{325, "sliceop", 0, 3, states_69,
"\000\000\000\002\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{326, "exprlist", 0, 3, states_70,
"\000\040\040\200\000\000\000\000\000\200\000\000\000\000\000\000\000\206\120\076\000\000"},
{327, "testlist", 0, 3, states_71,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{328, "dictorsetmaker", 0, 11, states_72,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{329, "classdef", 0, 8, states_73,
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\004\000"},
{330, "arglist", 0, 8, states_74,
"\000\040\040\200\001\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{331, "argument", 0, 4, states_75,
"\000\040\040\000\000\000\000\000\000\200\000\000\000\200\020\000\000\206\120\076\000\000"},
{332, "comp_iter", 0, 2, states_76,
"\000\000\000\000\000\000\000\000\000\000\000\020\001\000\000\000\000\000\000\000\000\000"},
{333, "comp_for", 0, 6, states_77,
"\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000"},
{334, "comp_if", 0, 4, states_78,
"\000\000\000\000\000\000\000\000\000\000\000\020\000\000\000\000\000\000\000\000\000\000"},
{335, "encoding_decl", 0, 2, states_79,
"\000\000\040\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"},
{336, "yield_expr", 0, 3, states_80,
"\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\200\000"},
{337, "yield_arg", 0, 3, states_81,
"\000\040\040\000\000\000\000\000\000\202\000\000\000\200\020\000\000\206\120\076\000\000"},
};
static label labels[169] = {
{0, "EMPTY"},
{256, 0},
{4, 0},
@ -2135,10 +2152,11 @@ static label labels[168] = {
{334, 0},
{335, 0},
{1, "yield"},
{337, 0},
};
grammar _PyParser_Grammar = {
81,
82,
dfas,
{168, labels},
{169, labels},
256
};

2
Python/opcode_targets.h
View File

@ -71,7 +71,7 @@ static void *opcode_targets[256] = {
&&TARGET_STORE_LOCALS,
&&TARGET_PRINT_EXPR,
&&TARGET_LOAD_BUILD_CLASS,
&&_unknown_opcode,
&&TARGET_YIELD_FROM,
&&_unknown_opcode,
&&_unknown_opcode,
&&TARGET_INPLACE_LSHIFT,

19
Python/symtable.c
View File

@ -19,10 +19,6 @@
#define IMPORT_STAR_WARNING "import * only allowed at module level"
#define RETURN_VAL_IN_GENERATOR \
"'return' with argument inside generator"
static PySTEntryObject *
ste_new(struct symtable *st, identifier name, _Py_block_ty block,
void *key, int lineno, int col_offset)
@ -1133,14 +1129,6 @@ symtable_visit_stmt(struct symtable *st, stmt_ty s)
if (s->v.Return.value) {
VISIT(st, expr, s->v.Return.value);
st->st_cur->ste_returns_value = 1;
if (st->st_cur->ste_generator) {
PyErr_SetString(PyExc_SyntaxError,
RETURN_VAL_IN_GENERATOR);
PyErr_SyntaxLocationEx(st->st_filename,
s->lineno,
s->col_offset);
return 0;
}
}
break;
case Delete_kind:
@ -1345,13 +1333,6 @@ symtable_visit_expr(struct symtable *st, expr_ty e)
if (e->v.Yield.value)
VISIT(st, expr, e->v.Yield.value);
st->st_cur->ste_generator = 1;
if (st->st_cur->ste_returns_value) {
PyErr_SetString(PyExc_SyntaxError,
RETURN_VAL_IN_GENERATOR);
PyErr_SyntaxLocationEx(st->st_filename,
e->lineno, e->col_offset);
return 0;
}
break;
case Compare_kind:
VISIT(st, expr, e->v.Compare.left);