diff --git a/Doc/glossary.rst b/Doc/glossary.rst index bdbb272051d..d00185e386f 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -169,18 +169,19 @@ Glossary statement by defining :meth:`__enter__` and :meth:`__exit__` methods. See :pep:`343`. - coroutine function - A function which returns a :term:`coroutine` object. It is defined - with an :keyword:`async def` keyword, and may contain :keyword:`await`, - :keyword:`async for`, and :keyword:`async with` keywords. Introduced - by :pep:`492`. - coroutine Coroutines is a more generalized form of subroutines. Subroutines are - entered at one point and exited at another point. Coroutines, can be - entered, exited, and resumed at many different points. See - :keyword:`await` expressions, and :keyword:`async for` and - :keyword:`async with` statements. See also :pep:`492`. + entered at one point and exited at another point. Coroutines can be + entered, exited, and resumed at many different points. They can be + implemented with the :keyword:`async def` statement. See also + :pep:`492`. + + coroutine function + A function which returns a :term:`coroutine` object. A coroutine + function may be defined with the :keyword:`async def` statement, + and may contain :keyword:`await`, :keyword:`async for`, and + :keyword:`async with` keywords. These were introduced + by :pep:`492`. CPython The canonical implementation of the Python programming language, as diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index e7ff7d2e2d4..fa9e96a3dc6 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -8,17 +8,23 @@ Tasks and coroutines Coroutines ---------- -A coroutine is a generator that follows certain conventions. For -documentation purposes, all coroutines should be decorated with -``@asyncio.coroutine``, but this cannot be strictly enforced. +Coroutines used with :mod:`asyncio` may be implemented using the +:keyword:`async def` statement, or by using :term:`generators `. +The :keyword:`async def` type of coroutine was added in Python 3.5, and +is recommended if there is no need to support older Python versions. -Coroutines use the ``yield from`` syntax introduced in :pep:`380`, +Generator-based coroutines should be decorated with :func:`@asyncio.coroutine +`, although this is not strictly enforced. +The decorator enables compatibility with :keyword:`async def` coroutines, +and also serves as documentation. Generator-based +coroutines use the ``yield from`` syntax introduced in :pep:`380`, instead of the original ``yield`` syntax. The word "coroutine", like the word "generator", is used for two different (though related) concepts: -- The function that defines a coroutine (a function definition +- The function that defines a coroutine + (a function definition using :keyword:`async def` or decorated with ``@asyncio.coroutine``). If disambiguation is needed we will call this a *coroutine function* (:func:`iscoroutinefunction` returns ``True``). @@ -30,27 +36,28 @@ different (though related) concepts: Things a coroutine can do: -- ``result = yield from future`` -- suspends the coroutine until the +- ``result = await future`` or ``result = yield from future`` -- + suspends the coroutine until the future is done, then returns the future's result, or raises an exception, which will be propagated. (If the future is cancelled, it will raise a ``CancelledError`` exception.) Note that tasks are futures, and everything said about futures also applies to tasks. -- ``result = yield from coroutine`` -- wait for another coroutine to +- ``result = await coroutine`` or ``result = yield from coroutine`` -- + wait for another coroutine to produce a result (or raise an exception, which will be propagated). The ``coroutine`` expression must be a *call* to another coroutine. - ``return expression`` -- produce a result to the coroutine that is - waiting for this one using ``yield from``. + waiting for this one using :keyword:`await` or ``yield from``. - ``raise exception`` -- raise an exception in the coroutine that is - waiting for this one using ``yield from``. + waiting for this one using :keyword:`await` or ``yield from``. -Calling a coroutine does not start its code running -- it is just a -generator, and the coroutine object returned by the call is really a -generator object, which doesn't do anything until you iterate over it. -In the case of a coroutine object, there are two basic ways to start -it running: call ``yield from coroutine`` from another coroutine +Calling a coroutine does not start its code running -- +the coroutine object returned by the call doesn't do anything until you +schedule its execution. There are two basic ways to start it running: +call ``await coroutine`` or ``yield from coroutine`` from another coroutine (assuming the other coroutine is already running!), or schedule its execution using the :func:`async` function or the :meth:`BaseEventLoop.create_task` method. @@ -60,9 +67,15 @@ Coroutines (and tasks) can only run when the event loop is running. .. decorator:: coroutine - Decorator to mark coroutines. + Decorator to mark generator-based coroutines. This enables + the generator use :keyword:`!yield from` to call :keyword:`async + def` coroutines, and also enables the generator to be called by + :keyword:`async def` coroutines, for instance using an + :keyword:`await` expression. - If the coroutine is not yielded from before it is destroyed, an error + There is no need to decorate :keyword:`async def` coroutines themselves. + + If the generator is not yielded from before it is destroyed, an error message is logged. See :ref:`Detect coroutines never scheduled `. @@ -84,8 +97,7 @@ Example of coroutine displaying ``"Hello World"``:: import asyncio - @asyncio.coroutine - def hello_world(): + async def hello_world(): print("Hello World!") loop = asyncio.get_event_loop() @@ -111,6 +123,21 @@ using the :meth:`sleep` function:: import asyncio import datetime + async def display_date(loop): + end_time = loop.time() + 5.0 + while True: + print(datetime.datetime.now()) + if (loop.time() + 1.0) >= end_time: + break + await asyncio.sleep(1) + + loop = asyncio.get_event_loop() + # Blocking call which returns when the display_date() coroutine is done + loop.run_until_complete(display_date(loop)) + loop.close() + +The same coroutine implemented using a generator:: + @asyncio.coroutine def display_date(loop): end_time = loop.time() + 5.0 @@ -120,11 +147,6 @@ using the :meth:`sleep` function:: break yield from asyncio.sleep(1) - loop = asyncio.get_event_loop() - # Blocking call which returns when the display_date() coroutine is done - loop.run_until_complete(display_date(loop)) - loop.close() - .. seealso:: The :ref:`display the current date with call_later() @@ -139,15 +161,13 @@ Example chaining coroutines:: import asyncio - @asyncio.coroutine - def compute(x, y): + async def compute(x, y): print("Compute %s + %s ..." % (x, y)) - yield from asyncio.sleep(1.0) + await asyncio.sleep(1.0) return x + y - @asyncio.coroutine - def print_sum(x, y): - result = yield from compute(x, y) + async def print_sum(x, y): + result = await compute(x, y) print("%s + %s = %s" % (x, y, result)) loop = asyncio.get_event_loop() @@ -550,12 +570,14 @@ Task functions .. function:: iscoroutine(obj) - Return ``True`` if *obj* is a :ref:`coroutine object `. + Return ``True`` if *obj* is a :ref:`coroutine object `, + which may be based on a generator or an :keyword:`async def` coroutine. -.. function:: iscoroutinefunction(obj) +.. function:: iscoroutinefunction(func) - Return ``True`` if *func* is a decorated :ref:`coroutine function - `. + Return ``True`` if *func* is determined to be a :ref:`coroutine function + `, which may be a decorated generator function or an + :keyword:`async def` function. .. coroutinefunction:: sleep(delay, result=None, \*, loop=None) diff --git a/Doc/library/collections.abc.rst b/Doc/library/collections.abc.rst index 8c710ef3807..0653d4e35c8 100644 --- a/Doc/library/collections.abc.rst +++ b/Doc/library/collections.abc.rst @@ -154,21 +154,22 @@ ABC Inherits from Abstract Methods Mixin .. class:: Awaitable - ABC for classes that provide ``__await__`` method. Instances - of such classes can be used in ``await`` expression. + ABC for :term:`awaitable` objects, which can be used in :keyword:`await` + expressions. Custom implementations must provide the :meth:`__await__` + method. - :term:`coroutine` objects and instances of - :class:`~collections.abc.Coroutine` are too instances of this ABC. + :term:`Coroutine` objects and instances of the + :class:`~collections.abc.Coroutine` ABC are all instances of this ABC. .. versionadded:: 3.5 .. class:: Coroutine - ABC for coroutine compatible classes that implement a subset of - generator methods defined in :pep:`342`, namely: - :meth:`~generator.send`, :meth:`~generator.throw`, - :meth:`~generator.close` methods. :meth:`__await__` must also be - implemented. All :class:`Coroutine` instances are also instances of + ABC for coroutine compatible classes. These implement the + following methods, defined in :ref:`coroutine-objects`: + :meth:`~coroutine.send`, :meth:`~coroutine.throw`, and + :meth:`~coroutine.close`. Custom implementations must also implement + :meth:`__await__`. All :class:`Coroutine` instances are also instances of :class:`Awaitable`. See also the definition of :term:`coroutine`. .. versionadded:: 3.5 diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst index 836c4c1ef7c..7a214edcc78 100644 --- a/Doc/library/dis.rst +++ b/Doc/library/dis.rst @@ -516,12 +516,14 @@ the original TOS1. Implements ``del TOS1[TOS]``. -**Coroutines opcodes** +**Coroutine opcodes** .. opcode:: GET_AWAITABLE - Implements ``TOS = get_awaitable(TOS)``; where ``get_awaitable(o)`` - returns ``o`` if ``o`` is a coroutine object; or resolved ``o.__await__``. + Implements ``TOS = get_awaitable(TOS)``, where ``get_awaitable(o)`` + returns ``o`` if ``o`` is a coroutine object or a generator object with + the CO_ITERABLE_COROUTINE flag, or resolves + ``o.__await__``. .. opcode:: GET_AITER diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst index 2209f167ea3..1a9d0299d5f 100644 --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -162,7 +162,8 @@ The following exceptions are the exceptions that are usually raised. .. exception:: GeneratorExit - Raised when a :term:`generator`\'s :meth:`close` method is called. It + Raised when a :term:`generator` or :term:`coroutine` is closed; + see :meth:`generator.close` and :meth:`coroutine.close`. It directly inherits from :exc:`BaseException` instead of :exc:`Exception` since it is technically not an error. @@ -306,7 +307,8 @@ The following exceptions are the exceptions that are usually raised. given as an argument when constructing the exception, and defaults to :const:`None`. - When a generator function returns, a new :exc:`StopIteration` instance is + When a :term:`generator` or :term:`coroutine` function + returns, a new :exc:`StopIteration` instance is raised, and the value returned by the function is used as the :attr:`value` parameter to the constructor of the exception. diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 144c986ab19..545e6742c13 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1080,7 +1080,7 @@ always available. :func:`types.coroutine` or :func:`asyncio.coroutine` will not be intercepted). - *wrapper* must be either: + The *wrapper* argument must be either: * a callable that accepts one argument (a coroutine object); * ``None``, to reset the wrapper. @@ -1096,7 +1096,8 @@ always available. return wrap(coro) sys.set_coroutine_wrapper(wrapper) - async def foo(): pass + async def foo(): + pass # The following line will fail with a RuntimeError, because # `wrapper` creates a `wrap(coro)` coroutine: diff --git a/Doc/library/tulip_coro.png b/Doc/library/tulip_coro.png index 65b6951550e..36ced8ddbfd 100644 Binary files a/Doc/library/tulip_coro.png and b/Doc/library/tulip_coro.png differ diff --git a/Doc/library/types.rst b/Doc/library/types.rst index d7b14e7a3d9..ff75de1ef55 100644 --- a/Doc/library/types.rst +++ b/Doc/library/types.rst @@ -281,15 +281,23 @@ Additional Utility Classes and Functions .. versionadded:: 3.4 -Coroutines Utility Functions ----------------------------- +Coroutine Utility Functions +--------------------------- .. function:: coroutine(gen_func) - The function transforms a generator function to a :term:`coroutine function`, - so that it returns a :term:`coroutine` object. + This function transforms a :term:`generator` function into a + :term:`coroutine function` which returns a generator-based coroutine. + The generator-based coroutine is still a :term:`generator iterator`, + but is also considered to be a :term:`coroutine` object and is + :term:`awaitable`. However, it may not necessarily implement + the :meth:`__await__` method. - *gen_func* is modified in-place, hence the function can be used as a - decorator. + If *gen_func* is a generator function, it will be modified in-place. + + If *gen_func* is not a generator function, it will be wrapped. If it + returns an instance of :class:`collections.abc.Generator`, the instance + will be wrapped in an *awaitable* proxy object. All other types + of objects will be returned as is. .. versionadded:: 3.5 diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index c73e8862a64..76b3850543a 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -666,15 +666,9 @@ can be used to create instance variables with different implementation details. Coroutines ========== -.. index:: - statement: async def - statement: async for - statement: async with - keyword: async - keyword: await - .. versionadded:: 3.5 +.. index:: statement: async def .. _`async def`: Coroutine function definition @@ -683,14 +677,23 @@ Coroutine function definition .. productionlist:: async_funcdef: "async" `funcdef` +.. index:: + keyword: async + keyword: await + Execution of Python coroutines can be suspended and resumed at many points -(see :term:`coroutine`.) :keyword:`await` expressions, :keyword:`async for` -and :keyword:`async with` can only be used in their bodies. +(see :term:`coroutine`). In the body of a coroutine, any ``await`` and +``async`` identifiers become reserved keywords; :keyword:`await` expressions, +:keyword:`async for` and :keyword:`async with` can only be used in +coroutine bodies. However, to simplify the parser, these keywords cannot +be used on the same line as a function or coroutine (:keyword:`def` +statement) header. Functions defined with ``async def`` syntax are always coroutine functions, even if they do not contain ``await`` or ``async`` keywords. -It is a :exc:`SyntaxError` to use :keyword:`yield` expressions in coroutines. +It is a :exc:`SyntaxError` to use :keyword:`yield` expressions in +``async def`` coroutines. An example of a coroutine function:: @@ -699,6 +702,7 @@ An example of a coroutine function:: await some_coroutine() +.. index:: statement: async for .. _`async for`: The :keyword:`async for` statement @@ -742,6 +746,7 @@ It is a :exc:`SyntaxError` to use ``async for`` statement outside of an :keyword:`async def` function. +.. index:: statement: async with .. _`async with`: The :keyword:`async with` statement diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 508b4b519cd..2ca1194c5ee 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -624,7 +624,7 @@ Callable types a :dfn:`coroutine function`. Such a function, when called, returns a :term:`coroutine` object. It may contain :keyword:`await` expressions, as well as :keyword:`async with` and :keyword:`async for` statements. See - also :ref:`coroutines` section. + also the :ref:`coroutine-objects` section. Built-in functions .. index:: @@ -2264,26 +2264,25 @@ special methods (the special method *must* be set on the class object itself in order to be consistently invoked by the interpreter). -.. _coroutines: +.. index:: + single: coroutine Coroutines ========== -.. index:: - single: coroutine - Awaitable Objects ----------------- -An *awaitable* object can be one of the following: +An :term:`awaitable` object generally implements an :meth:`__await__` method. +:term:`Coroutine` objects returned from :keyword:`async def` functions +are awaitable. -* A :term:`coroutine` object returned from a :term:`coroutine function`. +.. note:: -* A :term:`generator` decorated with :func:`types.coroutine` - (or :func:`asyncio.coroutine`) decorator. - -* An object that implements an ``__await__`` method. + The :term:`generator iterator` objects returned from generators + decorated with :func:`types.coroutine` or :func:`asyncio.coroutine` + are also awaitable, but they do not implement :meth:`__await__`. .. method:: object.__await__(self) @@ -2296,6 +2295,58 @@ An *awaitable* object can be one of the following: .. seealso:: :pep:`492` for additional information about awaitable objects. +.. _coroutine-objects: + +Coroutine Objects +----------------- + +:term:`Coroutine` objects are :term:`awaitable` objects. +A coroutine's execution can be controlled by calling :meth:`__await__` and +iterating over the result. When the coroutine has finished executing and +returns, the iterator raises :exc:`StopIteration`, and the exception's +:attr:`~StopIteration.value` attribute holds the return value. If the +coroutine raises an exception, it is propagated by the iterator. Coroutines +should not directly raise unhandled :exc:`StopIteration` exceptions. + +Coroutines also have the methods listed below, which are analogous to +those of generators (see :ref:`generator-methods`). However, unlike +generators, coroutines do not directly support iteration. + +.. method:: coroutine.send(value) + + Starts or resumes execution of the coroutine. If *value* is ``None``, + this is equivalent to advancing the iterator returned by + :meth:`__await__`. If *value* is not ``None``, this method delegates + to the :meth:`~generator.send` method of the iterator that caused + the coroutine to suspend. The result (return value, + :exc:`StopIteration`, or other exception) is the same as when + iterating over the :meth:`__await__` return value, described above. + +.. method:: coroutine.throw(type[, value[, traceback]]) + + Raises the specified exception in the coroutine. This method delegates + to the :meth:`~generator.throw` method of the iterator that caused + the coroutine to suspend, if it has such a method. Otherwise, + the exception is raised at the suspension point. The result + (return value, :exc:`StopIteration`, or other exception) is the same as + when iterating over the :meth:`__await__` return value, described + above. If the exception is not caught in the coroutine, it propagates + back to the caller. + +.. method:: coroutine.close() + + Causes the coroutine to clean itself up and exit. If the coroutine + is suspended, this method first delegates to the :meth:`~generator.close` + method of the iterator that caused the coroutine to suspend, if it + has such a method. Then it raises :exc:`GeneratorExit` at the + suspension point, causing the coroutine to immediately clean itself up. + Finally, the coroutine is marked as having finished executing, even if + it was never started. + + Coroutine objects are automatically closed using the above process when + they are about to be destroyed. + + Asynchronous Iterators ---------------------- diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 9d3fdf26fc0..d3face7f6a7 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -390,6 +390,7 @@ on the right hand side of an assignment statement. to sub-generators easy. .. index:: object: generator +.. _generator-methods: Generator-iterator methods ^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/Objects/genobject.c b/Objects/genobject.c index 40340b48742..3311c4e0e75 100644 --- a/Objects/genobject.c +++ b/Objects/genobject.c @@ -793,11 +793,11 @@ static PyMemberDef coro_memberlist[] = { PyDoc_STRVAR(coro_send_doc, "send(arg) -> send 'arg' into coroutine,\n\ -return next yielded value or raise StopIteration."); +return next iterated value or raise StopIteration."); PyDoc_STRVAR(coro_throw_doc, "throw(typ[,val[,tb]]) -> raise exception in coroutine,\n\ -return next yielded value or raise StopIteration."); +return next iterated value or raise StopIteration."); PyDoc_STRVAR(coro_close_doc, "close() -> raise GeneratorExit inside coroutine."); @@ -908,9 +908,9 @@ coro_wrapper_traverse(PyCoroWrapper *cw, visitproc visit, void *arg) } static PyMethodDef coro_wrapper_methods[] = { - {"send",(PyCFunction)coro_wrapper_send, METH_O, send_doc}, - {"throw",(PyCFunction)coro_wrapper_throw, METH_VARARGS, throw_doc}, - {"close",(PyCFunction)coro_wrapper_close, METH_NOARGS, close_doc}, + {"send",(PyCFunction)coro_wrapper_send, METH_O, coro_send_doc}, + {"throw",(PyCFunction)coro_wrapper_throw, METH_VARARGS, coro_throw_doc}, + {"close",(PyCFunction)coro_wrapper_close, METH_NOARGS, coro_close_doc}, {NULL, NULL} /* Sentinel */ };