traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Update asyncio documentation 

- Document the new create_task() method
- "Hide" the Task class: point to the create_task() method for interoperability
- Rewrite the documentation of the Task class
- Document the "Pending task destroyed"
- Update output in debug mode of examples in the dev section
- Replace Task() with create_task() in examples
This commit is contained in:
Victor Stinner 2014-07-08 12:39:10 +02:00
parent 896a25ab30
commit 530ef2f069
4 changed files with 131 additions and 49 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

95
Doc/library/asyncio-dev.rst
View File

@ -103,20 +103,11 @@ the logger ``'asyncio'``.
Detect coroutine objects never scheduled
----------------------------------------
When a coroutine function is called but not passed to :func:`async` or to the
:class:`Task` constructor, it is not scheduled and it is probably a bug.
To detect such bug, :ref:`enable the debug mode of asyncio
<asyncio-debug-mode>`. When the coroutine object is destroyed by the garbage
collector, a log will be emitted with the traceback where the coroutine
function was called. See the :ref:`asyncio logger <asyncio-logger>`.
The debug flag changes the behaviour of the :func:`coroutine` decorator. The
debug flag value is only used when then coroutine function is defined, not when
it is called. Coroutine functions defined before the debug flag is set to
``True`` will not be tracked. For example, it is not possible to debug
coroutines defined in the :mod:`asyncio` module, because the module must be
imported before the flag value can be changed.
When a coroutine function is called and its result is not passed to
:func:`async` or to the :meth:`BaseEventLoop.create_task` method: the execution
of the coroutine objet will never be scheduled and it is probably a bug.
:ref:`Enable the debug mode of asyncio <asyncio-debug-mode>` to :ref:`log a
warning <asyncio-logger>` to detect it.
Example with the bug::
@ -130,20 +121,27 @@ Example with the bug::
Output in debug mode::
Coroutine 'test' defined at test.py:4 was never yielded from
Coroutine test() at test.py:3 was never yielded from
Coroutine object created at (most recent call last):
File "test.py", line 7, in <module>
test()
The fix is to call the :func:`async` function or create a :class:`Task` object
with this coroutine object.
The fix is to call the :func:`async` function or the
:meth:`BaseEventLoop.create_task` method with the coroutine object.
.. seealso::
:ref:`Pending task destroyed <asyncio-pending-task-destroyed>`.
Detect exceptions not consumed
------------------------------
Detect exceptions never consumed
--------------------------------
Python usually calls :func:`sys.displayhook` on unhandled exceptions. If
:meth:`Future.set_exception` is called, but the exception is not consumed,
:func:`sys.displayhook` is not called. Instead, a log is emitted when the
future is deleted by the garbage collector, with the traceback where the
exception was raised. See the :ref:`asyncio logger <asyncio-logger>`.
:meth:`Future.set_exception` is called, but the exception is never consumed,
:func:`sys.displayhook` is not called. Instead, a :ref:`a log is emitted
<asyncio-logger>` when the future is deleted by the garbage collector, with the
traceback where the exception was raised.
Example of unhandled exception::
@ -159,16 +157,27 @@ Example of unhandled exception::
Output::
Future/Task exception was never retrieved:
Task exception was never retrieved
future: <Task finished bug() done at asyncio/coroutines.py:139 exception=Exception('not consumed',)>
source_traceback: Object created at (most recent call last):
File "test.py", line 10, in <module>
asyncio.async(bug())
File "asyncio/tasks.py", line 510, in async
task = loop.create_task(coro_or_future)
Traceback (most recent call last):
File "/usr/lib/python3.4/asyncio/tasks.py", line 279, in _step
File "asyncio/tasks.py", line 244, in _step
result = next(coro)
File "/usr/lib/python3.4/asyncio/tasks.py", line 80, in coro
File "coroutines.py", line 78, in __next__
return next(self.gen)
File "asyncio/coroutines.py", line 141, in coro
res = func(*args, **kw)
File "test.py", line 5, in bug
File "test.py", line 7, in bug
raise Exception("not consumed")
Exception: not consumed
:ref:`Enable the debug mode of asyncio <asyncio-debug-mode>` to get the
traceback where the task was created.
There are different options to fix this issue. The first option is to chain to
coroutine in another coroutine and use classic try/except::
@ -195,7 +204,7 @@ function::
See also the :meth:`Future.exception` method.
Chain coroutines correctly
Chain correctly coroutines
--------------------------
When a coroutine function calls other coroutine functions and tasks, they
@ -246,7 +255,9 @@ Actual output::
(3) close file
(2) write into file
Pending tasks at exit: {Task(<create>)<PENDING>}
Pending tasks at exit: {<Task pending create() at test.py:7 wait_for=<Future pending cb=[Task._wakeup()]>>}
Task was destroyed but it is pending!
task: <Task pending create() done at test.py:5 wait_for=<Future pending cb=[Task._wakeup()]>>
The loop stopped before the ``create()`` finished, ``close()`` has been called
before ``write()``, whereas coroutine functions were called in this order:
@ -272,3 +283,29 @@ Or without ``asyncio.async()``::
yield from asyncio.sleep(2.0)
loop.stop()
.. _asyncio-pending-task-destroyed:
Pending task destroyed
----------------------
If a pending task is destroyed, the execution of its wrapped :ref:`coroutine
<coroutine>` did not complete. It is probably a bug and so a warning is logged.
Example of log::
Task was destroyed but it is pending!
source_traceback: Object created at (most recent call last):
File "test.py", line 17, in <module>
task = asyncio.async(coro, loop=loop)
File "asyncio/tasks.py", line 510, in async
task = loop.create_task(coro_or_future)
task: <Task pending kill_me() done at test.py:5 wait_for=<Future pending cb=[Task._wakeup()]>>
:ref:`Enable the debug mode of asyncio <asyncio-debug-mode>` to get the
traceback where the task was created.
.. seealso::
:ref:`Detect coroutine objects never scheduled <asyncio-coroutine-not-scheduled>`.

23
Doc/library/asyncio-eventloop.rst
View File

@ -102,8 +102,8 @@ Run an event loop
Run until the :class:`Future` is done.
If the argument is a :ref:`coroutine <coroutine>`, it is wrapped
in a :class:`Task`.
If the argument is a :ref:`coroutine object <coroutine>`, it is wrapped by
:func:`async`.
Return the Future's result, or raise its exception.
@ -205,6 +205,25 @@ a different clock than :func:`time.time`.
The :func:`asyncio.sleep` function.
Coroutines
----------
.. method:: BaseEventLoop.create_task(coro)
Schedule the execution of a :ref:`coroutine object <coroutine>`: wrap it in
a future. Return a :class:`Task` object.
Third-party event loops can use their own subclass of :class:`Task` for
interoperability. In this case, the result type is a subclass of
:class:`Task`.
.. seealso::
The :meth:`async` function.
.. versionadded:: 3.4.2
Creating connections
--------------------

3
Doc/library/asyncio-stream.rst
View File

@ -41,7 +41,8 @@ Stream functions
:class:`StreamReader` object, while *client_writer* is a
:class:`StreamWriter` object. This parameter can either be a plain callback
function or a :ref:`coroutine function <coroutine>`; if it is a coroutine
function, it will be automatically converted into a :class:`Task`.
function, it will be automatically wrapped in a future using the
:meth:`BaseEventLoop.create_task` method.
The rest of the arguments are all the usual arguments to
:meth:`~BaseEventLoop.create_server()` except *protocol_factory*; most

59
Doc/library/asyncio-task.rst
View File

@ -51,8 +51,8 @@ generator, and the coroutine object returned by the call is really a
generator object, which doesn't do anything until you iterate over it.
In the case of a coroutine object, there are two basic ways to start
it running: call ``yield from coroutine`` from another coroutine
(assuming the other coroutine is already running!), or convert it to a
:class:`Task`.
(assuming the other coroutine is already running!), or schedule its execution
using the :meth:`BaseEventLoop.create_task` method.
Coroutines (and tasks) can only run when the event loop is running.
@ -256,7 +256,7 @@ Example combining a :class:`Future` and a :ref:`coroutine function
loop = asyncio.get_event_loop()
future = asyncio.Future()
asyncio.Task(slow_operation(future))
loop.create_task(slow_operation(future))
loop.run_until_complete(future)
print(future.result())
loop.close()
@ -292,7 +292,7 @@ flow::
loop = asyncio.get_event_loop()
future = asyncio.Future()
asyncio.Task(slow_operation(future))
loop.create_task(slow_operation(future))
future.add_done_callback(got_result)
try:
loop.run_forever()
@ -314,7 +314,33 @@ Task
.. class:: Task(coro, \*, loop=None)
A coroutine object wrapped in a :class:`Future`. Subclass of :class:`Future`.
Schedule the execution of a :ref:`coroutine <coroutine>`: wrap it in a
future. A task is a subclass of :class:`Future`.
A task is responsible to execute a coroutine object in an event loop. If
the wrapped coroutine yields from a future, the task suspends the execution
of the wrapped coroutine and waits for the completition of the future. When
the future is done, the execution of the wrapped coroutine restarts with the
result or the exception of the future.
Event loops use cooperative scheduling: an event loop only runs one task at
the same time. Other tasks may run in parallel if other event loops are
running in different threads. While a task waits for the completion of a
future, the event loop executes a new task.
The cancellation of a task is different than cancelling a future. Calling
:meth:`cancel` will throw a :exc:`~concurrent.futures.CancelledError` to the
wrapped coroutine. :meth:`~Future.cancelled` only returns ``True`` if the
wrapped coroutine did not catch the
:exc:`~concurrent.futures.CancelledError` exception, or raised a
:exc:`~concurrent.futures.CancelledError` exception.
If a pending task is destroyed, the execution of its wrapped :ref:`coroutine
<coroutine>` did not complete. It is probably a bug and a warning is
logged: see :ref:`Pending task destroyed <asyncio-pending-task-destroyed>`.
Don't create directly :class:`Task` instances: use the
:meth:`BaseEventLoop.create_task` method.
.. classmethod:: all_tasks(loop=None)
@ -396,12 +422,11 @@ Example executing 3 tasks (A, B, C) in parallel::
f *= i
print("Task %s: factorial(%s) = %s" % (name, number, f))
tasks = [
asyncio.Task(factorial("A", 2)),
asyncio.Task(factorial("B", 3)),
asyncio.Task(factorial("C", 4))]
loop = asyncio.get_event_loop()
tasks = [
loop.create_task(factorial("A", 2)),
loop.create_task(factorial("B", 3)),
loop.create_task(factorial("C", 4))]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()
@ -450,7 +475,8 @@ Task functions
.. function:: async(coro_or_future, \*, loop=None)
Wrap a :ref:`coroutine object <coroutine>` in a future.
Wrap a :ref:`coroutine object <coroutine>` in a future using the
:meth:`BaseEventLoop.create_task` method.
If the argument is a :class:`Future`, it is returned directly.
@ -566,18 +592,17 @@ Task functions
.. function:: wait_for(fut, timeout, \*, loop=None)
Wait for the single :class:`Future` or :ref:`coroutine object <coroutine>`
to complete, with timeout. If *timeout* is ``None``, block until the future
to complete with timeout. If *timeout* is ``None``, block until the future
completes.
Coroutine will be wrapped in :class:`Task`.
Coroutine objects are wrapped in a future using the
:meth:`BaseEventLoop.create_task` method.
Returns result of the Future or coroutine. When a timeout occurs, it
cancels the task and raises :exc:`asyncio.TimeoutError`. To avoid the task
cancellation, wrap it in :func:`shield`.
This function is a :ref:`coroutine <coroutine>`.
This function is a :ref:`coroutine <coroutine>`, usage::
Usage::
result = yield from asyncio.wait_for(fut, 60.0)
result = yield from asyncio.wait_for(fut, 60.0)