bpo-41229: Update docs for explicit aclose()-required cases and add contextlib.aclosing() method (GH-21545)
This is a PR to: * Add `contextlib.aclosing` which ia analogous to `contextlib.closing` but for async-generators with an explicit test case for [bpo-41229]() * Update the docs to describe when we need explicit `aclose()` invocation. which are motivated by the following issues, articles, and examples: * [bpo-41229]() * https://github.com/njsmith/async_generator * https://vorpus.org/blog/some-thoughts-on-asynchronous-api-design-in-a-post-asyncawait-world/#cleanup-in-generators-and-async-generators * https://www.python.org/dev/peps/pep-0533/ * https://github.com/achimnol/aiotools/blob/ef7bf0cea7af/src/aiotools/context.py#L152 Particuarly regarding [PEP-533](https://www.python.org/dev/peps/pep-0533/), its acceptance (`__aiterclose__()`) would make this little addition of `contextlib.aclosing()` unnecessary for most use cases, but until then this could serve as a good counterpart and analogy to `contextlib.closing()`. The same applies for `contextlib.closing` with `__iterclose__()`. Also, still there are other use cases, e.g., when working with non-generator objects with `aclose()` methods.
This commit is contained in:
parent
e9208f0e74
commit
6e8dcdaaa4
|
@ -154,6 +154,39 @@ Functions and classes provided:
|
|||
``page.close()`` will be called when the :keyword:`with` block is exited.
|
||||
|
||||
|
||||
.. class:: aclosing(thing)
|
||||
|
||||
Return an async context manager that calls the ``aclose()`` method of *thing*
|
||||
upon completion of the block. This is basically equivalent to::
|
||||
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
@asynccontextmanager
|
||||
async def aclosing(thing):
|
||||
try:
|
||||
yield thing
|
||||
finally:
|
||||
await thing.aclose()
|
||||
|
||||
Significantly, ``aclosing()`` supports deterministic cleanup of async
|
||||
generators when they happen to exit early by :keyword:`break` or an
|
||||
exception. For example::
|
||||
|
||||
from contextlib import aclosing
|
||||
|
||||
async with aclosing(my_generator()) as values:
|
||||
async for value in values:
|
||||
if value == 42:
|
||||
break
|
||||
|
||||
This pattern ensures that the generator's async exit code is executed in
|
||||
the same context as its iterations (so that exceptions and context
|
||||
variables work as expected, and the exit code isn't run after the
|
||||
lifetime of some task it depends on).
|
||||
|
||||
.. versionadded:: 3.10
|
||||
|
||||
|
||||
.. _simplifying-support-for-single-optional-context-managers:
|
||||
|
||||
.. function:: nullcontext(enter_result=None)
|
||||
|
|
|
@ -643,6 +643,16 @@ after resuming depends on the method which resumed the execution. If
|
|||
:meth:`~agen.asend` is used, then the result will be the value passed in to
|
||||
that method.
|
||||
|
||||
If an asynchronous generator happens to exit early by :keyword:`break`, the caller
|
||||
task being cancelled, or other exceptions, the generator's async cleanup code
|
||||
will run and possibly raise exceptions or access context variables in an
|
||||
unexpected context--perhaps after the lifetime of tasks it depends, or
|
||||
during the event loop shutdown when the async-generator garbage collection hook
|
||||
is called.
|
||||
To prevent this, the caller must explicitly close the async generator by calling
|
||||
:meth:`~agen.aclose` method to finalize the generator and ultimately detach it
|
||||
from the event loop.
|
||||
|
||||
In an asynchronous generator function, yield expressions are allowed anywhere
|
||||
in a :keyword:`try` construct. However, if an asynchronous generator is not
|
||||
resumed before it is finalized (by reaching a zero reference count or by
|
||||
|
@ -654,9 +664,9 @@ generator-iterator's :meth:`~agen.aclose` method and run the resulting
|
|||
coroutine object, thus allowing any pending :keyword:`!finally` clauses
|
||||
to execute.
|
||||
|
||||
To take care of finalization, an event loop should define
|
||||
a *finalizer* function which takes an asynchronous generator-iterator
|
||||
and presumably calls :meth:`~agen.aclose` and executes the coroutine.
|
||||
To take care of finalization upon event loop termination, an event loop should
|
||||
define a *finalizer* function which takes an asynchronous generator-iterator and
|
||||
presumably calls :meth:`~agen.aclose` and executes the coroutine.
|
||||
This *finalizer* may be registered by calling :func:`sys.set_asyncgen_hooks`.
|
||||
When first iterated over, an asynchronous generator-iterator will store the
|
||||
registered *finalizer* to be called upon finalization. For a reference example
|
||||
|
|
|
@ -303,6 +303,32 @@ class closing(AbstractContextManager):
|
|||
self.thing.close()
|
||||
|
||||
|
||||
class aclosing(AbstractAsyncContextManager):
|
||||
"""Async context manager for safely finalizing an asynchronously cleaned-up
|
||||
resource such as an async generator, calling its ``aclose()`` method.
|
||||
|
||||
Code like this:
|
||||
|
||||
async with aclosing(<module>.fetch(<arguments>)) as agen:
|
||||
<block>
|
||||
|
||||
is equivalent to this:
|
||||
|
||||
agen = <module>.fetch(<arguments>)
|
||||
try:
|
||||
<block>
|
||||
finally:
|
||||
await agen.aclose()
|
||||
|
||||
"""
|
||||
def __init__(self, thing):
|
||||
self.thing = thing
|
||||
async def __aenter__(self):
|
||||
return self.thing
|
||||
async def __aexit__(self, *exc_info):
|
||||
await self.thing.aclose()
|
||||
|
||||
|
||||
class _RedirectStream(AbstractContextManager):
|
||||
|
||||
_stream = None
|
||||
|
|
|
@ -1,5 +1,5 @@
|
|||
import asyncio
|
||||
from contextlib import asynccontextmanager, AbstractAsyncContextManager, AsyncExitStack
|
||||
from contextlib import aclosing, asynccontextmanager, AbstractAsyncContextManager, AsyncExitStack
|
||||
import functools
|
||||
from test import support
|
||||
import unittest
|
||||
|
@ -279,6 +279,63 @@ class AsyncContextManagerTestCase(unittest.TestCase):
|
|||
self.assertEqual(target, (11, 22, 33, 44))
|
||||
|
||||
|
||||
class AclosingTestCase(unittest.TestCase):
|
||||
|
||||
@support.requires_docstrings
|
||||
def test_instance_docs(self):
|
||||
cm_docstring = aclosing.__doc__
|
||||
obj = aclosing(None)
|
||||
self.assertEqual(obj.__doc__, cm_docstring)
|
||||
|
||||
@_async_test
|
||||
async def test_aclosing(self):
|
||||
state = []
|
||||
class C:
|
||||
async def aclose(self):
|
||||
state.append(1)
|
||||
x = C()
|
||||
self.assertEqual(state, [])
|
||||
async with aclosing(x) as y:
|
||||
self.assertEqual(x, y)
|
||||
self.assertEqual(state, [1])
|
||||
|
||||
@_async_test
|
||||
async def test_aclosing_error(self):
|
||||
state = []
|
||||
class C:
|
||||
async def aclose(self):
|
||||
state.append(1)
|
||||
x = C()
|
||||
self.assertEqual(state, [])
|
||||
with self.assertRaises(ZeroDivisionError):
|
||||
async with aclosing(x) as y:
|
||||
self.assertEqual(x, y)
|
||||
1 / 0
|
||||
self.assertEqual(state, [1])
|
||||
|
||||
@_async_test
|
||||
async def test_aclosing_bpo41229(self):
|
||||
state = []
|
||||
|
||||
class Resource:
|
||||
def __del__(self):
|
||||
state.append(1)
|
||||
|
||||
async def agenfunc():
|
||||
r = Resource()
|
||||
yield -1
|
||||
yield -2
|
||||
|
||||
x = agenfunc()
|
||||
self.assertEqual(state, [])
|
||||
with self.assertRaises(ZeroDivisionError):
|
||||
async with aclosing(x) as y:
|
||||
self.assertEqual(x, y)
|
||||
self.assertEqual(-1, await x.__anext__())
|
||||
1 / 0
|
||||
self.assertEqual(state, [1])
|
||||
|
||||
|
||||
class TestAsyncExitStack(TestBaseExitStack, unittest.TestCase):
|
||||
class SyncAsyncExitStack(AsyncExitStack):
|
||||
@staticmethod
|
||||
|
|
|
@ -0,0 +1,3 @@
|
|||
Add ``contextlib.aclosing`` for deterministic cleanup of async generators
|
||||
which is analogous to ``contextlib.closing`` for non-async generators.
|
||||
Patch by Joongi Kim and John Belmonte.
|
Loading…
Reference in New Issue