contextlib doc updates and refactoring

- explain single use, reusable and reentrant in docs
- converted suppress to a reentrant class based impl
- converted redirect_stdout to a reusable impl
- moved both suppress and redirect_stdout behind a functional
  facade
- added reentrancy tests for the updated suppress
- added reusability tests for the updated redirect_stdio
- slightly cleaned up an exception from contextmanager

Doc/library/contextlib.rst

@@ -128,6 +128,8 @@ Functions and classes provided:
        except FileNotFoundError:
            pass
 
+   This context manager is :ref:`reentrant <reentrant-cms>`.
+
    .. versionadded:: 3.4
 
 
@@ -165,6 +167,8 @@ Functions and classes provided:
    applications. It also has no effect on the output of subprocesses.
    However, it is still a useful approach for many utility scripts.
 
+   This context manager is :ref:`reusable but not reentrant <reusable-cms>`.
+
    .. versionadded:: 3.4
 
 
@@ -593,3 +597,115 @@ an explicit ``with`` statement.
       The specification, background, and examples for the Python :keyword:`with`
       statement.
 
+
+Reusable and reentrant context managers
+---------------------------------------
+
+Most context managers are written in a way that means they can only be
+used effectively in a :keyword:`with` statement once. These single use
+context managers must be created afresh each time they're used -
+attempting to use them a second time will trigger an exception or
+otherwise not work correctly.
+
+This common limitation means that it is generally advisable to create
+context managers directly in the header of the :keyword:`with` statement
+where they are used (as shown in all of the usage examples above).
+
+Files are an example of effectively single use context managers, since
+the first :keyword:`with` statement will close the file, preventing any
+further IO operations using that file object.
+
+Context managers created using :func:`contextmanager` are also single use
+context managers, and will complain about the underlying generator failing
+to yield if an attempt is made to use them a second time::
+
+    >>> from contextlib import contextmanager
+    >>> @contextmanager
+    ... def singleuse():
+    ...     print("Before")
+    ...     yield
+    ...     print("After")
+    ...
+    >>> cm = singleuse()
+    >>> with cm:
+    ...     pass
+    ...
+    Before
+    After
+    >>> with cm:
+    ...    pass
+    ...
+    Traceback (most recent call last):
+        ...
+    RuntimeError: generator didn't yield
+
+
+.. _reentrant-cms:
+
+Reentrant context managers
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+More sophisticated context managers may be "reentrant". These context
+managers can not only be used in multiple :keyword:`with` statements,
+but may also be used *inside* a :keyword:`with` statement that is already
+using the same context manager.
+
+:class:`threading.RLock` is an example of a reentrant context manager, as is
+:func:`suppress`. Here's a toy example of reentrant use (real world
+examples of reentrancy are more likely to occur with objects like recursive
+locks and are likely to be far more complicated than this example)::
+
+    >>> from contextlib import suppress
+    >>> ignore_raised_exception = suppress(ZeroDivisionError)
+    >>> with ignore_raised_exception:
+    ...     with ignore_raised_exception:
+    ...         1/0
+    ...     print("This line runs")
+    ...     1/0
+    ...     print("This is skipped")
+    ...
+    This line runs
+    >>> # The second exception is also suppressed
+
+
+.. _reusable-cms:
+
+Reusable context managers
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Distinct from both single use and reentrant context managers are "reusable"
+context managers (or, to be completely explicit, "reusable, but not
+reentrant" context managers, since reentrant context managers are also
+reusable). These context managers support being used multiple times, but
+will fail (or otherwise not work correctly) if the specific context manager
+instance has already been used in a containing with statement.
+
+An example of a reusable context manager is :func:`redirect_stdout`::
+
+    >>> from contextlib import redirect_stdout
+    >>> from io import StringIO
+    >>> f = StringIO()
+    >>> collect_output = redirect_stdout(f)
+    >>> with collect_output:
+    ...     print("Collected")
+    ...
+    >>> print("Not collected")
+    Not collected
+    >>> with collect_output:
+    ...     print("Also collected")
+    ...
+    >>> print(f.getvalue())
+    Collected
+    Also collected
+
+However, this context manager is not reentrant, so attempting to reuse it
+within a containing with statement fails:
+
+    >>> with collect_output:
+    ...     # Nested reuse is not permitted
+    ...     with collect_output:
+    ...         pass
+    ...
+    Traceback (most recent call last):
+      ...
+    RuntimeError: Cannot reenter <...>

Lib/contextlib.py

@@ -48,7 +48,7 @@ class _GeneratorContextManager(ContextDecorator):
         try:
             return next(self.gen)
         except StopIteration:
-            raise RuntimeError("generator didn't yield")
+            raise RuntimeError("generator didn't yield") from None
 
     def __exit__(self, type, value, traceback):
         if type is None:
@@ -117,6 +117,9 @@ def contextmanager(func):
     return helper
 
 
+# Unfortunately, this was originally published as a class, so
+# backwards compatibility prevents the use of the wrapper function
+# approach used for the other classes
 class closing(object):
     """Context to automatically close something at the end of a block.
 
@@ -141,55 +144,75 @@ class closing(object):
     def __exit__(self, *exc_info):
         self.thing.close()
 
-class redirect_stdout:
+class _RedirectStdout:
+    """Helper for redirect_stdout."""
+
+    def __init__(self, new_target):
+        self._new_target = new_target
+        self._old_target = self._sentinel = object()
+
+    def __enter__(self):
+        if self._old_target is not self._sentinel:
+            raise RuntimeError("Cannot reenter {!r}".format(self))
+        self._old_target = sys.stdout
+        sys.stdout = self._new_target
+        return self._new_target
+
+    def __exit__(self, exctype, excinst, exctb):
+        restore_stdout = self._old_target
+        self._old_target = self._sentinel
+        sys.stdout = restore_stdout
+
+# Use a wrapper function since we don't care about supporting inheritance
+# and a function gives much cleaner output in help()
+def redirect_stdout(target):
     """Context manager for temporarily redirecting stdout to another file
 
         # How to send help() to stderr
-
         with redirect_stdout(sys.stderr):
             help(dir)
 
         # How to write help() to a file
-
         with open('help.txt', 'w') as f:
             with redirect_stdout(f):
                 help(pow)
-
-        # How to capture disassembly to a string
-
-        import dis
-        import io
-
-        f = io.StringIO()
-        with redirect_stdout(f):
-            dis.dis('x**2 - y**2')
-        s = f.getvalue()
-
     """
+    return _RedirectStdout(target)
 
-    def __init__(self, new_target):
+
-        self.new_target = new_target
+class _SuppressExceptions:
+    """Helper for suppress."""
+    def __init__(self, *exceptions):
+        self._exceptions = exceptions
 
     def __enter__(self):
-        self.old_target = sys.stdout
+        pass
-        sys.stdout = self.new_target
-        return self.new_target
 
     def __exit__(self, exctype, excinst, exctb):
-        sys.stdout = self.old_target
+        # Unlike isinstance and issubclass, exception handling only
+        # looks at the concrete type heirarchy (ignoring the instance
+        # and subclass checking hooks). However, all exceptions are
+        # also required to be concrete subclasses of BaseException, so
+        # if there's a discrepancy in behaviour, we currently consider it
+        # the fault of the strange way the exception has been defined rather
+        # than the fact that issubclass can be customised while the
+        # exception checks can't.
+        # See http://bugs.python.org/issue12029 for more details
+        return exctype is not None and issubclass(exctype, self._exceptions)
 
-@contextmanager
+# Use a wrapper function since we don't care about supporting inheritance
+# and a function gives much cleaner output in help()
 def suppress(*exceptions):
     """Context manager to suppress specified exceptions
 
-         with suppress(OSError):
+    After the exception is suppressed, execution proceeds with the next
-             os.remove(somefile)
+    statement following the with statement.
 
+         with suppress(FileNotFoundError):
+             os.remove(somefile)
+         # Execution still resumes here if the file was already removed
     """
-    try:
+    return _SuppressExceptions(*exceptions)
-        yield
-    except exceptions:
-        pass
 
 # Inspired by discussions on http://bugs.python.org/issue13585
 class ExitStack(object):

Lib/test/test_contextlib.py

@@ -641,27 +641,67 @@ class TestRedirectStdout(unittest.TestCase):
         s = f.getvalue()
         self.assertIn('pow', s)
 
+    def test_enter_result_is_target(self):
+        f = io.StringIO()
+        with redirect_stdout(f) as enter_result:
+            self.assertIs(enter_result, f)
+
+    def test_cm_is_reusable(self):
+        f = io.StringIO()
+        write_to_f = redirect_stdout(f)
+        with write_to_f:
+            print("Hello", end=" ")
+        with write_to_f:
+            print("World!")
+        s = f.getvalue()
+        self.assertEqual(s, "Hello World!\n")
+
+    # If this is ever made reentrant, update the reusable-but-not-reentrant
+    # example at the end of the contextlib docs accordingly.
+    def test_nested_reentry_fails(self):
+        f = io.StringIO()
+        write_to_f = redirect_stdout(f)
+        with self.assertRaisesRegex(RuntimeError, "Cannot reenter"):
+            with write_to_f:
+                print("Hello", end=" ")
+                with write_to_f:
+                    print("World!")
+
+
 class TestSuppress(unittest.TestCase):
 
-    def test_no_exception(self):
+    def test_no_result_from_enter(self):
+        with suppress(ValueError) as enter_result:
+            self.assertIsNone(enter_result)
 
+    def test_no_exception(self):
         with suppress(ValueError):
             self.assertEqual(pow(2, 5), 32)
 
     def test_exact_exception(self):
-
         with suppress(TypeError):
             len(5)
 
     def test_multiple_exception_args(self):
-
+        with suppress(ZeroDivisionError, TypeError):
+            1/0
         with suppress(ZeroDivisionError, TypeError):
             len(5)
 
     def test_exception_hierarchy(self):
-
         with suppress(LookupError):
             'Hello'[50]
 
+    def test_cm_is_reentrant(self):
+        ignore_exceptions = suppress(Exception)
+        with ignore_exceptions:
+            pass
+        with ignore_exceptions:
+            len(5)
+        with ignore_exceptions:
+            1/0
+            with ignore_exceptions: # Check nested usage
+                len(5)
+
 if __name__ == "__main__":
     unittest.main()

Misc/NEWS

@@ -74,7 +74,7 @@ Library
 - Issue #19266: Rename the new-in-3.4 ``contextlib.ignore`` context manager
   to ``contextlib.suppress`` in order to be more consistent with existing
   descriptions of that operation elsewhere in the language and standard
-  library documentation (Patch by Zero Piraeus)
+  library documentation (Patch by Zero Piraeus).
 
 - Issue #18891: Completed the new email package (provisional) API additions
   by adding new classes EmailMessage, MIMEPart, and ContentManager.