From 988fb28431d3a3fc5dc6eb657c8a4baacc04d9ce Mon Sep 17 00:00:00 2001 From: Cheryl Sabella Date: Sun, 11 Feb 2018 08:10:42 -0500 Subject: [PATCH] bpo-11015: Update test.support documentation (GH-5610) --- Doc/library/test.rst | 728 +++++++++++++++++- .../2018-02-10-12-48-38.bpo-11015.-gUf34.rst | 1 + 2 files changed, 723 insertions(+), 6 deletions(-) create mode 100644 Misc/NEWS.d/next/Documentation/2018-02-10-12-48-38.bpo-11015.-gUf34.rst diff --git a/Doc/library/test.rst b/Doc/library/test.rst index 01ba1ec7062..a366fe2556b 100644 --- a/Doc/library/test.rst +++ b/Doc/library/test.rst @@ -237,12 +237,127 @@ The :mod:`test.support` module defines the following constants: ``True`` if the running interpreter is Jython. +.. data:: is_android + + ``True`` if the system is Android. + + +.. data:: unix_shell + + Path for shell if not on Windows; otherwise ``None``. + + +.. data:: FS_NONASCII + + A non-ASCII character encodable by :func:`os.fsencode`. + + .. data:: TESTFN Set to a name that is safe to use as the name of a temporary file. Any temporary file that is created should be closed and unlinked (removed). +.. data:: TESTFN_UNICODE + + Set to a non-ASCII name for a temporary file. + + +.. data:: TESTFN_ENCODING + + Set to :func:`sys.getfilesystemencoding`. + + +.. data:: TESTFN_UNENCODABLE + + Set to a filename (str type) that should not be able to be encoded by file + system encoding in strict mode. It may be ``None`` if it's not possible to + generate such a filename. + + +.. data:: TESTFN_UNDECODABLE + + Set to a filename (bytes type) that should not be able to be decoded by + file system encoding in strict mode. It may be ``None`` if it's not + possible to generate such a filename. + + +.. data:: TESTFN_NONASCII + + Set to a filename containing the :data:`FS_NONASCII` character. + + +.. data:: IPV6_ENABLED + + Set to ``True`` if IPV6 is enabled on this host, ``False`` otherwise. + + +.. data:: SAVEDCWD + + Set to :func:`os.getcwd`. + + +.. data:: PGO + + Set when tests can be skipped when they are not useful for PGO. + + +.. data:: PIPE_MAX_SIZE + + A constant that is likely larger than the underlying OS pipe buffer size, + to make writes blocking. + + +.. data:: SOCK_MAX_SIZE + + A constant that is likely larger than the underlying OS socket buffer size, + to make writes blocking. + + +.. data:: TEST_SUPPORT_DIR + + Set to the top level directory that contains :mod:`test.support`. + + +.. data:: TEST_HOME_DIR + + Set to the top level directory for the test package. + + +.. data:: TEST_DATA_DIR + + Set to the ``data`` directory within the test package. + + +.. data:: MAX_Py_ssize_t + + Set to :data:`sys.maxsize` for big memory tests. + + +.. data:: max_memuse + + Set by :func:`set_memlimit` as the memory limit for big memory tests. + Limited by :data:`MAX_Py_ssize_t`. + + +.. data:: real_max_memuse + + Set by :func:`set_memlimit` as the memory limit for big memory tests. Not + limited by :data:`MAX_Py_ssize_t`. + + +.. data:: MISSING_C_DOCSTRINGS + + Return ``True`` if running on CPython, not on Windows, and configuration + not set with ``WITH_DOC_STRINGS``. + + +.. data:: HAVE_DOCSTRINGS + + Check for presence of docstrings. + + + The :mod:`test.support` module defines the following functions: .. function:: forget(module_name) @@ -251,6 +366,38 @@ The :mod:`test.support` module defines the following functions: byte-compiled files of the module. +.. function:: unload(name) + + Delete *name* from ``sys.modules``. + + +.. function:: unlink(filename) + + Call :func:`os.unlink` on *filename*. On Windows platforms, this is + wrapped with a wait loop that checks for the existence fo the file. + + +.. function:: rmdir(filename) + + Call :func:`os.rmdir` on *filename*. On Windows platforms, this is + wrapped with a wait loop that checks for the existence of the file. + + +.. function:: rmtree(path) + + Call :func:`shutil.rmtree` on *path* or call :func:`os.lstat` and + :func:`os.rmdir` to remove a path and its contents. On Windows platforms, + this is wrapped with a wait loop that checks for the existence of the files. + + +.. function:: make_legacy_pyc(source) + + Move a PEP 3147/488 pyc file to its legacy pyc location and return the file + system path to the legacy pyc file. The *source* value is the file system + path to the source file. It does not need to exist, however the PEP + 3147/488 pyc file must exist. + + .. function:: is_resource_enabled(resource) Return ``True`` if *resource* is enabled and available. The list of @@ -258,6 +405,16 @@ The :mod:`test.support` module defines the following functions: tests. +.. function:: python_is_optimized() + + Return ``True`` if Python was not built with ``-O0`` or ``-Og``. + + +.. function:: with_pymalloc() + + Return :data:`_testcapi.WITH_PYMALLOC`. + + .. function:: requires(resource, msg=None) Raise :exc:`ResourceDenied` if *resource* is not available. *msg* is the @@ -266,14 +423,44 @@ The :mod:`test.support` module defines the following functions: Used when tests are executed by :mod:`test.regrtest`. +.. function:: system_must_validate_cert(f) + + Raise :exc:`unittest.SkipTest` on TLS certification validation failures. + + +.. function:: sortdict(dict) + + Return a repr of *dict* with keys sorted. + + .. function:: findfile(filename, subdir=None) Return the path to the file named *filename*. If no match is found *filename* is returned. This does not equal a failure since it could be the path to the file. - Setting *subdir* indicates a relative path to use to find the file - rather than looking directly in the path directories. + Setting *subdir* indicates a relative path to use to find the file + rather than looking directly in the path directories. + + +.. function:: create_empty_file(filename) + + Create an empty file with *filename*. If it already exists, truncate it. + + +.. function:: fd_count() + + Count the number of open file descriptors. + + +.. function:: match_test(test) + + Match *test* to patterns set in :func:`set_match_tests`. + + +.. function:: set_match_tests(patterns) + + Define match test with regular expression *patterns*. .. function:: run_unittest(\*classes) @@ -293,14 +480,32 @@ The :mod:`test.support` module defines the following functions: This will run all tests defined in the named module. -.. function:: run_doctest(module, verbosity=None) +.. function:: run_doctest(module, verbosity=None, optionflags=0) Run :func:`doctest.testmod` on the given *module*. Return ``(failure_count, test_count)``. If *verbosity* is ``None``, :func:`doctest.testmod` is run with verbosity set to :data:`verbose`. Otherwise, it is run with verbosity set to - ``None``. + ``None``. *optionflags* is passed as ``optionflags`` to + :func:`doctest.testmod`. + + +.. function:: setswitchinterval(interval) + + Set the :func:`sys.setswitchinterval` to the given *interval*. Defines + a minimum interval for Android systems to prevent the system from hanging. + + +.. function:: check_impl_detail(**guards) + + Use this check to guard CPython's implementation-specific tests or to + run them only on the implementations guarded by the arguments:: + + check_impl_detail() # Only on CPython (default). + check_impl_detail(jython=True) # Only on Jython. + check_impl_detail(cpython=False) # Everywhere except CPython. + .. function:: check_warnings(\*filters, quiet=True) @@ -368,6 +573,50 @@ The :mod:`test.support` module defines the following functions: New optional arguments *filters* and *quiet*. +.. function:: check_no_resource_warning(testcase) + + Context manager to check that no :exc:`ResourceWarning` was raised. You + must remove the object which may emit :exc:`ResourceWarning` before the + end of the context manager. + + +.. function:: set_memlimit(limit) + + Set the values for :data:`max_memuse` and :data:`real_max_memuse` for big + memory tests. + + +.. function:: record_original_stdout(stdout) + + Store the value from *stdout*. It is meant to hold the stdout at the + time the regrtest began. + + +.. function:: get_original_stdout + + Return the original stdout set by :func:`record_original_stdout` or + ``sys.stdout`` if it's not set. + + +.. function:: strip_python_strerr(stderr) + + Strip the *stderr* of a Python process from potential debug output + emitted by the interpreter. This will typically be run on the result of + :meth:`subprocess.Popen.communicate`. + + +.. function:: args_from_interpreter_flags() + + Return a list of command line arguments reproducing the current settings + in ``sys.flags`` and ``sys.warnoptions``. + + +.. function:: optim_args_from_interpreter_flags() + + Return a list of command line arguments reproducing the current + optimization settings in ``sys.flags``. + + .. function:: captured_stdin() captured_stdout() captured_stderr() @@ -434,17 +683,125 @@ The :mod:`test.support` module defines the following functions: A context manager that temporarily sets the process umask. +.. function:: transient_internet(resource_name, *, timeout=30.0, errnos=()) + + A context manager that raises :exc:`ResourceDenied` when various issues + with the internet connection manifest themselves as exceptions. + + +.. function:: disable_faulthandler() + + A context manager that replaces ``sys.stderr`` with ``sys.__stderr__``. + + +.. function:: gc_collect() + + Force as many objects as possible to be collected. This is needed because + timely deallocation is not guaranteed by the garbage collector. This means + that ``__del__`` methods may be called later than expected and weakrefs + may remain alive for longer than expected. + + +.. function:: disable_gc() + + A context manager that disables the garbage collector upon entry and + reenables it upon exit. + + +.. function:: swap_attr(obj, attr, new_val) + + Context manager to swap out an attribute with a new object. + + Usage:: + + with swap_attr(obj, "attr", 5): + ... + + This will set ``obj.attr`` to 5 for the duration of the ``with`` block, + restoring the old value at the end of the block. If ``attr`` doesn't + exist on ``obj``, it will be created and then deleted at the end of the + block. + + The old value (or ``None`` if it doesn't exist) will be assigned to the + target of the "as" clause, if there is one. + + +.. function:: swap_item(obj, attr, new_val) + + Context manager to swap out an item with a new object. + + Usage:: + + with swap_item(obj, "item", 5): + ... + + This will set ``obj["item"]`` to 5 for the duration of the ``with`` block, + restoring the old value at the end of the block. If ``item`` doesn't + exist on ``obj``, it will be created and then deleted at the end of the + block. + + The old value (or ``None`` if it doesn't exist) will be assigned to the + target of the "as" clause, if there is one. + + +.. function:: wait_threads_exit(timeout=60.0) + + Context manager to wait until all threads created in the ``with`` statment + exit. + + +.. function:: start_threads(threads, unlock=None) + + Context manager to start *threads*. It attempts to join the threads upon + exit. + + +.. function:: calcobjsize(fmt) + + Return :func:`struct.calcsize` for ``nP{fmt}0n`` or, if ``gettotalrefcount`` + exists, ``2PnP{fmt}0P``. + + +.. function:: calcvobjsize(fmt) + + Return :func:`struct.calcsize` for ``nPn{fmt}0n`` or, if ``gettotalrefcount`` + exists, ``2PnPn{fmt}0P``. + + +.. function:: checksizeof(test, o, size) + + For testcase *test*, assert that the ``sys.getsizeof`` for *o* plus the GC + header size equals *size*. + + .. function:: can_symlink() Return ``True`` if the OS supports symbolic links, ``False`` otherwise. +.. function:: can_xattr() + + Return ``True`` if the OS supports xattr, ``False`` + otherwise. + + .. decorator:: skip_unless_symlink A decorator for running tests that require support for symbolic links. +.. decorator:: skip_unless_xattr + + A decorator for running tests that require support for xattr. + + +.. decorator:: skip_unless_bind_unix_socket + + A decorator for running tests that require a functional bind() for Unix + sockets. + + .. decorator:: anticipate_failure(condition) A decorator to conditionally mark tests with @@ -460,20 +817,145 @@ The :mod:`test.support` module defines the following functions: sequentially, and the first valid locale will be used. +.. decorator:: run_with_tz(tz) + + A decorator for running a function in a specific timezone, correctly + resetting it after it has finished. + + +.. decorator:: requires_freebsd_version(*min_version) + + Decorator for the minimum version when running test on FreeBSD. If the + FreeBSD version is less than the minimum, raise :exc:`unittest.SkipTest`. + + +.. decorator:: requires_linux_version(*min_version) + + Decorator for the minimum version when running test on Linux. If the + Linux version is less than the minimum, raise :exc:`unittest.SkipTest`. + + +.. decorator:: requires_mac_version(*min_version) + + Decorator for the minimum version when running test on Mac OS X. If the + MAC OS X version is less than the minimum, raise :exc:`unittest.SkipTest`. + + +.. decorator:: requires_IEEE_754 + + Decorator for skipping tests on non-IEEE 754 platforms. + + +.. decorator:: requires_zlib + + Decorator for skipping tests if :mod:`zlib` doesn't exist. + + +.. decorator:: requires_gzip + + Decorator for skipping tests if :mod:`gzip` doesn't exist. + + +.. decorator:: requires_bz2 + + Decorator for skipping tests if :mod:`bz2` doesn't exist. + + +.. decorator:: requires_lzma + + Decorator for skipping tests if :mod:`lzma` doesn't exist. + + +.. decorator:: requires_resource(resource) + + Decorator for skipping tests if *resource* is not available. + + +.. decorator:: requires_docstrings + + Decorator for only running the test if :data:`HAVE_DOCSTRINGS`. + + +.. decorator:: cpython_only(test) + + Decorator for tests only applicable to CPython. + + +.. decorator:: impl_detail(msg=None, **guards) + + Decorator for invoking :func:`check_impl_detail` on *guards*. If that + returns ``False``, then uses *msg* as the reason for skipping the test. + + +.. decorator:: no_tracing(func) + + Decorator to temporarily turn off tracing for the duration of the test. + + +.. decorator:: refcount_test(test) + + Decorator for tests which involve reference counting. The decorator does + not run the test if it is not run by CPython. Any trace function is unset + for the duration of the test to prevent unexpected refcounts caused by + the trace function. + + +.. decorator:: reap_threads(func) + + Decorator to ensure the threads are cleaned up even if the test fails. + + +.. decorator:: bigmemtest(size, memuse, dry_run=True) + + Decorator for bigmem tests. + + *size* is a requested size for the test (in arbitrary, test-interpreted + units.) *memuse* is the number of bytes per unit for the test, or a good + estimate of it. For example, a test that needs two byte buffers, of 4 GiB + each, could be decorated with ``@bigmemtest(size=_4G, memuse=2)``. + + The *size* argument is normally passed to the decorated test method as an + extra argument. If *dry_run* is ``True``, the value passed to the test + method may be less than the requested value. If *dry_run* is ``False``, it + means the test doesn't support dummy runs when ``-M`` is not specified. + + +.. decorator:: bigaddrspacetest(f) + + Decorator for tests that fill the address space. *f* is the function to + wrap. + + .. function:: make_bad_fd() Create an invalid file descriptor by opening and closing a temporary file, and returning its descriptor. -.. function:: import_module(name, deprecated=False) +.. function:: check_syntax_error(testcase, statement, errtext='', *, lineno=None, offset=None) + + Test for syntax errors in *statement* by attempting to compile *statement*. + *testcase* is the :mod:`unittest` instance for the test. *errtext* is the + text of the error raised by :exc:`SyntaxError`. If *lineno* is not None, + compares to the line of the :exc:`SyntaxError`. If *offset* is not None, + compares to the offset of the :exc:`SyntaxError`. + + +.. function:: open_urlresource(url, *args, **kw) + + Open *url*. If open fails, raises :exc:`TestFailed`. + + +.. function:: import_module(name, deprecated=False, *, required_on()) This function imports and returns the named module. Unlike a normal import, this function raises :exc:`unittest.SkipTest` if the module cannot be imported. Module and package deprecation messages are suppressed during this import - if *deprecated* is ``True``. + if *deprecated* is ``True``. If a module is required on a platform but + optional for others, set *required_on* to an iterable of platform prefixes + which will be compared against :data:`sys.platform`. .. versionadded:: 3.1 @@ -514,6 +996,47 @@ The :mod:`test.support` module defines the following functions: .. versionadded:: 3.1 +.. function:: modules_setup() + + Return a copy of :data:`sys.modules`. + + +.. function:: modules_cleanup(oldmodules) + + Remove modules except for *oldmodules* and ``encodings`` in order to + preserve internal cache. + + +.. function:: threading_setup() + + Return current thread count and copy of dangling threads. + + +.. function:: threading_cleanup(*original_values) + + Cleanup up threads not specified in *original_values*. Designed to emit + a warning if a test leaves running threads in the background. + + +.. function:: join_thread(thread, timeout=30.0) + + Join a *thread* within *timeout*. Raise an :exc:`AssertionError` if thread + is still alive after *timeout* seconds. + + +.. function:: reap_children() + + Use this at the end of ``test_main`` whenever sub-processes are started. + This will help ensure that no extra children (zombies) stick around to + hog resources and create problems when looking for refleaks. + + +.. function:: get_attribute(obj, name) + + Get an attribute, raising :exc:`unittest.SkipTest` if :exc:`AttributeError` + is raised. + + .. function:: bind_port(sock, host=HOST) Bind the socket to a free port and return the port number. Relies on @@ -533,6 +1056,12 @@ The :mod:`test.support` module defines the following functions: test. +.. function:: bind_unix_socket(sock, addr) + + Bind a unix socket, raising :exc:`unittest.SkipTest` if + :exc:`PermissionError` is raised. + + .. function:: find_unused_port(family=socket.AF_INET, socktype=socket.SOCK_STREAM) Returns an unused port that should be suitable for binding. This is @@ -571,6 +1100,11 @@ The :mod:`test.support` module defines the following functions: return load_package_tests(os.path.dirname(__file__), *args) +.. function:: fs_is_case_insensitive(directory) + + Return ``True`` if the file system for *directory* is case-insensitive. + + .. function:: detect_api_mismatch(ref_api, other_api, *, ignore=()) Returns the set of attributes, functions or methods of *ref_api* not @@ -583,6 +1117,33 @@ The :mod:`test.support` module defines the following functions: .. versionadded:: 3.5 +.. function:: patch(test_instance, object_to_patch, attr_name, new_value) + + Override *object_to_patch.attr_name* with *new_value*. Also add + cleanup procedure to *test_instance* to restore *object_to_patch* for + *attr_name*. The *attr_name* should be a valid attribute for + *object_to_patch*. + + +.. function:: run_in_subinterp(code) + + Run *code* in subinterpreter. Raise :exc:`unittest.SkipTest` if + :mod:`tracemalloc` is enabled. + + +.. function:: check_free_after_iterating(test, iter, cls, args=()) + + Assert that *iter* is deallocated after iterating. + + +.. function:: missing_compiler_executable(cmd_names=[]) + + Check for the existence of the compiler executables whose names are listed + in *cmd_names* or all the compiler executables when *cmd_names* is empty + and return the first missing executable or ``None`` when none is found + missing. + + .. function:: check__all__(test_case, module, name_of_module=None, extra=(), blacklist=()) Assert that the ``__all__`` variable of *module* contains all public names. @@ -673,7 +1234,162 @@ The :mod:`test.support` module defines the following classes: On both platforms, the old value is restored by :meth:`__exit__`. +.. class:: CleanImport(*module_names) + + A context manager to force import to return a new module reference. This + is useful for testing module-level behaviors, such as the emission of a + DeprecationWarning on import. Example usage:: + + with CleanImport('foo'): + importlib.import_module('foo') # New reference. + + +.. class:: DirsOnSysPath(*paths) + + A context manager to temporarily add directories to sys.path. + + This makes a copy of :data:`sys.path`, appends any directories given + as positional arguments, then reverts :data:`sys.path` to the copied + settings when the context ends. + + Note that *all* :data:`sys.path` modifications in the body of the + context manager, including replacement of the object, + will be reverted at the end of the block. + + +.. class:: SaveSignals() + + Class to save and restore signal handlers registered by the Python signal + handler. + + +.. class:: Matcher() + + .. method:: matches(self, d, **kwargs) + + Try to match a single dict with the supplied arguments. + + + .. method:: match_value(self, k, dv, v) + + Try to match a single stored value (*dv*) with a supplied value (*v*). + + .. class:: WarningsRecorder() Class used to record warnings for unit tests. See documentation of :func:`check_warnings` above for more details. + + +.. class:: BasicTestRunner() + + .. method:: run(test) + + Run *test* and return the result. + + +.. class:: TestHandler(logging.handlers.BufferingHandler) + + Class for logging support. + + +:mod:`test.support.script_helper` --- Utilities for the Python execution tests +============================================================================== + +.. module:: test.support.script_helper + :synopsis: Support for Python's script execution tests. + + +The :mod:`test.support.script_helper` module provides support for Python's +script execution tests. + +.. function:: interpreter_requires_environment() + + Return ``True`` if ``sys.executable interpreter`` requires environment + variables in order to be able to run at all. + + This is designed to be used with ``@unittest.skipIf()`` to annotate tests + that need to use an ``assert_python*()`` function to launch an isolated + mode (``-I``) or no environment mode (``-E``) sub-interpreter process. + + A normal build & test does not run into this situation but it can happen + when trying to run the standard library test suite from an interpreter that + doesn't have an obvious home with Python's current home finding logic. + + Setting :envvar:`PYTHONHOME` is one way to get most of the testsuite to run + in that situation. :envvar:`PYTHONPATH` or :envvar:`PYTHONUSERSITE` are + other common environment variables that might impact whether or not the + interpreter can start. + + +.. function:: run_python_until_end(*args, **env_vars) + + Set up the environment based on *env_vars* for running the interpreter + in a subprocess. The values can include ``__isolated``, ``__cleavenv``, + and ``TERM``. + + +.. function:: assert_python_ok(*args, **env_vars) + + Assert that running the interpreter with *args* and optional environment + variables *env_vars* succeeds (``rc == 0``) and return a ``(return code, + stdout, stderr)`` tuple. + + If the ``__cleanenv`` keyword is set, *env_vars* is used as a fresh + environment. + + Python is started in isolated mode (command line option ``-I``), + except if the ``__isolated`` keyword is set to ``False``. + + +.. function:: assert_python_failure(*args, **env_vars) + + Assert that running the interpreter with *args* and optional environment + variables *env_vars* fails (``rc != 0``) and return a ``(return code, + stdout, stderr)`` tuple. + + See :func:`assert_python_ok` for more options. + + +.. function:: spawn_python(*args, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, **kw) + + Run a Python subprocess with the given arguments. + + *kw* is extra keyword args to pass to :func:`subprocess.Popen`. Returns a + :class:`subprocess.Popen` object. + + +.. function:: kill_python(p) + + Run the given :class:`subprocess.Popen` process until completion and return + stdout. + + +.. function:: make_script(script_dir, script_basename, source, omit_suffix=False) + + Create script containing *source* in path *script_dir* and *script_basename*. + If *omit_suffix* is ``False``, append ``.py`` to the name. Return the full + script path. + + +.. function:: make_zip_script(zip_dir, zip_basename, script_name, name_in_zip=None) + + Create zip file at *zip_dir* and *zip_basename* with extension ``zip`` which + contains the files in *script_name*. *name_in_zip* is the archive name. + Return a tuple containing ``(full path, full path of archive name)``. + + +.. function:: make_pkg(pkg_dir, init_source='') + + Create a directory named *pkg_dir* containing an ``__init__`` file with + *init_source* as its contents. + + +.. function:: make_zip_pkg(zip_dir, zip_basename, pkg_name, script_basename, \ + source, depth=1, compiled=False) + + Create a zip package directory with a path of *zip_dir* and *zip_basename* + containing an empty ``__init__`` file and a file *script_basename* + containing the *source*. If *compiled* is ``True``, both source files will + be compiled and added to the zip package. Return a tuple of the full zip + path and the archive name for the zip file. diff --git a/Misc/NEWS.d/next/Documentation/2018-02-10-12-48-38.bpo-11015.-gUf34.rst b/Misc/NEWS.d/next/Documentation/2018-02-10-12-48-38.bpo-11015.-gUf34.rst new file mode 100644 index 00000000000..73612dab69e --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2018-02-10-12-48-38.bpo-11015.-gUf34.rst @@ -0,0 +1 @@ +Update :mod:`test.support` documentation.