1393 lines
56 KiB
ReStructuredText
1393 lines
56 KiB
ReStructuredText
****************************
|
|
What's New In Python 3.7
|
|
****************************
|
|
|
|
.. Rules for maintenance:
|
|
|
|
* Anyone can add text to this document. Do not spend very much time
|
|
on the wording of your changes, because your text will probably
|
|
get rewritten to some degree.
|
|
|
|
* The maintainer will go through Misc/NEWS periodically and add
|
|
changes; it's therefore more important to add your changes to
|
|
Misc/NEWS than to this file.
|
|
|
|
* This is not a complete list of every single change; completeness
|
|
is the purpose of Misc/NEWS. Some changes I consider too small
|
|
or esoteric to include. If such a change is added to the text,
|
|
I'll just remove it. (This is another reason you shouldn't spend
|
|
too much time on writing your addition.)
|
|
|
|
* If you want to draw your new text to the attention of the
|
|
maintainer, add 'XXX' to the beginning of the paragraph or
|
|
section.
|
|
|
|
* It's OK to just add a fragmentary note about a change. For
|
|
example: "XXX Describe the transmogrify() function added to the
|
|
socket module." The maintainer will research the change and
|
|
write the necessary text.
|
|
|
|
* You can comment out your additions if you like, but it's not
|
|
necessary (especially when a final release is some months away).
|
|
|
|
* Credit the author of a patch or bugfix. Just the name is
|
|
sufficient; the e-mail address isn't necessary.
|
|
|
|
* It's helpful to add the bug/patch number as a comment:
|
|
|
|
XXX Describe the transmogrify() function added to the socket
|
|
module.
|
|
(Contributed by P.Y. Developer in :issue:`12345`.)
|
|
|
|
This saves the maintainer the effort of going through the Mercurial log
|
|
when researching a change.
|
|
|
|
This article explains the new features in Python 3.7, compared to 3.6.
|
|
|
|
For full details, see the :ref:`changelog <changelog>`.
|
|
|
|
.. note::
|
|
|
|
Prerelease users should be aware that this document is currently in draft
|
|
form. It will be updated substantially as Python 3.7 moves towards release,
|
|
so it's worth checking back even after reading earlier versions.
|
|
|
|
|
|
Summary -- Release highlights
|
|
=============================
|
|
|
|
.. This section singles out the most important changes in Python 3.7.
|
|
Brevity is key.
|
|
|
|
|
|
.. PEP-sized items next.
|
|
|
|
|
|
|
|
New Features
|
|
============
|
|
|
|
|
|
.. _whatsnew37-pep538:
|
|
|
|
PEP 538: Legacy C Locale Coercion
|
|
---------------------------------
|
|
|
|
An ongoing challenge within the Python 3 series has been determining a sensible
|
|
default strategy for handling the "7-bit ASCII" text encoding assumption
|
|
currently implied by the use of the default C locale on non-Windows platforms.
|
|
|
|
:pep:`538` updates the default interpreter command line interface to
|
|
automatically coerce that locale to an available UTF-8 based locale as
|
|
described in the documentation of the new :envvar:`PYTHONCOERCECLOCALE`
|
|
environment variable. Automatically setting ``LC_CTYPE`` this way means that
|
|
both the core interpreter and locale-aware C extensions (such as
|
|
:mod:`readline`) will assume the use of UTF-8 as the default text encoding,
|
|
rather than ASCII.
|
|
|
|
The platform support definition in :pep:`11` has also been updated to limit
|
|
full text handling support to suitably configured non-ASCII based locales.
|
|
|
|
As part of this change, the default error handler for :data:`~sys.stdin` and
|
|
:data:`~sys.stdout` is now ``surrogateescape`` (rather than ``strict``) when
|
|
using any of the defined coercion target locales (currently ``C.UTF-8``,
|
|
``C.utf8``, and ``UTF-8``). The default error handler for :data:`~sys.stderr`
|
|
continues to be ``backslashreplace``, regardless of locale.
|
|
|
|
Locale coercion is silent by default, but to assist in debugging potentially
|
|
locale related integration problems, explicit warnings (emitted directly on
|
|
:data:`~sys.stderr` can be requested by setting ``PYTHONCOERCECLOCALE=warn``.
|
|
This setting will also cause the Python runtime to emit a warning if the
|
|
legacy C locale remains active when the core interpreter is initialized.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`538` -- Coercing the legacy C locale to a UTF-8 based locale
|
|
PEP written and implemented by Nick Coghlan.
|
|
|
|
|
|
.. _whatsnew37-pep553:
|
|
|
|
PEP 553: Built-in breakpoint()
|
|
------------------------------
|
|
|
|
:pep:`553` describes a new built-in called :func:`breakpoint` which makes it
|
|
easy and consistent to enter the Python debugger. Built-in ``breakpoint()``
|
|
calls :func:`sys.breakpointhook`. By default, this latter imports :mod:`pdb` and
|
|
then calls ``pdb.set_trace()``, but by binding ``sys.breakpointhook()`` to the
|
|
function of your choosing, ``breakpoint()`` can enter any debugger. Or, the
|
|
environment variable :envvar:`PYTHONBREAKPOINT` can be set to the callable of
|
|
your debugger of choice. Set ``PYTHONBREAKPOINT=0`` to completely disable
|
|
built-in ``breakpoint()``.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`553` -- Built-in breakpoint()
|
|
PEP written and implemented by Barry Warsaw
|
|
|
|
|
|
.. _whatsnew37-pep539:
|
|
|
|
PEP 539: A New C-API for Thread-Local Storage in CPython
|
|
--------------------------------------------------------
|
|
|
|
While Python provides a C API for thread-local storage support; the existing
|
|
:ref:`Thread Local Storage (TLS) API <thread-local-storage-api>` has used
|
|
:c:type:`int` to represent TLS keys across all platforms. This has not
|
|
generally been a problem for officially-support platforms, but that is neither
|
|
POSIX-compliant, nor portable in any practical sense.
|
|
|
|
:pep:`539` changes this by providing a new :ref:`Thread Specific Storage (TSS)
|
|
API <thread-specific-storage-api>` to CPython which supersedes use of the
|
|
existing TLS API within the CPython interpreter, while deprecating the existing
|
|
API. The TSS API uses a new type :c:type:`Py_tss_t` instead of :c:type:`int`
|
|
to represent TSS keys--an opaque type the definition of which may depend on
|
|
the underlying TLS implementation. Therefore, this will allow to build CPython
|
|
on platforms where the native TLS key is defined in a way that cannot be safely
|
|
cast to :c:type:`int`.
|
|
|
|
Note that on platforms where the native TLS key is defined in a way that cannot
|
|
be safely cast to :c:type:`int`, all functions of the existing TLS API will be
|
|
no-op and immediately return failure. This indicates clearly that the old API
|
|
is not supported on platforms where it cannot be used reliably, and that no
|
|
effort will be made to add such support.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`539` -- A New C-API for Thread-Local Storage in CPython
|
|
PEP written by Erik M. Bray; implementation by Masayuki Yamamoto.
|
|
|
|
|
|
PEP 562: Customization of access to module attributes
|
|
-----------------------------------------------------
|
|
|
|
It is sometimes convenient to customize or otherwise have control over access
|
|
to module attributes. A typical example is managing deprecation warnings.
|
|
Typical workarounds are assigning :attr:`~instance.__class__` of a module
|
|
object to a custom subclass of :class:`types.ModuleType` or replacing the
|
|
:data:`sys.modules` item with a custom wrapper instance. This procedure is
|
|
now simplified by recognizing ``__getattr__`` defined directly in a module
|
|
that would act like a normal :meth:`__getattr__` method, except that it will
|
|
be defined on module *instances*.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`562` -- Module ``__getattr__`` and ``__dir__``
|
|
PEP written and implemented by Ivan Levkivskyi
|
|
|
|
|
|
PEP 563: Postponed evaluation of annotations
|
|
--------------------------------------------
|
|
|
|
The advent of type hints in Python uncovered two glaring usability issues
|
|
with the functionality of annotations added in :pep:`3107` and refined
|
|
further in :pep:`526`:
|
|
|
|
* annotations could only use names which were already available in the
|
|
current scope, in other words they didn't support forward references
|
|
of any kind; and
|
|
|
|
* annotating source code had adverse effects on startup time of Python
|
|
programs.
|
|
|
|
Both of these issues are fixed by postponing the evaluation of
|
|
annotations. Instead of compiling code which executes expressions in
|
|
annotations at their definition time, the compiler stores the annotation
|
|
in a string form equivalent to the AST of the expression in question.
|
|
If needed, annotations can be resolved at runtime using
|
|
``typing.get_type_hints()``. In the common case where this is not
|
|
required, the annotations are cheaper to store (since short strings
|
|
are interned by the interpreter) and make startup time faster.
|
|
|
|
Usability-wise, annotations now support forward references, making the
|
|
following syntax valid::
|
|
|
|
class C:
|
|
@classmethod
|
|
def from_string(cls, source: str) -> C:
|
|
...
|
|
|
|
def validate_b(self, obj: B) -> bool:
|
|
...
|
|
|
|
class B:
|
|
...
|
|
|
|
Since this change breaks compatibility, the new behavior can be enabled
|
|
on a per-module basis in Python 3.7 using a :mod:`__future__` import, like
|
|
this::
|
|
|
|
from __future__ import annotations
|
|
|
|
It will become the default in Python 4.0.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`563` -- Postponed evaluation of annotations
|
|
PEP written and implemented by Łukasz Langa.
|
|
|
|
|
|
PEP 564: Add new time functions with nanosecond resolution
|
|
----------------------------------------------------------
|
|
|
|
Add six new "nanosecond" variants of existing functions to the :mod:`time`
|
|
module:
|
|
|
|
* :func:`time.clock_gettime_ns`
|
|
* :func:`time.clock_settime_ns`
|
|
* :func:`time.monotonic_ns`
|
|
* :func:`time.perf_counter_ns`
|
|
* :func:`time.process_time_ns`
|
|
* :func:`time.time_ns`
|
|
|
|
While similar to the existing functions without the ``_ns`` suffix, they
|
|
provide nanosecond resolution: they return a number of nanoseconds as a Python
|
|
``int``.
|
|
|
|
The ``time.time_ns()`` resolution is 3 times better than the ``time.time()``
|
|
resolution on Linux and Windows.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`564` -- Add new time functions with nanosecond resolution
|
|
PEP written and implemented by Victor Stinner
|
|
|
|
|
|
.. _whatsnew37-pep565:
|
|
|
|
PEP 565: Show DeprecationWarning in ``__main__``
|
|
------------------------------------------------
|
|
|
|
The default handling of :exc:`DeprecationWarning` has been changed such that
|
|
these warnings are once more shown by default, but only when the code
|
|
triggering them is running directly in the :mod:`__main__` module. As a result,
|
|
developers of single file scripts and those using Python interactively should
|
|
once again start seeing deprecation warnings for the APIs they use, but
|
|
deprecation warnings triggered by imported application, library and framework
|
|
modules will continue to be hidden by default.
|
|
|
|
As a result of this change, the standard library now allows developers to choose
|
|
between three different deprecation warning behaviours:
|
|
|
|
* :exc:`FutureWarning`: always displayed by default, recommended for warnings
|
|
intended to be seen by application end users (e.g. for deprecated application
|
|
configuration settings).
|
|
* :exc:`DeprecationWarning`: displayed by default only in :mod:`__main__` and when
|
|
running tests, recommended for warnings intended to be seen by other Python
|
|
developers where a version upgrade may result in changed behaviour or an
|
|
error.
|
|
* :exc:`PendingDeprecationWarning`: displayed by default only when running
|
|
tests, intended for cases where a future version upgrade will change the
|
|
warning category to :exc:`DeprecationWarning` or :exc:`FutureWarning`.
|
|
|
|
Previously both :exc:`DeprecationWarning` and :exc:`PendingDeprecationWarning`
|
|
were only visible when running tests, which meant that developers primarily
|
|
writing single file scripts or using Python interactively could be surprised
|
|
by breaking changes in the APIs they used.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`565` -- Show DeprecationWarning in ``__main__``
|
|
PEP written and implemented by Nick Coghlan
|
|
|
|
|
|
PEP 540: Add a new UTF-8 mode
|
|
-----------------------------
|
|
|
|
Add a new UTF-8 mode to ignore the locale, use the UTF-8 encoding, and change
|
|
:data:`sys.stdin` and :data:`sys.stdout` error handlers to ``surrogateescape``.
|
|
This mode is enabled by default in the POSIX locale, but otherwise disabled by
|
|
default.
|
|
|
|
The new :option:`-X` ``utf8`` command line option and :envvar:`PYTHONUTF8`
|
|
environment variable are added to control the UTF-8 mode.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`540` -- Add a new UTF-8 mode
|
|
PEP written and implemented by Victor Stinner
|
|
|
|
|
|
.. _whatsnew37-pep557:
|
|
|
|
PEP 557: Data Classes
|
|
---------------------
|
|
|
|
Adds a new module :mod:`dataclasses`. It provides a class decorator
|
|
:func:`~dataclasses.dataclass` which inspects the class's variable annotations (see
|
|
:pep:`526`) and using them, adds methods such as ``__init__``,
|
|
``__repr__``, and ``__eq__`` to the class. It is similar to
|
|
:class:`typing.NamedTuple`, but also works on classes with mutable
|
|
instances, among other features.
|
|
|
|
For example::
|
|
|
|
@dataclass
|
|
class Point:
|
|
x: float
|
|
y: float
|
|
z: float = 0.0
|
|
|
|
p = Point(1.5, 2.5)
|
|
print(p) # produces "Point(x=1.5, y=2.5, z=0.0)"
|
|
|
|
.. seealso::
|
|
|
|
:pep:`557` -- Data Classes
|
|
PEP written and implemented by Eric V. Smith
|
|
|
|
|
|
PEP 567: Context Variables
|
|
--------------------------
|
|
|
|
Adds a new module :mod:`contextvars`, that provides APIs to manage,
|
|
store, and access non-local state.
|
|
|
|
Context variables are natively supported in :mod:`asyncio` and are
|
|
ready to be used without any extra configuration.
|
|
|
|
The :mod:`decimal` module was updated to use *contextvars* to store
|
|
the current decimal context. This allows decimal operations to work
|
|
with the correct context in async/await code.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`567` -- Context Variables
|
|
PEP written and implemented by Yury Selivanov
|
|
|
|
|
|
PEP 560: Core support for typing module and generic types
|
|
---------------------------------------------------------
|
|
|
|
Initially :pep:`484` was designed in such way that it would not introduce *any*
|
|
changes to the core CPython interpreter. Now type hints and the :mod:`typing`
|
|
module are extensively used by the community, so this restriction is removed.
|
|
The PEP introduces two special methods :meth:`__class_getitem__` and
|
|
``__mro_entries__``, these methods are now used by most classes and special
|
|
constructs in :mod:`typing`. As a result, the speed of various operations
|
|
with types increased up to 7 times, the generic types can be used without
|
|
metaclass conflicts, and several long standing bugs in :mod:`typing` module are
|
|
fixed.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`560` -- Core support for typing module and generic types
|
|
PEP written and implemented by Ivan Levkivskyi
|
|
|
|
|
|
New Development Mode: -X dev
|
|
----------------------------
|
|
|
|
Add a new "development mode": :option:`-X` ``dev`` command line option and
|
|
:envvar:`PYTHONDEVMODE` environment variable to enable CPython's "development
|
|
mode", introducing additional runtime checks which are too expensive to be
|
|
enabled by default. See :option:`-X` ``dev`` documentation for the effects of
|
|
the development mode.
|
|
|
|
Hash-based pycs
|
|
---------------
|
|
|
|
Python has traditionally checked the up-to-dateness of bytecode cache files
|
|
(i.e., ``.pyc`` files) by comparing the source metadata (last-modified timestamp
|
|
and size) with source metadata saved in the cache file header when it was
|
|
generated. While effective, this invalidation method has its drawbacks. When
|
|
filesystem timestamps are too coarse, Python can miss source updates, leading to
|
|
user confusion. Additionally, having a timestamp in the cache file is
|
|
problematic for `build reproduciblity <https://reproducible-builds.org/>`_ and
|
|
content-based build systems.
|
|
|
|
:pep:`552` extends the pyc format to allow the hash of the source file to be
|
|
used for invalidation instead of the source timestamp. Such ``.pyc`` files are
|
|
called "hash-based". By default, Python still uses timestamp-based invalidation
|
|
and does not generate hash-based ``.pyc`` files at runtime. Hash-based ``.pyc``
|
|
files may be generated with :mod:`py_compile` or :mod:`compileall`.
|
|
|
|
Hash-based ``.pyc`` files come in two variants: checked and unchecked. Python
|
|
validates checked hash-based ``.pyc`` files against the corresponding source
|
|
files at runtime but doesn't do so for unchecked hash-based pycs. Unchecked
|
|
hash-based ``.pyc`` files are a useful performance optimization for environments
|
|
where a system external to Python (e.g., the build system) is responsible for
|
|
keeping ``.pyc`` files up-to-date.
|
|
|
|
See :ref:`pyc-invalidation` for more information.
|
|
|
|
|
|
Other Language Changes
|
|
======================
|
|
|
|
* More than 255 arguments can now be passed to a function, and a function can
|
|
now have more than 255 parameters. (Contributed by Serhiy Storchaka in
|
|
:issue:`12844` and :issue:`18896`.)
|
|
|
|
* :meth:`bytes.fromhex` and :meth:`bytearray.fromhex` now ignore all ASCII
|
|
whitespace, not only spaces. (Contributed by Robert Xiao in :issue:`28927`.)
|
|
|
|
* :exc:`ImportError` now displays module name and module ``__file__`` path when
|
|
``from ... import ...`` fails. (Contributed by Matthias Bussonnier in
|
|
:issue:`29546`.)
|
|
|
|
* Circular imports involving absolute imports with binding a submodule to
|
|
a name are now supported.
|
|
(Contributed by Serhiy Storchaka in :issue:`30024`.)
|
|
|
|
* ``object.__format__(x, '')`` is now equivalent to ``str(x)`` rather than
|
|
``format(str(self), '')``.
|
|
(Contributed by Serhiy Storchaka in :issue:`28974`.)
|
|
|
|
* In order to better support dynamic creation of stack traces,
|
|
:class:`types.TracebackType` can now be instantiated from Python code, and
|
|
the ``tb_next`` attribute on :ref:`tracebacks <traceback-objects>` is now
|
|
writable.
|
|
(Contributed by Nathaniel J. Smith in :issue:`30579`.)
|
|
|
|
* When using the :option:`-m` switch, ``sys.path[0]`` is now eagerly expanded
|
|
to the full starting directory path, rather than being left as the empty
|
|
directory (which allows imports from the *current* working directory at the
|
|
time when an import occurs)
|
|
(Contributed by Nick Coghlan in :issue:`33053`.)
|
|
|
|
|
|
New Modules
|
|
===========
|
|
|
|
importlib.resources
|
|
-------------------
|
|
|
|
This module provides several new APIs and one new ABC for access to, opening,
|
|
and reading *resources* inside packages. Resources are roughly akin to files
|
|
inside of packages, but they needn't be actual files on the physical file
|
|
system. Module loaders can provide a :meth:`get_resource_reader()` function
|
|
which returns a :class:`importlib.abc.ResourceReader` instance to support this
|
|
new API. Built-in file path loaders and zip file loaders both support this.
|
|
(see the PyPI package
|
|
`importlib_resources <http://importlib-resources.readthedocs.io/en/latest/>`_
|
|
as a compatible back port for older Python versions).
|
|
|
|
|
|
Improved Modules
|
|
================
|
|
|
|
|
|
argparse
|
|
--------
|
|
|
|
The :meth:`~argparse.ArgumentParser.parse_intermixed_args` supports letting
|
|
the user intermix options and positional arguments on the command line,
|
|
as is possible in many unix commands. It supports most but not all
|
|
argparse features. (Contributed by paul.j3 in :issue:`14191`.)
|
|
|
|
binascii
|
|
--------
|
|
|
|
The :func:`~binascii.b2a_uu` function now accepts an optional *backtick*
|
|
keyword argument. When it's true, zeros are represented by ``'`'``
|
|
instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.)
|
|
|
|
calendar
|
|
--------
|
|
|
|
The class :class:`~calendar.HTMLCalendar` has new class attributes which ease
|
|
the customisation of the CSS classes in the produced HTML calendar.
|
|
(Contributed by Oz Tiram in :issue:`30095`.)
|
|
|
|
contextlib
|
|
----------
|
|
|
|
:func:`~contextlib.asynccontextmanager` and
|
|
:class:`~contextlib.AbstractAsyncContextManager` have been added. (Contributed
|
|
by Jelle Zijlstra in :issue:`29679` and :issue:`30241`.)
|
|
|
|
:class:`contextlib.AsyncExitStack` has been added. (Contributed by
|
|
Alexander Mohr and Ilya Kulakov in :issue:`29302`.)
|
|
|
|
cProfile
|
|
--------
|
|
|
|
:mod:`cProfile` command line now accepts ``-m module_name`` as an alternative
|
|
to script path. (Contributed by Sanyam Khurana in :issue:`21862`.)
|
|
|
|
crypt
|
|
-----
|
|
|
|
Added support for the Blowfish method.
|
|
(Contributed by Serhiy Storchaka in :issue:`31664`.)
|
|
|
|
The :func:`~crypt.mksalt` function now allows to specify the number of rounds
|
|
for hashing. (Contributed by Serhiy Storchaka in :issue:`31702`.)
|
|
|
|
datetime
|
|
--------
|
|
|
|
Added the :meth:`datetime.fromisoformat <datetime.datetime.fromisoformat>`
|
|
method, which constructs a :class:`~datetime.datetime` object from a string
|
|
in one of the formats output by
|
|
:meth:`datetime.isoformat <datetime.datetime.isoformat>`.
|
|
(Contributed by Paul Ganssle in :issue:`15873`.)
|
|
|
|
dis
|
|
---
|
|
|
|
The :func:`~dis.dis` function now is able to
|
|
disassemble nested code objects (the code of comprehensions, generator
|
|
expressions and nested functions, and the code used for building nested
|
|
classes). (Contributed by Serhiy Storchaka in :issue:`11822`.)
|
|
|
|
distutils
|
|
---------
|
|
|
|
``README.rst`` is now included in the list of distutils standard READMEs and
|
|
therefore included in source distributions.
|
|
(Contributed by Ryan Gonzalez in :issue:`11913`.)
|
|
|
|
:class:`distutils.core.setup` now warns if the ``classifiers``, ``keywords``
|
|
and ``platforms`` fields are not specified as a list or a string.
|
|
(Contributed by Berker Peksag in :issue:`19610`.)
|
|
|
|
The ``upload`` command no longer tries to change CR end-of-line characters
|
|
to CRLF. This fixes a corruption issue with sdists that ended with a byte
|
|
equivalent to CR.
|
|
(Contributed by Bo Bayles in :issue:`32304`.)
|
|
|
|
http.client
|
|
-----------
|
|
|
|
Add configurable *blocksize* to :class:`~http.client.HTTPConnection` and
|
|
:class:`~http.client.HTTPSConnection` for improved upload throughput.
|
|
(Contributed by Nir Soffer in :issue:`31945`.)
|
|
|
|
http.server
|
|
-----------
|
|
|
|
:class:`~http.server.SimpleHTTPRequestHandler` supports the HTTP
|
|
``If-Modified-Since`` header. The server returns the 304 response status if the
|
|
target file was not modified after the time specified in the header.
|
|
(Contributed by Pierre Quentel in :issue:`29654`.)
|
|
|
|
Add the parameter *directory* to the :class:`~http.server.SimpleHTTPRequestHandler`
|
|
and the ``--directory`` to the command line of the module :mod:`http.server`.
|
|
With this parameter, the server serves the specified directory, by default it
|
|
uses the current working directory.
|
|
(Contributed by Stéphane Wirtel and Julien Palard in :issue:`28707`.)
|
|
|
|
hmac
|
|
----
|
|
|
|
The :mod:`hmac` module now has an optimized one-shot :func:`~hmac.digest`
|
|
function, which is up to three times faster than :func:`~hmac.HMAC`.
|
|
(Contributed by Christian Heimes in :issue:`32433`.)
|
|
|
|
importlib
|
|
---------
|
|
|
|
The :class:`importlib.abc.ResourceReader` ABC was introduced to
|
|
support the loading of resource from packages.
|
|
|
|
locale
|
|
------
|
|
|
|
Added another argument *monetary* in :func:`~locale.format_string` of :mod:`locale`.
|
|
If *monetary* is true, the conversion uses monetary thousands separator and
|
|
grouping strings. (Contributed by Garvit in :issue:`10379`.)
|
|
|
|
The :func:`~locale.getpreferredencoding` function now always returns ``'UTF-8'``
|
|
on Android or in the UTF-8 mode (:option:`-X` ``utf8`` option), the locale and
|
|
the *do_setlocale* argument are ignored.
|
|
|
|
math
|
|
----
|
|
|
|
New :func:`~math.remainder` function, implementing the IEEE 754-style remainder
|
|
operation. (Contributed by Mark Dickinson in :issue:`29962`.)
|
|
|
|
os
|
|
--
|
|
|
|
Added support for :class:`bytes` paths in :func:`~os.fwalk`. (Contributed by
|
|
Serhiy Storchaka in :issue:`28682`.)
|
|
|
|
Added support for :ref:`file descriptors <path_fd>` in :func:`~os.scandir`
|
|
on Unix. (Contributed by Serhiy Storchaka in :issue:`25996`.)
|
|
|
|
New function :func:`~os.register_at_fork` allows registering Python callbacks
|
|
to be executed on a process fork. (Contributed by Antoine Pitrou in
|
|
:issue:`16500`.)
|
|
|
|
Exposed the system calls *preadv*, *preadv2*, *pwritev* and *pwritev2* through
|
|
the new functions :func:`~os.preadv` and :func:`~os.pwritev`. (Contributed by
|
|
Pablo Galindo in :issue:`31368`.)
|
|
|
|
Exposed the system call *posix_spawn* through the new function
|
|
:func:`~os.posix_spawn`. (Contributed by Pablo Galindo, Serhiy Storchaka and
|
|
Gregory P. Smith in :issue:`20104`.)
|
|
|
|
pdb
|
|
---
|
|
|
|
:func:`~pdb.set_trace` now takes an optional *header* keyword-only
|
|
argument. If given, this is printed to the console just before debugging
|
|
begins. (Contributed by Barry Warsaw in :issue:`31389`.)
|
|
|
|
:mod:`pdb` command line now accepts ``-m module_name`` as an alternative to
|
|
script file. (Contributed by Mario Corchero in :issue:`32206`.)
|
|
|
|
py_compile
|
|
----------
|
|
|
|
:func:`py_compile.compile` -- and by extension, :mod:`compileall` -- now
|
|
respects the :envvar:`SOURCE_DATE_EPOCH` environment variable by
|
|
unconditionally creating ``.pyc`` files for hash-based validation.
|
|
This allows for guaranteeing
|
|
`reproducible builds <https://reproducible-builds.org/>`_ of ``.pyc``
|
|
files when they are created eagerly. (Contributed by Bernhard M. Wiedemann
|
|
in :issue:`29708`.)
|
|
|
|
re
|
|
--
|
|
|
|
The flags :const:`re.ASCII`, :const:`re.LOCALE` and :const:`re.UNICODE`
|
|
can be set within the scope of a group.
|
|
(Contributed by Serhiy Storchaka in :issue:`31690`.)
|
|
|
|
:func:`re.split` now supports splitting on a pattern like ``r'\b'``,
|
|
``'^$'`` or ``(?=-)`` that matches an empty string.
|
|
(Contributed by Serhiy Storchaka in :issue:`25054`.)
|
|
|
|
|
|
sqlite3
|
|
-------
|
|
|
|
:class:`sqlite3.Connection` now exposes a :class:`~sqlite3.Connection.backup`
|
|
method, if the underlying SQLite library is at version 3.6.11 or higher.
|
|
(Contributed by Lele Gaifax in :issue:`27645`.)
|
|
|
|
|
|
ssl
|
|
---
|
|
|
|
The :mod:`ssl` module now uses OpenSSL's builtin API instead of
|
|
:func:`~ssl.match_hostname` to check host name or IP address. Values
|
|
are validated during TLS handshake. Any cert validation error including
|
|
a failing host name match now raises :exc:`~ssl.SSLCertVerificationError` and
|
|
aborts the handshake with a proper TLS Alert message. The new exception
|
|
contains additional information. Host name validation can be customized
|
|
with :attr:`~ssl.SSLContext.host_flags`.
|
|
(Contributed by Christian Heimes in :issue:`31399`.)
|
|
|
|
.. note::
|
|
The improved host name check requires an OpenSSL 1.0.2 or 1.1 compatible
|
|
libssl. OpenSSL 0.9.8 and 1.0.1 are no longer supported. LibreSSL is
|
|
temporarily not supported until it gains the necessary OpenSSL 1.0.2 APIs.
|
|
|
|
The ssl module no longer sends IP addresses in SNI TLS extension.
|
|
(Contributed by Christian Heimes in :issue:`32185`.)
|
|
|
|
:func:`~ssl.match_hostname` no longer supports partial wildcards like
|
|
``www*.example.org``. :attr:`~ssl.SSLContext.host_flags` has partial
|
|
wildcard matching disabled by default.
|
|
(Contributed by Mandeep Singh in :issue:`23033` and Christian Heimes in
|
|
:issue:`31399`.)
|
|
|
|
The default cipher suite selection of the ssl module now uses a blacklist
|
|
approach rather than a hard-coded whitelist. Python no longer re-enables
|
|
ciphers that have been blocked by OpenSSL security update. Default cipher
|
|
suite selection can be configured on compile time.
|
|
(Contributed by Christian Heimes in :issue:`31429`.)
|
|
|
|
Added support for validating server certificates containing
|
|
internationalized domain names (IDNs). As part of this change, the
|
|
:attr:`ssl.SSLSocket.server_hostname` attribute now stores the
|
|
expected hostname in A-label form (``"xn--pythn-mua.org"``), rather
|
|
than the U-label form (``"pythön.org"``). (Contributed by
|
|
Nathaniel J. Smith and Christian Heimes in :issue:`28414`.)
|
|
|
|
The ssl module has preliminary and experimental support for TLS 1.3 and
|
|
OpenSSL 1.1.1. (Contributed by Christian Heimes in :issue:`32947`,
|
|
:issue:`20995`, :issue:`29136`, and :issue:`30622`)
|
|
|
|
:class:`~ssl.SSLSocket` and :class:`~ssl.SSLObject` no longer have a public
|
|
constructor. Direct instantiation was never a documented and supported
|
|
feature. Instances must be created with :class:`~ssl.SSLContext` methods
|
|
:meth:`~ssl.SSLContext.wrap_socket` and :meth:`~ssl.SSLContext.wrap_bio`.
|
|
(Contributed by Christian Heimes in :issue:`32951`)
|
|
|
|
OpenSSL 1.1 APIs for setting the minimum and maximum TLS protocol version are
|
|
available as :attr:`~ssl.SSLContext.minimum_version` and
|
|
:attr:`~ssl.SSLContext.maximum_version`. Supported protocols are indicated
|
|
by new flags like :data:`~ssl.HAS_TLSv1_1`.
|
|
(Contributed by Christian Heimes in :issue:`32609`.)
|
|
|
|
string
|
|
------
|
|
|
|
:class:`string.Template` now lets you to optionally modify the regular
|
|
expression pattern for braced placeholders and non-braced placeholders
|
|
separately. (Contributed by Barry Warsaw in :issue:`1198569`.)
|
|
|
|
subprocess
|
|
----------
|
|
|
|
On Windows the default for *close_fds* was changed from ``False`` to
|
|
``True`` when redirecting the standard handles. It's now possible to set
|
|
*close_fds* to ``True`` when redirecting the standard handles. See
|
|
:class:`subprocess.Popen`.
|
|
|
|
This means that *close_fds* now defaults to ``True`` on all supported
|
|
platforms. (Contributed by Segev Finer in :issue:`19764`.)
|
|
|
|
sys
|
|
---
|
|
|
|
Added :attr:`sys.flags.dev_mode` flag for the new development mode.
|
|
|
|
tkinter
|
|
-------
|
|
|
|
Added :class:`tkinter.ttk.Spinbox`.
|
|
(Contributed by Alan Moore in :issue:`32585`.)
|
|
|
|
time
|
|
----
|
|
|
|
The :pep:`564` added six new functions with nanosecond resolution:
|
|
|
|
* :func:`time.clock_gettime_ns`
|
|
* :func:`time.clock_settime_ns`
|
|
* :func:`time.monotonic_ns`
|
|
* :func:`time.perf_counter_ns`
|
|
* :func:`time.process_time_ns`
|
|
* :func:`time.time_ns`
|
|
|
|
Add new clock identifiers:
|
|
|
|
* :data:`time.CLOCK_BOOTTIME` (Linux): Identical to
|
|
:data:`time.CLOCK_MONOTONIC`, except it also includes any time that the
|
|
system is suspended.
|
|
* :data:`time.CLOCK_PROF` (FreeBSD, NetBSD and OpenBSD): High-resolution
|
|
per-process timer from the CPU.
|
|
* :data:`time.CLOCK_UPTIME` (FreeBSD, OpenBSD): Time whose absolute value is
|
|
the time the system has been running and not suspended, providing accurate
|
|
uptime measurement, both absolute and interval.
|
|
|
|
Added functions :func:`time.thread_time` and :func:`time.thread_time_ns`
|
|
to get per-thread CPU time measurements.
|
|
(Contributed by Antoine Pitrou in :issue:`32025`.)
|
|
|
|
unicodedata
|
|
-----------
|
|
|
|
The internal :mod:`unicodedata` database has been upgraded to use `Unicode 10
|
|
<http://www.unicode.org/versions/Unicode10.0.0/>`_. (Contributed by Benjamin
|
|
Peterson.)
|
|
|
|
unittest
|
|
--------
|
|
|
|
Added new command-line option ``-k`` to filter tests to run with a substring or
|
|
Unix shell-like pattern. For example, ``python -m unittest -k foo`` runs the
|
|
tests ``foo_tests.SomeTest.test_something``, ``bar_tests.SomeTest.test_foo``,
|
|
but not ``bar_tests.FooTest.test_something``.
|
|
(Contributed by Jonas Haag in :issue:`32071`.)
|
|
|
|
unittest.mock
|
|
-------------
|
|
|
|
The :const:`~unittest.mock.sentinel` attributes now preserve their identity
|
|
when they are :mod:`copied <copy>` or :mod:`pickled <pickle>`. (Contributed by
|
|
Serhiy Storchaka in :issue:`20804`.)
|
|
|
|
New function :func:`~unittest.mock.seal` will disable the creation of mock
|
|
children by preventing to get or set any new attribute on the sealed mock.
|
|
The sealing process is performed recursively. (Contributed by Mario Corchero
|
|
in :issue:`30541`.)
|
|
|
|
urllib.parse
|
|
------------
|
|
|
|
:func:`urllib.parse.quote` has been updated from :rfc:`2396` to :rfc:`3986`,
|
|
adding ``~`` to the set of characters that is never quoted by default.
|
|
(Contributed by Christian Theune and Ratnadeep Debnath in :issue:`16285`.)
|
|
|
|
uu
|
|
--
|
|
|
|
Function :func:`~uu.encode` now accepts an optional *backtick*
|
|
keyword argument. When it's true, zeros are represented by ``'`'``
|
|
instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.)
|
|
|
|
warnings
|
|
--------
|
|
|
|
The initialization of the default warnings filters has changed as follows:
|
|
|
|
* warnings enabled via command line options (including those for :option:`-b`
|
|
and the new CPython-specific ``-X dev`` option) are always passed to the
|
|
warnings machinery via the ``sys.warnoptions`` attribute.
|
|
* warnings filters enabled via the command line or the environment now have the
|
|
following precedence order:
|
|
|
|
* the ``BytesWarning`` filter for :option:`-b` (or ``-bb``)
|
|
* any filters specified with :option:`-W`
|
|
* any filters specified with :envvar:`PYTHONWARNINGS`
|
|
* any other CPython specific filters (e.g. the ``default`` filter added
|
|
for the new ``-X dev`` mode)
|
|
* any implicit filters defined directly by the warnings machinery
|
|
* in CPython debug builds, all warnings are now displayed by default (the
|
|
implicit filter list is empty)
|
|
|
|
(Contributed by Nick Coghlan and Victor Stinner in :issue:`20361`,
|
|
:issue:`32043`, and :issue:`32230`)
|
|
|
|
xml.etree
|
|
---------
|
|
|
|
:ref:`ElementPath <elementtree-xpath>` predicates in the :meth:`find`
|
|
methods can now compare text of the current node with ``[. = "text"]``,
|
|
not only text in children. Predicates also allow adding spaces for
|
|
better readability. (Contributed by Stefan Behnel in :issue:`31648`.)
|
|
|
|
xmlrpc.server
|
|
-------------
|
|
|
|
:meth:`register_function` of :class:`~xmlrpc.server.SimpleXMLRPCDispatcher` and
|
|
its subclasses can be used as a decorator. (Contributed by Xiang Zhang in
|
|
:issue:`7769`.)
|
|
|
|
zipapp
|
|
------
|
|
|
|
Function :func:`~zipapp.create_archive` now accepts an optional *filter*
|
|
argument to allow the user to select which files should be included in the
|
|
archive. (Contributed by Irmen de Jong in :issue:`31072`.)
|
|
|
|
Function :func:`~zipapp.create_archive` now accepts an optional *compressed*
|
|
argument to generate a compressed archive. A command line option
|
|
``--compress`` has also been added to support compression.
|
|
(Contributed by Zhiming Wang in :issue:`31638`.)
|
|
|
|
|
|
Optimizations
|
|
=============
|
|
|
|
* Added two new opcodes: :opcode:`LOAD_METHOD` and :opcode:`CALL_METHOD` to avoid
|
|
instantiation of bound method objects for method calls, which results
|
|
in method calls being faster up to 20%. (Contributed by Yury Selivanov and
|
|
INADA Naoki in :issue:`26110`.)
|
|
|
|
* Searching some unlucky Unicode characters (like Ukrainian capital "Є")
|
|
in a string was up to 25 times slower than searching other characters.
|
|
Now it is slower only by 3 times in the worst case.
|
|
(Contributed by Serhiy Storchaka in :issue:`24821`.)
|
|
|
|
* Fast implementation from standard C library is now used for functions
|
|
:func:`~math.erf` and :func:`~math.erfc` in the :mod:`math` module.
|
|
(Contributed by Serhiy Storchaka in :issue:`26121`.)
|
|
|
|
* The :func:`os.fwalk` function has been sped up by 2 times. This was done
|
|
using the :func:`os.scandir` function.
|
|
(Contributed by Serhiy Storchaka in :issue:`25996`.)
|
|
|
|
* The :func:`shutil.rmtree` function has been sped up to 20--40%.
|
|
This was done using the :func:`os.scandir` function.
|
|
(Contributed by Serhiy Storchaka in :issue:`28564`.)
|
|
|
|
* Optimized case-insensitive matching and searching of :mod:`regular
|
|
expressions <re>`. Searching some patterns can now be up to 20 times faster.
|
|
(Contributed by Serhiy Storchaka in :issue:`30285`.)
|
|
|
|
* :func:`re.compile` now converts ``flags`` parameter to int object if
|
|
it is ``RegexFlag``. It is now as fast as Python 3.5, and faster than
|
|
Python 3.6 by about 10% depending on the pattern.
|
|
(Contributed by INADA Naoki in :issue:`31671`.)
|
|
|
|
* :meth:`~selectors.BaseSelector.modify` methods of classes
|
|
:class:`selectors.EpollSelector`, :class:`selectors.PollSelector`
|
|
and :class:`selectors.DevpollSelector` may be around 10% faster under
|
|
heavy loads. (Contributed by Giampaolo Rodola' in :issue:`30014`)
|
|
|
|
* Constant folding is moved from peephole optimizer to new AST optimizer.
|
|
(Contributed by Eugene Toder and INADA Naoki in :issue:`29469`)
|
|
|
|
* Most functions and methods in :mod:`abc` have been rewritten in C.
|
|
This makes creation of abstract base classes, and calling :func:`isinstance`
|
|
and :func:`issubclass` on them 1.5x faster. This also reduces Python
|
|
start-up time by up to 10%. (Contributed by Ivan Levkivskyi and INADA Naoki
|
|
in :issue:`31333`)
|
|
|
|
* Significant speed improvements to alternate constructors for
|
|
:class:`datetime.date` and :class:`datetime.datetime` by using fast-path
|
|
constructors when not constructing subclasses. (Contributed by Paul Ganssle
|
|
in :issue:`32403`)
|
|
|
|
|
|
Build and C API Changes
|
|
=======================
|
|
|
|
* :mod:`py_compile` and :mod:`compileall` now support the
|
|
:envvar:`SOURCE_DATE_EPOCH` environment variable by unconditionally
|
|
building ``.pyc`` files for hash verification instead of potentially
|
|
timestamp-based ``.pyc`` files. See the notes for the `py_compile`_
|
|
improvement notes for more details.
|
|
|
|
* A full copy of libffi is no longer bundled for use when building the
|
|
:mod:`_ctypes <ctypes>` module on non-OSX UNIX platforms. An installed copy
|
|
of libffi is now required when building ``_ctypes`` on such platforms.
|
|
(Contributed by Zachary Ware in :issue:`27979`.)
|
|
|
|
* The fields :c:member:`name` and :c:member:`doc` of structures
|
|
:c:type:`PyMemberDef`, :c:type:`PyGetSetDef`,
|
|
:c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`,
|
|
and :c:type:`wrapperbase` are now of type ``const char *`` rather of
|
|
``char *``. (Contributed by Serhiy Storchaka in :issue:`28761`.)
|
|
|
|
* The result of :c:func:`PyUnicode_AsUTF8AndSize` and :c:func:`PyUnicode_AsUTF8`
|
|
is now of type ``const char *`` rather of ``char *``. (Contributed by Serhiy
|
|
Storchaka in :issue:`28769`.)
|
|
|
|
* The result of :c:func:`PyMapping_Keys`, :c:func:`PyMapping_Values` and
|
|
:c:func:`PyMapping_Items` is now always a list, rather than a list or a
|
|
tuple. (Contributed by Oren Milman in :issue:`28280`.)
|
|
|
|
* Added functions :c:func:`PySlice_Unpack` and :c:func:`PySlice_AdjustIndices`.
|
|
(Contributed by Serhiy Storchaka in :issue:`27867`.)
|
|
|
|
* :c:func:`PyOS_AfterFork` is deprecated in favour of the new functions
|
|
:c:func:`PyOS_BeforeFork`, :c:func:`PyOS_AfterFork_Parent` and
|
|
:c:func:`PyOS_AfterFork_Child`. (Contributed by Antoine Pitrou in
|
|
:issue:`16500`.)
|
|
|
|
* The Windows build process no longer depends on Subversion to pull in external
|
|
sources, a Python script is used to download zipfiles from GitHub instead.
|
|
If Python 3.6 is not found on the system (via ``py -3.6``), NuGet is used to
|
|
download a copy of 32-bit Python for this purpose. (Contributed by Zachary
|
|
Ware in :issue:`30450`.)
|
|
|
|
* The ``PyExc_RecursionErrorInst`` singleton that was part of the public API
|
|
has been removed as its members being never cleared may cause a segfault
|
|
during finalization of the interpreter. Contributed by Xavier de Gaye in
|
|
:issue:`22898` and :issue:`30697`.
|
|
|
|
* Support for building ``--without-threads`` is removed.
|
|
(Contributed by Antoine Pitrou in :issue:`31370`.).
|
|
|
|
* Added C API support for timezones with timezone constructors
|
|
:c:func:`PyTimeZone_FromOffset` and :c:func:`PyTimeZone_FromOffsetAndName`,
|
|
and access to the UTC singleton with :c:data:`PyDateTime_TimeZone_UTC`.
|
|
Contributed by Paul Ganssle in :issue:`10381`.
|
|
|
|
- The type of results of :c:func:`PyThread_start_new_thread` and
|
|
:c:func:`PyThread_get_thread_ident`, and the *id* parameter of
|
|
:c:func:`PyThreadState_SetAsyncExc` changed from :c:type:`long` to
|
|
:c:type:`unsigned long`.
|
|
(Contributed by Serhiy Storchaka in :issue:`6532`.)
|
|
|
|
- :c:func:`PyUnicode_AsWideCharString` now raises a :exc:`ValueError` if the
|
|
second argument is *NULL* and the :c:type:`wchar_t*` string contains null
|
|
characters. (Contributed by Serhiy Storchaka in :issue:`30708`.)
|
|
|
|
- Changes to the startup sequence and the management of dynamic memory
|
|
allocators mean that the long documented requirement to call
|
|
:c:func:`Py_Initialize` before calling most C API functions is now
|
|
relied on more heavily, and failing to abide by it may lead to segfaults in
|
|
embedding applications. See the :ref:`porting-to-python-37` section in this
|
|
document and the :ref:`pre-init-safe` section in the C API documentation
|
|
for more details.
|
|
|
|
|
|
Other CPython Implementation Changes
|
|
====================================
|
|
|
|
* Trace hooks may now opt out of receiving ``line`` events from the interpreter
|
|
by setting the new ``f_trace_lines`` attribute to :const:`False` on the frame
|
|
being traced. (Contributed by Nick Coghlan in :issue:`31344`.)
|
|
|
|
* Trace hooks may now opt in to receiving ``opcode`` events from the interpreter
|
|
by setting the new ``f_trace_opcodes`` attribute to :const:`True` on the frame
|
|
being traced. (Contributed by Nick Coghlan in :issue:`31344`.)
|
|
|
|
* Fixed some consistency problems with namespace package module attributes.
|
|
Namespace module objects now have an ``__file__`` that is set to ``None``
|
|
(previously unset), and their ``__spec__.origin`` is also set to ``None``
|
|
(previously the string ``"namespace"``). See :issue:`32305`. Also, the
|
|
namespace module object's ``__spec__.loader`` is set to the same value as
|
|
``__loader__`` (previously, the former was set to ``None``). See
|
|
:issue:`32303`.
|
|
|
|
|
|
Deprecated
|
|
==========
|
|
|
|
* In Python 3.8, the abstract base classes in :mod:`collections.abc` will no
|
|
longer be exposed in the regular :mod:`collections` module. This will help
|
|
create a clearer distinction between the concrete classes and the abstract
|
|
base classes.
|
|
|
|
* Yield expressions (both ``yield`` and ``yield from`` clauses) are now deprecated
|
|
in comprehensions and generator expressions (aside from the iterable expression
|
|
in the leftmost :keyword:`for` clause). This ensures that comprehensions
|
|
always immediately return a container of the appropriate type (rather than
|
|
potentially returning a :term:`generator iterator` object), while generator
|
|
expressions won't attempt to interleave their implicit output with the output
|
|
from any explicit yield expressions.
|
|
|
|
In Python 3.7, such expressions emit :exc:`DeprecationWarning` when compiled,
|
|
in Python 3.8+ they will emit :exc:`SyntaxError`. (Contributed by Serhiy
|
|
Storchaka in :issue:`10544`.)
|
|
|
|
- Function :c:func:`PySlice_GetIndicesEx` is deprecated and replaced with
|
|
a macro if ``Py_LIMITED_API`` is not set or set to the value between
|
|
``0x03050400`` and ``0x03060000`` (not including) or ``0x03060100`` or
|
|
higher. (Contributed by Serhiy Storchaka in :issue:`27867`.)
|
|
|
|
- Deprecated :meth:`format` from :mod:`locale`, use the :meth:`format_string`
|
|
instead. (Contributed by Garvit in :issue:`10379`.)
|
|
|
|
- Methods
|
|
:meth:`MetaPathFinder.find_module() <importlib.abc.MetaPathFinder.find_module>`
|
|
(replaced by
|
|
:meth:`MetaPathFinder.find_spec() <importlib.abc.MetaPathFinder.find_spec>`)
|
|
and
|
|
:meth:`PathEntryFinder.find_loader() <importlib.abc.PathEntryFinder.find_loader>`
|
|
(replaced by
|
|
:meth:`PathEntryFinder.find_spec() <importlib.abc.PathEntryFinder.find_spec>`)
|
|
both deprecated in Python 3.4 now emit :exc:`DeprecationWarning`.
|
|
(Contributed by Matthias Bussonnier in :issue:`29576`)
|
|
|
|
- Using non-integer value for selecting a plural form in :mod:`gettext` is
|
|
now deprecated. It never correctly worked. (Contributed by Serhiy Storchaka
|
|
in :issue:`28692`.)
|
|
|
|
- The :mod:`macpath` is now deprecated and will be removed in Python 3.8.
|
|
|
|
- The :class:`importlib.abc.ResourceLoader` ABC has been deprecated in
|
|
favour of :class:`importlib.abc.ResourceReader`.
|
|
|
|
- Deprecated :func:`sys.set_coroutine_wrapper` and
|
|
:func:`sys.get_coroutine_wrapper`.
|
|
|
|
- :func:`ssl.wrap_socket` is deprecated. Use
|
|
:meth:`ssl.SSLContext.wrap_socket` instead.
|
|
(Contributed by Christian Heimes in :issue:`28124`.)
|
|
|
|
|
|
Windows Only
|
|
------------
|
|
|
|
- The python launcher, (py.exe), can accept 32 & 64 bit specifiers **without**
|
|
having to specify a minor version as well. So ``py -3-32`` and ``py -3-64``
|
|
become valid as well as ``py -3.7-32``, also the -*m*-64 and -*m.n*-64 forms
|
|
are now accepted to force 64 bit python even if 32 bit would have otherwise
|
|
been used. If the specified version is not available py.exe will error exit.
|
|
(Contributed by Steve Barnes in :issue:`30291`.)
|
|
|
|
- The launcher can be run as ``py -0`` to produce a list of the installed pythons,
|
|
*with default marked with an asterisk*. Running ``py -0p`` will include the paths.
|
|
If py is run with a version specifier that cannot be matched it will also print
|
|
the *short form* list of available specifiers.
|
|
(Contributed by Steve Barnes in :issue:`30362`.)
|
|
|
|
|
|
Removed
|
|
=======
|
|
|
|
Platform Support Removals
|
|
-------------------------
|
|
|
|
* FreeBSD 9 and older are no longer supported.
|
|
|
|
API and Feature Removals
|
|
------------------------
|
|
|
|
* The ``os.stat_float_times()`` function has been removed. It was introduced in
|
|
Python 2.3 for backward compatibility with Python 2.2, and was deprecated
|
|
since Python 3.1.
|
|
|
|
* Unknown escapes consisting of ``'\'`` and an ASCII letter in replacement
|
|
templates for :func:`re.sub` were deprecated in Python 3.5, and will now
|
|
cause an error.
|
|
|
|
* Removed support of the *exclude* argument in :meth:`tarfile.TarFile.add`.
|
|
It was deprecated in Python 2.7 and 3.2. Use the *filter* argument instead.
|
|
|
|
* The ``splitunc()`` function in the :mod:`ntpath` module was deprecated in
|
|
Python 3.1, and has now been removed. Use the :func:`~os.path.splitdrive`
|
|
function instead.
|
|
|
|
* :func:`collections.namedtuple` no longer supports the *verbose* parameter
|
|
or ``_source`` attribute which showed the generated source code for the
|
|
named tuple class. This was part of an optimization designed to speed-up
|
|
class creation. (Contributed by Jelle Zijlstra with further improvements
|
|
by INADA Naoki, Serhiy Storchaka, and Raymond Hettinger in :issue:`28638`.)
|
|
|
|
* Functions :func:`bool`, :func:`float`, :func:`list` and :func:`tuple` no
|
|
longer take keyword arguments. The first argument of :func:`int` can now
|
|
be passed only as positional argument.
|
|
|
|
* Removed previously deprecated in Python 2.4 classes ``Plist``, ``Dict`` and
|
|
``_InternalDict`` in the :mod:`plistlib` module. Dict values in the result
|
|
of functions :func:`~plistlib.readPlist` and
|
|
:func:`~plistlib.readPlistFromBytes` are now normal dicts. You no longer
|
|
can use attribute access to access items of these dictionaries.
|
|
|
|
* The ``asyncio.windows_utils.socketpair()`` function has been
|
|
removed: use directly :func:`socket.socketpair` which is available on all
|
|
platforms since Python 3.5 (before, it wasn't available on Windows).
|
|
``asyncio.windows_utils.socketpair`` was just an alias to
|
|
``socket.socketpair`` on Python 3.5 and newer.
|
|
|
|
* :mod:`asyncio`: The module doesn't export :mod:`selectors` and
|
|
:mod:`_overlapped` modules as ``asyncio.selectors`` and
|
|
``asyncio._overlapped``. Replace ``from asyncio import selectors`` with
|
|
``import selectors`` for example.
|
|
|
|
.. _porting-to-python-37:
|
|
|
|
Porting to Python 3.7
|
|
=====================
|
|
|
|
This section lists previously described changes and other bugfixes
|
|
that may require changes to your code.
|
|
|
|
|
|
Changes in Python behavior
|
|
--------------------------
|
|
|
|
* :pep:`479` is enabled for all code in Python 3.7, meaning that
|
|
:exc:`StopIteration` exceptions raised directly or indirectly in
|
|
coroutines and generators are transformed into :exc:`RuntimeError`
|
|
exceptions.
|
|
(Contributed by Yury Selivanov in :issue:`32670`.)
|
|
|
|
* Due to an oversight, earlier Python versions erroneously accepted the
|
|
following syntax::
|
|
|
|
f(1 for x in [1],)
|
|
|
|
class C(1 for x in [1]):
|
|
pass
|
|
|
|
Python 3.7 now correctly raises a :exc:`SyntaxError`, as a generator
|
|
expression always needs to be directly inside a set of parentheses
|
|
and cannot have a comma on either side, and the duplication of the
|
|
parentheses can be omitted only on calls.
|
|
(Contributed by Serhiy Storchaka in :issue:`32012` and :issue:`32023`.)
|
|
|
|
* When using the :option:`-m` switch, the initial working directory is now added
|
|
to :data:`sys.path`, rather than an empty string (which dynamically denoted
|
|
the current working directory at the time of each import). Any programs that
|
|
are checking for the empty string, or otherwise relying on the previous
|
|
behaviour, will need to be updated accordingly (e.g. by also checking for
|
|
``os.getcwd()`` or ``os.path.dirname(__main__.__file__)``, depending on why
|
|
the code was checking for the empty string in the first place).
|
|
|
|
|
|
Changes in the Python API
|
|
-------------------------
|
|
|
|
* :meth:`socketserver.ThreadingMixIn.server_close` now waits until all
|
|
non-daemon threads complete. Use daemonic threads by setting
|
|
:data:`socketserver.ThreadingMixIn.daemon_threads` to ``True`` to not
|
|
wait until threads complete.
|
|
(Contributed by Victor Stinner in :issue:`31233`.)
|
|
|
|
* :meth:`socketserver.ForkingMixIn.server_close` now waits until all
|
|
child processes complete. (Contributed by Victor Stinner in :issue:`31151`.)
|
|
|
|
* The :func:`locale.localeconv` function now sets temporarily the ``LC_CTYPE``
|
|
locale to the ``LC_NUMERIC`` locale in some cases.
|
|
(Contributed by Victor Stinner in :issue:`31900`.)
|
|
|
|
* :meth:`pkgutil.walk_packages` now raises :exc:`ValueError` if *path* is a string.
|
|
Previously an empty list was returned. (Contributed by Sanyam Khurana in
|
|
:issue:`24744`.)
|
|
|
|
* A format string argument for :meth:`string.Formatter.format`
|
|
is now :ref:`positional-only <positional-only_parameter>`.
|
|
Passing it as a keyword argument was deprecated in Python 3.5. (Contributed
|
|
by Serhiy Storchaka in :issue:`29193`.)
|
|
|
|
* Attributes :attr:`~http.cookies.Morsel.key`,
|
|
:attr:`~http.cookies.Morsel.value` and
|
|
:attr:`~http.cookies.Morsel.coded_value` of class
|
|
:class:`http.cookies.Morsel` are now read-only.
|
|
Assigning to them was deprecated in Python 3.5.
|
|
Use the :meth:`~http.cookies.Morsel.set` method for setting them.
|
|
(Contributed by Serhiy Storchaka in :issue:`29192`.)
|
|
|
|
* ``Module``, ``FunctionDef``, ``AsyncFunctionDef``, and
|
|
``ClassDef`` AST nodes now have a new ``docstring`` field.
|
|
The first statement in their body is not considered as a docstring
|
|
anymore. ``co_firstlineno`` and ``co_lnotab`` of code object for class
|
|
and module are affected by this change. (Contributed by INADA Naoki and
|
|
Eugene Toder in :issue:`29463`.)
|
|
|
|
* The *mode* argument of :func:`os.makedirs` no longer affects the file
|
|
permission bits of newly-created intermediate-level directories.
|
|
To set their file permission bits you can set the umask before invoking
|
|
``makedirs()``.
|
|
(Contributed by Serhiy Storchaka in :issue:`19930`.)
|
|
|
|
* The :attr:`struct.Struct.format` type is now :class:`str` instead of
|
|
:class:`bytes`. (Contributed by Victor Stinner in :issue:`21071`.)
|
|
|
|
* :func:`~cgi.parse_multipart` returns now the same results as
|
|
:class:`~FieldStorage`: for non-file fields, the value associated to a key
|
|
is a list of strings, not bytes.
|
|
(Contributed by Pierre Quentel in :issue:`29979`.)
|
|
|
|
* Due to internal changes in :mod:`socket` you won't be able to
|
|
:func:`socket.fromshare` a socket :func:`~socket.socket.share`-ed in older
|
|
Python versions.
|
|
|
|
* ``repr`` for :exc:`BaseException` has changed not to include trailing comma
|
|
in the output. Mind that most exceptions are affected by this change.
|
|
(Contributed by Serhiy Storchaka in :issue:`30399`.)
|
|
|
|
* ``repr`` for :class:`datetime.timedelta` has changed to include keyword arguments
|
|
in the output. (Contributed by Utkarsh Upadhyay in :issue:`30302`.)
|
|
|
|
* Because :func:`shutil.rmtree` is now implemented using the :func:`os.scandir`
|
|
function, the user specified handler *onerror* is now called with the first
|
|
argument ``os.scandir`` instead of ``os.listdir`` when listing the direcory
|
|
is failed.
|
|
|
|
* Support of nested sets and set operations in regular expressions as in
|
|
`Unicode Technical Standard #18`_ might be added in the future. This would
|
|
change the syntax, so to facilitate this change a :exc:`FutureWarning` will
|
|
be raised in ambiguous cases for the time being.
|
|
That include sets starting with a literal ``'['`` or containing literal
|
|
character sequences ``'--'``, ``'&&'``, ``'~~'``, and ``'||'``. To
|
|
avoid a warning escape them with a backslash.
|
|
(Contributed by Serhiy Storchaka in :issue:`30349`.)
|
|
|
|
.. _Unicode Technical Standard #18: https://unicode.org/reports/tr18/
|
|
|
|
* The result of splitting a string on a :mod:`regular expression <re>`
|
|
that could match an empty string has been changed. For example
|
|
splitting on ``r'\s*'`` will now split not only on whitespaces as it
|
|
did previously, but also on empty strings before all non-whitespace
|
|
characters and just before the end of the string.
|
|
The previous behavior can be restored by changing the pattern
|
|
to ``r'\s+'``. A :exc:`FutureWarning` was emitted for such patterns since
|
|
Python 3.5.
|
|
|
|
For patterns that match both empty and non-empty strings, the result of
|
|
searching for all matches may also be changed in other cases. For example
|
|
in the string ``'a\n\n'``, the pattern ``r'(?m)^\s*?$'`` will not only
|
|
match empty strings at positions 2 and 3, but also the string ``'\n'`` at
|
|
positions 2--3. To match only blank lines, the pattern should be rewritten
|
|
as ``r'(?m)^[^\S\n]*$'``.
|
|
|
|
:func:`re.sub()` now replaces empty matches adjacent to a previous
|
|
non-empty match. For example ``re.sub('x*', '-', 'abxd')`` returns now
|
|
``'-a-b--d-'`` instead of ``'-a-b-d-'`` (the first minus between 'b' and
|
|
'd' replaces 'x', and the second minus replaces an empty string between
|
|
'x' and 'd').
|
|
|
|
(Contributed by Serhiy Storchaka in :issue:`25054` and :issue:`32308`.)
|
|
|
|
* Change :func:`re.escape` to only escape regex special characters instead
|
|
of escaping all characters other than ASCII letters, numbers, and ``'_'``.
|
|
(Contributed by Serhiy Storchaka in :issue:`29995`.)
|
|
|
|
* :class:`tracemalloc.Traceback` frames are now sorted from oldest to most
|
|
recent to be more consistent with :mod:`traceback`.
|
|
(Contributed by Jesse Bakker in :issue:`32121`.)
|
|
|
|
* On OSes that support :const:`socket.SOCK_NONBLOCK` or
|
|
:const:`socket.SOCK_CLOEXEC` bit flags, the
|
|
:attr:`socket.type <socket.socket.type>` no longer has them applied.
|
|
Therefore, checks like ``if sock.type == socket.SOCK_STREAM``
|
|
work as expected on all platforms.
|
|
(Contributed by Yury Selivanov in :issue:`32331`.)
|
|
|
|
* On Windows the default for the *close_fds* argument of
|
|
:class:`subprocess.Popen` was changed from :const:`False` to :const:`True`
|
|
when redirecting the standard handles. If you previously depended on handles
|
|
being inherited when using :class:`subprocess.Popen` with standard io
|
|
redirection, you will have to pass ``close_fds=False`` to preserve the
|
|
previous behaviour, or use
|
|
:attr:`STARTUPINFO.lpAttributeList <subprocess.STARTUPINFO.lpAttributeList>`.
|
|
|
|
|
|
Changes in the C API
|
|
--------------------
|
|
|
|
* The function :c:func:`PySlice_GetIndicesEx` is considered not safe for
|
|
resizable sequences. If the slice indices are not instances of :class:`int`,
|
|
but objects that implement the :meth:`!__index__` method, the sequence can be
|
|
resized after passing its length to :c:func:`!PySlice_GetIndicesEx`. This
|
|
can lead to returning indices out of the length of the sequence. For
|
|
avoiding possible problems use new functions :c:func:`PySlice_Unpack` and
|
|
:c:func:`PySlice_AdjustIndices`.
|
|
(Contributed by Serhiy Storchaka in :issue:`27867`.)
|
|
|
|
|
|
CPython bytecode changes
|
|
------------------------
|
|
|
|
* Added two new opcodes: :opcode:`LOAD_METHOD` and :opcode:`CALL_METHOD`.
|
|
(Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.)
|
|
|
|
* Removed the :opcode:`STORE_ANNOTATION` opcode.
|
|
(Contributed by Mark Shannon in :issue:`32550`.)
|
|
|
|
|
|
Other CPython implementation changes
|
|
------------------------------------
|
|
|
|
* In preparation for potential future changes to the public CPython runtime
|
|
initialization API (see :pep:`432` for an initial, but somewhat outdated,
|
|
draft), CPython's internal startup
|
|
and configuration management logic has been significantly refactored. While
|
|
these updates are intended to be entirely transparent to both embedding
|
|
applications and users of the regular CPython CLI, they're being mentioned
|
|
here as the refactoring changes the internal order of various operations
|
|
during interpreter startup, and hence may uncover previously latent defects,
|
|
either in embedding applications, or in CPython itself.
|
|
(Initially contributed by Nick Coghlan and Eric Snow as part of
|
|
:issue:`22257`, and further updated by Nick, Eric, and Victor Stinner in a
|
|
number of other issues). Some known details affected:
|
|
|
|
* :c:func:`PySys_AddWarnOptionUnicode` is not currently usable by embedding
|
|
applications due to the requirement to create a Unicode object prior to
|
|
calling `Py_Initialize`. Use :c:func:`PySys_AddWarnOption` instead.
|
|
* warnings filters added by an embedding application with
|
|
:c:func:`PySys_AddWarnOption` should now more consistently take precedence
|
|
over the default filters set by the interpreter
|
|
|
|
* Due to changes in the way the default warnings filters are configured,
|
|
setting :c:data:`Py_BytesWarningFlag` to a value greater than one is no longer
|
|
sufficient to both emit :exc:`BytesWarning` messages and have them converted
|
|
to exceptions. Instead, the flag must be set (to cause the warnings to be
|
|
emitted in the first place), and an explicit ``error::BytesWarning``
|
|
warnings filter added to convert them to exceptions.
|
|
|
|
* CPython' :mod:`ssl` module requires OpenSSL 1.0.2 or 1.1 compatible libssl.
|
|
OpenSSL 1.0.1 has reached end of lifetime on 2016-12-31 and is no longer
|
|
supported. LibreSSL is temporarily not supported as well. LibreSSL releases
|
|
up to version 2.6.4 are missing required OpenSSL 1.0.2 APIs.
|
|
|
|
|
|
Documentation
|
|
=============
|
|
|
|
.. _whatsnew37-pep545:
|
|
|
|
PEP 545: Python Documentation Translations
|
|
------------------------------------------
|
|
|
|
:pep:`545` describes the process to translate Python documentation,
|
|
and three translations have been added:
|
|
|
|
- Japanese: https://docs.python.org/ja/ and associated GitHub
|
|
repository: https://github.com/python/python-docs-ja
|
|
|
|
- French: https://docs.python.org/fr/ and associated GitHub
|
|
repository: https://github.com/python/python-docs-fr
|
|
|
|
- Korean: https://docs.python.org/ko/ and associated GitHub
|
|
repository: https://github.com/python/python-docs-ko
|
|
|
|
(Contributed by Julien Palard, Inada Naoki, and Victor Stinner in
|
|
:issue:`26546`.)
|