mirror of https://github.com/python/cpython
946 lines
37 KiB
ReStructuredText
946 lines
37 KiB
ReStructuredText
****************************
|
|
What's New In Python 3.7
|
|
****************************
|
|
|
|
:Release: |release|
|
|
:Date: |today|
|
|
|
|
.. Rules for maintenance:
|
|
|
|
* Anyone can add text to this document. Do not spend very much time
|
|
on the wording of your changes, because your text will probably
|
|
get rewritten to some degree.
|
|
|
|
* The maintainer will go through Misc/NEWS periodically and add
|
|
changes; it's therefore more important to add your changes to
|
|
Misc/NEWS than to this file.
|
|
|
|
* This is not a complete list of every single change; completeness
|
|
is the purpose of Misc/NEWS. Some changes I consider too small
|
|
or esoteric to include. If such a change is added to the text,
|
|
I'll just remove it. (This is another reason you shouldn't spend
|
|
too much time on writing your addition.)
|
|
|
|
* If you want to draw your new text to the attention of the
|
|
maintainer, add 'XXX' to the beginning of the paragraph or
|
|
section.
|
|
|
|
* It's OK to just add a fragmentary note about a change. For
|
|
example: "XXX Describe the transmogrify() function added to the
|
|
socket module." The maintainer will research the change and
|
|
write the necessary text.
|
|
|
|
* You can comment out your additions if you like, but it's not
|
|
necessary (especially when a final release is some months away).
|
|
|
|
* Credit the author of a patch or bugfix. Just the name is
|
|
sufficient; the e-mail address isn't necessary.
|
|
|
|
* It's helpful to add the bug/patch number as a comment:
|
|
|
|
XXX Describe the transmogrify() function added to the socket
|
|
module.
|
|
(Contributed by P.Y. Developer in :issue:`12345`.)
|
|
|
|
This saves the maintainer the effort of going through the Mercurial log
|
|
when researching a change.
|
|
|
|
This article explains the new features in Python 3.7, compared to 3.6.
|
|
|
|
For full details, see the :ref:`changelog <changelog>`.
|
|
|
|
.. note::
|
|
|
|
Prerelease users should be aware that this document is currently in draft
|
|
form. It will be updated substantially as Python 3.7 moves towards release,
|
|
so it's worth checking back even after reading earlier versions.
|
|
|
|
|
|
Summary -- Release highlights
|
|
=============================
|
|
|
|
.. This section singles out the most important changes in Python 3.7.
|
|
Brevity is key.
|
|
|
|
|
|
.. PEP-sized items next.
|
|
|
|
|
|
|
|
New Features
|
|
============
|
|
|
|
.. _whatsnew37-pep538:
|
|
|
|
PEP 538: Legacy C Locale Coercion
|
|
---------------------------------
|
|
|
|
An ongoing challenge within the Python 3 series has been determining a sensible
|
|
default strategy for handling the "7-bit ASCII" text encoding assumption
|
|
currently implied by the use of the default C locale on non-Windows platforms.
|
|
|
|
:pep:`538` updates the default interpreter command line interface to
|
|
automatically coerce that locale to an available UTF-8 based locale as
|
|
described in the documentation of the new :envvar:`PYTHONCOERCECLOCALE`
|
|
environment variable. Automatically setting ``LC_CTYPE`` this way means that
|
|
both the core interpreter and locale-aware C extensions (such as
|
|
:mod:`readline`) will assume the use of UTF-8 as the default text encoding,
|
|
rather than ASCII.
|
|
|
|
The platform support definition in :pep:`11` has also been updated to limit
|
|
full text handling support to suitably configured non-ASCII based locales.
|
|
|
|
As part of this change, the default error handler for ``stdin`` and ``stdout``
|
|
is now ``surrogateescape`` (rather than ``strict``) when using any of the
|
|
defined coercion target locales (currently ``C.UTF-8``, ``C.utf8``, and
|
|
``UTF-8``). The default error handler for ``stderr`` continues to be
|
|
``backslashreplace``, regardless of locale.
|
|
|
|
Locale coercion is silent by default, but to assist in debugging potentially
|
|
locale related integration problems, explicit warnings (emitted directly on
|
|
``stderr`` can be requested by setting ``PYTHONCOERCECLOCALE=warn``. This
|
|
setting will also cause the Python runtime to emit a warning if the legacy C
|
|
locale remains active when the core interpreter is initialized.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`538` -- Coercing the legacy C locale to a UTF-8 based locale
|
|
PEP written and implemented by Nick Coghlan.
|
|
|
|
.. _whatsnew37-pep553:
|
|
|
|
PEP 553: Built-in breakpoint()
|
|
------------------------------
|
|
|
|
:pep:`553` describes a new built-in called ``breakpoint()`` which makes it
|
|
easy and consistent to enter the Python debugger. Built-in ``breakpoint()``
|
|
calls ``sys.breakpointhook()``. By default, this latter imports ``pdb`` and
|
|
then calls ``pdb.set_trace()``, but by binding ``sys.breakpointhook()`` to the
|
|
function of your choosing, ``breakpoint()`` can enter any debugger. Or, the
|
|
environment variable :envvar:`PYTHONBREAKPOINT` can be set to the callable of
|
|
your debugger of choice. Set ``PYTHONBREAKPOINT=0`` to completely disable
|
|
built-in ``breakpoint()``.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`553` -- Built-in breakpoint()
|
|
PEP written and implemented by Barry Warsaw
|
|
|
|
|
|
.. _whatsnew37-pep539:
|
|
|
|
PEP 539: A New C-API for Thread-Local Storage in CPython
|
|
--------------------------------------------------------
|
|
|
|
While Python provides a C API for thread-local storage support; the existing
|
|
:ref:`Thread Local Storage (TLS) API <thread-local-storage-api>` has used
|
|
:c:type:`int` to represent TLS keys across all platforms. This has not
|
|
generally been a problem for officially-support platforms, but that is neither
|
|
POSIX-compliant, nor portable in any practical sense.
|
|
|
|
:pep:`539` changes this by providing a new :ref:`Thread Specific Storage (TSS)
|
|
API <thread-specific-storage-api>` to CPython which supersedes use of the
|
|
existing TLS API within the CPython interpreter, while deprecating the existing
|
|
API. The TSS API uses a new type :c:type:`Py_tss_t` instead of :c:type:`int`
|
|
to represent TSS keys--an opaque type the definition of which may depend on
|
|
the underlying TLS implementation. Therefore, this will allow to build CPython
|
|
on platforms where the native TLS key is defined in a way that cannot be safely
|
|
cast to :c:type:`int`.
|
|
|
|
Note that on platforms where the native TLS key is defined in a way that cannot
|
|
be safely cast to :c:type:`int`, all functions of the existing TLS API will be
|
|
no-op and immediately return failure. This indicates clearly that the old API
|
|
is not supported on platforms where it cannot be used reliably, and that no
|
|
effort will be made to add such support.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`539` -- A New C-API for Thread-Local Storage in CPython
|
|
PEP written by Erik M. Bray; implementation by Masayuki Yamamoto.
|
|
|
|
|
|
PEP 562: Customization of access to module attributes
|
|
-----------------------------------------------------
|
|
|
|
It is sometimes convenient to customize or otherwise have control over access
|
|
to module attributes. A typical example is managing deprecation warnings.
|
|
Typical workarounds are assigning ``__class__`` of a module object to
|
|
a custom subclass of :class:`types.ModuleType` or replacing the ``sys.modules``
|
|
item with a custom wrapper instance. This procedure is now simplified by
|
|
recognizing ``__getattr__`` defined directly in a module that would act like
|
|
a normal ``__getattr__`` method, except that it will be defined on module
|
|
*instances*.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`562` -- Module ``__getattr__`` and ``__dir__``
|
|
PEP written and implemented by Ivan Levkivskyi
|
|
|
|
|
|
PEP 564: Add new time functions with nanosecond resolution
|
|
----------------------------------------------------------
|
|
|
|
Add six new "nanosecond" variants of existing functions to the :mod:`time`
|
|
module:
|
|
|
|
* :func:`time.clock_gettime_ns`
|
|
* :func:`time.clock_settime_ns`
|
|
* :func:`time.monotonic_ns`
|
|
* :func:`time.perf_counter_ns`
|
|
* :func:`time.process_time_ns`
|
|
* :func:`time.time_ns`
|
|
|
|
While similar to the existing functions without the ``_ns`` suffix, they
|
|
provide nanosecond resolution: they return a number of nanoseconds as a Python
|
|
``int``.
|
|
|
|
The ``time.time_ns()`` resolution is 3 times better than the ``time.time()``
|
|
resolution on Linux and Windows.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`564` -- Add new time functions with nanosecond resolution
|
|
PEP written and implemented by Victor Stinner
|
|
|
|
|
|
PEP 540: Add a new UTF-8 mode
|
|
-----------------------------
|
|
|
|
Add a new UTF-8 mode to ignore the locale, use the UTF-8 encoding, and change
|
|
:data:`sys.stdin` and :data:`sys.stdout` error handlers to ``surrogateescape``.
|
|
This mode is enabled by default in the POSIX locale, but otherwise disabled by
|
|
default.
|
|
|
|
The new :option:`-X` ``utf8`` command line option and :envvar:`PYTHONUTF8`
|
|
environment variable are added to control the UTF-8 mode.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`540` -- Add a new UTF-8 mode
|
|
PEP written and implemented by Victor Stinner
|
|
|
|
|
|
New Development Mode: -X dev
|
|
----------------------------
|
|
|
|
Add a new "development mode": :option:`-X` ``dev`` command line option and
|
|
:envvar:`PYTHONDEVMODE` environment variable to enable CPython's "development
|
|
mode", introducing additional runtime checks which are too expensive to be
|
|
enabled by default. See :option:`-X` ``dev`` documentation for the effects of
|
|
the development mode.
|
|
|
|
Hash-based pycs
|
|
---------------
|
|
|
|
Python has traditionally checked the up-to-dateness of bytecode cache files
|
|
(i.e., ``.pyc`` files) by comparing the source metadata (last-modified timestamp
|
|
and size) with source metadata saved in the cache file header when it was
|
|
generated. While effective, this invalidation method has its drawbacks. When
|
|
filesystem timestamps are too coarse, Python can miss source updates, leading to
|
|
user confusion. Additionally, having a timestamp in the cache file is
|
|
problematic for `build reproduciblity <https://reproducible-builds.org/>`_ and
|
|
content-based build systems.
|
|
|
|
:pep:`552` extends the pyc format to allow the hash of the source file to be
|
|
used for invalidation instead of the source timestamp. Such ``.pyc`` files are
|
|
called "hash-based". By default, Python still uses timestamp-based invalidation
|
|
and does not generate hash-based ``.pyc`` files at runtime. Hash-based ``.pyc``
|
|
files may be generated with :mod:`py_compile` or :mod:`compileall`.
|
|
|
|
Hash-based ``.pyc`` files come in two variants: checked and unchecked. Python
|
|
validates checked hash-based ``.pyc`` files against the corresponding source
|
|
files at runtime but doesn't do so for unchecked hash-based pycs. Unchecked
|
|
hash-based ``.pyc`` files are a useful performance optimization for environments
|
|
where a system external to Python (e.g., the build system) is responsible for
|
|
keeping ``.pyc`` files up-to-date.
|
|
|
|
See :ref:`pyc-invalidation` for more information.
|
|
|
|
|
|
Other Language Changes
|
|
======================
|
|
|
|
* More than 255 arguments can now be passed to a function, and a function can
|
|
now have more than 255 parameters. (Contributed by Serhiy Storchaka in
|
|
:issue:`12844` and :issue:`18896`.)
|
|
|
|
* :meth:`bytes.fromhex` and :meth:`bytearray.fromhex` now ignore all ASCII
|
|
whitespace, not only spaces. (Contributed by Robert Xiao in :issue:`28927`.)
|
|
|
|
* :exc:`ImportError` now displays module name and module ``__file__`` path when
|
|
``from ... import ...`` fails. (Contributed by Matthias Bussonnier in :issue:`29546`.)
|
|
|
|
* Circular imports involving absolute imports with binding a submodule to
|
|
a name are now supported.
|
|
(Contributed by Serhiy Storchaka in :issue:`30024`.)
|
|
|
|
* ``object.__format__(x, '')`` is now equivalent to ``str(x)`` rather than
|
|
``format(str(self), '')``.
|
|
(Contributed by Serhiy Storchaka in :issue:`28974`.)
|
|
|
|
|
|
New Modules
|
|
===========
|
|
|
|
* None yet.
|
|
|
|
|
|
Improved Modules
|
|
================
|
|
|
|
|
|
argparse
|
|
--------
|
|
|
|
The :meth:`~argparse.ArgumentParser.parse_intermixed_args` supports letting
|
|
the user intermix options and positional arguments on the command line,
|
|
as is possible in many unix commands. It supports most but not all
|
|
argparse features. (Contributed by paul.j3 in :issue:`14191`.)
|
|
|
|
|
|
binascii
|
|
--------
|
|
|
|
The :func:`~binascii.b2a_uu` function now accepts an optional *backtick*
|
|
keyword argument. When it's true, zeros are represented by ``'`'``
|
|
instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.)
|
|
|
|
|
|
calendar
|
|
--------
|
|
|
|
The class :class:`~calendar.HTMLCalendar` has new class attributes which ease
|
|
the customisation of the CSS classes in the produced HTML calendar.
|
|
(Contributed by Oz Tiram in :issue:`30095`.)
|
|
|
|
cgi
|
|
---
|
|
|
|
:func:`~cgi.parse_multipart` returns the same results as
|
|
:class:`~FieldStorage` : for non-file fields, the value associated to a key
|
|
is a list of strings, not bytes.
|
|
(Contributed by Pierre Quentel in :issue:`29979`.)
|
|
|
|
contextlib
|
|
----------
|
|
|
|
:func:`~contextlib.asynccontextmanager` and
|
|
:class:`~contextlib.AbstractAsyncContextManager` have been added. (Contributed
|
|
by Jelle Zijlstra in :issue:`29679` and :issue:`30241`.)
|
|
|
|
cProfile
|
|
--------
|
|
|
|
cProfile command line now accepts `-m module_name` as an alternative to
|
|
script path. (Contributed by Sanyam Khurana in :issue:`21862`.)
|
|
|
|
crypt
|
|
-----
|
|
|
|
Added support for the Blowfish method.
|
|
(Contributed by Serhiy Storchaka in :issue:`31664`.)
|
|
|
|
The :func:`~crypt.mksalt` function now allows to specify the number of rounds
|
|
for hashing. (Contributed by Serhiy Storchaka in :issue:`31702`.)
|
|
|
|
dis
|
|
---
|
|
|
|
The :func:`~dis.dis` function now is able to
|
|
disassemble nested code objects (the code of comprehensions, generator
|
|
expressions and nested functions, and the code used for building nested
|
|
classes). (Contributed by Serhiy Storchaka in :issue:`11822`.)
|
|
|
|
distutils
|
|
---------
|
|
|
|
README.rst is now included in the list of distutils standard READMEs and
|
|
therefore included in source distributions.
|
|
(Contributed by Ryan Gonzalez in :issue:`11913`.)
|
|
|
|
:class:`distutils.core.setup` now warns if the ``classifiers``, ``keywords``
|
|
and ``platforms`` fields are not specified as a list or a string.
|
|
(Contributed by Berker Peksag in :issue:`19610`.)
|
|
|
|
http.client
|
|
-----------
|
|
|
|
Add Configurable *blocksize* to ``HTTPConnection`` and
|
|
``HTTPSConnection`` for improved upload throughput.
|
|
(Contributed by Nir Soffer in :issue:`31945`.)
|
|
|
|
http.server
|
|
-----------
|
|
|
|
:class:`~http.server.SimpleHTTPRequestHandler` supports the HTTP
|
|
If-Modified-Since header. The server returns the 304 response status if the
|
|
target file was not modified after the time specified in the header.
|
|
(Contributed by Pierre Quentel in :issue:`29654`.)
|
|
|
|
Add the parameter ``directory`` to the :class:`~http.server.SimpleHTTPRequestHandler`
|
|
and the ``--directory`` to the command line of the module :mod:`~http.server`.
|
|
With this parameter, the server serves the specified directory, by default it uses the current working directory.
|
|
(Contributed by Stéphane Wirtel and Julien Palard in :issue:`28707`.)
|
|
|
|
locale
|
|
------
|
|
|
|
Added another argument *monetary* in :meth:`format_string` of :mod:`locale`.
|
|
If *monetary* is true, the conversion uses monetary thousands separator and
|
|
grouping strings. (Contributed by Garvit in :issue:`10379`.)
|
|
|
|
The :func:`locale.getpreferredencoding` function now always returns ``'UTF-8'``
|
|
on Android or in the UTF-8 mode (:option:`-X` ``utf8`` option), the locale and
|
|
the *do_setlocale* argument are ignored.
|
|
|
|
math
|
|
----
|
|
|
|
New :func:`~math.remainder` function, implementing the IEEE 754-style remainder
|
|
operation. (Contributed by Mark Dickinson in :issue:`29962`.)
|
|
|
|
os
|
|
--
|
|
|
|
Added support for :class:`bytes` paths in :func:`~os.fwalk`. (Contributed by
|
|
Serhiy Storchaka in :issue:`28682`.)
|
|
|
|
Added support for :ref:`file descriptors <path_fd>` in :func:`~os.scandir`
|
|
on Unix. (Contributed by Serhiy Storchaka in :issue:`25996`.)
|
|
|
|
New function :func:`os.register_at_fork` allows registering Python callbacks
|
|
to be executed on a process fork. (Contributed by Antoine Pitrou in
|
|
:issue:`16500`.)
|
|
|
|
pdb
|
|
---
|
|
|
|
:func:`~pdb.set_trace` now takes an optional *header* keyword-only
|
|
argument. If given, this is printed to the console just before debugging
|
|
begins. (Contributed by Barry Warsaw in :issue:`31389`.)
|
|
|
|
re
|
|
--
|
|
|
|
The flags :const:`re.ASCII`, :const:`re.LOCALE` and :const:`re.UNICODE`
|
|
can be set within the scope of a group.
|
|
(Contributed by Serhiy Storchaka in :issue:`31690`.)
|
|
|
|
:func:`re.split` now supports splitting on a pattern like ``r'\b'``,
|
|
``'^$'`` or ``(?=-)`` that matches an empty string.
|
|
(Contributed by Serhiy Storchaka in :issue:`25054`.)
|
|
|
|
string
|
|
------
|
|
|
|
:class:`string.Template` now lets you to optionally modify the regular
|
|
expression pattern for braced placeholders and non-braced placeholders
|
|
separately. (Contributed by Barry Warsaw in :issue:`1198569`.)
|
|
|
|
sys
|
|
---
|
|
|
|
Added :attr:`sys.flags.dev_mode` flag for the new development mode.
|
|
|
|
time
|
|
----
|
|
|
|
The :pep:`564` added six new functions with nanosecond resolution:
|
|
|
|
* :func:`time.clock_gettime_ns`
|
|
* :func:`time.clock_settime_ns`
|
|
* :func:`time.monotonic_ns`
|
|
* :func:`time.perf_counter_ns`
|
|
* :func:`time.process_time_ns`
|
|
* :func:`time.time_ns`
|
|
|
|
Add new clock identifiers:
|
|
|
|
* :data:`time.CLOCK_BOOTTIME` (Linux): Identical to
|
|
:data:`time.CLOCK_MONOTONIC`, except it also includes any time that the
|
|
system is suspended.
|
|
* :data:`time.CLOCK_PROF` (FreeBSD, NetBSD and OpenBSD): High-resolution
|
|
per-process timer from the CPU.
|
|
* :data:`time.CLOCK_UPTIME` (FreeBSD, OpenBSD): Time whose absolute value is
|
|
the time the system has been running and not suspended, providing accurate
|
|
uptime measurement, both absolute and interval.
|
|
|
|
Added functions :func:`time.thread_time` and :func:`time.thread_time_ns`
|
|
to get per-thread CPU time measurements.
|
|
(Contributed by Antoine Pitrou in :issue:`32025`.)
|
|
|
|
|
|
unittest
|
|
--------
|
|
Added new command-line option ``-k`` to filter tests to run with a substring or
|
|
Unix shell-like pattern. For example, ``python -m unittest -k foo`` runs the
|
|
tests ``foo_tests.SomeTest.test_something``, ``bar_tests.SomeTest.test_foo``,
|
|
but not ``bar_tests.FooTest.test_something``.
|
|
|
|
|
|
unittest.mock
|
|
-------------
|
|
|
|
The :const:`~unittest.mock.sentinel` attributes now preserve their identity
|
|
when they are :mod:`copied <copy>` or :mod:`pickled <pickle>`. (Contributed by
|
|
Serhiy Storchaka in :issue:`20804`.)
|
|
|
|
New function :const:`~unittest.mock.seal` will disable the creation of mock
|
|
children by preventing to get or set any new attribute on the sealed mock.
|
|
The sealing process is performed recursively. (Contributed by Mario Corchero
|
|
in :issue:`30541`.)
|
|
|
|
xmlrpc.server
|
|
-------------
|
|
|
|
:meth:`register_function` of :class:`xmlrpc.server.SimpleXMLRPCDispatcher` and
|
|
its subclasses can be used as a decorator. (Contributed by Xiang Zhang in
|
|
:issue:`7769`.)
|
|
|
|
unicodedata
|
|
-----------
|
|
|
|
The internal :mod:`unicodedata` database has been upgraded to use `Unicode 10
|
|
<http://www.unicode.org/versions/Unicode10.0.0/>`_. (Contributed by Benjamin
|
|
Peterson.)
|
|
|
|
urllib.parse
|
|
------------
|
|
|
|
:func:`urllib.parse.quote` has been updated from RFC 2396 to RFC 3986,
|
|
adding `~` to the set of characters that is never quoted by default.
|
|
(Contributed by Christian Theune and Ratnadeep Debnath in :issue:`16285`.)
|
|
|
|
uu
|
|
--
|
|
|
|
Function :func:`~uu.encode` now accepts an optional *backtick*
|
|
keyword argument. When it's true, zeros are represented by ``'`'``
|
|
instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.)
|
|
|
|
warnings
|
|
--------
|
|
|
|
The initialization of the default warnings filters has changed as follows:
|
|
|
|
* warnings enabled via command line options (including those for :option:`-b`
|
|
and the new CPython-specific ``-X dev`` option) are always passed to the
|
|
warnings machinery via the ``sys.warnoptions`` attribute.
|
|
* warnings filters enabled via the command line or the environment now have the
|
|
following precedence order:
|
|
|
|
* the ``BytesWarning`` filter for :option:`-b` (or ``-bb``)
|
|
* any filters specified with :option:`-W`
|
|
* any filters specified with :envvar:`PYTHONWARNINGS`
|
|
* any other CPython specific filters (e.g. the ``default`` filter added
|
|
for the new ``-X dev`` mode)
|
|
* any implicit filters defined directly by the warnings machinery
|
|
* in CPython debug builds, all warnings are now displayed by default (the
|
|
implicit filter list is empty)
|
|
|
|
(Contributed by Nick Coghlan and Victor Stinner in :issue:`20361`,
|
|
:issue:`32043`, and :issue:`32230`)
|
|
|
|
xml.etree
|
|
---------
|
|
|
|
:ref:`ElementPath <elementtree-xpath>` predicates in the :meth:`find`
|
|
methods can now compare text of the current node with ``[. = "text"]``,
|
|
not only text in children. Predicates also allow adding spaces for
|
|
better readability. (Contributed by Stefan Behnel in :issue:`31648`.)
|
|
|
|
zipapp
|
|
------
|
|
|
|
Function :func:`zipapp.create_archive` now accepts an optional *filter*
|
|
argument to allow the user to select which files should be included in the
|
|
archive, and an optional *compressed* argument to generate a compressed
|
|
archive.
|
|
|
|
A command line option ``--compress`` has also been added to support
|
|
compression.
|
|
|
|
|
|
Optimizations
|
|
=============
|
|
|
|
* Added two new opcodes: ``LOAD_METHOD`` and ``CALL_METHOD`` to avoid
|
|
instantiation of bound method objects for method calls, which results
|
|
in method calls being faster up to 20%. (Contributed by Yury Selivanov and
|
|
INADA Naoki in :issue:`26110`.)
|
|
|
|
* Searching some unlucky Unicode characters (like Ukrainian capital "Є")
|
|
in a string was up to 25 times slower than searching other characters.
|
|
Now it is slower only by 3 times in the worst case.
|
|
(Contributed by Serhiy Storchaka in :issue:`24821`.)
|
|
|
|
* Fast implementation from standard C library is now used for functions
|
|
:func:`~math.erf` and :func:`~math.erfc` in the :mod:`math` module.
|
|
(Contributed by Serhiy Storchaka in :issue:`26121`.)
|
|
|
|
* The :func:`os.fwalk` function has been sped up by 2 times. This was done
|
|
using the :func:`os.scandir` function.
|
|
(Contributed by Serhiy Storchaka in :issue:`25996`.)
|
|
|
|
* The :func:`shutil.rmtree` function has been sped up to 20--40%.
|
|
This was done using the :func:`os.scandir` function.
|
|
(Contributed by Serhiy Storchaka in :issue:`28564`.)
|
|
|
|
* Optimized case-insensitive matching and searching of :mod:`regular
|
|
expressions <re>`. Searching some patterns can now be up to 20 times faster.
|
|
(Contributed by Serhiy Storchaka in :issue:`30285`.)
|
|
|
|
* :func:`re.compile` now converts ``flags`` parameter to int object if
|
|
it is ``RegexFlag``. It is now as fast as Python 3.5, and faster than
|
|
Python 3.6 by about 10% depending on the pattern.
|
|
(Contributed by INADA Naoki in :issue:`31671`.)
|
|
|
|
* :meth:`selectors.EpollSelector.modify`, :meth:`selectors.PollSelector.modify`
|
|
and :meth:`selectors.DevpollSelector.modify` may be around 10% faster under
|
|
heavy loads. (Contributed by Giampaolo Rodola' in :issue:`30014`)
|
|
|
|
* Constant folding is moved from peephole optimizer to new AST optimizer.
|
|
(Contributed by Eugene Toder and INADA Naoki in :issue:`29469`)
|
|
|
|
Build and C API Changes
|
|
=======================
|
|
|
|
* A full copy of libffi is no longer bundled for use when building the
|
|
:mod:`_ctypes <ctypes>` module on non-OSX UNIX platforms. An installed copy
|
|
of libffi is now required when building ``_ctypes`` on such platforms.
|
|
Contributed by Zachary Ware in :issue:`27979`.
|
|
|
|
* The fields :c:member:`name` and :c:member:`doc` of structures
|
|
:c:type:`PyMemberDef`, :c:type:`PyGetSetDef`,
|
|
:c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`,
|
|
and :c:type:`wrapperbase` are now of type ``const char *`` rather of
|
|
``char *``. (Contributed by Serhiy Storchaka in :issue:`28761`.)
|
|
|
|
* The result of :c:func:`PyUnicode_AsUTF8AndSize` and :c:func:`PyUnicode_AsUTF8`
|
|
is now of type ``const char *`` rather of ``char *``. (Contributed by Serhiy
|
|
Storchaka in :issue:`28769`.)
|
|
|
|
* The result of :c:func:`PyMapping_Keys`, :c:func:`PyMapping_Values` and
|
|
:c:func:`PyMapping_Items` is now always a list, rather than a list or a
|
|
tuple. (Contributed by Oren Milman in :issue:`28280`.)
|
|
|
|
* Added functions :c:func:`PySlice_Unpack` and :c:func:`PySlice_AdjustIndices`.
|
|
(Contributed by Serhiy Storchaka in :issue:`27867`.)
|
|
|
|
* :c:func:`PyOS_AfterFork` is deprecated in favour of the new functions
|
|
:c:func:`PyOS_BeforeFork`, :c:func:`PyOS_AfterFork_Parent` and
|
|
:c:func:`PyOS_AfterFork_Child`. (Contributed by Antoine Pitrou in
|
|
:issue:`16500`.)
|
|
|
|
* The Windows build process no longer depends on Subversion to pull in external
|
|
sources, a Python script is used to download zipfiles from GitHub instead.
|
|
If Python 3.6 is not found on the system (via ``py -3.6``), NuGet is used to
|
|
download a copy of 32-bit Python for this purpose. (Contributed by Zachary
|
|
Ware in :issue:`30450`.)
|
|
|
|
* The ``PyExc_RecursionErrorInst`` singleton that was part of the public API
|
|
has been removed as its members being never cleared may cause a segfault
|
|
during finalization of the interpreter. Contributed by Xavier de Gaye in
|
|
:issue:`22898` and :issue:`30697`.
|
|
|
|
* Support for building ``--without-threads`` is removed.
|
|
(Contributed by Antoine Pitrou in :issue:`31370`.).
|
|
|
|
|
|
Other CPython Implementation Changes
|
|
====================================
|
|
|
|
* Trace hooks may now opt out of receiving ``line`` events from the interpreter
|
|
by setting the new ``f_trace_lines`` attribute to :const:`False` on the frame
|
|
being traced. (Contributed by Nick Coghlan in :issue:`31344`.)
|
|
|
|
* Trace hooks may now opt in to receiving ``opcode`` events from the interpreter
|
|
by setting the new ``f_trace_opcodes`` attribute to :const:`True` on the frame
|
|
being traced. (Contributed by Nick Coghlan in :issue:`31344`.)
|
|
|
|
|
|
Deprecated
|
|
==========
|
|
|
|
* Yield expressions (both ``yield`` and ``yield from`` clauses) are now deprecated
|
|
in comprehensions and generator expressions (aside from the iterable expression
|
|
in the leftmost :keyword:`for` clause). This ensures that comprehensions
|
|
always immediately return a container of the appropriate type (rather than
|
|
potentially returning a :term:`generator iterator` object), while generator
|
|
expressions won't attempt to interleave their implicit output with the output
|
|
from any explicit yield expressions.
|
|
|
|
In Python 3.7, such expressions emit :exc:`DeprecationWarning` when compiled,
|
|
in Python 3.8+ they will emit :exc:`SyntaxError`. (Contributed by Serhiy
|
|
Storchaka in :issue:`10544`.)
|
|
|
|
- Function :c:func:`PySlice_GetIndicesEx` is deprecated and replaced with
|
|
a macro if ``Py_LIMITED_API`` is not set or set to the value between
|
|
``0x03050400`` and ``0x03060000`` (not including) or ``0x03060100`` or
|
|
higher. (Contributed by Serhiy Storchaka in :issue:`27867`.)
|
|
|
|
- Deprecated :meth:`format` from :mod:`locale`, use the :meth:`format_string`
|
|
instead. (Contributed by Garvit in :issue:`10379`.)
|
|
|
|
- Methods
|
|
:meth:`MetaPathFinder.find_module() <importlib.abc.MetaPathFinder.find_module>`
|
|
(replaced by
|
|
:meth:`MetaPathFinder.find_spec() <importlib.abc.MetaPathFinder.find_spec>`
|
|
) and
|
|
:meth:`PathEntryFinder.find_loader() <importlib.abc.PathEntryFinder.find_loader>`
|
|
(replaced by
|
|
:meth:`PathEntryFinder.find_spec() <importlib.abc.PathEntryFinder.find_spec>`)
|
|
both deprecated in Python 3.4 now emit :exc:`DeprecationWarning`. (Contributed
|
|
by Matthias Bussonnier in :issue:`29576`)
|
|
|
|
- Using non-integer value for selecting a plural form in :mod:`gettext` is
|
|
now deprecated. It never correctly worked. (Contributed by Serhiy Storchaka
|
|
in :issue:`28692`.)
|
|
|
|
- The :mod:`macpath` is now deprecated and will be removed in Python 3.8.
|
|
|
|
|
|
Changes in the C API
|
|
--------------------
|
|
|
|
- The type of results of :c:func:`PyThread_start_new_thread` and
|
|
:c:func:`PyThread_get_thread_ident`, and the *id* parameter of
|
|
:c:func:`PyThreadState_SetAsyncExc` changed from :c:type:`long` to
|
|
:c:type:`unsigned long`.
|
|
(Contributed by Serhiy Storchaka in :issue:`6532`.)
|
|
|
|
- :c:func:`PyUnicode_AsWideCharString` now raises a :exc:`ValueError` if the
|
|
second argument is *NULL* and the :c:type:`wchar_t*` string contains null
|
|
characters. (Contributed by Serhiy Storchaka in :issue:`30708`.)
|
|
|
|
|
|
Windows Only
|
|
------------
|
|
- The python launcher, (py.exe), can accept 32 & 64 bit specifiers **without**
|
|
having to specify a minor version as well. So ``py -3-32`` and ``py -3-64``
|
|
become valid as well as ``py -3.7-32``, also the -*m*-64 and -*m.n*-64 forms
|
|
are now accepted to force 64 bit python even if 32 bit would have otherwise
|
|
been used. If the specified version is not available py.exe will error exit.
|
|
(Contributed by Steve Barnes in :issue:`30291`.)
|
|
|
|
- The launcher can be run as "py -0" to produce a list of the installed pythons,
|
|
*with default marked with an asterix*. Running "py -0p" will include the paths.
|
|
If py is run with a version specifier that cannot be matched it will also print
|
|
the *short form* list of available specifiers.
|
|
(Contributed by Steve Barnes in :issue:`30362`.)
|
|
|
|
Removed
|
|
=======
|
|
|
|
API and Feature Removals
|
|
------------------------
|
|
|
|
* The ``os.stat_float_times()`` function has been removed. It was introduced in
|
|
Python 2.3 for backward compatibility with Python 2.2, and was deprecated
|
|
since Python 3.1.
|
|
|
|
* Unknown escapes consisting of ``'\'`` and an ASCII letter in replacement
|
|
templates for :func:`re.sub` were deprecated in Python 3.5, and will now
|
|
cause an error.
|
|
|
|
* Removed support of the *exclude* argument in :meth:`tarfile.TarFile.add`.
|
|
It was deprecated in Python 2.7 and 3.2. Use the *filter* argument instead.
|
|
|
|
* The ``splitunc()`` function in the :mod:`ntpath` module was deprecated in
|
|
Python 3.1, and has now been removed. Use the :func:`~os.path.splitdrive`
|
|
function instead.
|
|
|
|
* :func:`collections.namedtuple` no longer supports the *verbose* parameter
|
|
or ``_source`` attribute which showed the generated source code for the
|
|
named tuple class. This was part of an optimization designed to speed-up
|
|
class creation. (Contributed by Jelle Zijlstra with further improvements
|
|
by INADA Naoki, Serhiy Storchaka, and Raymond Hettinger in :issue:`28638`.)
|
|
|
|
* Functions :func:`bool`, :func:`float`, :func:`list` and :func:`tuple` no
|
|
longer take keyword arguments. The first argument of :func:`int` can now
|
|
be passed only as positional argument.
|
|
|
|
* Removed previously deprecated in Python 2.4 classes ``Plist``, ``Dict`` and
|
|
``_InternalDict`` in the :mod:`plistlib` module. Dict values in the result
|
|
of functions :func:`~plistlib.readPlist` and
|
|
:func:`~plistlib.readPlistFromBytes` are now normal dicts. You no longer
|
|
can use attribute access to access items of these dictionaries.
|
|
|
|
|
|
Porting to Python 3.7
|
|
=====================
|
|
|
|
This section lists previously described changes and other bugfixes
|
|
that may require changes to your code.
|
|
|
|
|
|
Changes in Python behavior
|
|
--------------------------
|
|
|
|
* Due to an oversight, earlier Python versions erroneously accepted the
|
|
following syntax::
|
|
|
|
f(1 for x in [1],)
|
|
|
|
class C(1 for x in [1]):
|
|
pass
|
|
|
|
Python 3.7 now correctly raises a :exc:`SyntaxError`, as a generator
|
|
expression always needs to be directly inside a set of parentheses
|
|
and cannot have a comma on either side, and the duplication of the
|
|
parentheses can be omitted only on calls.
|
|
(Contributed by Serhiy Storchaka in :issue:`32012` and :issue:`32023`.)
|
|
|
|
|
|
Changes in the Python API
|
|
-------------------------
|
|
|
|
* The ``asyncio.windows_utils.socketpair()`` function has been
|
|
removed: use directly :func:`socket.socketpair` which is available on all
|
|
platforms since Python 3.5 (before, it wasn't available on Windows).
|
|
``asyncio.windows_utils.socketpair()`` was just an alias to
|
|
``socket.socketpair`` on Python 3.5 and newer.
|
|
|
|
* :mod:`asyncio`: The module doesn't export :mod:`selectors` and
|
|
:mod:`_overlapped` modules as ``asyncio.selectors`` and
|
|
``asyncio._overlapped``. Replace ``from asyncio import selectors`` with
|
|
``import selectors`` for example.
|
|
|
|
* :meth:`pkgutil.walk_packages` now raises ValueError if *path* is a string.
|
|
Previously an empty list was returned. (Contributed by Sanyam Khurana in
|
|
:issue:`24744`.)
|
|
|
|
* A format string argument for :meth:`string.Formatter.format`
|
|
is now :ref:`positional-only <positional-only_parameter>`.
|
|
Passing it as a keyword argument was deprecated in Python 3.5. (Contributed
|
|
by Serhiy Storchaka in :issue:`29193`.)
|
|
|
|
* Attributes :attr:`~http.cookies.Morsel.key`,
|
|
:attr:`~http.cookies.Morsel.value` and
|
|
:attr:`~http.cookies.Morsel.coded_value` of class
|
|
:class:`http.cookies.Morsel` are now read-only.
|
|
Assigning to them was deprecated in Python 3.5.
|
|
Use the :meth:`~http.cookies.Morsel.set` method for setting them.
|
|
(Contributed by Serhiy Storchaka in :issue:`29192`.)
|
|
|
|
* ``Module``, ``FunctionDef``, ``AsyncFunctionDef``, and
|
|
``ClassDef`` AST nodes now have a new ``docstring`` field.
|
|
The first statement in their body is not considered as a docstring
|
|
anymore. ``co_firstlineno`` and ``co_lnotab`` of code object for class
|
|
and module are affected by this change. (Contributed by INADA Naoki and
|
|
Eugene Toder in :issue:`29463`.)
|
|
|
|
* The *mode* argument of :func:`os.makedirs` no longer affects the file
|
|
permission bits of newly-created intermediate-level directories.
|
|
To set their file permission bits you can set the umask before invoking
|
|
``makedirs()``.
|
|
(Contributed by Serhiy Storchaka in :issue:`19930`.)
|
|
|
|
* The :attr:`struct.Struct.format` type is now :class:`str` instead of
|
|
:class:`bytes`. (Contributed by Victor Stinner in :issue:`21071`.)
|
|
|
|
* Due to internal changes in :mod:`socket` you won't be able to
|
|
:func:`socket.fromshare` a socket :func:`~socket.socket.share`-ed in older
|
|
Python versions.
|
|
|
|
* ``repr`` for :class:`datetime.timedelta` has changed to include keyword arguments
|
|
in the output. (Contributed by Utkarsh Upadhyay in :issue:`30302`.)
|
|
|
|
* Because :func:`shutil.rmtree` is now implemented using the :func:`os.scandir`
|
|
function, the user specified handler *onerror* is now called with the first
|
|
argument ``os.scandir`` instead of ``os.listdir`` when listing the direcory
|
|
is failed.
|
|
|
|
* Support of nested sets and set operations in regular expressions as in
|
|
`Unicode Technical Standard #18`_ might be added in the future. This would
|
|
change the syntax, so to facilitate this change a :exc:`FutureWarning` will
|
|
be raised in ambiguous cases for the time being.
|
|
That include sets starting with a literal ``'['`` or containing literal
|
|
character sequences ``'--'``, ``'&&'``, ``'~~'``, and ``'||'``. To
|
|
avoid a warning escape them with a backslash.
|
|
(Contributed by Serhiy Storchaka in :issue:`30349`.)
|
|
|
|
* The result of splitting a string on a :mod:`regular expression <re>`
|
|
that could match an empty string has been changed. For example
|
|
splitting on ``r'\s*'`` will now split not only on whitespaces as it
|
|
did previously, but also between any pair of non-whitespace
|
|
characters. The previous behavior can be restored by changing the pattern
|
|
to ``r'\s+'``. A :exc:`FutureWarning` was emitted for such patterns since
|
|
Python 3.5.
|
|
|
|
For patterns that match both empty and non-empty strings, the result of
|
|
searching for all matches may also be changed in other cases. For example
|
|
in the string ``'a\n\n'``, the pattern ``r'(?m)^\s*?$'`` will not only
|
|
match empty strings at positions 2 and 3, but also the string ``'\n'`` at
|
|
positions 2--3. To match only blank lines, the pattern should be rewritten
|
|
as ``r'(?m)^[^\S\n]*$'``.
|
|
|
|
(Contributed by Serhiy Storchaka in :issue:`25054`.)
|
|
|
|
* :class:`tracemalloc.Traceback` frames are now sorted from oldest to most
|
|
recent to be more consistent with :mod:`traceback`.
|
|
(Contributed by Jesse Bakker in :issue:`32121`.)
|
|
|
|
.. _Unicode Technical Standard #18: https://unicode.org/reports/tr18/
|
|
|
|
|
|
Changes in the C API
|
|
--------------------
|
|
|
|
* The function :c:func:`PySlice_GetIndicesEx` is considered not safe for
|
|
resizable sequences. If the slice indices are not instances of :class:`int`,
|
|
but objects that implement the :meth:`!__index__` method, the sequence can be
|
|
resized after passing its length to :c:func:`!PySlice_GetIndicesEx`. This
|
|
can lead to returning indices out of the length of the sequence. For
|
|
avoiding possible problems use new functions :c:func:`PySlice_Unpack` and
|
|
:c:func:`PySlice_AdjustIndices`.
|
|
(Contributed by Serhiy Storchaka in :issue:`27867`.)
|
|
|
|
|
|
CPython bytecode changes
|
|
------------------------
|
|
|
|
* Added two new opcodes: :opcode:`LOAD_METHOD` and :opcode:`CALL_METHOD`.
|
|
(Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.)
|
|
|
|
|
|
Other CPython implementation changes
|
|
------------------------------------
|
|
|
|
* In preparation for potential future changes to the public CPython runtime
|
|
initialization API (see :pep:`432` for details), CPython's internal startup
|
|
and configuration management logic has been significantly refactored. While
|
|
these updates are intended to be entirely transparent to both embedding
|
|
applications and users of the regular CPython CLI, they're being mentioned
|
|
here as the refactoring changes the internal order of various operations
|
|
during interpreter startup, and hence may uncover previously latent defects,
|
|
either in embedding applications, or in CPython itself.
|
|
(Contributed by Nick Coghlan and Eric Snow as part of :issue:`22257`.)
|
|
|
|
* Due to changes in the way the default warnings filters are configured,
|
|
setting ``Py_BytesWarningFlag`` to a value greater than one is no longer
|
|
sufficient to both emit ``BytesWarning`` messages and have them converted
|
|
to exceptions. Instead, the flag must be set (to cause the warnings to be
|
|
emitted in the first place), and an explicit ``error::BytesWarning``
|
|
warnings filter added to convert them to exceptions.
|
|
|
|
Documentation
|
|
=============
|
|
|
|
.. _whatsnew37-pep545:
|
|
|
|
PEP 545: Python Documentation Translations
|
|
------------------------------------------
|
|
|
|
:pep:`545` describes the process to translate Python documentation,
|
|
and two translations have been added:
|
|
|
|
- Japanese: https://docs.python.org/ja/ and associated GitHub
|
|
repository: https://github.com/python/python-docs-ja
|
|
|
|
- French: https://docs.python.org/fr/ and associated GitHub
|
|
repository: https://github.com/python/python-docs-fr
|
|
|
|
(Contributed by Julien Palard, Inada Naoki, and Victor Stinner in
|
|
:issue:`26546`.)
|