1076 lines
38 KiB
ReStructuredText
1076 lines
38 KiB
ReStructuredText
****************************
|
|
What's New In Python 3.5
|
|
****************************
|
|
|
|
:Release: |release|
|
|
:Date: |today|
|
|
|
|
.. Rules for maintenance:
|
|
|
|
* Anyone can add text to this document. Do not spend very much time
|
|
on the wording of your changes, because your text will probably
|
|
get rewritten to some degree.
|
|
|
|
* The maintainer will go through Misc/NEWS periodically and add
|
|
changes; it's therefore more important to add your changes to
|
|
Misc/NEWS than to this file.
|
|
|
|
* This is not a complete list of every single change; completeness
|
|
is the purpose of Misc/NEWS. Some changes I consider too small
|
|
or esoteric to include. If such a change is added to the text,
|
|
I'll just remove it. (This is another reason you shouldn't spend
|
|
too much time on writing your addition.)
|
|
|
|
* If you want to draw your new text to the attention of the
|
|
maintainer, add 'XXX' to the beginning of the paragraph or
|
|
section.
|
|
|
|
* It's OK to just add a fragmentary note about a change. For
|
|
example: "XXX Describe the transmogrify() function added to the
|
|
socket module." The maintainer will research the change and
|
|
write the necessary text.
|
|
|
|
* You can comment out your additions if you like, but it's not
|
|
necessary (especially when a final release is some months away).
|
|
|
|
* Credit the author of a patch or bugfix. Just the name is
|
|
sufficient; the e-mail address isn't necessary.
|
|
|
|
* It's helpful to add the bug/patch number as a comment:
|
|
|
|
XXX Describe the transmogrify() function added to the socket
|
|
module.
|
|
(Contributed by P.Y. Developer in :issue:`12345`.)
|
|
|
|
This saves the maintainer the effort of going through the Mercurial log
|
|
when researching a change.
|
|
|
|
This article explains the new features in Python 3.5, compared to 3.4.
|
|
|
|
For full details, see the :source:`Misc/NEWS` file.
|
|
|
|
.. note::
|
|
|
|
Prerelease users should be aware that this document is currently in draft
|
|
form. It will be updated substantially as Python 3.5 moves towards release,
|
|
so it's worth checking back even after reading earlier versions.
|
|
|
|
|
|
.. seealso::
|
|
|
|
:pep:`478` - Python 3.5 Release Schedule
|
|
|
|
|
|
Summary -- Release highlights
|
|
=============================
|
|
|
|
.. This section singles out the most important changes in Python 3.5.
|
|
Brevity is key.
|
|
|
|
New syntax features:
|
|
|
|
* :pep:`465`, a new matrix multiplication operator: ``a @ b``.
|
|
* :pep:`492`, coroutines with async and await syntax.
|
|
|
|
New library modules:
|
|
|
|
* :mod:`zipapp`: :ref:`Improving Python ZIP Application Support
|
|
<whatsnew-zipapp>` (:pep:`441`).
|
|
|
|
New built-in features:
|
|
|
|
* ``bytes % args``, ``bytearray % args``: :pep:`461` - Adding ``%`` formatting
|
|
to bytes and bytearray
|
|
* ``b'\xf0\x9f\x90\x8d'.hex()``, ``bytearray(b'\xf0\x9f\x90\x8d').hex()``,
|
|
``memoryview(b'\xf0\x9f\x90\x8d').hex()``: :issue:`9951` - A ``hex`` method
|
|
has been added to bytes, bytearray, and memoryview.
|
|
|
|
Implementation improvements:
|
|
|
|
* When the ``LC_TYPE`` locale is the POSIX locale (``C`` locale),
|
|
:py:data:`sys.stdin` and :py:data:`sys.stdout` are now using the
|
|
``surrogateescape`` error handler, instead of the ``strict`` error handler
|
|
(:issue:`19977`).
|
|
|
|
* :pep:`488`, the elimination of ``.pyo`` files.
|
|
* :pep:`489`, multi-phase initialization of extension modules.
|
|
|
|
Significantly Improved Library Modules:
|
|
|
|
* :class:`collections.OrderedDict` is now implemented in C, which improves
|
|
its performance between 4x to 100x times. Contributed by Eric Snow in
|
|
:issue:`16991`.
|
|
|
|
* You may now pass bytes to the :mod:`tempfile` module's APIs and it will
|
|
return the temporary pathname as bytes instead of str. It also accepts
|
|
a value of ``None`` on parameters where only str was accepted in the past to
|
|
do the right thing based on the types of the other inputs. Two functions,
|
|
:func:`gettempdirb` and :func:`gettempprefixb`, have been added to go along
|
|
with this. This behavior matches that of the :mod:`os` APIs.
|
|
|
|
Security improvements:
|
|
|
|
* None yet.
|
|
|
|
Please read on for a comprehensive list of user-facing changes.
|
|
|
|
|
|
.. PEP-sized items next.
|
|
|
|
.. _pep-4XX:
|
|
|
|
.. PEP 4XX: Virtual Environments
|
|
.. =============================
|
|
|
|
|
|
.. (Implemented by Foo Bar.)
|
|
|
|
.. .. seealso::
|
|
|
|
:pep:`4XX` - Python Virtual Environments
|
|
PEP written by Carl Meyer
|
|
|
|
|
|
PEP 492 - Coroutines with async and await syntax
|
|
------------------------------------------------
|
|
|
|
The PEP added dedicated syntax for declaring :term:`coroutines <coroutine>`,
|
|
:keyword:`await` expressions, new asynchronous :keyword:`async for`
|
|
and :keyword:`async with` statements.
|
|
|
|
Example::
|
|
|
|
async def read_data(db):
|
|
async with db.transaction():
|
|
data = await db.fetch('SELECT ...')
|
|
|
|
PEP written and implemented by Yury Selivanov.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`492` -- Coroutines with async and await syntax
|
|
|
|
|
|
PEP 461 - Adding formatting to bytes and bytearray
|
|
--------------------------------------------------
|
|
|
|
This PEP proposes adding % formatting operations similar to Python 2's ``str``
|
|
type to :class:`bytes` and :class:`bytearray`.
|
|
|
|
Examples::
|
|
|
|
>>> b'Hello %s!' % b'World'
|
|
b'Hello World!'
|
|
>>> b'x=%i y=%f' % (1, 2.5)
|
|
b'x=1 y=2.500000'
|
|
|
|
Unicode is not allowed for ``%s``, but it is accepted by ``%a`` (equivalent of
|
|
``repr(obj).encode('ascii', 'backslashreplace')``)::
|
|
|
|
>>> b'Hello %s!' % 'World'
|
|
Traceback (most recent call last):
|
|
File "<stdin>", line 1, in <module>
|
|
TypeError: %b requires bytes, or an object that implements __bytes__, not 'str'
|
|
>>> b'price: %a' % '10€'
|
|
b"price: '10\\u20ac'"
|
|
|
|
.. seealso::
|
|
|
|
:pep:`461` -- Adding % formatting to bytes and bytearray
|
|
|
|
|
|
PEP 465 - A dedicated infix operator for matrix multiplication
|
|
--------------------------------------------------------------
|
|
|
|
This PEP proposes a new binary operator to be used for matrix multiplication,
|
|
called ``@``. (Mnemonic: ``@`` is ``*`` for mATrices.)
|
|
|
|
.. seealso::
|
|
|
|
:pep:`465` -- A dedicated infix operator for matrix multiplication
|
|
|
|
|
|
PEP 471 - os.scandir() function -- a better and faster directory iterator
|
|
-------------------------------------------------------------------------
|
|
|
|
:pep:`471` adds a new directory iteration function, :func:`os.scandir`,
|
|
to the standard library. Additionally, :func:`os.walk` is now
|
|
implemented using :func:`os.scandir`, which speeds it up by 3-5 times
|
|
on POSIX systems and by 7-20 times on Windows systems.
|
|
|
|
PEP and implementation written by Ben Hoyt with the help of Victor Stinner.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`471` -- os.scandir() function -- a better and faster directory
|
|
iterator
|
|
|
|
|
|
PEP 475: Retry system calls failing with EINTR
|
|
----------------------------------------------
|
|
|
|
:pep:`475` adds support for automatic retry of system calls failing with
|
|
:py:data:`~errno.EINTR`: this means that user code doesn't have to deal with
|
|
EINTR or :exc:`InterruptedError` manually, and should make it more robust
|
|
against asynchronous signal reception.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`475` -- Retry system calls failing with EINTR
|
|
|
|
|
|
PEP 479: Change StopIteration handling inside generators
|
|
--------------------------------------------------------
|
|
|
|
:pep:`479` changes the behavior of generators: when a :exc:`StopIteration`
|
|
exception is raised inside a generator, it is replaced with a
|
|
:exc:`RuntimeError`. To enable the feature a ``__future__`` import should
|
|
be used::
|
|
|
|
from __future__ import generator_stop
|
|
|
|
Without a ``__future__`` import, a :exc:`PendingDeprecationWarning` will be
|
|
raised.
|
|
|
|
PEP written by Chris Angelico and Guido van Rossum. Implemented by
|
|
Chris Angelico, Yury Selivanov and Nick Coghlan.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`479` -- Change StopIteration handling inside generators
|
|
|
|
|
|
PEP 486: Make the Python Launcher aware of virtual environments
|
|
---------------------------------------------------------------
|
|
|
|
:pep:`486` makes the Windows launcher (see :pep:`397`) aware of an active
|
|
virtual environment. When the default interpreter would be used and the
|
|
``VIRTUAL_ENV`` environment variable is set, the interpreter in the virtual
|
|
environment will be used.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`486` -- Make the Python Launcher aware of virtual environments
|
|
|
|
|
|
PEP 488: Elimination of PYO files
|
|
---------------------------------
|
|
|
|
:pep:`488` does away with the concept of ``.pyo`` files. This means that
|
|
``.pyc`` files represent both unoptimized and optimized bytecode. To prevent the
|
|
need to constantly regenerate bytecode files, ``.pyc`` files now have an
|
|
optional ``opt-`` tag in their name when the bytecode is optimized. This has the
|
|
side-effect of no more bytecode file name clashes when running under either
|
|
``-O`` or ``-OO``. Consequently, bytecode files generated from ``-O``, and
|
|
``-OO`` may now exist simultaneously. :func:`importlib.util.cache_from_source`
|
|
has an updated API to help with this change.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`488` -- Elimination of PYO files
|
|
|
|
|
|
PEP 489: Multi-phase extension module initialization
|
|
----------------------------------------------------
|
|
|
|
:pep:`489` updates extension module initialization to take advantage of the
|
|
two step module loading mechanism introduced by :pep:`451` in Python 3.4.
|
|
|
|
This change brings the import semantics of extension modules that opt-in to
|
|
using the new mechanism much closer to those of Python source and bytecode
|
|
modules, including the ability to any valid identifier as a module name,
|
|
rather than being restricted to ASCII.
|
|
|
|
.. seealso::
|
|
|
|
:pep:`488` -- Multi-phase extension module initialization
|
|
|
|
Other Language Changes
|
|
======================
|
|
|
|
Some smaller changes made to the core Python language are:
|
|
|
|
* Added the ``'namereplace'`` error handlers. The ``'backslashreplace'``
|
|
error handlers now works with decoding and translating.
|
|
(Contributed by Serhiy Storchaka in :issue:`19676` and :issue:`22286`.)
|
|
|
|
* The :option:`-b` option now affects comparisons of :class:`bytes` with
|
|
:class:`int`. (Contributed by Serhiy Storchaka in :issue:`23681`)
|
|
|
|
* New Kazakh :ref:`codec <standard-encodings>` ``kz1048``. (Contributed by
|
|
Serhiy Storchaka in :issue:`22682`.)
|
|
|
|
* Property docstrings are now writable. This is especially useful for
|
|
:func:`collections.namedtuple` docstrings.
|
|
(Contributed by Berker Peksag in :issue:`24064`.)
|
|
|
|
* New Tajik :ref:`codec <standard-encodings>` ``koi8_t``. (Contributed by
|
|
Serhiy Storchaka in :issue:`22681`.)
|
|
|
|
|
|
New Modules
|
|
===========
|
|
|
|
.. _whatsnew-zipapp:
|
|
|
|
zipapp
|
|
------
|
|
|
|
The new :mod:`zipapp` module (specified in :pep:`441`) provides an API and
|
|
command line tool for creating executable Python Zip Applications, which
|
|
were introduced in Python 2.6 in :issue:`1739468` but which were not well
|
|
publicised, either at the time or since.
|
|
|
|
With the new module, bundling your application is as simple as putting all
|
|
the files, including a ``__main__.py`` file, into a directory ``myapp``
|
|
and running::
|
|
|
|
$ python -m zipapp myapp
|
|
$ python myapp.pyz
|
|
|
|
|
|
Improved Modules
|
|
================
|
|
|
|
argparse
|
|
--------
|
|
|
|
* :class:`~argparse.ArgumentParser` now allows to disable
|
|
:ref:`abbreviated usage <prefix-matching>` of long options by setting
|
|
:ref:`allow_abbrev` to ``False``.
|
|
(Contributed by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson.)
|
|
|
|
cgi
|
|
---
|
|
|
|
* :class:`~cgi.FieldStorage` now supports the context management protocol.
|
|
(Contributed by Berker Peksag in :issue:`20289`.)
|
|
|
|
code
|
|
----
|
|
|
|
* The :func:`code.InteractiveInterpreter.showtraceback` method now prints
|
|
the full chained traceback, just like the interactive interpreter.
|
|
(Contributed by Claudiu Popa in :issue:`17442`.)
|
|
|
|
collections
|
|
-----------
|
|
|
|
* You can now update docstrings produced by :func:`collections.namedtuple`::
|
|
|
|
Point = namedtuple('Point', ['x', 'y'])
|
|
Point.__doc__ = 'ordered pair'
|
|
Point.x.__doc__ = 'abscissa'
|
|
Point.y.__doc__ = 'ordinate'
|
|
|
|
(Contributed by Berker Peksag in :issue:`24064`.)
|
|
|
|
compileall
|
|
----------
|
|
|
|
* :func:`compileall.compile_dir` and :mod:`compileall`'s command-line interface
|
|
can now do parallel bytecode compilation.
|
|
(Contributed by Claudiu Popa in :issue:`16104`.)
|
|
|
|
contextlib
|
|
----------
|
|
|
|
* The new :func:`contextlib.redirect_stderr` context manager(similar to
|
|
:func:`contextlib.redirect_stdout`) makes it easier for utility scripts to
|
|
handle inflexible APIs that write their output to :data:`sys.stderr` and
|
|
don't provide any options to redirect it.
|
|
(Contributed by Berker Peksag in :issue:`22389`.)
|
|
|
|
curses
|
|
------
|
|
* The new :func:`curses.update_lines_cols` function updates the variables
|
|
:envvar:`curses.LINES` and :envvar:`curses.COLS`.
|
|
|
|
difflib
|
|
-------
|
|
|
|
* The charset of the HTML document generated by :meth:`difflib.HtmlDiff.make_file`
|
|
can now be customized by using *charset* keyword-only parameter. The default
|
|
charset of HTML document changed from ``'ISO-8859-1'`` to ``'utf-8'``.
|
|
(Contributed by Berker Peksag in :issue:`2052`.)
|
|
|
|
* It's now possible to compare lists of byte strings with
|
|
:func:`difflib.diff_bytes` (fixes a regression from Python 2).
|
|
|
|
distutils
|
|
---------
|
|
|
|
* The ``build`` and ``build_ext`` commands now accept a ``-j``
|
|
option to enable parallel building of extension modules.
|
|
(Contributed by Antoine Pitrou in :issue:`5309`.)
|
|
|
|
* Added support for the LZMA compression.
|
|
(Contributed by Serhiy Storchaka in :issue:`16314`.)
|
|
|
|
doctest
|
|
-------
|
|
|
|
* :func:`doctest.DocTestSuite` returns an empty :class:`unittest.TestSuite` if
|
|
*module* contains no docstrings instead of raising :exc:`ValueError`.
|
|
(Contributed by Glenn Jones in :issue:`15916`.)
|
|
|
|
email
|
|
-----
|
|
|
|
* A new policy option :attr:`~email.policy.Policy.mangle_from_` controls
|
|
whether or not lines that start with "From " in email bodies are prefixed with
|
|
a '>' character by generators. The default is ``True`` for
|
|
:attr:`~email.policy.compat32` and ``False`` for all other policies.
|
|
(Contributed by Milan Oberkirch in :issue:`20098`.)
|
|
|
|
* A new method :meth:`~email.message.Message.get_content_disposition` provides
|
|
easy access to a canonical value for the :mailheader:`Content-Disposition`
|
|
header (``None`` if there is no such header). (Contributed by Abhilash Raj
|
|
in :issue:`21083`.)
|
|
|
|
* A new policy option :attr:`~email.policy.EmailPolicy.utf8` can be set
|
|
``True`` to encode email headers using the utf8 charset instead of using
|
|
encoded words. This allows ``Messages`` to be formatted according to
|
|
:rfc:`6532` and used with an SMTP server that supports the :rfc:`6531`
|
|
``SMTPUTF8`` extension. (Contributed by R. David Murray in :issue:`24211`.)
|
|
|
|
glob
|
|
----
|
|
|
|
* :func:`~glob.iglob` and :func:`~glob.glob` now support recursive search in
|
|
subdirectories using the "``**``" pattern.
|
|
(Contributed by Serhiy Storchaka in :issue:`13968`.)
|
|
|
|
idlelib and IDLE
|
|
----------------
|
|
|
|
Since idlelib implements the IDLE shell and editor and is not intended for
|
|
import by other programs, it gets improvements with every release. See
|
|
:file:`Lib/idlelib/NEWS.txt` for a cumulative list of changes since 3.4.0,
|
|
as well as changes made in future 3.5.x releases. This file is also available
|
|
from the IDLE Help -> About Idle dialog.
|
|
|
|
imaplib
|
|
-------
|
|
|
|
* :class:`IMAP4` now supports the context management protocol. When used in a
|
|
:keyword:`with` statement, the IMAP4 ``LOGOUT`` command will be called
|
|
automatically at the end of the block. (Contributed by Tarek Ziadé and
|
|
Serhiy Storchaka in :issue:`4972`.)
|
|
|
|
* :mod:`imaplib` now supports :rfc:`5161`: the :meth:`~imaplib.IMAP4.enable`
|
|
extension), and :rfc:`6855`: utf-8 support (internationalized email, via the
|
|
``UTF8=ACCEPT`` argument to :meth:`~imaplib.IMAP4.enable`). A new attribute,
|
|
:attr:`~imaplib.IMAP4.utf8_enabled`, tracks whether or not :rfc:`6855`
|
|
support is enabled. Milan Oberkirch, R. David Murray, and Maciej Szulik in
|
|
:issue:`21800`.)
|
|
|
|
* :mod:`imaplib` now automatically encodes non-ASCII string usernames and
|
|
passwords using ``UTF8``, as recommended by the RFCs. (Contributed by Milan
|
|
Oberkirch in :issue:`21800`.)
|
|
|
|
imghdr
|
|
------
|
|
|
|
* :func:`~imghdr.what` now recognizes the `OpenEXR <http://www.openexr.com>`_
|
|
format. (Contributed by Martin Vignali and Claudiu Popa in :issue:`20295`.)
|
|
|
|
importlib
|
|
---------
|
|
|
|
* :class:`importlib.util.LazyLoader` allows for the lazy loading of modules in
|
|
applications where startup time is paramount.
|
|
(Contributed by Brett Cannon in :issue:`17621`.)
|
|
|
|
* :func:`importlib.abc.InspectLoader.source_to_code` is now a
|
|
static method to make it easier to work with source code in a string.
|
|
With a module object that you want to initialize you can then use
|
|
``exec(code, module.__dict__)`` to execute the code in the module.
|
|
|
|
* :func:`importlib.util.module_from_spec` is now the preferred way to create a
|
|
new module. Compared to :class:`types.ModuleType`, this new function will set
|
|
the various import-controlled attributes based on the passed-in spec object.
|
|
|
|
inspect
|
|
-------
|
|
|
|
* :class:`inspect.Signature` and :class:`inspect.Parameter` are now
|
|
picklable and hashable. (Contributed by Yury Selivanov in :issue:`20726`
|
|
and :issue:`20334`.)
|
|
|
|
* New method :meth:`inspect.BoundArguments.apply_defaults`. (Contributed
|
|
by Yury Selivanov in :issue:`24190`.)
|
|
|
|
* New class method :meth:`inspect.Signature.from_callable`, which makes
|
|
subclassing of :class:`~inspect.Signature` easier. (Contributed
|
|
by Yury Selivanov and Eric Snow in :issue:`17373`.)
|
|
|
|
* New argument ``follow_wrapped`` for :func:`inspect.signature`.
|
|
(Contributed by Yury Selivanov in :issue:`20691`.)
|
|
|
|
* New :func:`~inspect.iscoroutine`, :func:`~inspect.iscoroutinefunction`,
|
|
and :func:`~inspect.isawaitable` functions. (Contributed by Yury Selivanov
|
|
in :issue:`24017`.)
|
|
|
|
ipaddress
|
|
---------
|
|
|
|
* :class:`ipaddress.IPv4Network` and :class:`ipaddress.IPv6Network` now
|
|
accept an ``(address, netmask)`` tuple argument, so as to easily construct
|
|
network objects from existing addresses. (Contributed by Peter Moody
|
|
and Antoine Pitrou in :issue:`16531`.)
|
|
|
|
json
|
|
----
|
|
|
|
* The output of :mod:`json.tool` command line interface is now in the same
|
|
order as the input. Use the :option:`--sort-keys` option to sort the output
|
|
of dictionaries alphabetically by key. (Contributed by Berker Peksag in
|
|
:issue:`21650`.)
|
|
|
|
* JSON decoder now raises :exc:`json.JSONDecodeError` instead of
|
|
:exc:`ValueError`. (Contributed by Serhiy Storchaka in :issue:`19361`.)
|
|
|
|
os
|
|
--
|
|
|
|
* New :func:`os.scandir` function that exposes file information from
|
|
the operating system when listing a directory. :func:`os.scandir`
|
|
returns an iterator of :class:`os.DirEntry` objects corresponding to
|
|
the entries in the directory given by *path*. (Contributed by Ben
|
|
Hoyt with the help of Victor Stinner in :issue:`22524`.)
|
|
|
|
* :class:`os.stat_result` now has a :attr:`~os.stat_result.st_file_attributes`
|
|
attribute on Windows. (Contributed by Ben Hoyt in :issue:`21719`.)
|
|
|
|
os.path
|
|
-------
|
|
|
|
* New :func:`~os.path.commonpath` function that extracts common path prefix.
|
|
Unlike the :func:`~os.path.commonprefix` function, it always returns a valid
|
|
patch. (Contributed by Rafik Draoui and Serhiy Storchaka in :issue:`10395`.)
|
|
|
|
pickle
|
|
------
|
|
|
|
* Serializing more "lookupable" objects (such as unbound methods or nested
|
|
classes) now are supported with pickle protocols < 4.
|
|
(Contributed by Serhiy Storchaka in :issue:`23611`.)
|
|
|
|
poplib
|
|
------
|
|
|
|
* A new command :meth:`~poplib.POP3.utf8` enables :rfc:`6856`
|
|
(internationalized email) support if the POP server supports it. (Contributed
|
|
by Milan OberKirch in :issue:`21804`.)
|
|
|
|
re
|
|
--
|
|
|
|
* Number of capturing groups in regular expression is no longer limited by 100.
|
|
(Contributed by Serhiy Storchaka in :issue:`22437`.)
|
|
|
|
* Now unmatched groups are replaced with empty strings in :func:`re.sub`
|
|
and :func:`re.subn`. (Contributed by Serhiy Storchaka in :issue:`1519638`.)
|
|
|
|
math
|
|
----
|
|
|
|
* :data:`math.inf` and :data:`math.nan` constants added. (Contributed by Mark
|
|
Dickinson in :issue:`23185`.)
|
|
|
|
shutil
|
|
------
|
|
|
|
* :func:`~shutil.move` now accepts a *copy_function* argument, allowing,
|
|
for example, :func:`~shutil.copy` to be used instead of the default
|
|
:func:`~shutil.copy2` if there is a need to ignore metadata. (Contributed by
|
|
Claudiu Popa in :issue:`19840`.)
|
|
|
|
signal
|
|
------
|
|
|
|
* On Windows, :func:`signal.set_wakeup_fd` now also supports socket handles.
|
|
(Contributed by Victor Stinner in :issue:`22018`.)
|
|
|
|
* Different constants of :mod:`signal` module are now enumeration values using
|
|
the :mod:`enum` module. This allows meaningful names to be printed during
|
|
debugging, instead of integer “magic numbers”. (Contributed by Giampaolo
|
|
Rodola' in :issue:`21076`.)
|
|
|
|
smtpd
|
|
-----
|
|
|
|
* Both :class:`~smtpd.SMTPServer` and :class:`smtpd.SMTPChannel` now accept a
|
|
*decode_data* keyword to determine if the DATA portion of the SMTP
|
|
transaction is decoded using the ``utf-8`` codec or is instead provided to
|
|
:meth:`~smtpd.SMTPServer.process_message` as a byte string. The default
|
|
is ``True`` for backward compatibility reasons, but will change to ``False``
|
|
in Python 3.6. If *decode_data* is set to ``False``, the
|
|
:meth:`~smtpd.SMTPServer.process_message` method must be prepared to accept
|
|
keyword arguments. (Contributed by Maciej Szulik in :issue:`19662`.)
|
|
|
|
* :class:`~smtpd.SMTPServer` now advertises the ``8BITMIME`` extension
|
|
(:rfc:`6152`) if if *decode_data* has been set ``True``. If the client
|
|
specifies ``BODY=8BITMIME`` on the ``MAIL`` command, it is passed to
|
|
:meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword.
|
|
(Contributed by Milan Oberkirch and R. David Murray in :issue:`21795`.)
|
|
|
|
* :class:`~smtpd.SMTPServer` now supports the ``SMTPUTF8`` extension
|
|
(:rfc:`6531`: Internationalized Email). If the client specified ``SMTPUTF8
|
|
BODY=8BITMIME`` on the ``MAIL`` command, they are passed to
|
|
:meth:`~smtpd.SMTPServer.process_message` via the ``mail_options`` keyword.
|
|
It is the responsibility of the :meth:`~smtpd.SMTPServer.process_message`
|
|
method to correctly handle the ``SMTPUTF8`` data. (Contributed by Milan
|
|
Oberkirch in :issue:`21725`.)
|
|
|
|
* It is now possible to provide, directly or via name resolution, IPv6
|
|
addresses in the :class:`~smtpd.SMTPServer` constructor, and have it
|
|
successfully connect. (Contributed by Milan Oberkirch in :issue:`14758`.)
|
|
|
|
smtplib
|
|
-------
|
|
|
|
* A new :meth:`~smtplib.SMTP.auth` method provides a convenient way to
|
|
implement custom authentication mechanisms.
|
|
(Contributed by Milan Oberkirch in :issue:`15014`.)
|
|
|
|
* Additional debuglevel (2) shows timestamps for debug messages in
|
|
:class:`smtplib.SMTP`. (Contributed by Gavin Chappell and Maciej Szulik in
|
|
:issue:`16914`.)
|
|
|
|
* :mod:`smtplib` now supports :rfc:`6531` (SMTPUTF8) in both the
|
|
:meth:`~smtplib.SMTP.sendmail` and :meth:`~smtplib.SMTP.send_message`
|
|
commands. (Contributed by Milan Oberkirch and R. David Murray in
|
|
:issue:`22027`.)
|
|
|
|
sndhdr
|
|
------
|
|
|
|
* :func:`~sndhdr.what` and :func:`~sndhdr.whathdr` now return
|
|
:func:`~collections.namedtuple`.
|
|
(Contributed by Claudiu Popa in :issue:`18615`.)
|
|
|
|
socket
|
|
------
|
|
|
|
* New :meth:`socket.socket.sendfile` method allows to send a file over a socket
|
|
by using high-performance :func:`os.sendfile` function on UNIX resulting in
|
|
uploads being from 2x to 3x faster than when using plain
|
|
:meth:`socket.socket.send`.
|
|
(Contributed by Giampaolo Rodola' in :issue:`17552`.)
|
|
|
|
subprocess
|
|
----------
|
|
|
|
* The new :func:`subprocess.run` function runs subprocesses and returns a
|
|
:class:`subprocess.CompletedProcess` object. It Provides a more consistent
|
|
API than :func:`~subprocess.call`, :func:`~subprocess.check_call` and
|
|
:func:`~subprocess.check_output`.
|
|
|
|
sysconfig
|
|
---------
|
|
|
|
* The user scripts directory on Windows is now versioned.
|
|
(Contributed by Paul Moore in :issue:`23437`.)
|
|
|
|
tarfile
|
|
-------
|
|
|
|
* The :func:`tarfile.open` function now supports ``'x'`` (exclusive creation)
|
|
mode. (Contributed by Berker Peksag in :issue:`21717`.)
|
|
|
|
* The :meth:`~tarfile.TarFile.extractall` and :meth:`~tarfile.TarFile.extract`
|
|
methods now take a keyword parameter *numeric_only*. If set to ``True``,
|
|
the extracted files and directories will be owned by the numeric uid and gid
|
|
from the tarfile. If set to ``False`` (the default, and the behavior in
|
|
versions prior to 3.5), they will be owned bythe named user and group in the
|
|
tarfile. (Contributed by Michael Vogt and Eric Smith in :issue:`23193`.)
|
|
|
|
time
|
|
----
|
|
|
|
* The :func:`time.monotonic` function is now always available. (Contributed by
|
|
Victor Stinner in :issue:`22043`.)
|
|
|
|
tkinter
|
|
-------
|
|
|
|
* The :mod:`tkinter._fix` module used for setting up the Tcl/Tk environment
|
|
on Windows has been replaced by a private function in the :mod:`_tkinter`
|
|
module which makes no permanent changes to environment variables.
|
|
(Contributed by Zachary Ware in :issue:`20035`.)
|
|
|
|
types
|
|
-----
|
|
|
|
* New :func:`~types.coroutine` function. (Contributed by Yury Selivanov
|
|
in :issue:`24017`.)
|
|
|
|
urllib
|
|
------
|
|
|
|
* A new :class:`~urllib.request.HTTPPasswordMgrWithPriorAuth` allows HTTP Basic
|
|
Authentication credentials to be managed so as to eliminate unnecessary
|
|
``401`` response handling, or to unconditionally send credentials
|
|
on the first request in order to communicate with servers that return a
|
|
``404`` response instead of a ``401`` if the ``Authorization`` header is not
|
|
sent. (Contributed by Matej Cepl in :issue:`19494` and Akshit Khurana in
|
|
:issue:`7159`.)
|
|
|
|
* A new :func:`~urllib.parse.urlencode` parameter *quote_via* provides a way to
|
|
control the encoding of query parts if needed. (Contributed by Samwyse and
|
|
Arnon Yaari in :issue:`13866`.)
|
|
|
|
wsgiref
|
|
-------
|
|
|
|
* *headers* parameter of :class:`wsgiref.headers.Headers` is now optional.
|
|
(Contributed by Pablo Torres Navarrete and SilentGhost in :issue:`5800`.)
|
|
|
|
xmlrpc
|
|
------
|
|
|
|
* :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager`.
|
|
(Contributed by Claudiu Popa in :issue:`20627`.)
|
|
|
|
xml.sax
|
|
-------
|
|
|
|
* SAX parsers now support a character stream of
|
|
:class:`~xml.sax.xmlreader.InputSource` object.
|
|
(Contributed by Serhiy Storchaka in :issue:`2175`.)
|
|
|
|
faulthandler
|
|
------------
|
|
|
|
* :func:`~faulthandler.enable`, :func:`~faulthandler.register`,
|
|
:func:`~faulthandler.dump_traceback` and
|
|
:func:`~faulthandler.dump_traceback_later` functions now accept file
|
|
descriptors. (Contributed by Wei Wu in :issue:`23566`.)
|
|
|
|
zipfile
|
|
-------
|
|
|
|
* Added support for writing ZIP files to unseekable streams.
|
|
(Contributed by Serhiy Storchaka in :issue:`23252`.)
|
|
|
|
* The :func:`zipfile.ZipFile.open` function now supports ``'x'`` (exclusive
|
|
creation) mode. (Contributed by Serhiy Storchaka in :issue:`21717`.)
|
|
|
|
|
|
Optimizations
|
|
=============
|
|
|
|
The following performance enhancements have been added:
|
|
|
|
* :func:`os.walk` has been sped up by 3-5x on POSIX systems and 7-20x
|
|
on Windows. This was done using the new :func:`os.scandir` function,
|
|
which exposes file information from the underlying ``readdir`` and
|
|
``FindFirstFile``/``FindNextFile`` system calls. (Contributed by
|
|
Ben Hoyt with help from Victor Stinner in :issue:`23605`.)
|
|
|
|
* Construction of ``bytes(int)`` (filled by zero bytes) is faster and uses less
|
|
memory for large objects. ``calloc()`` is used instead of ``malloc()`` to
|
|
allocate memory for these objects.
|
|
|
|
* Some operations on :class:`~ipaddress.IPv4Network` and
|
|
:class:`~ipaddress.IPv6Network` have been massively sped up, such as
|
|
:meth:`~ipaddress.IPv4Network.subnets`, :meth:`~ipaddress.IPv4Network.supernet`,
|
|
:func:`~ipaddress.summarize_address_range`, :func:`~ipaddress.collapse_addresses`.
|
|
The speed up can range from 3x to 15x.
|
|
(:issue:`21486`, :issue:`21487`, :issue:`20826`)
|
|
|
|
* Many operations on :class:`io.BytesIO` are now 50% to 100% faster.
|
|
(Contributed by Serhiy Storchaka in :issue:`15381` and David Wilson in
|
|
:issue:`22003`.)
|
|
|
|
* :func:`marshal.dumps` is now faster (65%-85% with versions 3--4, 20-25% with
|
|
versions 0--2 on typical data, and up to 5x in best cases).
|
|
(Contributed by Serhiy Storchaka in :issue:`20416` and :issue:`23344`.)
|
|
|
|
* The UTF-32 encoder is now 3x to 7x faster. (Contributed by Serhiy Storchaka
|
|
in :issue:`15027`.)
|
|
|
|
|
|
Build and C API Changes
|
|
=======================
|
|
|
|
Changes to Python's build process and to the C API include:
|
|
|
|
* New ``calloc`` functions:
|
|
|
|
* :c:func:`PyMem_RawCalloc`
|
|
* :c:func:`PyMem_Calloc`
|
|
* :c:func:`PyObject_Calloc`
|
|
* :c:func:`_PyObject_GC_Calloc`
|
|
|
|
|
|
Deprecated
|
|
==========
|
|
|
|
New Keywords
|
|
------------
|
|
|
|
``async`` and ``await`` are not recommended to be used as variable, class or
|
|
function names. Introduced by :pep:`492` in Python 3.5, they will become
|
|
proper keywords in Python 3.7.
|
|
|
|
|
|
Unsupported Operating Systems
|
|
-----------------------------
|
|
|
|
* Windows XP - Per :PEP:`11`, Microsoft support of Windows XP has ended.
|
|
|
|
|
|
Deprecated Python modules, functions and methods
|
|
------------------------------------------------
|
|
|
|
* The :mod:`formatter` module has now graduated to full deprecation and is still
|
|
slated for removal in Python 3.6.
|
|
|
|
* :mod:`smtpd` has in the past always decoded the DATA portion of email
|
|
messages using the ``utf-8`` codec. This can now be controlled by the new
|
|
*decode_data* keyword to :class:`~smtpd.SMTPServer`. The default value is
|
|
``True``, but this default is deprecated. Specify the *decode_data* keyword
|
|
with an appropriate value to avoid the deprecation warning.
|
|
|
|
* Directly assigning values to the :attr:`~http.cookies.Morsel.key`,
|
|
:attr:`~http.cookies.Morsel.value` and
|
|
:attr:`~http.cookies.Morsel.coded_value` of :class:`~http.cookies.Morsel`
|
|
objects is deprecated. Use the :func:`~http.cookies.Morsel.set` method
|
|
instead. In addition, the undocumented *LegalChars* parameter of
|
|
:func:`~http.cookies.Morsel.set` is deprecated, and is now ignored.
|
|
|
|
* Passing a format string as keyword argument *format_string* to the
|
|
:meth:`~string.Formatter.format` method of the :class:`string.Formatter`
|
|
class has been deprecated.
|
|
|
|
* :func:`platform.dist` and :func:`platform.linux_distribution` functions are
|
|
now deprecated and will be removed in Python 3.7. Linux distributions use
|
|
too many different ways of describing themselves, so the functionality is
|
|
left to a package.
|
|
(Contributed by Vajrasky Kok and Berker Peksag in :issue:`1322`.)
|
|
|
|
* The previously undocumented ``from_function`` and ``from_builtin`` methods of
|
|
:class:`inspect.Signature` are deprecated. Use new
|
|
:meth:`inspect.Signature.from_callable` instead. (Contributed by Yury
|
|
Selivanov in :issue:`24248`.)
|
|
|
|
* :func:`inspect.getargspec` is deprecated and scheduled to be removed in
|
|
Python 3.6. (See :issue:`20438` for details.)
|
|
|
|
* :func:`~inspect.getfullargspec`, :func:`~inspect.getargvalues`,
|
|
:func:`~inspect.getcallargs`, :func:`~inspect.getargvalues`,
|
|
:func:`~inspect.formatargspec`, and :func:`~inspect.formatargvalues` are
|
|
deprecated in favor of :func:`inspect.signature` API. (See :issue:`20438`
|
|
for details.)
|
|
|
|
|
|
Deprecated functions and types of the C API
|
|
-------------------------------------------
|
|
|
|
* None yet.
|
|
|
|
|
|
Deprecated features
|
|
-------------------
|
|
|
|
* None yet.
|
|
|
|
|
|
Removed
|
|
=======
|
|
|
|
API and Feature Removals
|
|
------------------------
|
|
|
|
The following obsolete and previously deprecated APIs and features have been
|
|
removed:
|
|
|
|
* The ``__version__`` attribute has been dropped from the email package. The
|
|
email code hasn't been shipped separately from the stdlib for a long time,
|
|
and the ``__version__`` string was not updated in the last few releases.
|
|
|
|
* The internal ``Netrc`` class in the :mod:`ftplib` module was deprecated in
|
|
3.4, and has now been removed.
|
|
(Contributed by Matt Chaput in :issue:`6623`.)
|
|
|
|
* The concept of ``.pyo`` files has been removed.
|
|
|
|
* The JoinableQueue class in the provisional asyncio module was deprecated
|
|
in 3.4.4 and is now removed (:issue:`23464`).
|
|
|
|
|
|
Porting to Python 3.5
|
|
=====================
|
|
|
|
This section lists previously described changes and other bugfixes
|
|
that may require changes to your code.
|
|
|
|
Changes in the Python API
|
|
-------------------------
|
|
|
|
* :pep:`475`: Examples of functions which are now retried when interrupted
|
|
instead of raising :exc:`InterruptedError` if the signal handler does not
|
|
raise an exception:
|
|
|
|
- :func:`open`, :func:`os.open`, :func:`io.open`
|
|
- functions of the :mod:`faulthandler` module
|
|
- :mod:`os` functions:
|
|
|
|
* :func:`os.fchdir`
|
|
* :func:`os.fchmod`
|
|
* :func:`os.fchown`
|
|
* :func:`os.fdatasync`
|
|
* :func:`os.fstat`
|
|
* :func:`os.fstatvfs`
|
|
* :func:`os.fsync`
|
|
* :func:`os.ftruncate`
|
|
* :func:`os.mkfifo`
|
|
* :func:`os.mknod`
|
|
* :func:`os.posix_fadvise`
|
|
* :func:`os.posix_fallocate`
|
|
* :func:`os.pread`
|
|
* :func:`os.pwrite`
|
|
* :func:`os.read`
|
|
* :func:`os.readv`
|
|
* :func:`os.sendfile`
|
|
* :func:`os.wait3`
|
|
* :func:`os.wait4`
|
|
* :func:`os.wait`
|
|
* :func:`os.waitid`
|
|
* :func:`os.waitpid`
|
|
* :func:`os.write`
|
|
* :func:`os.writev`
|
|
* special cases: :func:`os.close` and :func:`os.dup2` now ignore
|
|
:py:data:`~errno.EINTR` error, the syscall is not retried (see the PEP
|
|
for the rationale)
|
|
|
|
- :func:`select.select`, :func:`select.poll.poll`, :func:`select.epoll.poll`,
|
|
:func:`select.kqueue.control`, :func:`select.devpoll.poll`
|
|
- :func:`socket.socket` methods:
|
|
|
|
* :meth:`~socket.socket.accept`
|
|
* :meth:`~socket.socket.connect` (except for non-blocking sockets)
|
|
* :meth:`~socket.socket.recv`
|
|
* :meth:`~socket.socket.recvfrom`
|
|
* :meth:`~socket.socket.recvmsg`
|
|
* :meth:`~socket.socket.send`
|
|
* :meth:`~socket.socket.sendall`
|
|
* :meth:`~socket.socket.sendmsg`
|
|
* :meth:`~socket.socket.sendto`
|
|
|
|
- :func:`signal.sigtimedwait`, :func:`signal.sigwaitinfo`
|
|
- :func:`time.sleep`
|
|
|
|
* Before Python 3.5, a :class:`datetime.time` object was considered to be false
|
|
if it represented midnight in UTC. This behavior was considered obscure and
|
|
error-prone and has been removed in Python 3.5. See :issue:`13936` for full
|
|
details.
|
|
|
|
* :meth:`ssl.SSLSocket.send()` now raises either :exc:`ssl.SSLWantReadError`
|
|
or :exc:`ssl.SSLWantWriteError` on a non-blocking socket if the operation
|
|
would block. Previously, it would return 0. See :issue:`20951`.
|
|
|
|
* The ``__name__`` attribute of generator is now set from the function name,
|
|
instead of being set from the code name. Use ``gen.gi_code.co_name`` to
|
|
retrieve the code name. Generators also have a new ``__qualname__``
|
|
attribute, the qualified name, which is now used for the representation
|
|
of a generator (``repr(gen)``). See :issue:`21205`.
|
|
|
|
* The deprecated "strict" mode and argument of :class:`~html.parser.HTMLParser`,
|
|
:meth:`HTMLParser.error`, and the :exc:`HTMLParserError` exception have been
|
|
removed. (Contributed by Ezio Melotti in :issue:`15114`.)
|
|
The *convert_charrefs* argument of :class:`~html.parser.HTMLParser` is
|
|
now ``True`` by default. (Contributed by Berker Peksag in :issue:`21047`.)
|
|
|
|
* Although it is not formally part of the API, it is worth noting for porting
|
|
purposes (ie: fixing tests) that error messages that were previously of the
|
|
form "'sometype' does not support the buffer protocol" are now of the form "a
|
|
bytes-like object is required, not 'sometype'". (Contributed by Ezio Melotti
|
|
in :issue:`16518`.)
|
|
|
|
* If the current directory is set to a directory that no longer exists then
|
|
:exc:`FileNotFoundError` will no longer be raised and instead
|
|
:meth:`~importlib.machinery.FileFinder.find_spec` will return ``None``
|
|
**without** caching ``None`` in :data:`sys.path_importer_cache` which is
|
|
different than the typical case (:issue:`22834`).
|
|
|
|
* HTTP status code and messages from :mod:`http.client` and :mod:`http.server`
|
|
were refactored into a common :class:`~http.HTTPStatus` enum. The values in
|
|
:mod:`http.client` and :mod:`http.server` remain available for backwards
|
|
compatibility. (Contributed by Demian Brecht in :issue:`21793`.)
|
|
|
|
* When an import loader defines :meth:`~importlib.machinery.Loader.exec_module`
|
|
it is now expected to also define
|
|
:meth:`~importlib.machinery.Loader.create_module` (raises a
|
|
:exc:`DeprecationWarning` now, will be an error in Python 3.6). If the loader
|
|
inherits from :class:`importlib.abc.Loader` then there is nothing to do, else
|
|
simply define :meth:`~importlib.machinery.Loader.create_module` to return
|
|
``None`` (:issue:`23014`).
|
|
|
|
* :func:`re.split` always ignored empty pattern matches, so the ``'x*'``
|
|
pattern worked the same as ``'x+'``, and the ``'\b'`` pattern never worked.
|
|
Now :func:`re.split` raises a warning if the pattern could match
|
|
an empty string. For compatibility use patterns that never match an empty
|
|
string (e.g. ``'x+'`` instead of ``'x*'``). Patterns that could only match
|
|
an empty string (such as ``'\b'``) now raise an error.
|
|
|
|
* The :class:`~http.cookies.Morsel` dict-like interface has been made self
|
|
consistent: morsel comparison now takes the :attr:`~http.cookies.Morsel.key`
|
|
and :attr:`~http.cookies.Morsel.value` into account,
|
|
:meth:`~http.cookies.Morsel.copy` now results in a
|
|
:class:`~http.cookies.Morsel` instance rather than a :class:`dict`, and
|
|
:meth:`~http.cookies.Morsel.update` will now raise an exception if any of the
|
|
keys in the update dictionary are invalid. In addition, the undocumented
|
|
*LegalChars* parameter of :func:`~http.cookies.Morsel.set` is deprecated and
|
|
is now ignored. (:issue:`2211`)
|
|
|
|
* :pep:`488` has removed ``.pyo`` files from Python and introduced the optional
|
|
``opt-`` tag in ``.pyc`` file names. The
|
|
:func:`importlib.util.cache_from_source` has gained an *optimization*
|
|
parameter to help control the ``opt-`` tag. Because of this, the
|
|
*debug_override* parameter of the function is now deprecated. `.pyo` files
|
|
are also no longer supported as a file argument to the Python interpreter and
|
|
thus serve no purpose when distributed on their own (i.e. sourcless code
|
|
distribution). Due to the fact that the magic number for bytecode has changed
|
|
in Python 3.5, all old `.pyo` files from previous versions of Python are
|
|
invalid regardless of this PEP.
|
|
|
|
* The :mod:`socket` module now exports the CAN_RAW_FD_FRAMES constant on linux
|
|
3.6 and greater.
|
|
|
|
* The `pygettext.py` Tool now uses the standard +NNNN format for timezones in
|
|
the POT-Creation-Date header.
|
|
|
|
* The :mod:`smtplib` module now uses :data:`sys.stderr` instead of previous
|
|
module level :data:`stderr` variable for debug output. If your (test)
|
|
program depends on patching the module level variable to capture the debug
|
|
output, you will need to update it to capture sys.stderr instead.
|
|
|
|
Changes in the C API
|
|
--------------------
|
|
|
|
* The undocumented :c:member:`~PyMemoryViewObject.format` member of the
|
|
(non-public) :c:type:`PyMemoryViewObject` structure has been removed.
|
|
|
|
All extensions relying on the relevant parts in ``memoryobject.h``
|
|
must be rebuilt.
|
|
|
|
* The :c:type:`PyMemAllocator` structure was renamed to
|
|
:c:type:`PyMemAllocatorEx` and a new ``calloc`` field was added.
|
|
|
|
* Removed non-documented macro :c:macro:`PyObject_REPR` which leaked references.
|
|
Use format character ``%R`` in :c:func:`PyUnicode_FromFormat`-like functions
|
|
to format the :func:`repr` of the object.
|
|
|
|
* Because the lack of the :attr:`__module__` attribute breaks pickling and
|
|
introspection, a deprecation warning now is raised for builtin type without
|
|
the :attr:`__module__` attribute. Would be an AttributeError in future.
|
|
(:issue:`20204`)
|
|
|
|
* As part of PEP 492 implementation, ``tp_reserved`` slot of
|
|
:c:type:`PyTypeObject` was replaced with a
|
|
:c:member:`PyTypeObject.tp_as_async` slot.
|