diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst index d66ac657cab..2337823f3a8 100644 --- a/Doc/whatsnew/3.6.rst +++ b/Doc/whatsnew/3.6.rst @@ -4,6 +4,7 @@ :Release: |release| :Date: |today| +:Editors: Elvis Pranskevichus , Yury Selivanov .. Rules for maintenance: @@ -45,16 +46,17 @@ This saves the maintainer the effort of going through the Mercurial log when researching a change. -This article explains the new features in Python 3.6, compared to 3.5. - -For full details, see the -`changelog `_. - .. note:: - Prerelease users should be aware that this document is currently in draft - form. It will be updated substantially as Python 3.6 moves towards release, - so it's worth checking back even after reading earlier versions. + Prerelease users should be aware that this document is currently in draft + form. It will be updated substantially as Python 3.6 moves towards release, + so it's worth checking back even after reading earlier versions. + +This article explains the new features in Python 3.6, compared to 3.5. + +.. seealso:: + + :pep:`494` - Python 3.6 Release Schedule Summary -- Release highlights @@ -65,40 +67,73 @@ Summary -- Release highlights New syntax features: -* A ``global`` or ``nonlocal`` statement must now textually appear - before the first use of the affected name in the same scope. - Previously this was a SyntaxWarning. +* :ref:`PEP 498 `, formatted string literals. -* PEP 498: :ref:`Formatted string literals ` +* :ref:`PEP 515 `, underscores in numeric literals. -* PEP 515: Underscores in Numeric Literals +* :ref:`PEP 526 `, syntax for variable annotations. -* PEP 526: :ref:`Syntax for Variable Annotations ` +* :ref:`PEP 525 `, asynchronous generators. -* PEP 525: Asynchronous Generators +* :ref:`PEP 530 `: asynchronous comprehensions. -* PEP 530: Asynchronous Comprehensions -Standard library improvements: +New library modules: + +* :mod:`secrets`: :ref:`PEP 506 -- Adding A Secrets Module To The Standard Library `. + + +CPython implementation improvements: + +* The :ref:`dict ` type has been reimplemented to use + a :ref:`faster, more compact representation ` + similar to the `PyPy dict implementation`_. This resulted in dictionaries + using 20% to 25% less memory when compared to Python 3.5. + +* Customization of class creation has been simplified with the + :ref:`new protocol `. + +* The class attibute definition order is + :ref:`now preserved `. + +* The order of elements in ``**kwargs`` now corresponds to the order in + the function signature: + :ref:`Preserving Keyword Argument Order `. + +* DTrace and SystemTap :ref:`probing support `. + + +Significant improvements in the standard library: + +* A new :ref:`file system path protocol ` has been + implemented to support :term:`path-like objects `. + All standard library functions operating on paths have been updated to + work with the new protocol. + +* The overhead of :mod:`asyncio` implementation has been reduced by + up to 50% thanks to the new C implementation of the :class:`asyncio.Future` + and :class:`asyncio.Task` classes and other optimizations. + Security improvements: -* On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool - is initialized to increase the security. See the :pep:`524` for the +* On Linux, :func:`os.urandom` now blocks until the system urandom entropy + pool is initialized to increase the security. See the :pep:`524` for the rationale. -* :mod:`hashlib` and :mod:`ssl` now support OpenSSL 1.1.0. +* The :mod:`hashlib` and :mod:`ssl` modules now support OpenSSL 1.1.0. -* The default settings and feature set of the :mod:`ssl` have been improved. +* The default settings and feature set of the :mod:`ssl` module have been + improved. + +* The :mod:`hashlib` module received support for the BLAKE2, SHA-3 and SHAKE + hash algorithms and the :func:`~hashlib.scrypt` key derivation function. -* The :mod:`hashlib` module has got support for BLAKE2, SHA-3 and SHAKE hash - algorithms and :func:`~hashlib.scrypt` key derivation function. Windows improvements: -* PEP 529: :ref:`Change Windows filesystem encoding to UTF-8 ` - -* PEP 528: :ref:`Change Windows console encoding to UTF-8 ` +* :ref:`PEP 528 ` and :ref:`PEP 529 `, + Windows filesystem and console encoding changed to UTF-8. * The ``py.exe`` launcher, when used interactively, no longer prefers Python 2 over Python 3 when the user doesn't specify a version (via @@ -117,66 +152,111 @@ Windows improvements: :envvar:`PYTHONHOME`. See :ref:`the documentation ` for more information. -.. PEP-sized items next. - -.. _pep-4XX: - -.. PEP 4XX: Virtual Environments -.. ============================= - - -.. (Implemented by Foo Bar.) - -.. .. seealso:: - - :pep:`4XX` - Python Virtual Environments - PEP written by Carl Meyer - -New built-in features: - -* PEP 520: :ref:`Preserving Class Attribute Definition Order` - -* PEP 468: :ref:`Preserving Keyword Argument Order` A complete list of PEP's implemented in Python 3.6: -* :pep:`468`, :ref:`Preserving Keyword Argument Order` -* :pep:`487`, :ref:`Simpler customization of class creation` +* :pep:`468`, :ref:`Preserving Keyword Argument Order ` +* :pep:`487`, :ref:`Simpler customization of class creation ` * :pep:`495`, Local Time Disambiguation -* :pep:`498`, :ref:`Formatted string literals ` +* :pep:`498`, :ref:`Formatted string literals ` * :pep:`506`, Adding A Secrets Module To The Standard Library -* :pep:`509`, :ref:`Add a private version to dict` -* :pep:`515`, :ref:`Underscores in Numeric Literals` -* :pep:`519`, :ref:`Adding a file system path protocol` -* :pep:`520`, :ref:`Preserving Class Attribute Definition Order` -* :pep:`523`, :ref:`Adding a frame evaluation API to CPython` +* :pep:`509`, :ref:`Add a private version to dict ` +* :pep:`515`, :ref:`Underscores in Numeric Literals ` +* :pep:`519`, :ref:`Adding a file system path protocol ` +* :pep:`520`, :ref:`Preserving Class Attribute Definition Order ` +* :pep:`523`, :ref:`Adding a frame evaluation API to CPython ` * :pep:`524`, Make os.urandom() blocking on Linux (during system startup) * :pep:`525`, Asynchronous Generators (provisional) -* :pep:`526`, :ref:`Syntax for Variable Annotations (provisional)` -* :pep:`528`, :ref:`Change Windows console encoding to UTF-8 (provisional)` -* :pep:`529`, :ref:`Change Windows filesystem encoding to UTF-8 (provisional)` +* :pep:`526`, :ref:`Syntax for Variable Annotations (provisional) ` +* :pep:`528`, :ref:`Change Windows console encoding to UTF-8 (provisional) ` +* :pep:`529`, :ref:`Change Windows filesystem encoding to UTF-8 (provisional) ` * :pep:`530`, Asynchronous Comprehensions +.. _PyPy dict implementation: https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html + + New Features ============ -.. _pep-515: +.. _whatsnew36-pep498: + +PEP 498: Formatted string literals +---------------------------------- + +:pep:`498` introduces a new kind of string literals: *f-strings*, or +*formatted string literals*. + +Formatted string literals are prefixed with ``'f'`` and are similar to +the format strings accepted by :meth:`str.format`. They contain replacement +fields surrounded by curly braces. The replacement fields are expressions, +which are evaluated at run time, and then formatted using the +:func:`format` protocol:: + + >>> name = "Fred" + >>> f"He said his name is {name}." + 'He said his name is Fred.' + +.. seealso:: + + :pep:`498` -- Literal String Interpolation. + PEP written and implemented by Eric V. Smith. + + :ref:`Feature documentation `. + + +.. _whatsnew36-pep526: + +PEP 526: Syntax for variable annotations +---------------------------------------- + +:pep:`484` introduced the standard for type annotations of function +parameters, a.k.a. type hints. This PEP adds syntax to Python for annotating +the types of variables including class variables and instance variables:: + + primes: List[int] = [] + + captain: str # Note: no initial value! + + class Starship: + stats: Dict[str, int] = {} + +Just as for function annotations, the Python interpreter does not attach any +particular meaning to variable annotations and only stores them in the +``__annotations__`` attribute of a class or module. + +In contrast to variable declarations in statically typed languages, +the goal of annotation syntax is to provide an easy way to specify structured +type metadata for third party tools and libraries via the abstract syntax tree +and the ``__annotations__`` attribute. + +.. seealso:: + + :pep:`526` -- Syntax for variable annotations. + PEP written by Ryan Gonzalez, Philip House, Ivan Levkivskyi, Lisa Roach, + and Guido van Rossum. Implemented by Ivan Levkivskyi. + + Tools that use or will use the new syntax: + `mypy `_, + `pytype `_, PyCharm, etc. + + +.. _whatsnew36-pep515: PEP 515: Underscores in Numeric Literals ---------------------------------------- -Prior to PEP 515, there was no support for writing long numeric -literals with some form of separator to improve readability. For -instance, how big is ``1000000000000000``? With :pep:`515`, though, -you can use underscores to separate digits as desired to make numeric -literals easier to read: ``1_000_000_000_000_000``. Underscores can be -used with other numeric literals beyond integers, e.g. -``0x_FF_FF_FF_FF``. +:pep:`515` adds the ability to use underscores in numeric literals for +improved readability. For example:: + + >>> 1_000_000_000_000_000 + 1000000000000000 + >>> 0x_FF_FF_FF_FF + 4294967295 Single underscores are allowed between digits and after any base -specifier. More than a single underscore in a row, leading, or -trailing underscores are not allowed. +specifier. Leading, trailing, or multiple underscores in a row are not +allowed. .. seealso:: @@ -184,36 +264,115 @@ trailing underscores are not allowed. PEP written by Georg Brandl and Serhiy Storchaka. -.. _pep-523: +.. _whatsnew36-pep525: -PEP 523: Adding a frame evaluation API to CPython -------------------------------------------------- +PEP 525: Asynchronous Generators +-------------------------------- -While Python provides extensive support to customize how code -executes, one place it has not done so is in the evaluation of frame -objects. If you wanted some way to intercept frame evaluation in -Python there really wasn't any way without directly manipulating -function pointers for defined functions. +:pep:`492` introduced support for native coroutines and ``async`` / ``await`` +syntax to Python 3.5. A notable limitation of the Python 3.5 implementation +is that it was not possible to use ``await`` and ``yield`` in the same +function body. In Python 3.6 this restriction has been lifted, making it +possible to define *asynchronous generators*:: -:pep:`523` changes this by providing an API to make frame -evaluation pluggable at the C level. This will allow for tools such -as debuggers and JITs to intercept frame evaluation before the -execution of Python code begins. This enables the use of alternative -evaluation implementations for Python code, tracking frame -evaluation, etc. + async def ticker(delay, to): + """Yield numbers from 0 to `to` every `delay` seconds.""" + for i in range(to): + yield i + await asyncio.sleep(delay) -This API is not part of the limited C API and is marked as private to -signal that usage of this API is expected to be limited and only -applicable to very select, low-level use-cases. Semantics of the -API will change with Python as necessary. +The new syntax allows for faster and more concise code. .. seealso:: - :pep:`523` -- Adding a frame evaluation API to CPython - PEP written by Brett Cannon and Dino Viehland. + :pep:`525` -- Asynchronous Generators + PEP written and implemented by Yury Selivanov. -.. _pep-519: +.. _whatsnew36-pep530: + +PEP 530: Asynchronous Comprehensions +------------------------------------ + +:pep:`530` adds support for using ``async for`` in list, set, dict +comprehensions and generator expressions:: + + result = [i async for i in aiter() if i % 2] + +Additionally, ``await`` expressions are supported in all kinds +of comprehensions:: + + result = [await fun() for fun in funcs] + +.. seealso:: + + :pep:`530` -- Asynchronous Comprehensions + PEP written and implemented by Yury Selivanov. + + +.. _whatsnew36-pep487: + +PEP 487: Simpler customization of class creation +------------------------------------------------ + +It is now possible to customize subclass creation without using a metaclass. +The new ``__init_subclass__`` classmethod will be called on the base class +whenever a new subclass is created:: + + >>> class QuestBase: + ... # this is implicitly a @classmethod + ... def __init_subclass__(cls, swallow, **kwargs): + ... cls.swallow = swallow + ... super().__init_subclass__(**kwargs) + + >>> class Quest(QuestBase, swallow="african"): + ... pass + + >>> Quest.swallow + 'african' + +.. seealso:: + + :pep:`487` -- Simpler customization of class creation + PEP written and implemented by Martin Teichmann. + + :ref:`Feature documentation ` + + +.. _whatsnew36-pep487-descriptors: + +PEP 487: Descriptor Protocol Enhancements +----------------------------------------- + +:pep:`487` extends the descriptor protocol has to include the new optional +``__set_name__`` method. Whenever a new class is defined, the new method +will be called on all descriptors included in the definition, providing +them with a reference to the class being defined and the name given to the +descriptor within the class namespace. + +.. seealso:: + + :pep:`487` -- Simpler customization of class creation + PEP written and implemented by Martin Teichmann. + + :ref:`Feature documentation ` + + +.. _whatsnew36-pep506: + +PEP 506: Adding A Secrets Module To The Standard Library +-------------------------------------------------------- + + + +.. seealso:: + + :pep:`506` -- Adding A Secrets Module To The Standard Library + PEP written and implemented by Steven D'Aprano. + + + +.. _whatsnew36-pep519: PEP 519: Adding a file system path protocol ------------------------------------------- @@ -280,60 +439,7 @@ pre-existing code:: PEP written by Brett Cannon and Koos Zevenhoven. -.. _whatsnew-fstrings: - -PEP 498: Formatted string literals ----------------------------------- - -Formatted string literals are a new kind of string literal, prefixed -with ``'f'``. They are similar to the format strings accepted by -:meth:`str.format`. They contain replacement fields surrounded by -curly braces. The replacement fields are expressions, which are -evaluated at run time, and then formatted using the :func:`format` protocol:: - - >>> name = "Fred" - >>> f"He said his name is {name}." - 'He said his name is Fred.' - -See :pep:`498` and the main documentation at :ref:`f-strings`. - - -.. _variable-annotations: - -PEP 526: Syntax for variable annotations ----------------------------------------- - -:pep:`484` introduced standard for type annotations of function parameters, -a.k.a. type hints. This PEP adds syntax to Python for annotating the -types of variables including class variables and instance variables:: - - primes: List[int] = [] - - captain: str # Note: no initial value! - - class Starship: - stats: Dict[str, int] = {} - -Just as for function annotations, the Python interpreter does not attach any -particular meaning to variable annotations and only stores them in a special -attribute ``__annotations__`` of a class or module. -In contrast to variable declarations in statically typed languages, -the goal of annotation syntax is to provide an easy way to specify structured -type metadata for third party tools and libraries via the abstract syntax tree -and the ``__annotations__`` attribute. - -.. seealso:: - - :pep:`526` -- Syntax for variable annotations. - PEP written by Ryan Gonzalez, Philip House, Ivan Levkivskyi, Lisa Roach, - and Guido van Rossum. Implemented by Ivan Levkivskyi. - - Tools that use or will use the new syntax: - `mypy `_, - `pytype `_, PyCharm, etc. - - -.. _pep-529: +.. _whatsnew36-pep529: PEP 529: Change Windows filesystem encoding to UTF-8 ---------------------------------------------------- @@ -356,28 +462,8 @@ correctly encoded. To revert to the previous behaviour, set See :pep:`529` for more information and discussion of code modifications that may be required. -.. _whatsnew-pep487: -PEP 487: Simpler customization of class creation ------------------------------------------------- - -Upon subclassing a class, the ``__init_subclass__`` classmethod (if defined) is -called on the base class. This makes it straightforward to write classes that -customize initialization of future subclasses without introducing the -complexity of a full custom metaclass. - -The descriptor protocol has also been expanded to include a new optional method, -``__set_name__``. Whenever a new class is defined, the new method will be called -on all descriptors included in the definition, providing them with a reference -to the class being defined and the name given to the descriptor within the -class namespace. - -Also see :pep:`487` and the updated class customization documentation at -:ref:`class-customization` and :ref:`descriptors`. - -(Contributed by Martin Teichmann in :issue:`27366`) - -.. _pep-528: +.. _whatsnew36-pep528: PEP 528: Change Windows console encoding to UTF-8 ------------------------------------------------- @@ -395,6 +481,104 @@ console use, set :envvar:`PYTHONLEGACYWINDOWSIOENCODING`. :pep:`528` -- Change Windows console encoding to UTF-8 PEP written and implemented by Steve Dower. + +.. _whatsnew36-pep520: + +PEP 520: Preserving Class Attribute Definition Order +---------------------------------------------------- + +Attributes in a class definition body have a natural ordering: the same +order in which the names appear in the source. This order is now +preserved in the new class's ``__dict__`` attribute. + +Also, the effective default class *execution* namespace (returned from +``type.__prepare__()``) is now an insertion-order-preserving mapping. + +.. seealso:: + + :pep:`520` -- Preserving Class Attribute Definition Order + PEP written and implemented by Eric Snow. + + +.. _whatsnew36-pep468: + +PEP 468: Preserving Keyword Argument Order +------------------------------------------ + +``**kwargs`` in a function signature is now guaranteed to be an +insertion-order-preserving mapping. + +.. seealso:: + + :pep:`468` -- Preserving Keyword Argument Order + PEP written and implemented by Eric Snow. + + +.. _whatsnew36-compactdict: + +New :ref:`dict ` implementation +--------------------------------------------- + +The :ref:`dict ` type now uses a "compact" representation +`pioneered by PyPy `_. +The memory usage of the new :func:`dict` is between 20% and 25% smaller +compared to Python 3.5. +:pep:`468` (Preserving the order of ``**kwargs`` in a function.) is +implemented by this. The order-preserving aspect of this new +implementation is considered an implementation detail and should +not be relied upon (this may change in the future, but it is desired +to have this new dict implementation in the language for a few +releases before changing the language spec to mandate +order-preserving semantics for all current and future Python +implementations; this also helps preserve backwards-compatibility +with older versions of the language where random iteration order is +still in effect, e.g. Python 3.5). +(Contributed by INADA Naoki in :issue:`27350`. Idea +`originally suggested by Raymond Hettinger +`_.) + + +.. _whatsnew36-pep523: + +PEP 523: Adding a frame evaluation API to CPython +------------------------------------------------- + +While Python provides extensive support to customize how code +executes, one place it has not done so is in the evaluation of frame +objects. If you wanted some way to intercept frame evaluation in +Python there really wasn't any way without directly manipulating +function pointers for defined functions. + +:pep:`523` changes this by providing an API to make frame +evaluation pluggable at the C level. This will allow for tools such +as debuggers and JITs to intercept frame evaluation before the +execution of Python code begins. This enables the use of alternative +evaluation implementations for Python code, tracking frame +evaluation, etc. + +This API is not part of the limited C API and is marked as private to +signal that usage of this API is expected to be limited and only +applicable to very select, low-level use-cases. Semantics of the +API will change with Python as necessary. + +.. seealso:: + + :pep:`523` -- Adding a frame evaluation API to CPython + PEP written by Brett Cannon and Dino Viehland. + + +.. _whatsnew36-pep509: + +PEP 509: Add a private version to dict +-------------------------------------- + +Add a new private version to the builtin ``dict`` type, incremented at +each dictionary creation and at each dictionary change, to implement +fast guards on namespaces. + +(Contributed by Victor Stinner in :issue:`26058`.) + + PYTHONMALLOC environment variable --------------------------------- @@ -469,6 +653,8 @@ Example of fatal error on buffer overflow using (Contributed by Victor Stinner in :issue:`26516` and :issue:`26564`.) +.. _whatsnew36-tracing: + DTrace and SystemTap probing support ------------------------------------ @@ -494,71 +680,14 @@ markers may be added in the future. Jesús Cea Avión, David Malcolm, and Nikhil Benesch.) -.. _whatsnew-deforder: - -PEP 520: Preserving Class Attribute Definition Order ----------------------------------------------------- - -Attributes in a class definition body have a natural ordering: the same -order in which the names appear in the source. This order is now -preserved in the new class's ``__dict__`` attribute. - -Also, the effective default class *execution* namespace (returned from -``type.__prepare__()``) is now an insertion-order-preserving mapping. - -.. seealso:: - - :pep:`520` -- Preserving Class Attribute Definition Order - PEP written and implemented by Eric Snow. - - -.. _whatsnew-kwargs: - -PEP 468: Preserving Keyword Argument Order ------------------------------------------- - -``**kwargs`` in a function signature is now guaranteed to be an -insertion-order-preserving mapping. - -.. seealso:: - - :pep:`468` -- Preserving Keyword Argument Order - PEP written and implemented by Eric Snow. - -.. _whatsnew-pep509: - -PEP 509: Add a private version to dict --------------------------------------- - -Add a new private version to the builtin ``dict`` type, incremented at -each dictionary creation and at each dictionary change, to implement -fast guards on namespaces. - -(Contributed by Victor Stinner in :issue:`26058`.) - - Other Language Changes ====================== Some smaller changes made to the core Python language are: -* :func:`dict` now uses a "compact" representation `pioneered by PyPy - `_. - The memory usage of the new :func:`dict` is between 20% and 25% smaller - compared to Python 3.5. - :pep:`468` (Preserving the order of ``**kwargs`` in a function.) is - implemented by this. The order-preserving aspect of this new - implementation is considered an implementation detail and should - not be relied upon (this may change in the future, but it is desired - to have this new dict implementation in the language for a few - releases before changing the language spec to mandate - order-preserving semantics for all current and future Python - implementations; this also helps preserve backwards-compatibility - with older versions of the language where random iteration order is - still in effect, e.g. Python 3.5). - (Contributed by INADA Naoki in :issue:`27350`. Idea - `originally suggested by Raymond Hettinger - `_.) +* A ``global`` or ``nonlocal`` statement must now textually appear + before the first use of the affected name in the same scope. + Previously this was a ``SyntaxWarning``. * Long sequences of repeated traceback lines are now abbreviated as ``"[Previous line repeated {count} more times]"`` (see @@ -573,16 +702,15 @@ Some smaller changes made to the core Python language are: New Modules =========== -* None yet. +secrets +------- + +The new :mod:`secrets` module. Improved Modules ================ -On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool -is initialized to increase the security. See the :pep:`524` for the rationale. - - asyncio ------- @@ -784,11 +912,14 @@ iterator is neither exhausted nor explicitly closed a :exc:`ResourceWarning` will be emitted in its destructor. (Contributed by Serhiy Storchaka in :issue:`25994`.) +On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool +is initialized to increase the security. See the :pep:`524` for the rationale. + The Linux ``getrandom()`` syscall (get random bytes) is now exposed as the new :func:`os.getrandom` function. (Contributed by Victor Stinner, part of the :pep:`524`) -See the summary for :ref:`PEP 519 ` for details on how the +See the summary for :ref:`PEP 519 ` for details on how the :mod:`os` and :mod:`os.path` modules now support :term:`path-like objects `. @@ -1160,6 +1291,7 @@ Optimizations it is now about 1.5--4 times faster. (Contributed by Serhiy Storchaka in :issue:`26032`). + Build and C API Changes =======================