**************************** What's New In Python 3.4 **************************** .. :Author: Someone (uncomment if there is a principal author) .. Rules for maintenance: * Anyone can add text to this document, but the maintainer reserves the right to rewrite any additions. In particular, for obscure or esoteric features, the maintainer may reduce any addition to a simple reference to the new documentation rather than explaining the feature inline. * While the maintainer will periodically go through Misc/NEWS and add changes, it's best not to rely on this. We know from experience that any changes that aren't in the What's New documentation around the time of the original release will remain largely unknown to the community for years, even if they're added later. We also know from experience that other priorities can arise, and the maintainer will run out of time to do updates - in such cases, end users will be much better served by partial notifications that at least give a hint about new features to investigate. * This is not a complete list of every single change; completeness is the purpose of Misc/NEWS. The What's New should focus on changes that are visible to Python *users* and that *require* a feature release (i.e. most bug fixes should only be recorded in Misc/NEWS) * PEPs should not be marked Final until they have an entry in What's New. A placeholder entry that is just a section header and a link to the PEP (e.g ":pep:`397` has been implemented") is acceptable. If a PEP has been implemented and noted in What's New, don't forget to mark it as Final! * If you want to draw your new text to the attention of the maintainer, add 'XXX' to the beginning of the paragraph or section. * It's OK to add just a very brief note about a change. For example: "The :ref:`~socket.transmogrify()` function was added to the :mod:`socket` module." The maintainer will research the change and write the necessary text (if appropriate). The advantage of doing this is that even if no more descriptive text is ever added, readers will at least have a notification that the new feature exists and a link to the relevant documentation. * You can comment out your additions if you like, but it's not necessary (especially when a final release is some months away). * Credit the author of a patch or bugfix. Just the name is sufficient; the e-mail address isn't necessary. * It's helpful to add the bug/patch number as a comment: The :ref:`~socket.transmogrify()` function was added to the :mod:`socket` module. (Contributed by P.Y. Developer in :issue:`12345`.) This saves the maintainer the effort of going through the Mercurial log when researching a change. * Cross referencing tip: :ref:`mod.attr` will display as ``mod.attr``, while :ref:`~mod.attr` will display as ``attr``. This article explains the new features in Python 3.4, compared to 3.3. .. Python 3.4 was released on TBD. For full details, see the `changelog `_. .. note:: Prerelease users should be aware that this document is currently in draft form. It will be updated substantially as Python 3.4 moves towards release, so it's worth checking back even after reading earlier versions. .. seealso:: .. :pep:`4XX` - Python 3.4 Release Schedule Summary -- Release highlights ============================= .. This section singles out the most important changes in Python 3.4. Brevity is key. New syntax features: * None yet. New library modules: * :mod:`enum`: Implementation of the :pep:`435`. * :mod:`selectors`: High-level and efficient I/O multiplexing, built upon the :mod:`select` module primitives. New built-in features: * :ref:`PEP 442: Safe object finalization `. * :ref:`PEP 445: Configurable memory allocators `. * :ref:`PEP 446: Make newly created file descriptors non-inheritable `. Implementation improvements: * A more efficient :mod:`marshal` format (:issue:`16475`). * Improve finalization of Python modules to avoid setting their globals to None, in most cases (:issue:`18214`). Significantly Improved Library Modules: * Single-dispatch generic functions (:pep:`443`) * SHA-3 (Keccak) support for :mod:`hashlib`. * TLSv1.1 and TLSv1.2 support for :mod:`ssl`. * :mod:`multiprocessing` now has option to avoid using :func:`os.fork` on Unix (:issue:`8713`). Security improvements: * command line option for :ref:`isolated mode `, :issue:`16499`. Please read on for a comprehensive list of user-facing changes. .. _pep-446: PEP 446: Make newly created file descriptors non-inheritable ============================================================ The :pep:`446` makes newly created file descriptors :ref:`non-inheritable `. New functions and methods: * :func:`os.get_inheritable`, :func:`os.set_inheritable` * :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable` * :meth:`socket.socket.get_inheritable`, :meth:`socket.socket.set_inheritable` .. _pep-445: PEP 445: Add new APIs to customize Python memory allocators =========================================================== The :pep:`445` adds new Application Programming Interfaces (API) to customize Python memory allocators. .. _pep-442: PEP 442: Safe object finalization ================================= This PEP removes the current limitations and quirks of object finalization. With it, objects with :meth:`__del__` methods, as well as generators with :keyword:`finally` clauses, can be finalized when they are part of a reference cycle. .. seealso:: :pep:`442` - Safe object finalization PEP written and implemented by Antoine Pitrou Other Language Changes ====================== Some smaller changes made to the core Python language are: * Unicode database updated to UCD version 6.2. * :func:`min` and :func:`max` now accept a *default* argument that can be used to specify the value they return if the iterable they are evaluating has no elements. Contributed by Julian Berman in :issue:`18111`. * Module objects are now :mod:`weakref`'able. New Modules =========== selectors --------- The new :mod:`selectors` module allows high-level and efficient I/O multiplexing, built upon the :mod:`select` module primitives. Improved Modules ================ aifc ---- The :meth:`~aifc.getparams` method now returns a namedtuple rather than a plain tuple. (Contributed by Claudiu Popa in :issue:`17818`.) colorsys -------- The number of digits in the coefficients for the RGB --- YIQ conversions have been expanded so that they match the FCC NTSC versions. The change in results should be less than 1% and may better match results found elsewhere. dis --- The :mod:`dis` module is now built around an :class:`Instruction` class that provides details of individual bytecode operations and a :func:`get_instructions` iterator that emits the Instruction stream for a given piece of Python code. The various display tools in the :mod:`dis` module have been updated to be based on these new components. The new :class:`dis.Bytecode` class provides an object-oriented API for inspecting bytecode, both in human-readable form and for iterating over instructions. (Contributed by Nick Coghlan, Ryan Kelly and Thomas Kluyver in :issue:`11816`) doctest ------- Added :data:`~doctest.FAIL_FAST` flag to halt test running as soon as the first failure is detected. (Contributed by R. David Murray and Daniel Urban in :issue:`16522`.) Updated the doctest command line interface to use :mod:`argparse`, and added ``-o`` and ``-f`` options to the interface. ``-o`` allows doctest options to be specified on the command line, and ``-f`` is a shorthand for ``-o FAIL_FAST`` (to parallel the similar option supported by the :mod:`unittest` CLI). (Contributed by R. David Murray in :issue:`11390`.) email ----- :meth:`~email.message.Message.as_string` now accepts a *policy* argument to override the default policy of the message when generating a string representation of it. This means that ``as_string`` can now be used in more circumstances, instead of having to create and use a :mod:`~email.generator` in order to pass formatting parameters to its ``flatten`` method. New method :meth:`~email.message.Message.as_bytes` added to produce a bytes representation of the message in a fashion similar to how ``as_string`` produces a string representation. It does not accept the *maxheaderlen* argument, but does accept the *unixfrom* and *policy* arguments. The :class:`~email.message.Message` :meth:`~email.message.Message.__bytes__` method calls it, meaning that ``bytes(mymsg)`` will now produce the intuitive result: a bytes object containing the fully formatted message. (Contributed by R. David Murray in :issue:`18600`.) functools --------- New :func:`functools.singledispatch` decorator: see the :pep:`443`. inspect ------- :func:`~inspect.unwrap` makes it easy to unravel wrapper function chains created by :func:`functools.wraps` (and any other API that sets the ``__wrapped__`` attribute on a wrapper function). mmap ---- mmap objects can now be weakref'ed. (Contributed by Valerie Lambert in :issue:`4885`.) multiprocessing --------------- On Unix two new *start methods* have been added for starting processes using :mod:`multiprocessing`. These make the mixing of processes with threads more robust. See :issue:`8713`. Also, except when using the old *fork* start method, child processes will no longer inherit unneeded handles/file descriptors from their parents. os -- New functions to get and set the `inheritable flag `_ of a file descriptors or a Windows handle: * :func:`os.get_inheritable`, :func:`os.set_inheritable` * :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable` poplib ------ New :meth:`~poplib.POP3.stls` method to switch a clear-text POP3 session into an encrypted POP3 session. New :meth:`~poplib.POP3.capa` method to query the capabilities advertised by the POP3 server. (Contributed by Lorenzo Catucci in :issue:`4473`.) smtplib ------- :exc:`~smtplib.SMTPException` is now a subclass of :exc:`OSError`, which allows both socket level errors and SMTP protocol level errors to be caught in one try/except statement by code that only cares whether or not an error occurred. (:issue:`2118`). socket ------ Socket objects have new methods to get or set their :ref:`inheritable flag `: * :meth:`socket.socket.get_inheritable`, :meth:`socket.socket.set_inheritable` The ``socket.AF_*`` and ``socket.SOCK_*`` constants are enumeration values, using the new :mod:`enum` module. This allows descriptive reporting during debugging, instead of seeing integer "magic numbers". ssl --- TLSv1.1 and TLSv1.2 support. (Contributed by Michele Orrù and Antoine Pitrou in :issue:`16692`) * New diagnostic functions :func:`~ssl.get_default_verify_paths`, :meth:`~ssl.SSLContext.cert_store_stats` and :meth:`~ssl.SSLContext.get_ca_certs` * Add :func:`ssl.enum_cert_store` to retrieve certificates and CRL from Windows' cert store. (Contributed by Christian Heimes in :issue:`18143`, :issue:`18147` and :issue:`17134`.) Support for server-side SNI using the new :meth:`ssl.SSLContext.set_servername_callback` method. (Contributed by Daniel Black in :issue:`8109`.) stat ---- The :mod:`stat` module is now backed by a C implementation in :mod:`_stat`. A C implementation is required as most of the values aren't standardized and platform-dependent. (Contributed by Christian Heimes in :issue:`11016`.) The module supports new file types: door, event port and whiteout. struct ------ Streaming struct unpacking using :func:`struct.iter_unpack`. (Contributed by Antoine Pitrou in :issue:`17804`.) sunau ----- The :meth:`~sunau.getparams` method now returns a namedtuple rather than a plain tuple. (Contributed by Claudiu Popa in :issue:`18901`.) :meth:`sunau.open` now supports the context manager protocol (:issue:`18878`). urllib ------ Add support.for ``data:`` URLs in :mod:`urllib.request`. (Contributed by Mathias Panzenböck in :issue:`16423`.) unittest -------- Support for easy dynamically-generated subtests using the :meth:`~unittest.TestCase.subTest` context manager. (Contributed by Antoine Pitrou in :issue:`16997`.) wave ---- The :meth:`~wave.getparams` method now returns a namedtuple rather than a plain tuple. (Contributed by Claudiu Popa in :issue:`17487`.) :meth:`wave.open` now supports the context manager protocol. (Contributed by Claudiu Popa in :issue:`17616`.) weakref ------- New :class:`~weakref.WeakMethod` class simulates weak references to bound methods. (Contributed by Antoine Pitrou in :issue:`14631`.) xml.etree --------- Add an event-driven parser for non-blocking applications, :class:`~xml.etree.ElementTree.XMLPullParser`. (Contributed by Antoine Pitrou in :issue:`17741`.) Other improvements ================== Tab-completion is now enabled by default in the interactive interpreter. (Contributed by Antoine Pitrou and Éric Araujo in :issue:`5845`.) Python invocation changes ========================= Invoking the Python interpreter with ``--version`` now outputs the version to standard output instead of standard error (:issue:`18338`). Similar changes were made to :mod:`argparse` (:issue:`18920`) and other modules that have script-like invocation capabilities (:issue:`18922`). Optimizations ============= Major performance enhancements have been added: * The UTF-32 decoder is now 3x to 4x faster. * The cost of hash collisions for sets is now reduced. Each hash table probe now checks a second key/hash pair for each cache line retrieved. This exploits cache locality to make collision resolution less expensive. (Contributed by Raymond Hettinger in :issue"`18771`.) Build and C API Changes ======================= Changes to Python's build process and to the C API include: * None yet. Deprecated ========== Unsupported Operating Systems ----------------------------- * OS/2 * Windows 2000 Deprecated Python modules, functions and methods ------------------------------------------------ * :meth:`difflib.SequenceMatcher.isbjunk` and :meth:`difflib.SequenceMatcher.isbpopular` were removed: use ``x in sm.bjunk`` and ``x in sm.bpopular``, where *sm* is a :class:`~difflib.SequenceMatcher` object. * :func:`importlib.util.module_for_loader` is pending deprecation. Using :func:`importlib.util.module_to_load` and :meth:`importlib.abc.Loader.init_module_attrs` allows subclasses of a loader to more easily customize module loading. * The :mod:`imp` module is pending deprecation. To keep compatibility with Python 2/3 code bases, the module's removal is currently not scheduled. Deprecated functions and types of the C API ------------------------------------------- * None yet. Deprecated features ------------------- * None yet. Porting to Python 3.4 ===================== This section lists previously described changes and other bugfixes that may require changes to your code. * The ABCs defined in :mod:`importlib.abc` now either raise the appropriate exception or return a default value instead of raising :exc:`NotImplementedError` blindly. This will only affect code calling :func:`super` and falling through all the way to the ABCs. For compatibility, catch both :exc:`NotImplementedError` or the appropriate exception as needed. * The module type now initializes the :attr:`__package__` and :attr:`__loader__` attributes to ``None`` by default. To determine if these attributes were set in a backwards-compatible fashion, use e.g. ``getattr(module, '__loader__', None) is not None``. * :meth:`importlib.util.module_for_loader` now sets ``__loader__`` and ``__package__`` unconditionally to properly support reloading. If this is not desired then you will need to set these attributes manually. You can use :func:`importlib.util.module_to_load` for module management. * Import now resets relevant attributes (e.g. ``__name__``, ``__loader__``, ``__package__``, ``__file__``, ``__cached__``) unconditionally when reloading. * Frozen packages no longer set ``__path__`` to a list containg the package name but an empty list instead. Determing if a module is a package should be done using ``hasattr(module, '__path__')``. * :c:func:`PyErr_SetImportError` now sets :exc:`TypeError` when its **msg** argument is not set. Previously only ``NULL`` was returned with no exception set. * :func:`py_compile.compile` now raises :exc:`FileExistsError` if the file path it would write to is a symlink or a non-regular file. This is to act as a warning that import will overwrite those files with a regular file regardless of what type of file path they were originally. * :meth:`importlib.abc.SourceLoader.get_source` no longer raises :exc:`ImportError` when the source code being loaded triggers a :exc:`SyntaxError` or :exc:`UnicodeDecodeError`. As :exc:`ImportError` is meant to be raised only when source code cannot be found but it should, it was felt to be over-reaching/overloading of that meaning when the source code is found but improperly structured. If you were catching ImportError before and wish to continue to ignore syntax or decoding issues, catch all three exceptions now. * :func:`functools.update_wrapper` and :func:`functools.wraps` now correctly set the ``__wrapped__`` attribute even if the wrapped function had a wrapped attribute set. This means ``__wrapped__`` attributes now correctly link a stack of decorated functions rather than every ``__wrapped__`` attribute in the chain referring to the innermost function. Introspection libraries that assumed the previous behaviour was intentional can use :func:`inspect.unwrap` to gain equivalent behaviour.