.. bpo: 42988 .. date: 2021-03-24-14-16-56 .. nonce: P2aNco .. release date: 2021-04-05 .. section: Security :cve:`2021-3426`: Remove the ``getfile`` feature of the :mod:`pydoc` module which could be abused to read arbitrary files on the disk (directory traversal vulnerability). Moreover, even source code of Python modules can contain sensitive data like passwords. Vulnerability reported by David Schwörer. .. .. bpo: 43285 .. date: 2021-03-13-03-48-14 .. nonce: g-Hah3 .. section: Security :mod:`ftplib` no longer trusts the IP address value returned from the server in response to the PASV command by default. This prevents a malicious FTP server from using the response to probe IPv4 address and port combinations on the client network. Code that requires the former vulnerable behavior may set a ``trust_server_pasv_ipv4_address`` attribute on their :class:`ftplib.FTP` instances to ``True`` to re-enable it. .. .. bpo: 43439 .. date: 2021-03-08-23-06-07 .. nonce: 5U3lXm .. section: Security Add audit hooks for :func:`gc.get_objects`, :func:`gc.get_referrers` and :func:`gc.get_referents`. Patch by Pablo Galindo. .. .. bpo: 27129 .. date: 2021-04-03-22-04-46 .. nonce: u_ehHb .. section: Core and Builtins Update CPython bytecode magic number. .. .. bpo: 43672 .. date: 2021-03-31-09-12-54 .. nonce: jTT5uG .. section: Core and Builtins Raise ImportWarning when calling find_loader(). .. .. bpo: 43660 .. date: 2021-03-29-19-50-34 .. nonce: scTgag .. section: Core and Builtins Fix crash that happens when replacing ``sys.stderr`` with a callable that can remove the object while an exception is being printed. Patch by Pablo Galindo. .. .. bpo: 27129 .. date: 2021-03-29-16-20-29 .. nonce: BF03A5 .. section: Core and Builtins The bytecode interpreter uses instruction, rather byte, offsets internally. This reduces the number of EXTENDED_ARG instructions needed and streamlines instruction dispatch a bit. .. .. bpo: 40645 .. date: 2021-03-29-11-55-06 .. nonce: PhaT-B .. section: Core and Builtins Fix reference leak in the :mod:`!_hashopenssl` extension. Patch by Pablo Galindo. .. .. bpo: 42134 .. date: 2021-03-26-17-30-19 .. nonce: G4Sjxg .. section: Core and Builtins Calls to find_module() by the import system now raise ImportWarning. .. .. bpo: 41064 .. date: 2021-03-24-00-32-20 .. nonce: _H0K_g .. section: Core and Builtins Improve the syntax error for invalid usage of double starred elements ('**') in f-strings. Patch by Pablo Galindo. .. .. bpo: 43575 .. date: 2021-03-21-12-26-32 .. nonce: pl-nSg .. section: Core and Builtins Speed up calls to ``map()`` by using the :pep:`590` ``vectorcall`` calling convention. Patch by Donghee Na. .. .. bpo: 42137 .. date: 2021-03-20-19-54-47 .. nonce: A8aQvj .. section: Core and Builtins The import system now prefers using ``__spec__`` for ``ModuleType.__repr__`` over ``module_repr()``. .. .. bpo: 43452 .. date: 2021-03-20-01-21-37 .. nonce: tDVJkc .. section: Core and Builtins Added micro-optimizations to ``_PyType_Lookup()`` to improve cache lookup performance in the common case of cache hits. .. .. bpo: 43555 .. date: 2021-03-19-22-49-40 .. nonce: ZmhYSA .. section: Core and Builtins Report the column offset for :exc:`SyntaxError` for invalid line continuation characters. Patch by Pablo Galindo. .. .. bpo: 43517 .. date: 2021-03-16-17-12-54 .. nonce: zAo6Ws .. section: Core and Builtins Fix misdetection of circular imports when using ``from pkg.mod import attr``, which caused false positives in non-trivial multi-threaded code. .. .. bpo: 43497 .. date: 2021-03-15-07-50-30 .. nonce: Uc5ZCJ .. section: Core and Builtins Emit SyntaxWarnings for assertions with tuple constants, this is a regression introduced in python3.7 .. .. bpo: 39316 .. date: 2021-03-14-16-44-50 .. nonce: Ns3a_F .. section: Core and Builtins Tracing now has correct line numbers for attribute accesses when the attribute is on a different line from the object. Improves debugging and profiling for multi-line method chains. .. .. bpo: 35883 .. date: 2021-03-13-13-57-21 .. nonce: UyGpdG .. section: Core and Builtins Python no longer fails at startup with a fatal error if a command line argument contains an invalid Unicode character. The :c:func:`Py_DecodeLocale` function now escapes byte sequences which would be decoded as Unicode characters outside the [U+0000; U+10ffff] range. .. .. bpo: 43410 .. date: 2021-03-05-17-23-36 .. nonce: lCzIg0 .. section: Core and Builtins Fix a bug that was causing the parser to crash when emitting syntax errors when reading input from stdin. Patch by Pablo Galindo .. .. bpo: 43406 .. date: 2021-03-04-22-53-10 .. nonce: Na_VpA .. section: Core and Builtins Fix a possible race condition where ``PyErr_CheckSignals`` tries to execute a non-Python signal handler. .. .. bpo: 42128 .. date: 2021-03-03-19-04-23 .. nonce: VouZjn .. section: Core and Builtins Add ``__match_args__`` to :ref:`struct sequence objects `. Patch by Pablo Galindo. .. .. bpo: 43390 .. date: 2021-03-03-17-58-49 .. nonce: epPpwV .. section: Core and Builtins CPython now sets the ``SA_ONSTACK`` flag in ``PyOS_setsig`` for the VM's default signal handlers. This is friendlier to other in-process code that an extension module or embedding use could pull in (such as Golang's cgo) where tiny thread stacks are the norm and ``sigaltstack()`` has been used to provide for signal handlers. This is a no-op change for the vast majority of processes that don't use sigaltstack. .. .. bpo: 43287 .. date: 2021-02-22-03-01-02 .. nonce: aTs6fO .. section: Core and Builtins Speed up calls to ``filter()`` by using the :pep:`590` ``vectorcall`` calling convention. Patch by Donghee Na. .. .. bpo: 37448 .. date: 2021-02-21-14-19-35 .. nonce: btl7vO .. section: Core and Builtins Add a radix tree based memory map to track in-use obmalloc arenas. Use to replace the old implementation of address_in_range(). The radix tree approach makes it easy to increase pool sizes beyond the OS page size. Boosting the pool and arena size allows obmalloc to handle a significantly higher percentage of requests from its ultra-fast paths. It also has the advantage of eliminating the memory unsanitary behavior of the previous address_in_range(). The old address_in_range() was marked with the annotations _Py_NO_SANITIZE_ADDRESS, _Py_NO_SANITIZE_THREAD, and _Py_NO_SANITIZE_MEMORY. Those annotations are no longer needed. To disable the radix tree map, set a preprocessor flag as follows: ``-DWITH_PYMALLOC_RADIX_TREE=0``. Co-authored-by: Tim Peters .. .. bpo: 29988 .. date: 2020-02-03-13-23-10 .. nonce: 8_UB5w .. section: Core and Builtins Only handle asynchronous exceptions and requests to drop the GIL when returning from a call or on the back edges of loops. Makes sure that :meth:`~object.__exit__` is always called in with statements, even for interrupts. .. .. bpo: 43720 .. date: 2021-04-03-13-45-51 .. nonce: FDZ5cZ .. section: Library Document various stdlib deprecations in imp, pkgutil, and importlib.util for removal in Python 3.12. .. .. bpo: 43433 .. date: 2021-03-28-23-50-20 .. nonce: so9j5G .. section: Library :class:`xmlrpc.client.ServerProxy` no longer ignores query and fragment in the URL of the server. .. .. bpo: 31956 .. date: 2021-03-28-16-53-25 .. nonce: Lt_67U .. section: Library The :meth:`~array.array.index` method of :class:`array.array` now has optional *start* and *stop* parameters. .. .. bpo: 40066 .. date: 2021-03-25-21-26-30 .. nonce: 7EBQ3_ .. section: Library Enum: adjust ``repr()`` to show only enum and member name (not value, nor angle brackets) and ``str()`` to show only member name. Update and improve documentation to match. .. .. bpo: 42136 .. date: 2021-03-25-08-44-26 .. nonce: rRY9e1 .. section: Library Deprecate all module_repr() methods found in importlib as their use is being phased out by Python 3.12. .. .. bpo: 35930 .. date: 2021-03-23-17-18-56 .. nonce: RZ51pM .. section: Library Raising an exception raised in a "future" instance will create reference cycles. .. .. bpo: 41369 .. date: 2021-03-21-17-50-42 .. nonce: -fpmYZ .. section: Library Finish updating the vendored libmpdec to version 2.5.1. Patch by Stefan Krah. .. .. bpo: 43422 .. date: 2021-03-21-17-02-52 .. nonce: POk6cU .. section: Library Revert the _decimal C API which was added in bpo-41324. .. .. bpo: 43577 .. date: 2021-03-21-10-13-17 .. nonce: m7JnAV .. section: Library Fix deadlock when using :class:`ssl.SSLContext` debug callback with :meth:`ssl.SSLContext.sni_callback`. .. .. bpo: 43571 .. date: 2021-03-20-17-40-35 .. nonce: acAL0W .. section: Library It's now possible to create MPTCP sockets with IPPROTO_MPTCP .. .. bpo: 43542 .. date: 2021-03-20-15-43-25 .. nonce: 6bt2F6 .. section: Library ``image/heic`` and ``image/heif`` were added to :mod:`mimetypes`. .. .. bpo: 40645 .. date: 2021-03-19-10-22-17 .. nonce: 5pXhb- .. section: Library The :mod:`hmac` module now uses OpenSSL's HMAC implementation when digestmod argument is a hash name or builtin hash function. .. .. bpo: 43510 .. date: 2021-03-16-17-20-33 .. nonce: -BeQH_ .. section: Library Implement :pep:`597`: Add ``EncodingWarning`` warning, ``-X warn_default_encoding`` option, :envvar:`PYTHONWARNDEFAULTENCODING` environment variable and ``encoding="locale"`` argument value. .. .. bpo: 43521 .. date: 2021-03-16-16-05-02 .. nonce: mRT6fh .. section: Library ``ast.unparse`` can now render NaNs and empty sets. .. .. bpo: 42914 .. date: 2021-03-14-21-47-28 .. nonce: 9U1o33 .. section: Library :func:`pprint.pprint` gains a new boolean ``underscore_numbers`` optional argument to emit integers with thousands separated by an underscore character for improved readability (for example ``1_000_000`` instead of ``1000000``). .. .. bpo: 41361 .. date: 2021-03-13-08-18-01 .. nonce: lXDIlr .. section: Library :meth:`~collections.deque.rotate` calls are now slightly faster due to faster argument parsing. .. .. bpo: 43423 .. date: 2021-03-11-15-44-18 .. nonce: rRomRD .. section: Library :func:`subprocess.communicate` no longer raises an IndexError when there is an empty stdout or stderr IO buffer during a timeout on Windows. .. .. bpo: 27820 .. date: 2021-03-10-14-07-44 .. nonce: Wwdy-r .. section: Library Fixed long-standing bug of smtplib.SMTP where doing AUTH LOGIN with initial_response_ok=False will fail. The cause is that SMTP.auth_login _always_ returns a password if provided with a challenge string, thus non-compliant with the standard for AUTH LOGIN. Also fixes bug with the test for smtpd. .. .. bpo: 43445 .. date: 2021-03-09-11-36-19 .. nonce: jnj-UB .. section: Library Add frozen modules to :data:`sys.stdlib_module_names`. For example, add ``"_frozen_importlib"`` and ``"_frozen_importlib_external"`` names. .. .. bpo: 43245 .. date: 2021-03-08-22-14-37 .. nonce: nXL-MC .. section: Library Add keyword arguments support to ``ChainMap.new_child()``. .. .. bpo: 29982 .. date: 2021-03-07-23-23-03 .. nonce: Q9iszT .. section: Library Add optional parameter *ignore_cleanup_errors* to :func:`tempfile.TemporaryDirectory` and allow multiple :func:`cleanup` attempts. Contributed by C.A.M. Gerlach. .. .. bpo: 43428 .. date: 2021-03-07-18-54-39 .. nonce: br0XmX .. section: Library Include changes from `importlib_metadata 3.7 `_: Performance enhancements to distribution discovery. ``entry_points`` only returns unique distributions. Introduces new ``EntryPoints`` object for containing a set of entry points with convenience methods for selecting entry points by group or name. ``entry_points`` now returns this object if selection parameters are supplied but continues to return a dict object for compatibility. Users are encouraged to rely on the selection interface. The dict object result is likely to be deprecated in the future. Added packages_distributions function to return a mapping of packages to the distributions that provide them. .. .. bpo: 43332 .. date: 2021-03-07-11-23-20 .. nonce: weatsh .. section: Library Improves the networking efficiency of :mod:`http.client` when using a proxy via :meth:`~HTTPConnection.set_tunnel`. Fewer small send calls are made during connection setup. .. .. bpo: 43420 .. date: 2021-03-07-08-03-31 .. nonce: cee_X5 .. section: Library Improve performance of :class:`fractions.Fraction` arithmetics for large components. Contributed by Sergey B. Kirpichev. .. .. bpo: 43356 .. date: 2021-03-04-21-51-20 .. nonce: X7IGBM .. section: Library Allow passing a signal number to ``_thread.interrupt_main()``. .. .. bpo: 43399 .. date: 2021-03-04-17-53-46 .. nonce: Wn95u- .. section: Library Fix ``ElementTree.extend`` not working on iterators when using the Python implementation .. .. bpo: 43369 .. date: 2021-03-02-15-25-28 .. nonce: F4knlQ .. section: Library Improve :mod:`sqlite3` error handling: If ``sqlite3_column_text()`` and ``sqlite3_column_blob()`` set ``SQLITE_NOMEM``, :exc:`MemoryError` is now raised. Patch by Erlend E. Aasland. .. .. bpo: 43368 .. date: 2021-03-02-13-45-05 .. nonce: t9XEkQ .. section: Library Fix a regression introduced in PR 24562, where an empty bytestring was fetched as ``None`` instead of ``b''`` in :mod:`sqlite3`. Patch by Mariusz Felisiak. .. .. bpo: 41282 .. date: 2021-03-02-09-54-22 .. nonce: xL4h94 .. section: Library Fixed stacklevel of ``DeprecationWarning`` emitted from ``import distutils``. .. .. bpo: 42129 .. date: 2021-02-28-04-21-35 .. nonce: V0KifQ .. section: Library ``importlib.resources`` now honors namespace packages, merging resources from each location in the namespace as introduced in ``importlib_resources`` 3.2 and including incidental changes through 5.0.3. .. .. bpo: 43295 .. date: 2021-02-22-22-54-40 .. nonce: h_ffu7 .. section: Library :meth:`datetime.datetime.strptime` now raises ``ValueError`` instead of ``IndexError`` when matching ``'z'`` with the ``%z`` format specifier. .. .. bpo: 43125 .. date: 2021-02-07-19-13-30 .. nonce: AqNoMa .. section: Library Return empty string if base64mime.body_encode receive empty bytes .. .. bpo: 43084 .. date: 2021-01-31-17-31-13 .. nonce: i8nLpK .. section: Library :func:`curses.window.enclose` returns now ``True`` or ``False`` (as was documented) instead of ``1`` or ``0``. .. .. bpo: 42994 .. date: 2021-01-21-16-58-34 .. nonce: El0Ksp .. section: Library Add MIME types for opus, AAC, 3gpp and 3gpp2 .. .. bpo: 14678 .. date: 2021-01-07-21-25-49 .. nonce: 1zniCH .. section: Library Add an invalidate_caches() method to the zipimport.zipimporter class to support importlib.invalidate_caches(). Patch by Desmond Cheong. .. .. bpo: 42782 .. date: 2020-12-29-13-46-57 .. nonce: 3r0HFY .. section: Library Fail fast in :func:`shutil.move()` to avoid creating destination directories on failure. .. .. bpo: 40066 .. date: 2020-09-23-21-58-34 .. nonce: f1dr_5 .. section: Library Enum's ``repr()`` and ``str()`` have changed: ``repr()`` is now *EnumClass.MemberName* and ``str()`` is *MemberName*. Additionally, stdlib Enum's whose contents are available as module attributes, such as ``RegexFlag.IGNORECASE``, have their ``repr()`` as *module.name*, e.g. ``re.IGNORECASE``. .. .. bpo: 26053 .. date: 2020-09-01-10-12-13 .. nonce: hXikw_ .. section: Library Fixed bug where the :mod:`pdb` interactive run command echoed the args from the shell command line, even if those have been overridden at the pdb prompt. .. .. bpo: 24160 .. date: 2020-08-28-23-07-53 .. nonce: MSGnKr .. section: Library Fixed bug where breakpoints did not persist across multiple debugger sessions in :mod:`pdb`'s interactive mode. .. .. bpo: 40701 .. date: 2020-05-27-05-42-39 .. nonce: PBIgW1 .. section: Library When the :data:`tempfile.tempdir` global variable is set to a value of type bytes, it is now handled consistently. Previously exceptions could be raised from some tempfile APIs when the directory did not already exist in this situation. Also ensures that the :func:`tempfile.gettempdir()` and :func:`tempfile.gettempdirb()` functions *always* return ``str`` and ``bytes`` respectively. .. .. bpo: 39342 .. date: 2020-01-15-11-15-35 .. nonce: S8PuJO .. section: Library Expose ``X509_V_FLAG_ALLOW_PROXY_CERTS`` as :const:`~ssl.VERIFY_ALLOW_PROXY_CERTS` to allow proxy certificate validation as explained in https://www.openssl.org/docs/man1.1.1/man7/proxy-certificates.html. .. .. bpo: 31861 .. date: 2018-08-24-01-08-09 .. nonce: -q9RKJ .. section: Library Add builtins.aiter and builtins.anext. Patch by Joshua Bronson (@jab), Daniel Pope (@lordmauve), and Justin Wang (@justin39). .. .. bpo: 43199 .. date: 2021-03-13-18-43-54 .. nonce: ZWA6KX .. section: Documentation Answer "Why is there no goto?" in the Design and History FAQ. .. .. bpo: 43407 .. date: 2021-03-04-22-53-03 .. nonce: x570l5 .. section: Documentation Clarified that a result from :func:`time.monotonic`, :func:`time.perf_counter`, :func:`time.process_time`, or :func:`time.thread_time` can be compared with the result from any following call to the same function - not just the next immediate call. .. .. bpo: 43354 .. date: 2021-03-02-12-55-34 .. nonce: ezZYkx .. section: Documentation Fix type documentation for ``Fault.faultCode``; the type has to be ``int`` instead of ``str``. .. .. bpo: 41933 .. date: 2020-10-05-20-04-43 .. nonce: Pff94- .. section: Documentation Clarified wording of s * n in the Common Sequence Operations .. .. bpo: 37945 .. date: 2021-03-31-11-38-42 .. nonce: HTUYhv .. section: Tests Fix test_getsetlocale_issue1813() of test_locale: skip the test if ``setlocale()`` fails. Patch by Victor Stinner. .. .. bpo: 41561 .. date: 2021-03-18-10-34-42 .. nonce: pDg4w- .. section: Tests Add workaround for Ubuntu's custom OpenSSL security level policy. .. .. bpo: 43179 .. date: 2021-03-31-12-20-23 .. nonce: Qbe1OD .. section: Build Introduce and correctly use ALIGNOF_X in place of SIZEOF_X for alignment-related code in optimized string routines. Patch by Jessica Clarke. .. .. bpo: 43631 .. date: 2021-03-26-09-16-34 .. nonce: msJyPi .. section: Build Update macOS, Windows, and CI to OpenSSL 1.1.1k. .. .. bpo: 43617 .. date: 2021-03-24-16-55-55 .. nonce: d69KAv .. section: Build Improve configure.ac: Check for presence of autoconf-archive package and remove our copies of M4 macros. .. .. bpo: 43466 .. date: 2021-03-11-00-14-47 .. nonce: N861Z5 .. section: Build The ``configure`` script now supports ``--with-openssl-rpath`` option. .. .. bpo: 43372 .. date: 2021-03-04-17-13-57 .. nonce: FfqDVL .. section: Build Use ``_freeze_importlib`` to generate code for the ``__hello__`` module. This approach ensures the code matches the interpreter version. Previously, PYTHON_FOR_REGEN was used to generate the code, which might be wrong. The marshal format for code objects has changed with bpo-42246, commit 877df851. Update the code and the expected code sizes in ctypes test_frozentable. .. .. bpo: 43440 .. date: 2021-03-09-11-15-41 .. nonce: igy2Mn .. section: Windows Build :mod:`sqlite3` with the ``R*Tree`` module enabled. Patch by Erlend E. Aasland. .. .. bpo: 42225 .. date: 2021-03-29-16-22-27 .. nonce: iIeiLg .. section: IDLE Document that IDLE can fail on Unix either from misconfigured IP masquerade rules or failure displaying complex colored (non-ascii) characters. .. .. bpo: 43688 .. date: 2021-04-01-09-10-42 .. nonce: G4gs6k .. section: C API The limited C API is now supported if Python is built in debug mode (if the ``Py_DEBUG`` macro is defined). In the limited C API, the :c:func:`Py_INCREF` and :c:func:`Py_DECREF` functions are now implemented as opaque function calls, rather than accessing directly the :c:member:`PyObject.ob_refcnt` member, if Python is built in debug mode and the ``Py_LIMITED_API`` macro targets Python 3.10 or newer. It became possible to support the limited C API in debug mode because the :c:type:`PyObject` structure is the same in release and debug mode since Python 3.8 (see :issue:`36465`). The limited C API is still not supported in the ``--with-trace-refs`` special build (``Py_TRACE_REFS`` macro). Patch by Victor Stinner. .. .. bpo: 43244 .. date: 2021-03-24-01-22-14 .. nonce: 31-97x .. section: C API Remove the ``pyarena.h`` header file with functions: * ``PyArena_New()`` * ``PyArena_Free()`` * ``PyArena_Malloc()`` * ``PyArena_AddPyObject()`` These functions were undocumented, excluded from the limited C API, and were only used internally by the compiler. Patch by Victor Stinner. .. .. bpo: 43244 .. date: 2021-03-23-20-53-41 .. nonce: VK3sLH .. section: C API Remove the compiler and parser functions using ``struct _mod`` type, because the public AST C API was removed: * ``PyAST_Compile()`` * ``PyAST_CompileEx()`` * ``PyAST_CompileObject()`` * ``PyFuture_FromAST()`` * ``PyFuture_FromASTObject()`` * ``PyParser_ASTFromFile()`` * ``PyParser_ASTFromFileObject()`` * ``PyParser_ASTFromFilename()`` * ``PyParser_ASTFromString()`` * ``PyParser_ASTFromStringObject()`` These functions were undocumented and excluded from the limited C API. Patch by Victor Stinner. .. .. bpo: 43244 .. date: 2021-03-19-12-56-11 .. nonce: VuIyOD .. section: C API Remove ``ast.h``, ``asdl.h``, and ``Python-ast.h`` header files. These functions were undocumented and excluded from the limited C API. Most names defined by these header files were not prefixed by ``Py`` and so could create names conflicts. For example, ``Python-ast.h`` defined a ``Yield`` macro which was conflict with the ``Yield`` name used by the Windows ```` header. Use the Python :mod:`ast` module instead. Patch by Victor Stinner. .. .. bpo: 43541 .. date: 2021-03-18-12-44-33 .. nonce: ICigzd .. section: C API Fix a ``PyEval_EvalCodeEx()`` regression: fix reference counting on builtins. Patch by Victor Stinner. .. .. bpo: 43244 .. date: 2021-03-17-23-53-14 .. nonce: kfPqA_ .. section: C API Remove the ``symtable.h`` header file and the undocumented functions: * ``PyST_GetScope()`` * ``PySymtable_Build()`` * ``PySymtable_BuildObject()`` * ``PySymtable_Free()`` * ``Py_SymtableString()`` * ``Py_SymtableStringObject()`` The ``Py_SymtableString()`` function was part the stable ABI by mistake but it could not be used, because the ``symtable.h`` header file was excluded from the limited C API. The Python :mod:`symtable` module remains available and is unchanged. Patch by Victor Stinner. .. .. bpo: 43244 .. date: 2021-03-17-23-20-07 .. nonce: diyn2C .. section: C API Remove the ``PyAST_Validate()`` function. It is no longer possible to build a AST object (``mod_ty`` type) with the public C API. The function was already excluded from the limited C API (:pep:`384`). Patch by Victor Stinner.