Check that the current default heap is initialized in
`_mi_os_get_aligned_hint` and `mi_os_claim_huge_pages`.
The mimalloc function `_mi_os_get_aligned_hint` assumes that there is an
initialized default heap. This is true for our main thread, but not for
background threads. The problematic code path is usually called during
initialization (i.e., `Py_Initialize`), but it may also be called if the
program allocates large amounts of memory in total.
The crash only affected the free-threaded build.
(cherry picked from commit d061ffea7b)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-116622: Android logging fixes (GH-122698)
Modifies the handling of stdout/stderr redirection on Android to accomodate
the rate and buffer size limits imposed by Android's logging infrastructure.
(cherry picked from commit b0c48b8fd8)
Co-authored-by: Malcolm Smith <smith@chaquo.com>
gh-116622: Rename build variable MODULE_LDFLAGS back to LIBPYTHON (GH-122764)
(LIBPYTHON was renamed MODULE_LDFLAGS in commit 7f5e3f04f.)
(cherry picked from commit 2f5c3b09e4)
Co-authored-by: Malcolm Smith <smith@chaquo.com>
gh-105376: Restore deprecated logging warn() method (GH-122775)
This reverts commit dcc028d924 and
commit 6c54e5d721.
Keep the deprecated logging warn() method in Python 3.13.
(cherry picked from commit d3239976a8)
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Before, the exception caused by calling non-default sys.excepthook
in code.InteractiveInterpreter bubbled up to the caller, ending the REPL.
(cherry picked from commit bd3d31f380)
Co-authored-by: CF Bolz-Tereick <cfbolz@gmx.de>
Fix _PyArg_UnpackKeywordsWithVararg for the case when argument for
positional-or-keyword parameter is passed by keyword.
There was only one such case in the stdlib -- the TypeVar constructor.
(cherry picked from commit 540fcc62f5)
gh-122728: Fix SystemError in PyEval_GetLocals() (GH-122735)
Fix PyEval_GetLocals() to avoid SystemError ("bad argument to
internal function"). Don't redefine the 'ret' variable in the if
block.
Add an unit test on PyEval_GetLocals().
(cherry picked from commit 4767a6e31c)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-121650: Encode newlines in headers, and verify headers are sound (GH-122233)
GH-GH- Encode header parts that contain newlines
Per RFC 2047:
> [...] these encoding schemes allow the
> encoding of arbitrary octet values, mail readers that implement this
> decoding should also ensure that display of the decoded data on the
> recipient's terminal will not cause unwanted side-effects
It seems that the "quoted-word" scheme is a valid way to include
a newline character in a header value, just like we already allow
undecodable bytes or control characters.
They do need to be properly quoted when serialized to text, though.
GH-GH- Verify that email headers are well-formed
This should fail for custom fold() implementations that aren't careful
about newlines.
(cherry picked from commit 0976339818)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Bas Bloemsaat <bas@bloemsaat.org>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-122573: Require Python 3.10 or newer for Windows builds (GH-122574)
Match statements in tooling require a more recent Python. Tools/cases_generator/*.py (and `Tools/jit/*.py` in 3.13+).
(cherry picked from commit d0b92dd5ca)
Co-authored-by: Jonathan Protzenko <protz@microsoft.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
gh-122334: Fix crash when importing ssl after re-initialization (GH-122481)
* Fix crash when importing ssl after re-initialization
(cherry picked from commit 9fc1c992d6)
Co-authored-by: neonene <53406459+neonene@users.noreply.github.com>
gh-116622: Fix testPyObjectPrintOSError on Android (GH-122487)
Adds extra handling for way BSD/Android return errors from calls to fwrite.
(cherry picked from commit 82db572813)
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Currently, idle-dev@python.org and idle-dev mailing list
serve to collect spam (90+%). Change About IDLE to direct
discussions to discuss.python.org. Users are already
doing so.
(cherry picked from commit 29c04dfa27)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
gh-105733: Soft-deprecate ctypes.ARRAY, rather than hard-deprecating it. (GH-122281)
Soft-deprecate ctypes.ARRAY, rather than hard-deprecating it.
Partially reverts 2211454fe2
(cherry picked from commit 3833d27f98)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Authenticate socket connection for `socket.socketpair()` fallback when the platform does not have a native `socketpair` C API. We authenticate in-process using `getsocketname` and `getpeername` (thanks to Nathaniel J Smith for that suggestion).
(cherry picked from commit 78df1043db)
Co-authored-by: Seth Michael Larson <seth@python.org>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
gh-121489: Export private _PyBytes_Join() again (GH-122267)
(cherry picked from commit aef95eb107)
Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
Serializing objects with complex __qualname__ (such as unbound methods and
nested classes) by name no longer involves serializing parent objects by value
in pickle protocols < 4.
(cherry picked from commit dc07f65a53)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-122029: Log call events in sys.setprofile when it's a method with c function (GH-122072)
Log call events in sys.setprofile when it is a method with a C function.
(cherry picked from commit e91ef13861)
Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>
Relatedly, emit the `cpython.run_startup` event from the Python version of
`PYTHONSTARTUP` handling.
(cherry picked from commit dc93d1125f)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
gh-120522: Apply App Store compliance patch during installation (GH-121947)
Adds a --with-app-store-compliance configuration option that patches out code known to be an issue with App Store review processes. This option is applied automatically on iOS, and optionally on macOS.
(cherry picked from commit 728432c804)
Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
gh-120930: Remove extra blank occuring in wrapped encoded words in email headers (GH-121747)
(cherry picked from commit cecaceea31)
Co-authored-by: Matthieu Caneill <matthieucan@users.noreply.github.com>
This adds tests for the documented behaviour of `pkgutil.extend_path`
regarding different argument types as well as for `*.pkg` files.
(cherry picked from commit 8f2532168b)
Co-authored-by: Andreas Stocker <andreas@stocker.co.it>
gh-78889: Stop IDLE Shell freezes from sys.stdout.shell.xyz (GH-121876)
Problem occurred when attribute xyz could not be pickled.
Since this is not trivial to selectively fix, block all
attributes (other than 'width'). IDLE does not access them
and they are private implementation details.
(cherry picked from commit 58753f33e4)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
* Switch PyUnicode_InternInPlace to _PyUnicode_InternMortal, clarify docs
* Document immortality in some functions that take `const char *`
This is PyUnicode_InternFromString;
PyDict_SetItemString, PyObject_SetAttrString;
PyObject_DelAttrString; PyUnicode_InternFromString;
and the PyModule_Add convenience functions.
Always point out a non-immortalizing alternative.
* Don't immortalize user-provided attr names in _ctypes
(cherry picked from commit b4aedb23ae)
The object's inline values may be marked invalid if the materialized
dict was already initialized and then deleted.
(cherry picked from commit 162b41f577)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-113993: For string interning, do not rely on (or assert) _Py_IsImmortal (GH-121358)
Older stable ABI extensions are allowed to make immortal objects mortal.
Instead, use `_PyUnicode_STATE` (`interned` and `statically_allocated`).
(cherry picked from commit 956270d08d)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
This reverts commit 0dfb437a32 prior
to the release of 3.13.0b4 to allow for additional review time.
(cherry picked from commit f27593a87c)
Co-authored-by: Ned Deily <nad@python.org>
When using the ** operator or pow() with Fraction as the base
and an exponent that is not rational, a float, or a complex, the
fraction is no longer converted to a float.
(cherry picked from commit b9965ef282)
Co-authored-by: Joshua Herman <30265+zitterbewegung@users.noreply.github.com>
gh-120831: Increase the default minimum supported iOS version to 13.0 (GH-121250)
Increases the default minimum iOS version to 13.0.
(cherry picked from commit 7e91e0dcfe)
Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
We should maintain the invariant that a zero `ob_tid` implies the
refcount fields are merged.
* Move the assignment in `_Py_MergeZeroLocalRefcount` to immediately
before the refcount merge.
* Update `_PyTrash_thread_destroy_chain` to set `ob_ref_shared` to
`_Py_REF_MERGED` when setting `ob_tid` to zero.
Also check this invariant with assertions in the GC in debug builds.
That uncovered a bug when running out of memory during GC.
(cherry picked from commit d23be3947c)
Co-authored-by: Sam Gross <colesbury@gmail.com>
It is our general practice to make new optional parameters keyword-only,
even if the existing parameters are all positional-or-keyword. Passing
this parameter as positional would look confusing and could be error-prone
if additional parameters are added in the future.
(cherry picked from commit 50eec501fe)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
gh-121497: Make Pyrepl respect correctly the history with input hook set (GH-121498)
(cherry picked from commit 4e36dd7d87)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
gh-121499: Fix multi-line history rendering in the REPL (GH-121531)
(cherry picked from commit 4b9e10d0ea)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
On POSIX systems, excluding macOS framework installs, the lib directory
for the free-threaded build now includes a "t" suffix to avoid conflicts
with a co-located default build installation.
(cherry picked from commit e8c91d90ba)
Co-authored-by: Sam Gross <colesbury@gmail.com>
When builtin static types are initialized for a subinterpreter, various "tp" slots have already been inherited (for the main interpreter). This was interfering with the logic in add_operators() (in Objects/typeobject.c), causing a wrapper to get created when it shouldn't. This change fixes that by preserving the original data from the static type struct and checking that.
(cherry picked from commit 5250a03133, AKA gh-121602)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-89364: Export PySignal_SetWakeupFd() function (GH-121537)
Export the PySignal_SetWakeupFd() function. Previously, the function
was documented but it couldn't be used in 3rd party code.
(cherry picked from commit ca0fb3423c)
Co-authored-by: Victor Stinner <vstinner@python.org>
GH-121439: Allow PyTupleObjects with an ob_size of 20 in the free_list to be reused (gh-121428)
(cherry picked from commit 9585a1a2a2)
Co-authored-by: satori1995 <132636720+satori1995@users.noreply.github.com>
The `_PySeqLock_EndRead` function needs an acquire fence to ensure that
the load of the sequence happens after any loads within the read side
critical section. The missing fence can trigger bugs on macOS arm64.
Additionally, we need a release fence in `_PySeqLock_LockWrite` to
ensure that the sequence update is visible before any modifications to
the cache entry.
(cherry picked from commit 1d3cf79a50)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-121467: Fix makefile to include mimalloc headers (GH-121469)
(cherry picked from commit 5aa1e60e0c)
Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
gh-121084: Fix test_typing random leaks (GH-121360)
Clear typing ABC caches when running tests for refleaks (-R option):
call _abc_caches_clear() on typing abstract classes and their
subclasses.
(cherry picked from commit 5f660e8e2c)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-121201: Disable perf_trampoline on riscv64 for now (GH-121328)
Disable perf_trampoline on riscv64 for now
Until support is added in perf_jit_trampoline.c
gh-120089 was incomplete.
(cherry picked from commit ca2e876500)
Co-authored-by: Stefano Rivera <stefano@rivera.za.net>
gh-117983: Defer import of threading for lazy module loading (GH-120233)
As noted in gh-117983, the import importlib.util can be triggered at
interpreter startup under some circumstances, so adding threading makes
it a potentially obligatory load.
Lazy loading is not used in the stdlib, so this removes an unnecessary
load for the majority of users and slightly increases the cost of the
first lazily loaded module.
An obligatory threading load breaks gevent, which monkeypatches the
stdlib. Although unsupported, there doesn't seem to be an offsetting
benefit to breaking their use case.
For reference, here are benchmarks for the current main branch:
```
❯ hyperfine -w 8 './python -c "import importlib.util"'
Benchmark 1: ./python -c "import importlib.util"
Time (mean ± σ): 9.7 ms ± 0.7 ms [User: 7.7 ms, System: 1.8 ms]
Range (min … max): 8.4 ms … 13.1 ms 313 runs
```
And with this patch:
```
❯ hyperfine -w 8 './python -c "import importlib.util"'
Benchmark 1: ./python -c "import importlib.util"
Time (mean ± σ): 8.4 ms ± 0.7 ms [User: 6.8 ms, System: 1.4 ms]
Range (min … max): 7.2 ms … 11.7 ms 352 runs
```
Compare to:
```
❯ hyperfine -w 8 './python -c pass'
Benchmark 1: ./python -c pass
Time (mean ± σ): 7.6 ms ± 0.6 ms [User: 5.9 ms, System: 1.6 ms]
Range (min … max): 6.7 ms … 11.3 ms 390 runs
```
This roughly halves the import time of importlib.util.
(cherry picked from commit 94f50f8ee6)
Co-authored-by: Chris Markiewicz <effigies@gmail.com>
gh-118714: Make the pdb post-mortem restart/quit behavior more reasonable (GH-118725)
(cherry picked from commit e245ed7d1e)
Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>
* [3.13] gh-120782: Update internal type cache when reloading datetime
When reloading _datetime module, the single-phase version did not invoke the PyInit__datetime function, whereas the current multi-phase version updates the static types through the module init. The outdated static type cache in the interpreter state needs to be invalidated at the end of reloading the multi-phase module.
gh-121200: Fix test_expanduser_pwd2() of test_posixpath (GH-121228)
Call getpwnam() to get pw_dir, since it can be different than
getpwall() pw_dir.
(cherry picked from commit 02cb5fdee3)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-121188: Sanitize invalid XML characters in regrtest (GH-121195)
When creating the JUnit XML file, regrtest now escapes characters
which are invalid in XML, such as the chr(27) control character used
in ANSI escape sequences.
(cherry picked from commit af8c3d7a26)
Co-authored-by: Victor Stinner <vstinner@python.org>
1. Use pkg-config to check for ncursesw/panelw. If that fails, use
pkg-config to check for ncurses/panel.
2. Regardless of pkg-config output, search for curses/panel headers, so
we're sure we have all defines in pyconfig.h.
3. Regardless of pkg-config output, check if libncurses or libncursesw
contains the 'initscr' symbol; if it does _and_ pkg-config failed
earlier, add the resulting -llib linker option to CURSES_LIBS.
Ditto for 'update_panels' and PANEL_LIBS.
4. Wrap the rest of the checks with WITH_SAVE_ENV and make sure we're
using updated LIBS and CPPFLAGS for those.
Add the PY_CHECK_CURSES convenience macro.
(cherry picked from commit f80376b129)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
gh-87744: fix waitpid race while calling send_signal in asyncio (GH-121126)
asyncio earlier relied on subprocess module to send signals to the process, this has some drawbacks one being that subprocess module unnecessarily calls waitpid on child processes and hence it races with asyncio implementation which internally uses child watchers. To mitigate this, now asyncio sends signals directly to the process without going through the subprocess on non windows systems. On Windows it fallbacks to subprocess module handling but on windows there are no child watchers so this issue doesn't exists altogether.
(cherry picked from commit bd473aa598)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
Miss Islington (bot)
Peter Bierma
Serhiy Storchaka
Pradyun Gedam
Irit Katriel
Thomas Wouters
Petr Viktorin
Dino Viehland
Brandt Bucher
Pablo Galindo Salgado
Kirill Podoprigora
Hugo van Kemenade
Jelle Zijlstra
Steve Dower
Miro Hrončok
neonene