GH-121439: Allow PyTupleObjects with an ob_size of 20 in the free_list to be reused (gh-121428)
(cherry picked from commit 9585a1a2a2)
Co-authored-by: satori1995 <132636720+satori1995@users.noreply.github.com>
The `_PySeqLock_EndRead` function needs an acquire fence to ensure that
the load of the sequence happens after any loads within the read side
critical section. The missing fence can trigger bugs on macOS arm64.
Additionally, we need a release fence in `_PySeqLock_LockWrite` to
ensure that the sequence update is visible before any modifications to
the cache entry.
(cherry picked from commit 1d3cf79a50)
Co-authored-by: Sam Gross <colesbury@gmail.com>
gh-121467: Fix makefile to include mimalloc headers (GH-121469)
(cherry picked from commit 5aa1e60e0c)
Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
gh-121084: Fix test_typing random leaks (GH-121360)
Clear typing ABC caches when running tests for refleaks (-R option):
call _abc_caches_clear() on typing abstract classes and their
subclasses.
(cherry picked from commit 5f660e8e2c)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-121201: Disable perf_trampoline on riscv64 for now (GH-121328)
Disable perf_trampoline on riscv64 for now
Until support is added in perf_jit_trampoline.c
gh-120089 was incomplete.
(cherry picked from commit ca2e876500)
Co-authored-by: Stefano Rivera <stefano@rivera.za.net>
gh-117983: Defer import of threading for lazy module loading (GH-120233)
As noted in gh-117983, the import importlib.util can be triggered at
interpreter startup under some circumstances, so adding threading makes
it a potentially obligatory load.
Lazy loading is not used in the stdlib, so this removes an unnecessary
load for the majority of users and slightly increases the cost of the
first lazily loaded module.
An obligatory threading load breaks gevent, which monkeypatches the
stdlib. Although unsupported, there doesn't seem to be an offsetting
benefit to breaking their use case.
For reference, here are benchmarks for the current main branch:
```
❯ hyperfine -w 8 './python -c "import importlib.util"'
Benchmark 1: ./python -c "import importlib.util"
Time (mean ± σ): 9.7 ms ± 0.7 ms [User: 7.7 ms, System: 1.8 ms]
Range (min … max): 8.4 ms … 13.1 ms 313 runs
```
And with this patch:
```
❯ hyperfine -w 8 './python -c "import importlib.util"'
Benchmark 1: ./python -c "import importlib.util"
Time (mean ± σ): 8.4 ms ± 0.7 ms [User: 6.8 ms, System: 1.4 ms]
Range (min … max): 7.2 ms … 11.7 ms 352 runs
```
Compare to:
```
❯ hyperfine -w 8 './python -c pass'
Benchmark 1: ./python -c pass
Time (mean ± σ): 7.6 ms ± 0.6 ms [User: 5.9 ms, System: 1.6 ms]
Range (min … max): 6.7 ms … 11.3 ms 390 runs
```
This roughly halves the import time of importlib.util.
(cherry picked from commit 94f50f8ee6)
Co-authored-by: Chris Markiewicz <effigies@gmail.com>
gh-118714: Make the pdb post-mortem restart/quit behavior more reasonable (GH-118725)
(cherry picked from commit e245ed7d1e)
Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>
* [3.13] gh-120782: Update internal type cache when reloading datetime
When reloading _datetime module, the single-phase version did not invoke the PyInit__datetime function, whereas the current multi-phase version updates the static types through the module init. The outdated static type cache in the interpreter state needs to be invalidated at the end of reloading the multi-phase module.
gh-121200: Fix test_expanduser_pwd2() of test_posixpath (GH-121228)
Call getpwnam() to get pw_dir, since it can be different than
getpwall() pw_dir.
(cherry picked from commit 02cb5fdee3)
Co-authored-by: Victor Stinner <vstinner@python.org>
gh-121188: Sanitize invalid XML characters in regrtest (GH-121195)
When creating the JUnit XML file, regrtest now escapes characters
which are invalid in XML, such as the chr(27) control character used
in ANSI escape sequences.
(cherry picked from commit af8c3d7a26)
Co-authored-by: Victor Stinner <vstinner@python.org>
1. Use pkg-config to check for ncursesw/panelw. If that fails, use
pkg-config to check for ncurses/panel.
2. Regardless of pkg-config output, search for curses/panel headers, so
we're sure we have all defines in pyconfig.h.
3. Regardless of pkg-config output, check if libncurses or libncursesw
contains the 'initscr' symbol; if it does _and_ pkg-config failed
earlier, add the resulting -llib linker option to CURSES_LIBS.
Ditto for 'update_panels' and PANEL_LIBS.
4. Wrap the rest of the checks with WITH_SAVE_ENV and make sure we're
using updated LIBS and CPPFLAGS for those.
Add the PY_CHECK_CURSES convenience macro.
(cherry picked from commit f80376b129)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
gh-87744: fix waitpid race while calling send_signal in asyncio (GH-121126)
asyncio earlier relied on subprocess module to send signals to the process, this has some drawbacks one being that subprocess module unnecessarily calls waitpid on child processes and hence it races with asyncio implementation which internally uses child watchers. To mitigate this, now asyncio sends signals directly to the process without going through the subprocess on non windows systems. On Windows it fallbacks to subprocess module handling but on windows there are no child watchers so this issue doesn't exists altogether.
(cherry picked from commit bd473aa598)
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
gh-120522: Add a `--with-app-store-compliance` configure option to patch out problematic code (GH-120984)
* Add --app-store-compliance configuration option.
* Added blurb.
* Correct tab-vs-spaces formatting issue.
* Correct source file name in docs.
* Correct source code reference in Mac docs
* Only apply the patch forward, and ensure the working directory is correct.
* Make patching reslient to multiple builds.
* Documentation fixes found during review
* Documentation and configure.ac syntax improvements
* Regenerate configure script.
* Silence the patch echo output.
---------
(cherry picked from commit 48cd104b0c)
Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
Co-authored-by: Nice Zombies <nineteendo19d0@gmail.com>
Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
* parse_intermixed_args() now raises ArgumentError instead of calling
error() if exit_on_error is false.
* Internal code now always raises ArgumentError instead of calling
error(). It is then caught at the higher level and error() is called if
exit_on_error is true.
(cherry picked from commit 81a654a342)
The check for whether the log file is a real file is expensive on NFS
filesystems. This commit reorders the rollover condition checking to
not do the file type check if the expected file size is less than the
rotation threshold.
(cherry picked from commit e9b4ec614b)
Co-authored-by: Craig Robson <craig@zhatt.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
This change makes things a little less painful for some users. It also fixes a failing assert (gh-120765), by making sure all subinterpreters are destroyed before the main interpreter. As part of that, we make sure Py_Finalize() always runs with the main interpreter active.
(cherry picked from commit 4be1f37b20, AKA gh-121060)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
gh-120380: fix Python implementation of `pickle.Pickler` for `bytes` and `bytearray` objects in protocol version 5. (GH-120422)
(cherry picked from commit 7595e6743a)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
gh-120671: Fix PY_CHECK_CC_WARNING() in configure.ac (GH-120822)
Add missing space in AS_VAR_APPEND() on CFLAGS.
(cherry picked from commit 2106c9bef0)
Co-authored-by: Michael Allwright <allsey87@gmail.com>
PyDict_Next no longer locks the dictionary in the free-threaded build. Locking
around individual PyDict_Next calls is not sufficient because the function
returns borrowed references and because it allows concurrent modifications
during the iteraiton loop.
The internal locking also interferes with correct external synchronization
because it may suspend outer critical sections created by the caller.
(cherry picked from commit 375b723d58)
Co-authored-by: Sam Gross <colesbury@gmail.com>
* Add an InternalDocs file describing how interning should work and how to use it.
* Add internal functions to *explicitly* request what kind of interning is done:
- `_PyUnicode_InternMortal`
- `_PyUnicode_InternImmortal`
- `_PyUnicode_InternStatic`
* Switch uses of `PyUnicode_InternInPlace` to those.
* Disallow using `_Py_SetImmortal` on strings directly.
You should use `_PyUnicode_InternImmortal` instead:
- Strings should be interned before immortalization, otherwise you're possibly
interning a immortalizing copy.
- `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to
`SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in
backports, as they are now part of public API and version-specific ABI.
* Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery.
* Make sure the statically allocated string singletons are unique. This means these sets are now disjoint:
- `_Py_ID`
- `_Py_STR` (including the empty string)
- one-character latin-1 singletons
Now, when you intern a singleton, that exact singleton will be interned.
* Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic).
* Intern `_Py_STR` singletons at startup.
* For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup.
* Beef up the tests. Cover internal details (marked with `@cpython_only`).
* Add lots of assertions
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
- gh-119521: Rename IncompleteInputError to _IncompleteInputError and remove from public API/ABI (GH-119680)
(cherry picked from commit ce1064e4c9)
- gh-119521: Use `PyAPI_DATA`, not `extern`, for `_PyExc_IncompleteInputError` (GH-120955)
(cherry picked from commit ac61d58db0)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Now the null character is always represented as \xc0\x80 for
Tcl_NewStringObj().
(cherry picked from commit c38e2f64d0)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
The integer part of the timestamp can be rounded up, while the millisecond
calculation truncates, causing the log timestamp to be wrong by up to 999 ms
(affected roughly 1 in 8 million timestamps).
(cherry picked from commit 1500a23f33)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Accessing the Tkinter object's string representation no longer converts
the underlying Tcl object to a string on Windows.
(cherry picked from commit f4ddaa3967)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
This makes the following macros public as part of the non-limited C-API for
locking a single object or two objects at once.
* `Py_BEGIN_CRITICAL_SECTION(op)` / `Py_END_CRITICAL_SECTION()`
* `Py_BEGIN_CRITICAL_SECTION2(a, b)` / `Py_END_CRITICAL_SECTION2()`
The supporting functions and structs used by the macros are also exposed for
cases where C macros are not available.
(cherry picked from commit 8f17d69b7b)
Miss Islington (bot)
Steve Dower
Miro Hrončok
neonene
Serhiy Storchaka
Thomas Wouters
Victor Stinner
Jelle Zijlstra
Petr Viktorin
Tian Gao
Nice Zombies
Sam Gross