* gh-112529: Implement GC for free-threaded builds
This implements a mark and sweep GC for the free-threaded builds of
CPython. The implementation relies on mimalloc to find GC tracked
objects (i.e., "containers").
* Bring in a subset of biased reference counting:
https://github.com/colesbury/nogil/commit/b6b12a9a94e
The NoGIL branch has functions for attempting to do an incref on an object which may or may not be in flight. This just brings those functions over so that they will be usable from in the dict implementation to get items w/o holding a lock.
There's a handful of small simple modifications:
Adding inline to the force inline functions to avoid a warning, and switching from _Py_ALWAYS_INLINE to Py_ALWAYS_INLINE as that's available
Remove _Py_REF_LOCAL_SHIFT as it doesn't exist yet (and is currently 0 in the 3.12 nogil branch anyway)
ob_ref_shared is currently Py_ssize_t and not uint32_t, so use that
_PY_LIKELY doesn't exist, so drop it
_Py_ThreadLocal becomes _Py_IsOwnedByCurrentThread
Add '_PyInterpreterState_GET()' to _Py_IncRefTotal calls.
Co-Authored-By: Sam Gross <colesbury@gmail.com>
The iterator returned by ElementTree.iterparse() may hold on to a file
descriptor. The reference cycle prevented prompt clean-up of the file
descriptor if the returned iterator was not exhausted.
* use the ParkingLot API to manage waiting threads
* use Argument Clinic's critical section directive to protect queue methods
* remove unnecessary overflow check
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
The `--disable-gil` builds occasionally need to pause all but one thread. Some
examples include:
* Cyclic garbage collection, where this is often called a "stop the world event"
* Before calling `fork()`, to ensure a consistent state for internal data structures
* During interpreter shutdown, to ensure that daemon threads aren't accessing Python objects
This adds the following functions to implement global and per-interpreter pauses:
* `_PyEval_StopTheWorldAll()` and `_PyEval_StartTheWorldAll()` (for the global runtime)
* `_PyEval_StopTheWorld()` and `_PyEval_StartTheWorld()` (per-interpreter)
(The function names may change.)
These functions are no-ops outside of the `--disable-gil` build.
The test_peg_generator test tried to link the python313_d.lib library,
which failed because the library is now named python313t_d.lib. The
underlying problem is that the "compiler" attribute was not set when
we call get_libraries() from distutils.
Use rst substitutions to reduce raw text duplication.
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Make it possible for a converter to have multiple includes, by collecting
them in a list on the converter instance. This implies converter includes
are added during template generation, so we have to add them to the
clinic instance at the end of the template generation instead of in the
beginning.
This improves the implementation in gh-106643.
Previously, venv passed "(<prompt>) " to the activation scripts, but we want
to provide the original value so that users can inspect it in the
$VIRTUAL_ENV_PROMPT env var.
Note: Lib/venv/scripts/common/Activate.ps1 surrounded the prompt value with
parens a second time, so no change was necessary in that file.
Remove a double negative in the documentation of `mkdir()`'s *exist_ok*
parameter.
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
I already effectively own these. (multiprocessing reluctantly, but I've
spent enough time in the code of late, it is important, and and
championing some changes, so I may as well be looped in there).
This adds support for visiting abandoned pages in mimalloc and improves
the performance of the page visiting code. Abandoned pages contain
memory blocks from threads that have exited. At some point, they may be
later reclaimed by other threads. We still need to visit those pages in
the free-threaded GC because they contain live objects.
This also reduces the overhead of visiting mimalloc pages:
* Special cases for full, empty, and pages containing only a single
block.
* Fix free_map to use one bit instead of one byte per block.
* Use fast integer division by a constant algorithm when computing
block offset from block size and index.
Sam Gross
Dino Viehland
Tian Gao
Ned Batchelder
AN Long
Irit Katriel
Michael Droettboom
Steve Dower
Vincent Cunningham
plokmijnuhby
Kirill Podoprigora
Mark Shannon
Erlend E. Aasland
Thomas Grainger
Hugo van Kemenade
Nikita Sobolev
Serhiy Storchaka
Daniel Hollas
Brett Cannon
Dino Viehland
mpage
Fabio Zadrozny
Jim Porter
Barney Gale
Ville Skyttä
Géry Ogam
Gregory P. Smith