Makes sys.settrace, sys.setprofile, and monitoring generally thread-safe.
Mostly uses a stop-the-world approach and synchronization around the code object's _co_instrumentation_version. There may be a little bit of extra synchronization around the monitoring data that's required to be TSAN clean.
Introduce a unified 16-bit backoff counter type (``_Py_BackoffCounter``),
shared between the Tier 1 adaptive specializer and the Tier 2 optimizer. The
API used for adaptive specialization counters is changed but the behavior is
(supposed to be) identical.
The behavior of the Tier 2 counters is changed:
- There are no longer dynamic thresholds (we never varied these).
- All counters now use the same exponential backoff.
- The counter for ``JUMP_BACKWARD`` starts counting down from 16.
- The ``temperature`` in side exits starts counting down from 64.
These helpers make it easier to customize and inspect the config used to initialize interpreters. This is especially valuable in our tests. I found inspiration from the PyConfig API for the PyInterpreterConfig dict conversion stuff. As part of this PR I've also added a bunch of tests.
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Co-authored-by: Ned Deily <nad@python.org>
I added it quite a while ago as a strategy for managing interpreter lifetimes relative to the PEP 554 (now 734) implementation. Relatively recently I refactored that implementation to no longer rely on InterpreterID objects. Thus now I'm removing it.
* Ensure importlib.metadata tests do not leak references in sys.modules.
* Move importlib.metadata tests to their own package for easier syncing with importlib_metadata.
* Update owners and makefile for new directories.
* Add blurb
* Split long.c tests of _testcapi into two parts: limited C API tests
in _testlimitedcapi and non-limited C API tests in _testcapi.
* Move testcapi_long.h from Modules/_testcapi/ to
Modules/_testlimitedcapi/.
* Add MODULE__TESTLIMITEDCAPI_DEPS to Makefile.pre.in.
Keep Tools/build/deepfreeze.py around (we may repurpose it for deepfreezing non-code objects),
and keep basic "clean" targets that remove the output of former deep-freeze activities,
to keep the build directories of current devs clean.
Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
If awailable, enable -fstrict-overflow for libmpdec. Also
shut off false positive warnings (-Warray-bounds).
The later was backported from mpdecimal-4.0.0.
This change is part of the work on PEP-738: Adding Android as a
supported platform.
* Remove the "1.0" suffix from libpython's filename on Android, which
would prevent Gradle from packaging it into an app.
* Simplify the build command in the Makefile so that libpython always
gets given an SONAME with the `-Wl-h` argument, even if the SONAME is
identical to the actual filename.
* Disable a number of functions on Android which can be compiled and
linked against, but always fail at runtime. As a result, the native
_multiprocessing module is no longer built for Android.
* gh-115390 (bee7bb331) added some pre-determined results to the
configure script for things that can't be autodetected when
cross-compiling; this change adds Android to these where appropriate.
* Add a couple more pre-determined results for Android, and making them
cover iOS as well. This means the --enable-ipv6 configure option will
no longer be required on either platform.
Part of the work on PEP 738: Adding Android as a supported platform.
* Rename the LIBPYTHON variable to MODULE_LDFLAGS, to more accurately
reflect its purpose.
* Edit makesetup to use MODULE_LDFLAGS when linking extension modules.
* Edit the Makefile so that extension modules depend on libpython on
Android and Cygwin.
* Restore `-fPIC` on Android. It was removed several years ago with a
note that the toolchain used it automatically, but this is no longer
the case. Omitting it causes all linker commands to fail with an error
like `relocation R_AARCH64_ADR_PREL_PG_HI21 cannot be used against
symbol '_Py_FalseStruct'; recompile with -fPIC`.
This adds a safe memory reclamation scheme based on FreeBSD's "GUS" and
quiescent state based reclamation (QSBR). The API provides a mechanism
for callers to detect when it is safe to free memory that may be
concurrently accessed by readers.
These are intended to be used in places where atomics are required in
free-threaded builds but not in the default build. We don't want to
introduce the potential performance overhead of an atomic operation in the
default build.
Update the list of installed test subdirectories with all newly added
subdirectories of Lib/test, so that the tests in those directories are
properly installed.
For the most part, these changes make is substantially easier to backport subinterpreter-related code to 3.12, especially the related modules (e.g. _xxsubinterpreters). The main motivation is to support releasing a PyPI package with the 3.13 capabilities compiled for 3.12.
A lot of the changes here involve either hiding details behind macros/functions or splitting up some files.
Part of the PEP 730 work to add iOS support.
This change lays the groundwork for introducing iOS/tvOS/watchOS
frameworks; it includes the structural refactoring needed so that iOS
branches can be added into in a subsequent PR.
Summary of changes:
* Updates config.sub to the 2024-01-01 release. This is the "as
released" version of config.sub.
* Adds a RESSRCDIR variable to allow sharing of macOS and iOS Makefile
steps.
* Adds an INSTALLTARGETS variable so platforms can customise which
targets are actually installed. This will be used to exclude certain
targets (e.g., binaries, manfiles) from iOS framework installs.
* Adds a PYTHONFRAMEWORKINSTALLNAMEPREFIX variable; this is used as
the install name for the library. This is needed to allow for iOS
frameworks to specify an @rpath-based install name.
* Evaluates MACHDEP earlier in the configure process so that
ac_sys_system is available.
* Modifies _PYTHON_HOST_PLATFORM evaluation for cross-platform builds
so that the CPU architecture is differentiated from the host
identifier. This will be used to generate a _PYTHON_HOST_PLATFORM
definition that includes ABI information, not just CPU architecture.
* Differentiates between SOABI_PLATFORM and PLATFORM_TRIPLET.
SOABI_PLATFORM is used in binary module names, and includes the ABI,
but not the OS or CPU architecture (e.g.,
math.cpython-313-iphonesimulator.dylib). PLATFORM_TRIPLET is used
as the sys._multiarch value, and on iOS will contains the ABI and
architecture (e.g., iphoneos-arm64). This differentiation hasn't
historically been needed because while macOS is a multiarch platform,
it uses a bare darwin as PLATFORM_TRIPLE.
* Removes the use of the deprecated -Wl,-single_module flag when
compiling macOS frameworks.
* Some whitespace normalisation where there was a mix of spaces and tabs
in a single block.
Biased reference counting maintains two refcount fields in each object:
`ob_ref_local` and `ob_ref_shared`. The true refcount is the sum of these two
fields. In some cases, when refcounting operations are split across threads,
the ob_ref_shared field can be negative (although the total refcount must be
at least zero). In this case, the thread that decremented the refcount
requests that the owning thread give up ownership and merge the refcount
fields.
Add an option (--enable-experimental-jit for configure-based builds
or --experimental-jit for PCbuild-based ones) to build an
*experimental* just-in-time compiler, based on copy-and-patch (https://fredrikbk.com/publications/copy-and-patch.pdf).
See Tools/jit/README.md for more information on how to install the required build-time tooling.
* gh-112529: Implement GC for free-threaded builds
This implements a mark and sweep GC for the free-threaded builds of
CPython. The implementation relies on mimalloc to find GC tracked
objects (i.e., "containers").
* gh-112529: Use GC heaps for GC allocations in free-threaded builds
The free-threaded build's garbage collector implementation will need to
find GC objects by traversing mimalloc heaps. This hooks up the
allocation calls with the correct heaps by using a thread-local
"current_obj_heap" variable.
* Refactor out setting heap based on type
Remove LibreSSL specific workaround ifdefs from `_ssl.c` and delete the non-version-specific `_ssl_data.h` file (relevant for OpenSSL < 1.1.1, which we no longer support per PEP 644).
Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
This splits part of Modules/gcmodule.c of into Python/gc.c, which
now contains the core garbage collection implementation. The Python
module remain in the Modules/gcmodule.c file.
A typo left this check broken so many of us who do out-of-tree builds
were seeing strange failures due to bad `Python/frozen_modules/*.h`
files being picked up from the source tree and used at build time from
different Python versions leading to errors like:
`Fatal Python error: _PyImport_InitCore: failed to initialize importlib`
Or similar once our build got to an "invoke the interpreter"
bootstrapping step due to incorrect bytecode being embedded.
The `MIMALLOC_HEADERS` variable is defined in the Makefile.pre.in, not
the configure script, so we should use the `$(MIMALLOC_HEADERS)` syntax
instead of the `@MIMALLOC_HEADERS@` syntax.
Eric Snow
Yichen Yan
Dino Viehland
Irit Katriel
Guido van Rossum
Sam Gross
Russell Keith-Magee
Jason R. Coombs
Victor Stinner
Skip Montanaro
Sergey B Kirpichev
Malcolm Smith
Mark Shannon
Seth Michael Larson
mpage
T. Wouters
Ken Jin
Nikita Sobolev
Zachary Ware
Brandt Bucher
Donghee Na
Jakub Kulík
Rami
Gregory P. Smith