Commit Graph

1602 Commits

Author SHA1 Message Date
Steve Dower 460546529b
gh-118734: Fixes Windows build when Use_TIER2 is unspecified (#118735) 2024-05-07 23:01:18 +02:00
Pablo Galindo Salgado 1b22d801b8
gh-118518: Allow perf to work without frame pointers (#112254) 2024-05-05 03:07:29 +02:00
Irit Katriel 85af789961
gh-111997: C-API for signalling monitoring events (#116413) 2024-05-04 08:23:50 +00:00
Erlend E. Aasland 1324502752
gh-115119: Update Windows installer to mpdecimal 4.0.0 (#118506) 2024-05-03 18:00:43 +02:00
Guido van Rossum a37b093228
gh-118335: Rename --experimental-interpreter on Windows to --experimental-jit-interpreter (#118497)
Also fix docs for this in whatsnew.
2024-05-02 00:48:34 +00:00
Guido van Rossum 7d83f7bcc4
gh-118335: Configure Tier 2 interpreter at build time (#118339)
The code for Tier 2 is now only compiled when configured
with `--enable-experimental-jit[=yes|interpreter]`.

We drop support for `PYTHON_UOPS` and -`Xuops`,
but you can disable the interpreter or JIT
at runtime by setting `PYTHON_JIT=0`.
You can also build it without enabling it by default
using `--enable-experimental-jit=yes-off`;
enable with `PYTHON_JIT=1`.

On Windows, the `build.bat` script supports
`--experimental-jit`, `--experimental-jit-off`,
`--experimental-interpreter`.

In the C code, `_Py_JIT` is defined as before
when the JIT is enabled; the new variable
`_Py_TIER2` is defined when the JIT *or* the
interpreter is enabled. It is actually a bitmask:
1: JIT; 2: default-off; 4: interpreter.
2024-04-30 18:26:34 -07:00
Ken Jin dc6b12d1b2
gh-117139: Add header for tagged pointers (GH-118330)
---------

Co-authored-by: Sam Gross <655866+colesbury@users.noreply.github.com>
2024-05-01 04:46:13 +08:00
Seth Michael Larson 72dae53e09
gh-116122: Add SBOM generation to PCbuild/build.bat (GH-116138) 2024-04-30 16:05:05 +01:00
Eric Snow 03e3e31723
gh-76785: Rename _xxsubinterpreters to _interpreters (gh-117791)
See https://discuss.python.org/t/pep-734-multiple-interpreters-in-the-stdlib/41147/26.
2024-04-24 16:18:24 +00:00
Serhiy Storchaka 6078f2033e
gh-117968: Add tests for the part of the PyRun family of the C API (GH-117982)
Co-authored-by: NGRsoftlab <78017794+NGRsoftlab@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-04-17 23:45:35 +03:00
Mariusz Felisiak de0dc68b82
gh-115009: Update Windows installer to use SQLite 3.45.3 (GH-117445) 2024-04-16 23:10:05 +01:00
Irit Katriel 04697bcfaf
gh-117494: extract the Instruction Sequence data structure into a separate file (#117496) 2024-04-04 15:47:26 +00:00
Guido van Rossum 060a96f1a9
gh-116968: Reimplement Tier 2 counters (#117144)
Introduce a unified 16-bit backoff counter type (``_Py_BackoffCounter``),
shared between the Tier 1 adaptive specializer and the Tier 2 optimizer. The
API used for adaptive specialization counters is changed but the behavior is
(supposed to be) identical.

The behavior of the Tier 2 counters is changed:
- There are no longer dynamic thresholds (we never varied these).
- All counters now use the same exponential backoff.
- The counter for ``JUMP_BACKWARD`` starts counting down from 16.
- The ``temperature`` in side exits starts counting down from 64.
2024-04-04 15:03:27 +00:00
Eric Snow f341d6017d
gh-76785: Add PyInterpreterConfig Helpers (gh-117170)
These helpers make it easier to customize and inspect the config used to initialize interpreters.  This is especially valuable in our tests.  I found inspiration from the PyConfig API for the PyInterpreterConfig dict conversion stuff.  As part of this PR I've also added a bunch of tests.
2024-04-02 20:35:52 +00:00
MonadChains 90c3c68a65
gh-94808:Improve coverage of PyObject_Print (GH-98749) 2024-04-01 12:52:25 +00:00
Sam Gross 19c1dd60c5
gh-117323: Make `cell` thread-safe in free-threaded builds (#117330)
Use critical sections to lock around accesses to cell contents. The critical sections are no-ops in the default (with GIL) build.
2024-03-29 13:35:43 -04:00
Serhiy Storchaka 0c1a42cf9c
gh-87193: Support bytes objects with refcount > 1 in _PyBytes_Resize() (GH-117160)
Create a new bytes object and destroy the old one if it has refcount > 1.
2024-03-25 16:32:11 +01:00
Eric Snow 617158e078
gh-76785: Drop PyInterpreterID_Type (gh-117101)
I added it quite a while ago as a strategy for managing interpreter lifetimes relative to the PEP 554 (now 734) implementation.  Relatively recently I refactored that implementation to no longer rely on InterpreterID objects.  Thus now I'm removing it.
2024-03-21 17:15:02 +00:00
Victor Stinner 8bea6c411d
gh-115754: Add Py_GetConstant() function (#116883)
Add Py_GetConstant() and Py_GetConstantBorrowed() functions.

In the limited C API version 3.13, getting Py_None, Py_False,
Py_True, Py_Ellipsis and Py_NotImplemented singletons is now
implemented as function calls at the stable ABI level to hide
implementation details. Getting these constants still return borrowed
references.

Add _testlimitedcapi/object.c and test_capi/test_object.py to test
Py_GetConstant() and Py_GetConstantBorrowed() functions.
2024-03-21 16:07:00 +00:00
Victor Stinner 61c659e2dc
gh-116417: Move limited C API complex.c tests to _testlimitedcapi (#117014)
Split complex.c tests of _testcapi into two parts: limited C API
tests in _testlimitedcapi and non-limited C API tests in _testcapi.
2024-03-19 17:23:12 +01:00
Victor Stinner f55e1880c1
gh-116417: Move limited C API dict.c tests to _testlimitedcapi (#117006)
Split dict.c tests of _testcapi into two parts: limited C API tests
in _testlimitedcapi and non-limited C API tests in _testcapi.
2024-03-19 15:06:20 +00:00
Victor Stinner 3cac2af5ec
gh-116417: Move limited C API long.c tests to _testlimitedcapi (#117001)
* Split long.c tests of _testcapi into two parts: limited C API tests
  in _testlimitedcapi and non-limited C API tests in _testcapi.
* Move testcapi_long.h from Modules/_testcapi/ to
  Modules/_testlimitedcapi/.
* Add MODULE__TESTLIMITEDCAPI_DEPS to Makefile.pre.in.
2024-03-19 14:04:23 +00:00
Victor Stinner a557478987
gh-116417: Move limited C API unicode.c tests to _testlimitedcapi (#116993)
Split unicode.c tests of _testcapi into two parts: limited C API
tests in _testlimitedcapi and non-limited C API tests in _testcapi.

Update test_codecs.
2024-03-19 12:30:39 +00:00
Victor Stinner 039d20ae54
gh-116417: Move limited C API abstract.c tests to _testlimitedcapi (#116986)
Split abstract.c and float.c tests of _testcapi into two parts:
limited C API tests in _testlimitedcapi and non-limited C API tests
in _testcapi.

Update test_bytes and test_class.
2024-03-19 10:44:13 +00:00
Victor Stinner ecb4a2b711
gh-116417: Move limited C API list.c tests to _testlimitedcapi (#116602)
Split list.c and set.c tests of _testcapi into two parts: limited C
API tests in _testlimitedcapi and non-limited C API tests in
_testcapi.
2024-03-18 22:03:55 +01:00
Guido van Rossum 9c7b3688e6
gh-108716: Cleanup remaining deepfreeze infrastructure (#116919)
Keep Tools/build/deepfreeze.py around (we may repurpose it for deepfreezing non-code objects),
and keep basic "clean" targets that remove the output of former deep-freeze activities,
to keep the build directories of current devs clean.
2024-03-18 11:13:11 -07:00
Zachary Ware 849e0716d3
gh-115119: Switch Windows build to mpdecimal external (GH-115182)
This includes adding what should be a relatively temporary
`Modules/_decimal/windows/mpdecimal.h` shim to choose between `mpdecimal32vc.h`
or `mpdecimal64vc.h` based on which of `CONFIG_64` or `CONFIG_32` is defined.
2024-03-18 12:07:25 -05:00
Victor Stinner 1cc02ca063
gh-116417: Move 4 limited C API test files to _testlimitedcapi (#116571)
Move the following files from Modules/_testcapi/ to
Modules/_testlimitedcapi/:

* bytearray.c
* bytes.c
* pyos.c
* sys.c

Changes:

* Replace PyBytes_AS_STRING() with PyBytes_AsString().
* Replace PyBytes_GET_SIZE() with PyBytes_Size().
* Update related test_capi tests.
* Copy Modules/_testcapi/util.h to Modules/_testlimitedcapi/util.h.
2024-03-11 10:28:16 +00:00
Dan Albert 5d0cdfe519
gh-116472: Replace literal newlines with escape characters in MSBuild files (GH-116473) 2024-03-07 21:53:13 +00:00
Victor Stinner d9bcdda39c
gh-116417: Add _testlimitedcapi C extension (#116419)
Add a new C extension "_testlimitedcapi" which is only built with the
limited C API.

Move heaptype_relative.c and vectorcall_limited.c from
Modules/_testcapi/ to Modules/_testlimitedcapi/.

* configure: add _testlimitedcapi test extension.
* Update generate_stdlib_module_names.py.
* Update make check-c-globals.

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-03-07 18:31:12 +00:00
Steve Dower 83c5ecdeec
gh-108051: Update versions found by find_python.bat and clarify readme (GH-116118) 2024-02-29 21:24:05 +00:00
Mark Shannon 10fbcd6c5d
GH-115816: Make tier2 optimizer symbols testable, and add a few tests. (GH-115953) 2024-02-27 10:51:26 +00:00
Steve Dower 37a13b9413
gh-115582: Make default PC/pyconfig.h work for free-threaded builds with manual /DPy_GIL_DISABLED (GH-115850) 2024-02-26 19:14:14 +00:00
Guido van Rossum c0fdfba7ff
Rename tier 2 redundancy eliminator to optimizer (#115888)
The original name is just too much of a mouthful.
2024-02-26 08:42:53 -08:00
Kirill Podoprigora a2bb8ad144
gh-115700: Add target `_RegenCases` in Windows build for cases regeneration. (GH-115708) 2024-02-20 19:10:47 +08:00
Sam Gross 5903190727
gh-115103: Implement delayed memory reclamation (QSBR) (#115180)
This adds a safe memory reclamation scheme based on FreeBSD's "GUS" and
quiescent state based reclamation (QSBR). The API provides a mechanism
for callers to detect when it is safe to free memory that may be
concurrently accessed by readers.
2024-02-16 15:25:19 -05:00
Łukasz Langa 711f42de2e
gh-115556: Remove quotes from command-line arguments in test.bat and rt.bat (#115557)
This change essentially replaces usage of `%1` with `%~1`, which removes
quotes, if any. Without this change, the if statements fail due to
the quotes mangling the syntax.

Additionally, this change works around comma being treated as a parameter
delimiter in test.bat by escaping commas at time of parsing. Tested
combinations of rt and regrtest arguments, all seems to work as before
but now you can specify commas in arguments like "-uall,extralargefile".
2024-02-16 21:24:56 +01:00
mpage a95b1a56bb
gh-115041: Add wrappers that are atomic only in free-threaded builds (#115046)
These are intended to be used in places where atomics are required in
free-threaded builds but not in the default build. We don't want to
introduce the potential performance overhead of an atomic operation in the
default build.
2024-02-14 15:15:05 -05:00
Petr Viktorin 879f4546bf
gh-110850: Add PyTime_t C API (GH-115215)
* gh-110850: Add PyTime_t C API

Add PyTime_t API:

* PyTime_t type.
* PyTime_MIN and PyTime_MAX constants.
* PyTime_AsSecondsDouble(), PyTime_Monotonic(),
  PyTime_PerfCounter() and PyTime_GetSystemClock() functions.

Co-authored-by: Victor Stinner <vstinner@python.org>
2024-02-12 18:13:10 +01:00
Sam Gross a3af3cb4f4
gh-110481: Implement inter-thread queue for biased reference counting (#114824)
Biased reference counting maintains two refcount fields in each object:
`ob_ref_local` and `ob_ref_shared`. The true refcount is the sum of these two
fields. In some cases, when refcounting operations are split across threads,
the ob_ref_shared field can be negative (although the total refcount must be
at least zero). In this case, the thread that decremented the refcount
requests that the owning thread give up ownership and merge the refcount
fields.
2024-02-09 17:08:32 -05:00
adang1345 5914a211ef
gh-115167: Exclude vcruntime140_threads.dll from Windows build output (GH-115176) 2024-02-08 21:42:45 +00:00
Erlend E. Aasland 11ac6f5354
gh-115009: Update Windows installer to use SQLite 3.45.1 (#115065) 2024-02-06 23:44:14 +01:00
Zachary Ware 01dceba13e
gh-109991: Update Windows build to use OpenSSL 3.0.13 (#115043) 2024-02-05 23:10:55 +00:00
Zachary Ware 618d7256e7
gh-111239: Update Windows build to use zlib 1.3.1 (GH-114877) 2024-02-01 17:54:02 -06:00
Brandt Bucher f6d9e5926b
GH-113464: Add a JIT backend for tier 2 (GH-113465)
Add an option (--enable-experimental-jit for configure-based builds
or --experimental-jit for PCbuild-based ones) to build an
*experimental* just-in-time compiler, based on copy-and-patch (https://fredrikbk.com/publications/copy-and-patch.pdf).

See Tools/jit/README.md for more information on how to install the required build-time tooling.
2024-01-28 18:48:48 -08:00
Sam Gross b52fc70d1a
gh-112529: Implement GC for free-threaded builds (#114262)
* gh-112529: Implement GC for free-threaded builds

This implements a mark and sweep GC for the free-threaded builds of
CPython. The implementation relies on mimalloc to find GC tracked
objects (i.e., "containers").
2024-01-25 10:27:36 -08:00
Steve Dower 665b8f365e
gh-113655: Revert extra stack reserve in PGO builds unless UseExtraStackReserve=true (GH-114263) 2024-01-22 21:19:16 +00:00
Sam Gross 1d6d5e854c
gh-112529: Use GC heaps for GC allocations in free-threaded builds (gh-114157)
* gh-112529: Use GC heaps for GC allocations in free-threaded builds

The free-threaded build's garbage collector implementation will need to
find GC objects by traversing mimalloc heaps. This hooks up the
allocation calls with the correct heaps by using a thread-local
"current_obj_heap" variable.

* Refactor out setting heap based on type
2024-01-21 01:14:45 +09:00
Steve Dower f56d132deb
gh-112984 Update Windows build and installer for free-threaded builds (GH-113129) 2024-01-17 21:52:23 +00:00
Steve Dower 2e672f7ca6
gh-113655: Increase default stack size for PGO builds to avoid C stack exhaustion (GH-114148) 2024-01-16 22:02:20 +00:00