Commit Graph

1619 Commits

Author SHA1 Message Date
Sergey B Kirpichev dbc1752d41
gh-111495: Add tests for PyTuple C API (#118757)
Co-authored-by: kalyanr <kalyan.ben10@live.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2024-08-26 11:57:52 +02:00
Jonathan Protzenko 325e9b8ef4
gh-99108: Add HACL* Blake2 implementation to hashlib (GH-119316)
This replaces the existing hashlib Blake2 module with a single implementation that uses HACL\*'s Blake2b/Blake2s implementations. We added support for all the modes exposed by the Python API, including tree hashing, leaf nodes, and so on. We ported and merged all of these changes upstream in HACL\*, added test vectors based on Python's existing implementation, and exposed everything needed for hashlib.

This was joint work done with @R1kM.

See the PR for much discussion and benchmarking details.   TL;DR: On many systems, 8-50% faster (!) than `libb2`, on some systems it appeared 10-20% slower than `libb2`.
2024-08-13 21:42:19 +00:00
Victor Stinner bf8b374639
gh-111495: Add more tests on PyEval C APIs (#122789)
* Add Lib/test/test_capi/test_eval.py
* Add Modules/_testlimitedcapi/eval.c
2024-08-08 14:16:20 +02:00
Sam Gross dc09301067
gh-122417: Implement per-thread heap type refcounts (#122418)
The free-threaded build partially stores heap type reference counts in
distributed manner in per-thread arrays. This avoids reference count
contention when creating or destroying instances.

Co-authored-by: Ken Jin <kenjin@python.org>
2024-08-06 14:36:57 -04:00
Jonathan Protzenko d0b92dd5ca
gh-122573: Require Python 3.10 or newer for Windows builds (GH-122574)
Match statements in tooling require a more recent Python. Tools/cases_generator/*.py (and `Tools/jit/*.py` in 3.13+).

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2024-08-04 23:22:51 +00:00
Sam Gross 5716cc3529
gh-100240: Use a consistent implementation for freelists (#121934)
This combines and updates our freelist handling to use a consistent
implementation. Objects in the freelist are linked together using the
first word of memory block.

If configured with freelists disabled, these operations are essentially
no-ops.
2024-07-22 12:08:27 -04:00
AraHaan 0e77540d86
Fixed regenerating files in a checkout path with spaces (GH-121384) 2024-07-05 17:33:52 +01:00
Victor Stinner f8373db153
gh-112136: Restore removed _PyArg_Parser (#121262)
Restore the private _PyArg_Parser structure and the private
_PyArg_ParseTupleAndKeywordsFast() function, previously removed
in Python 3.13 alpha 1.

Recreate Include/cpython/modsupport.h header file.
2024-07-03 18:36:57 +02:00
Sergey B Kirpichev 6988ff02a5
gh-61103: Support double complex (_Complex) type in ctypes (#120894)
Example:

```pycon
>>> import ctypes
>>> ctypes.__STDC_IEC_559_COMPLEX__
1
>>> libm = ctypes.CDLL('libm.so.6')
>>> libm.clog.argtypes = [ctypes.c_double_complex]
>>> libm.clog.restype = ctypes.c_double_complex
>>> libm.clog(1+1j)
(0.34657359027997264+0.7853981633974483j)
```

Co-authored-by: Nice Zombies <nineteendo19d0@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
2024-07-01 10:54:33 +02:00
Ken Jin 22b0de2755
gh-117139: Convert the evaluation stack to stack refs (#118450)
This PR sets up tagged pointers for CPython.

The general idea is to create a separate struct _PyStackRef for everything on the evaluation stack to store the bits. This forces the C compiler to warn us if we try to cast things or pull things out of the struct directly.

Only for free threading: We tag the low bit if something is deferred - that means we skip incref and decref operations on it. This behavior may change in the future if Mark's plans to defer all objects in the interpreter loop pans out.

This implies a strict stack reference discipline is required. ALL incref and decref operations on stackrefs must use the stackref variants. It is unsafe to untag something then do normal incref/decref ops on it.

The new incref and decref variants are called dup and close. They mimic a "handle" API operating on these stackrefs.

Please read Include/internal/pycore_stackref.h for more information!

---------

Co-authored-by: Mark Shannon <9448417+markshannon@users.noreply.github.com>
2024-06-27 03:10:43 +08:00
Victor Stinner 9e4a81f00f
gh-120642: Move private PyCode APIs to the internal C API (#120643)
* Move _Py_CODEUNIT and related functions to pycore_code.h.
* Move _Py_BackoffCounter to pycore_backoff.h.
* Move Include/cpython/optimizer.h content to pycore_optimizer.h.
* Remove Include/cpython/optimizer.h.
* Remove PyUnstable_Replace_Executor().

Rename functions:

* PyUnstable_GetExecutor() => _Py_GetExecutor()
* PyUnstable_GetOptimizer() => _Py_GetOptimizer()
* PyUnstable_SetOptimizer() => _Py_SetTier2Optimizer()
* PyUnstable_Optimizer_NewCounter() => _PyOptimizer_NewCounter()
* PyUnstable_Optimizer_NewUOpOptimizer() => _PyOptimizer_NewUOpOptimizer()
2024-06-26 13:54:03 +02:00
Sam Gross 8f17d69b7b
gh-119344: Make critical section API public (#119353)
This makes the following macros public as part of the non-limited C-API for
locking a single object or two objects at once.

* `Py_BEGIN_CRITICAL_SECTION(op)` / `Py_END_CRITICAL_SECTION()`
* `Py_BEGIN_CRITICAL_SECTION2(a, b)` / `Py_END_CRITICAL_SECTION2()`

The supporting functions and structs used by the macros are also exposed for
cases where C macros are not available.
2024-06-21 15:50:18 -04:00
Sam Gross 3af7263037
gh-117511: Make PyMutex public in the non-limited API (#117731) 2024-06-20 11:29:08 -04:00
Victor Stinner f9d47fed9f
gh-119853: Add Include/refcount.h to projects (#119860) 2024-05-31 21:21:30 +02:00
Steve Dower 0e8d35b931
gh-116145: Updated bundled Tcl/Tk on Windows to 8.6.14 (GH-117030) 2024-05-31 13:07:19 +01:00
Matthias Görgens 18c1a8d3a8
gh-97588: Align ctypes struct layout to GCC/MSVC (GH-97702)
Structure layout, and especially bitfields, sometimes resulted in clearly
wrong behaviour like overlapping fields. This fixes

Co-authored-by: Gregory P. Smith <gps@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-05-29 12:02:53 +02:00
Kirill Podoprigora ed2b0fb044
Update Windows library names for the Python version bump (#118766) 2024-05-08 16:00:38 +00:00
Steve Dower 460546529b
gh-118734: Fixes Windows build when Use_TIER2 is unspecified (#118735) 2024-05-07 23:01:18 +02:00
Pablo Galindo Salgado 1b22d801b8
gh-118518: Allow perf to work without frame pointers (#112254) 2024-05-05 03:07:29 +02:00
Irit Katriel 85af789961
gh-111997: C-API for signalling monitoring events (#116413) 2024-05-04 08:23:50 +00:00
Erlend E. Aasland 1324502752
gh-115119: Update Windows installer to mpdecimal 4.0.0 (#118506) 2024-05-03 18:00:43 +02:00
Guido van Rossum a37b093228
gh-118335: Rename --experimental-interpreter on Windows to --experimental-jit-interpreter (#118497)
Also fix docs for this in whatsnew.
2024-05-02 00:48:34 +00:00
Guido van Rossum 7d83f7bcc4
gh-118335: Configure Tier 2 interpreter at build time (#118339)
The code for Tier 2 is now only compiled when configured
with `--enable-experimental-jit[=yes|interpreter]`.

We drop support for `PYTHON_UOPS` and -`Xuops`,
but you can disable the interpreter or JIT
at runtime by setting `PYTHON_JIT=0`.
You can also build it without enabling it by default
using `--enable-experimental-jit=yes-off`;
enable with `PYTHON_JIT=1`.

On Windows, the `build.bat` script supports
`--experimental-jit`, `--experimental-jit-off`,
`--experimental-interpreter`.

In the C code, `_Py_JIT` is defined as before
when the JIT is enabled; the new variable
`_Py_TIER2` is defined when the JIT *or* the
interpreter is enabled. It is actually a bitmask:
1: JIT; 2: default-off; 4: interpreter.
2024-04-30 18:26:34 -07:00
Ken Jin dc6b12d1b2
gh-117139: Add header for tagged pointers (GH-118330)
---------

Co-authored-by: Sam Gross <655866+colesbury@users.noreply.github.com>
2024-05-01 04:46:13 +08:00
Seth Michael Larson 72dae53e09
gh-116122: Add SBOM generation to PCbuild/build.bat (GH-116138) 2024-04-30 16:05:05 +01:00
Eric Snow 03e3e31723
gh-76785: Rename _xxsubinterpreters to _interpreters (gh-117791)
See https://discuss.python.org/t/pep-734-multiple-interpreters-in-the-stdlib/41147/26.
2024-04-24 16:18:24 +00:00
Serhiy Storchaka 6078f2033e
gh-117968: Add tests for the part of the PyRun family of the C API (GH-117982)
Co-authored-by: NGRsoftlab <78017794+NGRsoftlab@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-04-17 23:45:35 +03:00
Mariusz Felisiak de0dc68b82
gh-115009: Update Windows installer to use SQLite 3.45.3 (GH-117445) 2024-04-16 23:10:05 +01:00
Irit Katriel 04697bcfaf
gh-117494: extract the Instruction Sequence data structure into a separate file (#117496) 2024-04-04 15:47:26 +00:00
Guido van Rossum 060a96f1a9
gh-116968: Reimplement Tier 2 counters (#117144)
Introduce a unified 16-bit backoff counter type (``_Py_BackoffCounter``),
shared between the Tier 1 adaptive specializer and the Tier 2 optimizer. The
API used for adaptive specialization counters is changed but the behavior is
(supposed to be) identical.

The behavior of the Tier 2 counters is changed:
- There are no longer dynamic thresholds (we never varied these).
- All counters now use the same exponential backoff.
- The counter for ``JUMP_BACKWARD`` starts counting down from 16.
- The ``temperature`` in side exits starts counting down from 64.
2024-04-04 15:03:27 +00:00
Eric Snow f341d6017d
gh-76785: Add PyInterpreterConfig Helpers (gh-117170)
These helpers make it easier to customize and inspect the config used to initialize interpreters.  This is especially valuable in our tests.  I found inspiration from the PyConfig API for the PyInterpreterConfig dict conversion stuff.  As part of this PR I've also added a bunch of tests.
2024-04-02 20:35:52 +00:00
MonadChains 90c3c68a65
gh-94808:Improve coverage of PyObject_Print (GH-98749) 2024-04-01 12:52:25 +00:00
Sam Gross 19c1dd60c5
gh-117323: Make `cell` thread-safe in free-threaded builds (#117330)
Use critical sections to lock around accesses to cell contents. The critical sections are no-ops in the default (with GIL) build.
2024-03-29 13:35:43 -04:00
Serhiy Storchaka 0c1a42cf9c
gh-87193: Support bytes objects with refcount > 1 in _PyBytes_Resize() (GH-117160)
Create a new bytes object and destroy the old one if it has refcount > 1.
2024-03-25 16:32:11 +01:00
Eric Snow 617158e078
gh-76785: Drop PyInterpreterID_Type (gh-117101)
I added it quite a while ago as a strategy for managing interpreter lifetimes relative to the PEP 554 (now 734) implementation.  Relatively recently I refactored that implementation to no longer rely on InterpreterID objects.  Thus now I'm removing it.
2024-03-21 17:15:02 +00:00
Victor Stinner 8bea6c411d
gh-115754: Add Py_GetConstant() function (#116883)
Add Py_GetConstant() and Py_GetConstantBorrowed() functions.

In the limited C API version 3.13, getting Py_None, Py_False,
Py_True, Py_Ellipsis and Py_NotImplemented singletons is now
implemented as function calls at the stable ABI level to hide
implementation details. Getting these constants still return borrowed
references.

Add _testlimitedcapi/object.c and test_capi/test_object.py to test
Py_GetConstant() and Py_GetConstantBorrowed() functions.
2024-03-21 16:07:00 +00:00
Victor Stinner 61c659e2dc
gh-116417: Move limited C API complex.c tests to _testlimitedcapi (#117014)
Split complex.c tests of _testcapi into two parts: limited C API
tests in _testlimitedcapi and non-limited C API tests in _testcapi.
2024-03-19 17:23:12 +01:00
Victor Stinner f55e1880c1
gh-116417: Move limited C API dict.c tests to _testlimitedcapi (#117006)
Split dict.c tests of _testcapi into two parts: limited C API tests
in _testlimitedcapi and non-limited C API tests in _testcapi.
2024-03-19 15:06:20 +00:00
Victor Stinner 3cac2af5ec
gh-116417: Move limited C API long.c tests to _testlimitedcapi (#117001)
* Split long.c tests of _testcapi into two parts: limited C API tests
  in _testlimitedcapi and non-limited C API tests in _testcapi.
* Move testcapi_long.h from Modules/_testcapi/ to
  Modules/_testlimitedcapi/.
* Add MODULE__TESTLIMITEDCAPI_DEPS to Makefile.pre.in.
2024-03-19 14:04:23 +00:00
Victor Stinner a557478987
gh-116417: Move limited C API unicode.c tests to _testlimitedcapi (#116993)
Split unicode.c tests of _testcapi into two parts: limited C API
tests in _testlimitedcapi and non-limited C API tests in _testcapi.

Update test_codecs.
2024-03-19 12:30:39 +00:00
Victor Stinner 039d20ae54
gh-116417: Move limited C API abstract.c tests to _testlimitedcapi (#116986)
Split abstract.c and float.c tests of _testcapi into two parts:
limited C API tests in _testlimitedcapi and non-limited C API tests
in _testcapi.

Update test_bytes and test_class.
2024-03-19 10:44:13 +00:00
Victor Stinner ecb4a2b711
gh-116417: Move limited C API list.c tests to _testlimitedcapi (#116602)
Split list.c and set.c tests of _testcapi into two parts: limited C
API tests in _testlimitedcapi and non-limited C API tests in
_testcapi.
2024-03-18 22:03:55 +01:00
Guido van Rossum 9c7b3688e6
gh-108716: Cleanup remaining deepfreeze infrastructure (#116919)
Keep Tools/build/deepfreeze.py around (we may repurpose it for deepfreezing non-code objects),
and keep basic "clean" targets that remove the output of former deep-freeze activities,
to keep the build directories of current devs clean.
2024-03-18 11:13:11 -07:00
Zachary Ware 849e0716d3
gh-115119: Switch Windows build to mpdecimal external (GH-115182)
This includes adding what should be a relatively temporary
`Modules/_decimal/windows/mpdecimal.h` shim to choose between `mpdecimal32vc.h`
or `mpdecimal64vc.h` based on which of `CONFIG_64` or `CONFIG_32` is defined.
2024-03-18 12:07:25 -05:00
Victor Stinner 1cc02ca063
gh-116417: Move 4 limited C API test files to _testlimitedcapi (#116571)
Move the following files from Modules/_testcapi/ to
Modules/_testlimitedcapi/:

* bytearray.c
* bytes.c
* pyos.c
* sys.c

Changes:

* Replace PyBytes_AS_STRING() with PyBytes_AsString().
* Replace PyBytes_GET_SIZE() with PyBytes_Size().
* Update related test_capi tests.
* Copy Modules/_testcapi/util.h to Modules/_testlimitedcapi/util.h.
2024-03-11 10:28:16 +00:00
Dan Albert 5d0cdfe519
gh-116472: Replace literal newlines with escape characters in MSBuild files (GH-116473) 2024-03-07 21:53:13 +00:00
Victor Stinner d9bcdda39c
gh-116417: Add _testlimitedcapi C extension (#116419)
Add a new C extension "_testlimitedcapi" which is only built with the
limited C API.

Move heaptype_relative.c and vectorcall_limited.c from
Modules/_testcapi/ to Modules/_testlimitedcapi/.

* configure: add _testlimitedcapi test extension.
* Update generate_stdlib_module_names.py.
* Update make check-c-globals.

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-03-07 18:31:12 +00:00
Steve Dower 83c5ecdeec
gh-108051: Update versions found by find_python.bat and clarify readme (GH-116118) 2024-02-29 21:24:05 +00:00
Mark Shannon 10fbcd6c5d
GH-115816: Make tier2 optimizer symbols testable, and add a few tests. (GH-115953) 2024-02-27 10:51:26 +00:00
Steve Dower 37a13b9413
gh-115582: Make default PC/pyconfig.h work for free-threaded builds with manual /DPy_GIL_DISABLED (GH-115850) 2024-02-26 19:14:14 +00:00