Commit Graph

8420 Commits

Author SHA1 Message Date
Sam Gross 608192ee2f
gh-118527: Use deferred reference counting for C functions on modules (#118529)
This addresses a scaling bottleneck in the free-threaded build when
calling functions like `math.floor()` concurrently from multiple
threads.
2024-05-03 11:33:05 -04:00
Brett Simmers c2627d6eea
gh-116322: Add Py_mod_gil module slot (#116882)
This PR adds the ability to enable the GIL if it was disabled at
interpreter startup, and modifies the multi-phase module initialization
path to enable the GIL when loading a module, unless that module's spec
includes a slot indicating it can run safely without the GIL.

PEP 703 called the constant for the slot `Py_mod_gil_not_used`; I went
with `Py_MOD_GIL_NOT_USED` for consistency with gh-104148.

A warning will be issued up to once per interpreter for the first
GIL-using module that is loaded. If `-v` is given, a shorter message
will be printed to stderr every time a GIL-using module is loaded
(including the first one that issues a warning).
2024-05-03 11:30:55 -04:00
Jelle Zijlstra ca269e58c2
gh-116126: Implement PEP 696 (#116129)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2024-05-03 06:17:32 -07:00
Brett Simmers f8290df63f
gh-116738: Make `_codecs` module thread-safe (#117530)
The module itself is a thin wrapper around calls to functions in
`Python/codecs.c`, so that's where the meaningful changes happened:

- Move codecs-related state that lives on `PyInterpreterState` to a
  struct declared in `pycore_codecs.h`.

- In free-threaded builds, add a mutex to `codecs_state` to synchronize
  operations on `search_path`. Because `search_path_mutex` is used as a
  normal mutex and not a critical section, we must be extremely careful
  with operations called while holding it.

- The codec registry is explicitly initialized as part of
  `_PyUnicode_InitEncodings` to simplify thread-safety.
2024-05-02 18:25:36 -04:00
Dino Viehland e54b0c8a4a
gh-118519: Fix empty weakref list check (#118520)
Fix empty list check
2024-05-02 13:03:29 -07:00
Dino Viehland 1e67b9207c
gh-117657: Fix TSAN list set failure (#118260)
* Fix TSAN list set failure

* Relaxed atomic is sufficient, add targetted test

* More list

* Remove atomic assign in list

* Fixup white space
2024-05-02 13:03:05 -07:00
Pablo Galindo Salgado 6bcbee09df
gh-93502: Add new C-API functions to trace object creation and destruction (#115945) 2024-05-02 19:30:00 +02:00
Mark Shannon f6fab21721
GH-118095: Make invalidating and clearing executors memory safe (GH-118459) 2024-05-01 11:34:50 +01:00
Thomas Grainger fc7e1aa3c0
GH-117881: fix athrow().throw()/asend().throw() concurrent access (GH-117882) 2024-05-01 08:44:01 +02:00
Guido van Rossum 7d83f7bcc4
gh-118335: Configure Tier 2 interpreter at build time (#118339)
The code for Tier 2 is now only compiled when configured
with `--enable-experimental-jit[=yes|interpreter]`.

We drop support for `PYTHON_UOPS` and -`Xuops`,
but you can disable the interpreter or JIT
at runtime by setting `PYTHON_JIT=0`.
You can also build it without enabling it by default
using `--enable-experimental-jit=yes-off`;
enable with `PYTHON_JIT=1`.

On Windows, the `build.bat` script supports
`--experimental-jit`, `--experimental-jit-off`,
`--experimental-interpreter`.

In the C code, `_Py_JIT` is defined as before
when the JIT is enabled; the new variable
`_Py_TIER2` is defined when the JIT *or* the
interpreter is enabled. It is actually a bitmask:
1: JIT; 2: default-off; 4: interpreter.
2024-04-30 18:26:34 -07:00
Dino Viehland 19d468a3ab
[gh-117657] Fix some issues with TSAN in typeobject (#118249)
Fix some racing reads in typebobject.c
2024-04-30 12:37:38 -07:00
Dino Viehland 4a1cf66c5c
gh-117657: Fix small issues with instrumentation and TSAN (#118064)
Small TSAN fixups for instrumentation
2024-04-30 11:38:05 -07:00
Irit Katriel 1f16b4ce56
gh-118272: Clear generator frame's locals when the generator is closed (#118277)
Co-authored-by: Thomas Grainger <tagrain@gmail.com>
2024-04-30 19:32:25 +01:00
Sam Gross 79688b5b0e
gh-118331: Handle errors in _PyObject_SetManagedDict (#118334)
When detaching a dict, the `copy_values` call may fail due to
out-of-memory errors. This can be triggered by test_no_memory in
test_repl.
2024-04-29 15:49:01 -04:00
Sam Gross 7ccacb220d
gh-117783: Immortalize objects that use deferred reference counting (#118112)
Deferred reference counting is not fully implemented yet. As a temporary
measure, we immortalize objects that would use deferred reference
counting to avoid multi-threaded scaling bottlenecks.

This is only performed in the free-threaded build once the first
non-main thread is started. Additionally, some tests, including refleak
tests, suppress this behavior.
2024-04-29 14:36:02 -04:00
mpage 43fa76638f
gh-118331: Don't raise an error if tuple allocation fails when clearing weakrefs (#118338)
It's not safe to raise an exception in `PyObject_ClearWeakRefs()` if one
is not already set, since it may be called by `_Py_Dealloc()`, which
requires that the active exception does not change.

Additionally, make sure we clear the weakrefs even when tuple allocation
fails.
2024-04-29 16:56:51 +00:00
Tian Gao 375c94c75d
gh-107674: Lazy load line number to improve performance of tracing (GH-118127) 2024-04-29 09:54:52 +01:00
Dino Viehland d5df25268b
gh-112075: _Py_dict_lookup needs to lock shared keys (#117528)
Lock shared keys in `Py_dict_lookup` and use thread-safe lookup in `insertdict`

Co-authored-by: Sam Gross <colesbury@gmail.com>
2024-04-25 15:34:05 -07:00
Dino Viehland 5da0280648
gh-117657: Fixes a few small TSAN issues in dictobject (#118200)
Fixup TSAN errors for dict
2024-04-25 08:53:29 -07:00
Donghee Na eb20a7d12c
gh-112069: Do not require lock if the set has never been exposed. (gh-118069) 2024-04-26 00:13:57 +09:00
neonene 2c45148912
gh-117578: Introduce _PyType_GetModuleByDef2 private function (GH-117661)
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-04-25 13:51:31 +02:00
Mark Shannon f180b31e76
GH-118095: Handle `RETURN_GENERATOR` in tier 2 (GH-118180) 2024-04-25 11:32:47 +01:00
Serhiy Storchaka 93b7ed7c6b
gh-108191: Add support of positional argument in SimpleNamespace constructor (GH-108195)
SimpleNamespace({'a': 1, 'b': 2}) and SimpleNamespace([('a', 1), ('b', 2)])
are now the same as SimpleNamespace(a=1, b=2).
2024-04-25 00:39:54 +03:00
Thomas Grainger 7d369d471c
GH-117536: GH-117894: fix athrow().throw(...) unawaited warning (GH-117851) 2024-04-24 14:20:19 +02:00
Shantanu 8e86579cae
gh-95754: Better error when script shadows a standard library or third party module (#113769) 2024-04-22 18:24:21 -07:00
Dino Viehland 8b541c017e
gh-112075: Make instance attributes stored in inline "dict" thread safe (#114742)
Make instance attributes stored in inline "dict" thread safe on free-threaded builds
2024-04-21 22:57:05 -07:00
Kirill Podoprigora 92c84ef831
``Objects/typeobject.c``: Fix typo (#118126) 2024-04-21 02:25:39 +00:00
Dino Viehland b45af00bad
[gh-117657] _Py_MergeZeroLocalRefcount isn't loading ob_ref_shared with strong enough semantics (#118111)
Use acquire for load of ob_ref_shared
2024-04-19 14:21:01 -07:00
Noah Kim b624490dee
Fix a typo in dictobject.c documentation (#117515) 2024-04-19 19:19:12 +00:00
Dino Viehland 1e4a4c4897
gh-117657: use relaxed loads for checking dict keys immortality (#118067)
Use relaxed load to check if dictkeys are immortal
2024-04-19 09:25:08 -07:00
Rostyslav Lobov 3e7d990a09
setobject: remove out of date docstring info (GH-118048) 2024-04-19 13:33:13 +02:00
Donghee Na 710c01be94
gh-112069: Make PySet_GET_SIZE to be atomic safe. (gh-118053)
gh-112069: Make PySet_GET_SIZE to be atomic operation
2024-04-19 06:40:28 +09:00
Donghee Na 94444ea45a
gh-112069: Add _PySet_NextEntryRef to be thread-safe. (gh-117990) 2024-04-19 00:18:22 +09:00
Jens Hedegaard Nielsen 1539854c3d
fix formatting of literal in docstring of int.from_bytes and int.to_bytes (#117847) 2024-04-18 09:01:58 +00:00
Nikita Sobolev 17ed54bc96
Fix two typos in `typeobject.c` (#118024) 2024-04-18 08:46:05 +01:00
Erlend E. Aasland 353ea0b273
gh-94673: Fix compiler warning in typeobject.c (#117980) 2024-04-17 17:56:59 +00:00
mpage b6c62c79e7
gh-117657: Fix data races in the method cache in free-threaded builds (#117954)
Fix data races in the method cache in free-threaded builds

These are technically data races, but I think they're benign (to
the extent that that is actually possible). We update cache entries
non-atomically but read them atomically from another thread, and there's
nothing that establishes a happens-before relationship between the
reads and writes that I can see.
2024-04-17 09:42:56 -07:00
Irit Katriel c179c0e6cb
gh-117680: make _PyInstructionSequence a PyObject and use it in tests (#117629) 2024-04-17 16:42:04 +01:00
Mark Shannon 147cd0581e
GH-117760: Streamline the trashcan mechanism (GH-117763) 2024-04-17 11:08:05 +01:00
Victor Stinner 3fe03ccea6
gh-117755: Fix mimalloc for huge allocation on s390x (#117809)
Fix mimalloc allocator for huge memory allocation (around
8,589,934,592 GiB) on s390x.

Abort allocation early in mimalloc if the number of slices doesn't
fit into uint32_t, to prevent a integer overflow (cast 64-bit
size_t to uint32_t).
2024-04-16 20:34:12 +00:00
Sam Gross 241ed5f2cd
gh-117376: Make code objects use deferred reference counting (#117823)
We want code objects to use deferred reference counting in the
free-threaded build. This requires them to be tracked by the GC, so we
set `Py_TPFLAGS_HAVE_GC` in the free-threaded build, but not the default
build.
2024-04-16 12:42:53 -04:00
Donghee Na a734fd5cf7
gh-112069: Make setiter_iternext to be thread-safe (gh-117935) 2024-04-16 16:31:42 +00:00
Mark Shannon 784e076a10
GH-117750: When clearing object's dict, clear inline values but leave dict attached (GH-117808) 2024-04-15 14:45:05 +01:00
Donghee Na 7bcc257e97
gh-117826: Remove lookdict_index from delitemif_lock_held (gh-117869) 2024-04-15 14:26:59 +09:00
Eric Snow eca53620e3
gh-94673: Clarify About Runtime State Related to Static Builtin Types (gh-117761)
Guido pointed out to me that some details about the per-interpreter state for the builtin types aren't especially clear.  I'm addressing that by:

* adding a comment explaining that state
* adding some asserts to point out the relationship between each index and the interp/global runtime state
2024-04-12 16:39:27 -06:00
Sam Gross 4ad8f090cc
gh-117376: Partial implementation of deferred reference counting (#117696)
This marks objects as using deferred refrence counting using the
`ob_gc_bits` field in the free-threaded build and collects those objects
during GC.
2024-04-12 17:36:20 +00:00
Serhiy Storchaka 2d3d9b4461
gh-117764: Add docstrings and signatures for the types of None, Ellipsis and NotImplemented (GH-117813) 2024-04-12 15:45:23 +03:00
Serhiy Storchaka 3a8c1ca7e7
gh-117764: Fix and add signatures for many builtins (GH-117769) 2024-04-12 13:56:41 +03:00
Serhiy Storchaka 6e05537676
gh-117764: Add docstrings and signatures for the __replace__ methods (GH-117768) 2024-04-12 08:46:20 +00:00
Erlend E. Aasland deb921f851
gh-117431: Adapt bytes and bytearray .find() and friends to Argument Clinic (#117502)
This change gives a significant speedup, as the METH_FASTCALL calling
convention is now used. The following bytes and bytearray methods are adapted:

- count()
- find()
- index()
- rfind()
- rindex()

Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2024-04-12 07:40:55 +00:00
Brett Simmers f268e328ed
gh-116738: Make _abc module thread-safe (#117488)
A collection of small changes aimed at making the `_abc` module safe to
use in a free-threaded build.
2024-04-11 18:13:25 -04:00
Erlend E. Aasland 044dc496e0
gh-117709: Add vectorcall support for str() with positional-only arguments (#117746)
Fall back to tp_call() for cases when arguments are passed by name.

Co-authored-by: Donghee Na <donghee.na@python.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
2024-04-11 13:55:37 +00:00
Erlend E. Aasland 0d42ac9474
gh-117431: Argument Clinic: copy forced text signature when cloning (#117591) 2024-04-10 10:12:05 +02:00
Serhiy Storchaka 24a2bd0481
gh-117642: Fix PEP 737 implementation (GH-117643)
* Fix implementation of %#T and %#N (they were implemented as %T# and
  %N#).
* Restore tests removed in gh-116417.
2024-04-08 16:27:25 +00:00
Sam Gross 1a6594f661
gh-117439: Make refleak checking thread-safe without the GIL (#117469)
This keeps track of the per-thread total reference count operations in
PyThreadState in the free-threaded builds. The count is merged into the
interpreter's total when the thread exits.
2024-04-08 12:11:36 -04:00
Sam Gross 2067da2579
gh-117547: Fix mimalloc compile error on OpenBSD (#117548) 2024-04-08 11:53:13 -04:00
mpage df73179048
gh-111926: Make weakrefs thread-safe in free-threaded builds (#117168)
Most mutable data is protected by a striped lock that is keyed on the
referenced object's address. The weakref's hash is protected using the
weakref's per-object lock.
 
Note that this only affects free-threaded builds. Apart from some minor
refactoring, the added code is all either gated by `ifdef`s or is a no-op
(e.g. `Py_BEGIN_CRITICAL_SECTION`).
2024-04-08 10:58:38 -04:00
Steve Dower 687616877b
gh-111140: PyLong_From/AsNativeBytes: Take *flags* rather than just *endianness* (GH-116053) 2024-04-05 16:21:16 +02:00
Dino Viehland 434bc593df
gh-112075: Make _PyDict_LoadGlobal thread safe (#117529)
Make _PyDict_LoadGlobal threadsafe
2024-04-04 12:26:07 -07:00
Eric Snow 976bcb2379
gh-76785: Raise InterpreterError, Not RuntimeError (gh-117489)
I had meant to switch everything to InterpreterError when I added it a while back.  At the time I missed a few key spots.

As part of this, I've added print-the-exception to _PyXI_InitTypes() and fixed an error case in `_PyStaticType_InitBuiltin().
2024-04-03 10:58:39 -06:00
Erlend E. Aasland 7ecd55d604
gh-117431: Adapt str.find and friends to Argument Clinic (#117468)
This change gives a significant speedup, as the METH_FASTCALL calling
convention is now used. The following methods are adapted:

- str.count
- str.find
- str.index
- str.rfind
- str.rindex
2024-04-03 17:59:18 +02:00
Erlend E. Aasland 595bb496b0
gh-117431: Adapt bytes and bytearray .startswith() and .endswith() to Argument Clinic (#117495)
This change gives a significant speedup, as the METH_FASTCALL calling
convention is now used.
2024-04-03 13:11:14 +02:00
Erlend E. Aasland 1dc1521042
gh-117431: Fix str.endswith docstring (#117499)
The first parameter is named 'suffix', not 'prefix'.

Regression introduced by commit 444156ed
2024-04-03 12:33:20 +02:00
Victor Stinner 8ef98924d3
gh-117281: Change weakref repr() to fully qualified name (#117285)
Use the fully qualified type name in repr() of weakref.ref and
weakref.proxy types.

Fix a crash in proxy_repr() when the reference is dead.

Add also test_ref_repr() and test_proxy_repr().
2024-04-03 12:18:05 +02:00
Erlend E. Aasland 444156ede4
gh-117431: Adapt str.startswith and str.endswith to Argument Clinic (#117466)
This change gives a significant speedup, as the METH_FASTCALL calling
convention is now used.
2024-04-03 09:11:39 +02:00
Sam Gross 027fa2eccf
gh-112087: Make `list.extend(dict)` behave atomically (#117438)
Add a special case for `list.extend(dict)` and `list(dict)` so that those
patterns behave atomically with respect to modifications to the list or
dictionary.

This is required by multiprocessing, which assumes that
`list(_finalizer_registry)` is atomic.
2024-04-02 10:45:00 -04:00
Mark Shannon c32dc47aca
GH-115776: Embed the values array into the object, for "normal" Python objects. (GH-116115) 2024-04-02 11:59:21 +01:00
mpage 5d21d884b6
gh-111926: Avoid locking in PyType_IsSubtype (#117275)
Read the MRO in a thread-unsafe way in `PyType_IsSubtype` to avoid locking. Fixing this is tracked in #117306. 

The motivation for this change is in support of making weakrefs thread-safe in free-threaded builds:

`WeakValueDictionary` uses a special dictionary function, `_PyDict_DelItemIf`
to remove dead weakrefs from the dictionary. `_PyDict_DelItemIf` removes a key
if a user supplied predicate evaluates to true for the value associated with
the key. Crucially for the `WeakValueDictionary` use case, the predicate
evaluation + deletion sequence is atomic, provided that the predicate doesn’t
suspend. The predicate used by `WeakValueDictionary` includes a subtype check,
which we must ensure doesn't suspend in free-threaded builds.
2024-03-29 13:42:02 -04:00
Sam Gross 19c1dd60c5
gh-117323: Make `cell` thread-safe in free-threaded builds (#117330)
Use critical sections to lock around accesses to cell contents. The critical sections are no-ops in the default (with GIL) build.
2024-03-29 13:35:43 -04:00
Serhiy Storchaka 0c1a42cf9c
gh-87193: Support bytes objects with refcount > 1 in _PyBytes_Resize() (GH-117160)
Create a new bytes object and destroy the old one if it has refcount > 1.
2024-03-25 16:32:11 +01:00
Serhiy Storchaka e2e0b4b4b9
gh-113024: C API: Add PyObject_GenericHash() function (GH-113025) 2024-03-22 20:19:10 +02:00
Tim Peters 8383915031
GH-116939: Rewrite binarysort() (#116940)
Rewrote binarysort() for clarity.

Also changed the signature to be more coherent (it was mixing sortslice with raw pointers).

No change in method or functionality. However, I left some experiments in, disabled for now
via `#if` tricks. Since this code was first written, some kinds of comparisons have gotten
enormously faster (like for lists of floats), which changes the tradeoffs.

For example, plain insertion sort's simpler innermost loop and highly predictable branches
leave it very competitive (even beating, by a bit) binary insertion when comparisons are
very cheap, despite that it can do many more compares. And it wins big on runs that
are already sorted (moving the next one in takes only 1 compare then).

So I left code for a plain insertion sort, to make future experimenting easier.

Also made the maximum value of minrun a `#define` (``MAX_MINRUN`) to make
experimenting with that easier too.

And another bit of `#if``-disabled code rewrites binary insertion's innermost loop to
remove its unpredictable branch. Surprisingly, this doesn't really seem to help
overall. I'm unclear on why not. It certainly adds more instructions, but they're very
simple, and it's hard to be believe they cost as much as a branch miss.
2024-03-21 22:27:25 -05:00
Guido van Rossum 97ba910e47
gh-108716:: Remove _PyStaticCode_Init/Fini (#117141)
More deepfreeze cleanup.
2024-03-22 01:27:48 +00:00
Guido van Rossum 570a82d46a
gh-117045: Add code object to function version cache (#117028)
Changes to the function version cache:

- In addition to the function object, also store the code object,
  and allow the latter to be retrieved even if the function has been evicted.
- Stop assigning new function versions after a critical attribute (e.g. `__code__`)
  has been modified; the version is permanently reset to zero in this case.
- Changes to `__annotations__` are no longer considered critical. (This fixes gh-109998.)

Changes to the Tier 2 optimization machinery:

- If we cannot map a function version to a function, but it is still mapped to a code object,
  we continue projecting the trace.
  The operand of the `_PUSH_FRAME` and `_POP_FRAME` opcodes can be either NULL,
  a function object, or a code object with the lowest bit set.

This allows us to trace through code that calls an ephemeral function,
i.e., a function that may not be alive when we are constructing the executor,
e.g. a generator expression or certain nested functions.
We will lose globals removal inside such functions,
but we can still do other peephole operations
(and even possibly [call inlining](https://github.com/python/cpython/pull/116290),
if we decide to do it), which only need the code object.
As before, if we cannot retrieve the code object from the cache, we stop projecting.
2024-03-21 12:37:41 -07:00
Eric Snow 617158e078
gh-76785: Drop PyInterpreterID_Type (gh-117101)
I added it quite a while ago as a strategy for managing interpreter lifetimes relative to the PEP 554 (now 734) implementation.  Relatively recently I refactored that implementation to no longer rely on InterpreterID objects.  Thus now I'm removing it.
2024-03-21 17:15:02 +00:00
Victor Stinner 8bea6c411d
gh-115754: Add Py_GetConstant() function (#116883)
Add Py_GetConstant() and Py_GetConstantBorrowed() functions.

In the limited C API version 3.13, getting Py_None, Py_False,
Py_True, Py_Ellipsis and Py_NotImplemented singletons is now
implemented as function calls at the stable ABI level to hide
implementation details. Getting these constants still return borrowed
references.

Add _testlimitedcapi/object.c and test_capi/test_object.py to test
Py_GetConstant() and Py_GetConstantBorrowed() functions.
2024-03-21 16:07:00 +00:00
Eric Snow bbee57fa8c
gh-76785: Clean Up Interpreter ID Conversions (gh-117048)
Mostly we unify the two different implementations of the conversion code (from PyObject * to int64_t.  We also drop the PyArg_ParseTuple()-style converter function, as well as rename and move PyInterpreterID_LookUp().
2024-03-21 09:56:12 -06:00
Mark Shannon 15309329b6
GH-108362: Incremental Cycle GC (GH-116206) 2024-03-20 08:54:42 +00:00
Sam Gross 60e105c1c1
gh-113964: Don't prevent new threads until all non-daemon threads exit (#116677)
Starting in Python 3.12, we prevented calling fork() and starting new threads
during interpreter finalization (shutdown). This has led to a number of
regressions and flaky tests. We should not prevent starting new threads
(or `fork()`) until all non-daemon threads exit and finalization starts in
earnest.

This changes the checks to use `_PyInterpreterState_GetFinalizing(interp)`,
which is set immediately before terminating non-daemon threads.
2024-03-19 14:40:20 -04:00
Bogdan Romanyuk a8e93d3dca
gh-115756: make PyCode_GetFirstFree an unstable API (GH-115781) 2024-03-19 09:20:38 +00:00
Donghee Na a3cf0fada0
gh-116621: Specialize list.extend for dict items (gh-116888) 2024-03-19 12:18:07 +09:00
Guido van Rossum 7e1f38f2de
gh-116916: Remove separate next_func_version counter (#116918)
Somehow we ended up with two separate counter variables tracking "the next function version".
Most likely this was a historical accident where an old branch was updated incorrectly.
This PR merges the two counters into a single one: `interp->func_state.next_version`.
2024-03-18 11:11:10 -07:00
Victor Stinner f6cdc6b4a1
Revert "gh-96844: Improve error message of list.remove (gh-106455)" (#116956)
This reverts commit 217f47d6e5.
2024-03-18 13:54:45 +00:00
Dino Viehland 280de3661b
gh-116868: Avoid locking in PyType_IsSubtype (#116829)
Make PyType_IsSubType not acquire lock
2024-03-15 12:35:29 -04:00
Donghee Na 8da83f3386
gh-116621: Specialize list.extend for dict keys/values (gh-116816) 2024-03-15 23:48:34 +09:00
mpage ce2c996b2f
gh-111926: Simplify proxy creation logic (#116844)
Since 3.12, allocating a GC-able object cannot trigger GC. This allows
us to simplify the logic for creating the canonical callback-less
proxy object.
2024-03-15 09:58:40 -04:00
mpage 001b21d1c5
gh-111926: Simplify weakref creation logic (#116843)
Since 3.12, allocating a GC object cannot immediately trigger GC. This
allows us to simplify the logic for creating the canonical callback-less
weakref.
2024-03-15 09:56:13 -04:00
Victor Stinner 7bbb9b57e6
gh-111696, PEP 737: Add %T and %N to PyUnicode_FromFormat() (#116839) 2024-03-14 22:23:00 +00:00
Victor Stinner c432df6d56
gh-111696, PEP 737: Add PyType_GetModuleName() function (#116824)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2024-03-14 18:17:43 +00:00
Victor Stinner 19c3a2ff91
gh-111696, PEP 737: Add PyType_GetFullyQualifiedName() function (#116815)
Rewrite tests on type names in Python, they were written in C.
2024-03-14 16:19:36 +00:00
Nikita Sobolev fcd49b4f47
gh-116714: Handle errors correctly in `PyFloat_GetInfo` (#116715)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2024-03-13 12:38:03 +00:00
Tim Peters bf121d6a69
GH-116554: Relax list.sort()'s notion of "descending" runs (#116578)
* GH-116554: Relax list.sort()'s notion of "descending" run

Rewrote `count_run()` so that sub-runs of equal elements no longer end a descending run. Both ascending and descending runs can have arbitrarily many sub-runs of arbitrarily many equal elements now. This is tricky, because we only use ``<`` comparisons, so checking for equality doesn't come "for free". Surprisingly, it turned out there's a very cheap (one comparison) way to determine whether an ascending run consisted of all-equal elements. That sealed the deal.

In addition, after a descending run is reversed in-place, we now go on to see whether it can be extended by an ascending run that just happens to be adjacent. This succeeds in finding at least one additional element to append about half the time, and so appears to more than repay its cost (the savings come from getting to skip a binary search, when a short run is artificially forced to length MIINRUN later, for each new element `count_run()` can add to the initial run).

While these have been in the back of my mind for years, a question on StackOverflow pushed it to action:

https://stackoverflow.com/questions/78108792/

They were wondering why it took about 4x longer to sort a list like:

[999_999, 999_999, ..., 2, 2, 1, 1, 0, 0]

than "similar" lists. Of course that runs very much faster after this patch.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
2024-03-12 19:59:42 -05:00
Donghee Na 3325699ffa
gh-116621: Set manual critical section for list.extend (gh-116657) 2024-03-13 07:28:23 +09:00
Nikita Sobolev eb947cdc13
gh-110819: Fix ‘kind’ may be used uninitialized warning in `longobject` (#116599) 2024-03-12 13:50:06 +03:00
Donghee Na 5b2f21faf3
gh-112087: Make list.sort to be thread-safe for PEP 703. (gh-116553) 2024-03-10 00:45:42 +00:00
Donghee Na 17d31bf384
gh-112087: Store memory allocation information into _PyListArray (gh-116529) 2024-03-09 23:50:28 +00:00
Tomas R c951e25c24
gh-112069: Make sets thread-safe with the GIL disabled (#113800)
This makes nearly all the operations on set thread-safe in the free-threaded build, with the exception of `_PySet_NextEntry` and `setiter_iternext`.

Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-03-08 16:25:34 -05:00
Dino Viehland 7db871e4fa
gh-112075: Support freeing object memory via QSBR (#116344)
Free objects with qsbr if shared
2024-03-08 09:56:36 -08:00
Ken Jin 41457c7fdb
gh-116381: Remove bad specializations, add fail stats (GH-116464)
* Remove bad specializations, add fail stats
2024-03-08 00:21:21 +08:00
Serhiy Storchaka 72d3cc94cd
gh-116437: Use new C API PyDict_Pop() to simplify the code (GH-116438) 2024-03-07 11:21:08 +02:00
Ken Jin 7114cf20c0
gh-116381: Specialize CONTAINS_OP (GH-116385)
* Specialize CONTAINS_OP

* 📜🤖 Added by blurb_it.

* Add PyAPI_FUNC for JIT

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2024-03-07 03:30:11 +08:00
Sam Gross c012c8ab7b
gh-115103: Delay reuse of mimalloc pages that store PyObjects (#115435)
This implements the delayed reuse of mimalloc pages that contain Python
objects in the free-threaded build.

Allocations of the same size class are grouped in data structures called
pages. These are different from operating system pages. For thread-safety, we
want to ensure that memory used to store PyObjects remains valid as long as
there may be concurrent lock-free readers; we want to delay using it for
other size classes, in other heaps, or returning it to the operating system.

When a mimalloc page becomes empty, instead of immediately freeing it, we tag
it with a QSBR goal and insert it into a per-thread state linked list of
pages to be freed. When mimalloc needs a fresh page, we process the queue and
free any still empty pages that are now deemed safe to be freed. Pages
waiting to be freed are still available for allocations of the same size
class and allocating from a page prevent it from being freed. There is
additional logic to handle abandoned pages when threads exit.
2024-03-06 09:42:11 -05:00
Donghee Na d2f1b0eb49
gh-112087: Update list_get_item_ref to optimistically avoid locking (gh-116353)
Co-authored-by: Sam Gross <colesbury@gmail.com>
2024-03-06 08:21:33 +09:00
Sam Gross 72714c0266
gh-115103: Enable internal mimalloc assertions in debug builds (#116343)
This sets `MI_DEBUG` to `2` in debug builds to enable `mi_assert_internal()`
calls. Expensive internal assertions are not enabled.

This also disables an assertion in free-threaded builds that would be
triggered by the free-threaded GC because we traverse heaps that are not
owned by the current thread.
2024-03-05 13:54:20 -05:00
cui fliter e7ba6e9dbe
chore: fix typos (#116345)
Signed-off-by: cui fliter <imcusg@gmail.com>
2024-03-05 09:05:52 -07:00
Dino Viehland ffcc450a9b
gh-112075: Enable freeing with qsbr and fallback to lock on key changed (GH-116336) 2024-03-05 09:08:18 +00:00
Donghee Na 6cddc731fb
gh-112087: Make list_{slice, ass_slice, subscript} to be threadsafe (gh-116233) 2024-03-05 04:58:14 +00:00
Sam Gross 58c7919d05
gh-116029: Fix unused function warning on macOS (#116340) 2024-03-04 22:06:21 -05:00
Kirill Podoprigora 8a84eb75a9
gh-116316: Fix typo in `UNARY_FUNC(PyNumber_Positive)` macros (GH-116317) 2024-03-04 17:57:01 +00:00
Erlend E. Aasland 17c4849981
gh-116296: Fix refleak in reduce_newobj() corner case (#116297)
Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
2024-03-04 09:46:18 +00:00
Donghee Na fb5e0344e4
gh-112087: Use QSBR technique for list_new/clear for free-thread build (gh-115875) 2024-03-02 08:30:35 +09:00
Humbulani 5e0c7bc1d3
gh-115941: fixes in dictobject.c doc block(#116196) 2024-03-01 12:48:26 -07:00
Sam Gross d7ddd90308
gh-115491: Fix Clang compiler warning (#116153)
gh-115491: Fix compiler warning on macOS
2024-02-29 16:02:44 -08:00
Dino Viehland 556749c3e3
gh-112075: Avoid locking shared keys on every assignment (#116087) 2024-02-29 13:38:50 -08:00
Dino Viehland 81c79961d2
gh-112075: Use relaxed stores for places where we may race with when reading lock-free (#115786) 2024-02-28 14:53:19 -08:00
Sam Gross 75c6c05fea
gh-115891: Fix debug byte filling in free-threaded build (#116018)
The previous code had two bugs. First, the debug offset in the mimalloc
heap includes the two pymalloc debug words, but the pointer passed to
fill_mem_debug does not include them. Second, the current object heap is
correct source for allocations, but not deallocations.
2024-02-28 12:50:09 -08:00
Donghee Na f58f8cef74
gh-112075: Remove compiler warning from apple clang (gh-115855) 2024-02-29 02:51:59 +09:00
Jay Ting 948acd6ed8
gh-115323: Add meaningful error message for using bytearray.extend with str (#115332)
Perform str check after TypeError is raised
---------

Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2024-02-24 18:34:45 -05:00
Dino Viehland 1002fbe12e
gh-112075: Iterating a dict shouldn't require locks (#115108)
Makes iteration of a dict be lock free for the forward iteration case.
2024-02-22 12:02:39 -08:00
Nikita Sobolev 465df8855e
gh-115827: Fix compile warning in `longobject.c` (#115828)
Objects/longobject.c:1186:42: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
2024-02-22 17:57:12 +00:00
Sam Gross 347acded84
gh-115491: Keep some fields valid across allocations in obmalloc (free-threading) (#115745) 2024-02-21 10:00:08 -08:00
Donghee Na 259730bbb5
gh-112087: Make list_{concat, repeat, inplace_repeat, ass_item) to be thread-safe (gh-115605) 2024-02-21 01:38:09 +00:00
Dino Viehland 54071460d7
gh-112075: Accessing a single element should optimistically avoid locking (#115109)
Makes accessing a single element thread safe and typically lock free
2024-02-20 17:08:14 -08:00
Dino Viehland 176df09adb
gh-112075: Make PyDictKeysObject thread-safe (#114741)
Adds locking for shared PyDictKeysObject's for dictionaries
2024-02-20 16:40:37 -08:00
Sam Gross 520403ed4c
gh-115733: Fix crash involving exhausted list iterator (#115740)
* gh-115733: Fix crash involving exhausted iterator

* Add blurb
2024-02-21 05:18:44 +09:00
Sam Gross e3ad6ca56f
gh-115103: Implement delayed free mechanism for free-threaded builds (#115367)
This adds `_PyMem_FreeDelayed()` and supporting functions. The
`_PyMem_FreeDelayed()` function frees memory with the same allocator as
`PyMem_Free()`, but after some delay to ensure that concurrent lock-free
readers have finished.
2024-02-20 13:04:37 -05:00
Sam Gross cc82e33af9
gh-115491: Keep some fields valid across allocations (free-threading) (#115573)
This avoids filling the memory occupied by ob_tid, ob_ref_local, and
ob_ref_shared with debug bytes (e.g., 0xDD) in mimalloc in the
free-threaded build.
2024-02-20 10:36:40 -05:00
Eugene Toder c0b0c2f201
gh-101860: Expose __name__ on property (GH-101876)
Useful for introspection and consistent with functions and other
descriptors.
2024-02-20 17:14:34 +02:00
Serhiy Storchaka 090dd21ab9
gh-115618: Remove improper Py_XDECREFs in property methods (GH-115619) 2024-02-17 23:18:30 +02:00
Donghee Na 8db8d7118e
gh-111968: Split _Py_async_gen_asend_freelist out of _Py_async_gen_fr… (gh-115546) 2024-02-17 10:03:10 +09:00
Benjamin Peterson 2ac9d9f2fb
gh-113743: Give _PyTypes_AfterFork a prototype. (gh-115563)
Fixes a compiler warning.
2024-02-16 08:49:41 -08:00
Sam Gross b24c9161a6
gh-112529: Make the GC scheduling thread-safe (#114880)
The GC keeps track of the number of allocations (less deallocations)
since the last GC. This buffers the count in thread-local state and uses
atomic operations to modify the per-interpreter count. The thread-local
buffering avoids contention on shared state.

A consequence is that the GC scheduling is not as precise, so
"test_sneaky_frame_object" is skipped because it requires that the GC be
run exactly after allocating a frame object.
2024-02-16 11:22:27 -05:00
Donghee Na 321d13fd2b
gh-111968: Split _Py_dictkeys_freelist out of _Py_dict_freelist (gh-115505) 2024-02-16 01:01:36 +00:00
Dino Viehland 454d7963e3
gh-113743: Use per-interpreter locks for types (#115541)
Move type-lock to per-interpreter lock to avoid heavy contention in interpreters test
2024-02-15 16:28:31 -08:00
Dino Viehland ae460d450a
gh-113743: Make the MRO cache thread-safe in free-threaded builds (#113930)
Makes _PyType_Lookup thread safe, including:
    Thread safety of the underlying cache.
    Make mutation of mro and type members thread safe
    Also _PyType_GetMRO and _PyType_GetBases are currently returning borrowed references which aren't safe.
2024-02-15 10:54:57 -08:00
David Hewitt 9e3729bbd7
gh-114626: add PyCFunctionFast and PyCFunctionFastWithKeywords (GH-114627)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-02-15 11:05:20 +01:00
Victor Stinner 3e7b7df5cb
gh-114570: Add PythonFinalizationError exception (#115352)
Add PythonFinalizationError exception. This exception derived from
RuntimeError is raised when an operation is blocked during the Python
finalization.

The following functions now raise PythonFinalizationError, instead of
RuntimeError:

* _thread.start_new_thread()
* subprocess.Popen
* os.fork()
* os.fork1()
* os.forkpty()

Morever, _winapi.Overlapped finalizer now logs an unraisable
PythonFinalizationError, instead of an unraisable RuntimeError.
2024-02-14 23:35:06 +01:00
Donghee Na a2d4281415
gh-112087: Make __sizeof__ and listiter_{len, next} to be threadsafe (gh-114843) 2024-02-15 02:00:50 +09:00
Donghee Na f15795c9a0
gh-111968: Rename freelist related struct names to Eric's suggestion (gh-115329) 2024-02-14 00:32:51 +00:00
Kirill Podoprigora 206f73dc5f
gh-115391: Fix compiler warning in `Objects/longobject.c` (GH-115368) 2024-02-13 20:49:13 +00:00
Steve Dower ea25f32d5f
gh-89240: Enable multiprocessing on Windows to use large process pools (GH-107873)
We add _winapi.BatchedWaitForMultipleObjects to wait for larger numbers of handles.
This is an internal module, hence undocumented, and should be used with caution.
Check the docstring for info before using BatchedWaitForMultipleObjects.
2024-02-13 00:28:35 +00:00
Steve Dower 7861dfd26a
gh-111140: Adds PyLong_AsNativeBytes and PyLong_FromNative[Unsigned]Bytes functions (GH-114886) 2024-02-12 20:13:13 +00:00
Brandt Bucher 235cacff81
GH-114695: Add `sys._clear_internal_caches` (GH-115152) 2024-02-12 09:04:36 +00:00
Donghee Na d4d5bae147
gh-111968: Refactor _PyXXX_Fini to integrate with _PyObject_ClearFreeLists (gh-114899) 2024-02-10 00:57:04 +00:00
Sam Gross a3af3cb4f4
gh-110481: Implement inter-thread queue for biased reference counting (#114824)
Biased reference counting maintains two refcount fields in each object:
`ob_ref_local` and `ob_ref_shared`. The true refcount is the sum of these two
fields. In some cases, when refcounting operations are split across threads,
the ob_ref_shared field can be negative (although the total refcount must be
at least zero). In this case, the thread that decremented the refcount
requests that the owning thread give up ownership and merge the refcount
fields.
2024-02-09 17:08:32 -05:00
Sam Gross 31633f4473
gh-115184: Fix refleak tracking issues in free-threaded build (#115188)
Fixes a few issues related to refleak tracking in the free-threaded build:

- Count blocks in abandoned segments
- Call `_mi_page_free_collect` earlier during heap traversal in order to get an accurate count of blocks in use.
- Add missing refcount tracking in `_Py_DecRefSharedDebug` and `_Py_ExplicitMergeRefcount`.
- Pause threads in  `get_num_global_allocated_blocks` to ensure that traversing the mimalloc heaps is safe.
2024-02-09 09:23:12 -05:00
Tomas R ed1a8daf10
gh-112069: Adapt set/frozenset methods to Argument Clinic (#115112) 2024-02-08 17:47:27 +01:00
Sam Gross ef3ceab09d
gh-112066: Use `PyDict_SetDefaultRef` in place of `PyDict_SetDefault`. (#112211)
This changes a number of internal usages of `PyDict_SetDefault` to use `PyDict_SetDefaultRef`.

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-02-07 13:43:18 -05:00
Mark Shannon 8a3c499ffe
GH-108362: Revert "GH-108362: Incremental GC implementation (GH-108038)" (#115132)
Revert "GH-108362: Incremental GC implementation (GH-108038)"

This reverts commit 36518e69d7.
2024-02-07 12:38:34 +00:00
Dino Viehland 92abb01240
gh-112075: Add critical sections for most dict APIs (#114508)
Starts adding thread safety to dict objects.


Use @critical_section for APIs which are exposed via argument clinic and don't directly correlate with a public C API which needs to acquire the lock
Use a _lock_held suffix for keeping changes to complicated functions simple and just wrapping them with a critical section
Acquire and release the lock in an existing function where it won't be overly disruptive to the existing logic
2024-02-06 14:03:43 -08:00
Donghee Na f7a22a7055
gh-112087: Make list_{count, index, contains} to be thread-safe. (gh-114916) 2024-02-06 16:41:18 +00:00
Sam Gross de61d4bd4d
gh-112066: Add `PyDict_SetDefaultRef` function. (#112123)
The `PyDict_SetDefaultRef` function is similar to `PyDict_SetDefault`,
but returns a strong reference through the optional `**result` pointer
instead of a borrowed reference.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-02-06 11:36:23 -05:00
Serhiy Storchaka 652fbf88c4
gh-82626: Emit a warning when bool is used as a file descriptor (GH-111275) 2024-02-05 22:51:11 +02:00
Mark Shannon 36518e69d7
GH-108362: Incremental GC implementation (GH-108038) 2024-02-05 18:28:51 +00:00
Mark Shannon 992446dd5b
GH-113462: Limit the number of versions that a single class can use. (GH-114900) 2024-02-05 16:20:54 +00:00
Nikita Sobolev 929d44e15a
gh-114685: PyBuffer_FillInfo() now raises on PyBUF_{READ,WRITE} (GH-114802) 2024-02-04 19:16:43 +00:00
Sam Gross d0f1307580
gh-114329: Add `PyList_GetItemRef` function (GH-114504)
The new `PyList_GetItemRef` is similar to `PyList_GetItem`, but returns
a strong reference instead of a borrowed reference. Additionally, if the
passed "list" object is not a list, the function sets a `TypeError`
instead of calling `PyErr_BadInternalCall()`.
2024-02-02 14:03:15 +01:00
Mark Shannon 0e71a295e9
GH-113710: Add a "globals to constants" pass (GH-114592)
Converts specializations of `LOAD_GLOBAL` into constants during tier 2 optimization.
2024-02-02 12:14:34 +00:00
Donghee Na 13907968d7
gh-111968: Use per-thread freelists for dict in free-threading (gh-114323) 2024-02-01 20:53:53 +00:00
Sam Gross 587d480203
gh-112529: Remove PyGC_Head from object pre-header in free-threaded build (#114564)
* gh-112529: Remove PyGC_Head from object pre-header in free-threaded build

This avoids allocating space for PyGC_Head in the free-threaded build.
The GC implementation for free-threaded CPython does not use the
PyGC_Head structure.

 * The trashcan mechanism uses the `ob_tid` field instead of `_gc_prev`
   in the free-threaded build.
 * The GDB libpython.py file now determines the offset of the managed
   dict field based on whether the running process is a free-threaded
   build. Those are identified by the `ob_ref_local` field in PyObject.
 * Fixes `_PySys_GetSizeOf()` which incorrectly incorrectly included the
   size of `PyGC_Head` in the size of static `PyTypeObject`.
2024-02-01 12:29:19 -08:00
Donghee Na 7b9d406729
gh-112087: Make PyList_{Append,Size,GetSlice} to be thread-safe (gh-114651) 2024-02-01 08:58:08 +09:00
Albert Zeyer 78c254582b
gh-113939: Frame clear, clear locals (#113940) 2024-01-31 19:14:44 +00:00
Serhiy Storchaka b7688ef71e
gh-114685: Check flags in PyObject_GetBuffer() (GH-114707)
PyObject_GetBuffer() now raises a SystemError if called with
PyBUF_READ or PyBUF_WRITE as flags. These flags should
only be used with the PyMemoryView_* C API.
2024-01-31 13:11:35 +02:00
Dino Viehland 0990d55725
gh-112075: refactor dictionary lookup functions for better re-usability (#114629)
Refactor dict lookup functions to use force inline helpers
2024-01-30 09:33:36 -08:00
Pablo Galindo Salgado 39d102c2ee
gh-113744: Add a new IncompleteInputError exception to improve incomplete input detection in the codeop module (#113745)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2024-01-30 16:21:30 +00:00
Erlend E. Aasland 53d921ed96
gh-114569: Use PyMem_* APIs for non-PyObjects in unicodeobject.c (#114690) 2024-01-29 21:48:49 +01:00
Dino Viehland 3d716655d2
gh-112075: Use PyMem_* for allocating dict keys objects (#114543)
Use PyMem_* for keys allocation
2024-01-29 09:38:03 -08:00
Furkan Onder f7c05d7ad3
gh-55664: Add warning when creating a type using a namespace dictionary with non-string keys. (GH-105338)
Co-authored-by: Daniel Urban <durban@users.noreply.github.com>
2024-01-29 01:05:29 +02:00
Charlie Zhao 11c582235d
gh-113560: Improve docstrings for set.issubset() and set.issuperset() (GH-113562) 2024-01-27 09:53:01 +00:00
Neil Schemenauer 7a7bce5a0a
gh-113055: Use pointer for interp->obmalloc state (gh-113412)
For interpreters that share state with the main interpreter, this points
to the same static memory structure.  For interpreters with their own
obmalloc state, it is heap allocated.  Add free_obmalloc_arenas() which
will free the obmalloc arenas and radix tree structures for interpreters
with their own obmalloc state.

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2024-01-26 19:38:14 -08:00
Donghee Na f9c505698a
gh-112087: Make list_repr and list_length to be thread-safe (gh-114582) 2024-01-27 01:20:21 +09:00
Donghee Na 699779256e
gh-111968: Unify freelist naming schema to Eric's suggestion (gh-114581) 2024-01-27 00:25:16 +09:00
Erlend E. Aasland dcd28b5c35
gh-114569: Use PyMem_* APIs for most non-PyObject uses (#114574)
Fix usage in Modules, Objects, and Parser subdirectories.
2024-01-26 10:11:35 +00:00
Michael Droettboom ea3cd0498c
gh-114312: Collect stats for unlikely events (GH-114493) 2024-01-25 11:10:51 +00:00
Dino Viehland afe8f376c0
gh-112075: Adapt more dict methods to Argument Clinic (#114256)
* Move more dict objects to argument clinic

* Improve doc strings

* More doc string improvements

* Update Objects/dictobject.c

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

* Update Objects/dictobject.c

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

* Update Objects/dictobject.c

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

* Update Objects/dictobject.c

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

* Update Objects/dictobject.c

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

* Update Objects/dictobject.c

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

* Update Objects/dictobject.c

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

---------

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-01-23 22:10:04 +00:00
Sam Gross 412920a41e
gh-112532: Improve mimalloc page visiting (#114133)
This adds support for visiting abandoned pages in mimalloc and improves
the performance of the page visiting code. Abandoned pages contain
memory blocks from threads that have exited. At some point, they may be
later reclaimed by other threads. We still need to visit those pages in
the free-threaded GC because they contain live objects.

This also reduces the overhead of visiting mimalloc pages:

 * Special cases for full, empty, and pages containing only a single
   block.
 * Fix free_map to use one bit instead of one byte per block.
 * Use fast integer division by a constant algorithm when computing
   block offset from block size and index.
2024-01-22 13:10:21 -08:00
Sam Gross 1d6d5e854c
gh-112529: Use GC heaps for GC allocations in free-threaded builds (gh-114157)
* gh-112529: Use GC heaps for GC allocations in free-threaded builds

The free-threaded build's garbage collector implementation will need to
find GC objects by traversing mimalloc heaps. This hooks up the
allocation calls with the correct heaps by using a thread-local
"current_obj_heap" variable.

* Refactor out setting heap based on type
2024-01-21 01:14:45 +09:00
Donghee Na a34e4db28a
gh-111968: Fix --without-freelists build (gh-114270) 2024-01-18 23:25:13 +00:00
Donghee Na d5442851a6
gh-112087: Remove duplicated critical_section (gh-114268) 2024-01-18 23:03:28 +00:00
Donghee Na 7fa511ba57
gh-111968: Use per-thread freelists for generator in free-threading (gh-114189) 2024-01-18 18:15:00 +00:00
kcatss a571a2fd3f
gh-114050: Fix crash when more than two arguments are passed to int() (GH-114067)
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
2024-01-18 13:27:44 +02:00
Mano Sriram 60ca37fdee
gh-106293: Fix typos in Objects/object_layout.md (#106294)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2024-01-16 23:11:15 +00:00
Sam Gross b331381485
gh-112529: Track if debug allocator is used as underlying allocator (#113747)
* gh-112529: Track if debug allocator is used as underlying allocator

The GC implementation for free-threaded builds will need to accurately
detect if the debug allocator is used because it affects the offset of
the Python object from the beginning of the memory allocation. The
current implementation of `_PyMem_DebugEnabled` only considers if the
debug allocator is the outer-most allocator; it doesn't handle the case
of "hooks" like tracemalloc being used on top of the debug allocator.

This change enables more accurate detection of the debug allocator by
tracking when debug hooks are enabled.

* Simplify _PyMem_DebugEnabled
2024-01-16 13:42:15 -08:00
Donghee Na 42b90cf0d6
gh-112087: Update list impl to be thread-safe with manual CS (gh-113863) 2024-01-16 09:11:14 +09:00
Donghee Na 3eae76554b
gh-111968: Use per-thread slice_cache in free-threading (gh-113972) 2024-01-16 00:38:57 +09:00
Sam Gross 44e47dfba5
gh-112532: Fix memory block count for free-threaded build (gh-113995)
This fixes `_PyInterpreterState_GetAllocatedBlocks()` and
`_Py_GetGlobalAllocatedBlocks()` in the free-threaded builds. The
gh-113263 change that introduced multiple mimalloc heaps per-thread
broke the logic for counting the number of allocated blocks. For subtle
reasons, this led to reported reference count leaks in the refleaks
buildbots.
2024-01-16 00:37:36 +09:00
Sergey B Kirpichev 0f2fa6150b
gh-109598: make PyComplex_RealAsDouble/ImagAsDouble use __complex__ (GH-109647)
`PyComplex_RealAsDouble()`/`PyComplex_ImagAsDouble` now try to convert
an object to a `complex` instance using its `__complex__()` method
before falling back to the ``__float__()`` method.

PyComplex_ImagAsDouble() also will not silently return 0.0 for
non-complex types anymore.  Instead we try to call PyFloat_AsDouble()
and return 0.0 only if this call is successful.
2024-01-15 16:04:17 +01:00
Donghee Na efa738e862
gh-111968: Explicit handling for finalized freelist (gh-113929) 2024-01-12 00:31:28 +00:00
Donghee Na 2e7577b622
gh-111968: Use per-thread freelists for tuple in free-threading (gh-113921) 2024-01-12 03:46:28 +09:00
Nikita Sobolev 2ac4cf4743
gh-112640: Add `kwdefaults` parameter to `types.FunctionType.__new__` (#112641) 2024-01-11 00:42:30 -08:00
Donghee Na c65ae26f2b
gh-111968: Unify naming scheme for freelist (gh-113919) 2024-01-11 08:51:51 +09:00
Sam Gross 73ae2023a7
gh-113753: Clear finalized bit when putting PyAsyncGenASend back into free list (#113754) 2024-01-10 10:18:38 -08:00
Donghee Na f728f7242c
gh-111968: Use per-thread freelists for float in free-threading (gh-113886) 2024-01-10 15:47:13 +00:00
Donghee Na 57bdc6c30d
gh-111968: Introduce _PyFreeListState and _PyFreeListState_GET API (gh-113584) 2024-01-10 08:04:41 +09:00
Donghee Na a023bc252d
gh-112087: Update list.{pop,clear,reverse,remove} to use CS (gh-113764) 2024-01-09 09:00:55 +09:00
Jakub Kulík 10d3f04aec
gh-112808: Fix mimalloc build on Solaris (#112809) 2024-01-08 15:50:56 -08:00
Jakub Kulík ae968d1032
gh-112806: Remove unused function warnings during mimalloc build on Solaris (#112807) 2024-01-08 15:49:33 -08:00
Sam Gross d0f0308a37
gh-113750: Fix object resurrection in free-threaded builds (gh-113751)
gh-113750: Fix object resurrection on free-threaded builds

This avoids the undesired re-initializing of fields like `ob_gc_bits`,
`ob_mutex`, and `ob_tid` when an object is resurrected due to its
finalizer being called.

This change has no effect on the default (with GIL) build.
2024-01-06 12:12:26 +09:00
Sam Gross 0b7476080b
gh-112532: Tag mimalloc heaps and pages (#113742)
* gh-112532: Tag mimalloc heaps and pages

Mimalloc pages are data structures that contain contiguous allocations
of the same block size. Note that they are distinct from operating
system pages. Mimalloc pages are contained in segments.

When a thread exits, it abandons any segments and contained pages that
have live allocations. These segments and pages may be later reclaimed
by another thread. To support GC and certain thread-safety guarantees in
free-threaded builds, we want pages to only be reclaimed by the
corresponding heap in the claimant thread. For example, we want pages
containing GC objects to only be claimed by GC heaps.

This allows heaps and pages to be tagged with an integer tag that is
used to ensure that abandoned pages are only claimed by heaps with the
same tag. Heaps can be initialized with a tag (0-15); any page allocated
by that heap copies the corresponding tag.

* Fix conversion warning
2024-01-05 12:08:50 -08:00
Sam Gross fcb3c2a444
gh-112532: Isolate abandoned segments by interpreter (#113717)
* gh-112532: Isolate abandoned segments by interpreter

Mimalloc segments are data structures that contain memory allocations along
with metadata. Each segment is "owned" by a thread. When a thread exits,
it abandons its segments to a global pool to be later reclaimed by other
threads. This changes the pool to be per-interpreter instead of process-wide.

This will be important for when we use mimalloc to find GC objects in the
`--disable-gil` builds. We want heaps to only store Python objects from a
single interpreter. Absent this change, the abandoning and reclaiming process
could break this isolation.

* Add missing '&_mi_abandoned_default' to 'tld_empty'
2024-01-04 22:21:40 +00:00
Alex Waygood f1f8392432
Document the `co_lines` method on code objects (#113682)
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2024-01-03 19:29:24 +00:00
Christopher Chavez a1eea1d032
gh-111178: Avoid calling functions from incompatible pointer types in dictobject.c (#112892)
Fix undefined behavior warnings (UBSan  -fsanitize=function).
2024-01-02 15:32:37 +01:00
Christopher Chavez acf4cf5ca5
gh-111178: Avoid calling functions from incompatible pointer types in descrobject.c (GH-112861)
Fix undefined behavior warnings (UBSan  -fsanitize=function), for example:

Python/generated_cases.c.h:3315:13: runtime error: call to function mappingproxy_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'
descrobject.c:1160: note: mappingproxy_dealloc defined here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:3315:13 in
2024-01-02 15:03:39 +01:00
Christopher Chavez 5d36a95e64
gh-111178: Avoid calling functions from incompatible pointer types in listobject.c (GH-112820)
Fix undefined behavior warnings (UBSan  -fsanitize=function), for example:

Objects/object.c:674:11: runtime error: call to function list_repr through pointer to incorrect function type 'struct _object *(*)(struct _object *)'
listobject.c:382: note: list_repr defined here
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/object.c:674:11 in
2024-01-02 14:41:32 +01:00
Sam Gross acf3bcc886
gh-112532: Use separate mimalloc heaps for GC objects (gh-113263)
* gh-112532: Use separate mimalloc heaps for GC objects

In `--disable-gil` builds, we now use four separate heaps in
anticipation of using mimalloc to find GC objects when the GIL is
disabled. To support this, we also make a few changes to mimalloc:

* `mi_heap_t` and `mi_tld_t` initialization is split from allocation.
  This allows us to have a `mi_tld_t` per-`PyThreadState`, which is
  important to keep interpreter isolation, since the same OS thread may
  run in multiple interpreters (using different PyThreadStates.)

* Heap abandoning (mi_heap_collect_ex) can now be called from a
  different thread than the one that created the heap. This is necessary
  because we may clear and delete the containing PyThreadStates from a
  different thread during finalization and after fork().

* Use enum instead of defines and guard mimalloc includes.

* The enum typedef will be convenient for future PRs that use the type.
* Guarding the mimalloc includes allows us to unconditionally include
  pycore_mimalloc.h from other header files that rely on things like
  `struct _mimalloc_thread_state`.

* Only define _mimalloc_thread_state in Py_GIL_DISABLED builds
2023-12-27 01:53:20 +09:00
Donghee Na 8f5b998706
gh-111971: Make _PyUnicode_FromId thread-safe in --disable-gil (gh-113489) 2023-12-26 16:48:33 +00:00
Yan Yanchii 4a3d2419bb
gh-113212: Improve error message & document zero-arg super inside nested functions and generator expressions (GH-113307) 2023-12-22 17:12:08 +02:00
Sam Gross 9afb0e1606
gh-112027: Don't print mimalloc warning after mmap() call (gh-113372)
gh-112027: Don't print mimalloc warning after mmap

This changes the warning to a "verbose"-level message in prim.c. The
address passed to mmap is only a hint -- it's normal for mmap() to
sometimes not respect the hint and return a different address.
2023-12-22 09:38:27 +09:00
Raymond Hettinger d058eaeed4
gh-113157 gh-89519: Fix method descriptors (gh-113233)
Restore behaviors before classmethod descriptor chaining was introduced.
2023-12-21 16:08:35 -06:00
Shantanu 61e8184095
gh-95754: Better AttributeError on partially initialised module (#112577)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-12-21 13:24:10 -08:00
Erlend E. Aasland 526d0a9b6e
gh-110383: Improve accuracy of str.split() and str.rsplit() docstrings (#113355)
Clarify split direction in the docstring body,
instead of in the 'maxsplit' param docstring.
2023-12-21 15:22:39 +01:00
Carey Metcalfe a2dd0e7038
gh-111375: Use `NULL` rather than `None` in the exception stack to indicate that an exception was handled (#113302) 2023-12-21 01:46:41 +00:00
Christopher Chavez a545a86ec6
gh-111178: Make slot functions in typeobject.c have compatible types (GH-112752) 2023-12-20 15:13:44 +01:00
Sam Gross fdee7b7b3e
gh-112532: Require mimalloc in `--disable-gil` builds (gh-112883) 2023-12-12 09:04:48 +09:00
Christopher Chavez f4fe65e2dd
gh-111178: Avoid calling functions from incompatible pointer types in memoryobject.c (GH-112863)
* Make memory_clear() compatible with inquiry
* Make memory_traverse() compatible with traverseproc
* Make memory_dealloc() compatible with destructor
* Make memory_repr() compatible with reprfunc
* Make memory_hash() compatible with hashfunc
* Make memoryiter_next() compatible with iternextfunc
* Make memoryiter_traverse() compatible with traverseproc
* Make memoryiter_dealloc() compatible with destructor
* Make several functions compatible with getter
* Make a few functions compatible with getter
* Make memory_item() compatible with ssizeargfunc
* Make memory_subscript() compatible with binaryfunc
* Make memory_length() compatible with lenfunc
* Make memory_ass_sub() compatible with objobjargproc
* Make memory_releasebuf() compatible with releasebufferproc
* Make memory_getbuf() compatible with getbufferproc
* Make mbuf_clear() compatible with inquiry
* Make mbuf_traverse() compatible with traverseproc
* Make mbuf_dealloc() compatible with destructor
2023-12-11 17:43:07 +01:00
Sam Gross cf6110ba13
gh-111924: Use PyMutex for Runtime-global Locks. (gh-112207)
This replaces some usages of PyThread_type_lock with PyMutex, which does not require memory allocation to initialize.

This simplifies some of the runtime initialization and is also one step towards avoiding changing the default raw memory allocator during initialize/finalization, which can be non-thread-safe in some circumstances.
2023-12-07 12:33:40 -07:00
andrewluotechnologies 9c3458e058
gh-112125: Fix None.__ne__(None) returning NotImplemented instead of False (#112504) 2023-12-07 13:56:01 +01:00
Serhiy Storchaka 8660fb7fd7
gh-112660: Do not clear arbitrary errors on import (GH-112661)
Previously arbitrary errors could be cleared during formatting error
messages for ImportError or AttributeError for modules. Now all
unexpected errors are reported.
2023-12-07 12:19:43 +02:00
Sergey B Kirpichev 9f92b31339
Minor refactoring of Object/abstract.c (UNARY_FUNC macro and more cases for BINARY_FUNC) (GH-112145)
* Use BINARY_FUNC macro for some remaining ops
* Add UNARY_FUNC macro to define unary PyNumber_* functions
2023-12-05 09:34:13 +02:00
chilaxan 0e732d0997
gh-112625: Protect bytearray from being freed by misbehaving iterator inside bytearray.join (GH-112626) 2023-12-04 08:15:43 +00:00
Irit Katriel bfb576ee23
gh-111058: Change coro.cr_frame/gen.gi_frame to be None for a closed coroutine/generator. (#112428) 2023-12-01 12:57:31 +00:00
Kirill Podoprigora 0785c68559
gh-111972: Make Unicode name C APIcapsule initialization thread-safe (#112249) 2023-11-30 11:12:49 +01:00
Eric Snow 790db85c77
gh-76785: Add _PyType_GetModuleName() to the Internal C-API (gh-112323)
The new function corresponds to the existing (public) PyType_GetName() and PyType_GetQualName().
2023-11-22 15:03:33 -07:00
Hugo van Kemenade 3b3ec0d77f
gh-111863: Rename `Py_NOGIL` to `Py_GIL_DISABLED` (#111864)
Rename Py_NOGIL to Py_GIL_DISABLED
2023-11-20 15:52:00 +02:00
Nikita Sobolev f8129146ef
gh-112266: Remove `(if defined)` part from `__dict__` and `__weakref__` docstrings (#112268) 2023-11-19 18:30:07 +00:00
Victor Stinner bd2f1485b0
gh-112026: Restore removed _PyDict_GetItemStringWithError() (#112119)
Restore the removed _PyDict_GetItemStringWithError() function. It is
used by numpy.
2023-11-15 17:10:06 +00:00
Miro Hrončok ec05507367
gh-96954: Fix `make regen-unicodedata` in out-of-tree builds (#112118)
This avoids:

    python3.13 Tools/unicode/makeunicodedata.py
    python3.13: can't open file '.../build/debug/Tools/unicode/makeunicodedata.py': [Errno 2] No such file or directory
    make: *** [Makefile:1498: regen-unicodedata] Error 2

Re-run `make regen-unicodedata` to update the script path in generated files.
2023-11-15 16:42:17 +00:00
Victor Stinner 58469244be
gh-112026: Restore removed private C API (#112115)
Restore removed private C API functions, macros and structures which
have no simple replacement for now:

* _PyDict_GetItem_KnownHash()
* _PyDict_NewPresized()
* _PyHASH_BITS
* _PyHASH_IMAG
* _PyHASH_INF
* _PyHASH_MODULUS
* _PyHASH_MULTIPLIER
* _PyLong_Copy()
* _PyLong_FromDigits()
* _PyLong_New()
* _PyLong_Sign()
* _PyObject_CallMethodId()
* _PyObject_CallMethodNoArgs()
* _PyObject_CallMethodOneArg()
* _PyObject_CallOneArg()
* _PyObject_EXTRA_INIT
* _PyObject_FastCallDict()
* _PyObject_GetAttrId()
* _PyObject_Vectorcall()
* _PyObject_VectorcallMethod()
* _PyStack_AsDict()
* _PyThread_CurrentFrames()
* _PyUnicodeWriter structure
* _PyUnicodeWriter_Dealloc()
* _PyUnicodeWriter_Finish()
* _PyUnicodeWriter_Init()
* _PyUnicodeWriter_Prepare()
* _PyUnicodeWriter_PrepareKind()
* _PyUnicodeWriter_WriteASCIIString()
* _PyUnicodeWriter_WriteChar()
* _PyUnicodeWriter_WriteLatin1String()
* _PyUnicodeWriter_WriteStr()
* _PyUnicodeWriter_WriteSubstring()
* _PyUnicode_AsString()
* _PyUnicode_FromId()
* _PyVectorcall_Function()
* _Py_HashDouble()
* _Py_HashPointer()
* _Py_IDENTIFIER()
* _Py_c_abs()
* _Py_c_diff()
* _Py_c_neg()
* _Py_c_pow()
* _Py_c_prod()
* _Py_c_quot()
* _Py_c_sum()
* _Py_static_string()
* _Py_static_string_init()
2023-11-15 16:38:31 +00:00
Furkan Onder 0ff6368519
gh-111906: Fix warnings during mimalloc build on FreeBSD (#111907)
Fix `unused function` warnings during mimalloc build on FreeBSD.
2023-11-14 17:07:46 +01:00
Victor Stinner 4f04172c92
gh-111262: Add PyDict_Pop() function (#112028)
_PyDict_Pop_KnownHash(): remove the default value and the return type
becomes an int.

Co-authored-by: Stefan Behnel <stefan_ml@behnel.de>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
2023-11-14 12:51:00 +00:00
Serhiy Storchaka 18203a6bc9
gh-111789: Use PyDict_GetItemRef() in Objects/ (GH-111827) 2023-11-14 11:25:39 +02:00
Victor Stinner babb787047
gh-111138: Add PyList_Extend() and PyList_Clear() functions (#111862)
* Split list_extend() into two sub-functions: list_extend_fast() and
  list_extend_iter().
* list_inplace_concat() no longer has to call Py_DECREF() on the
  list_extend() result, since list_extend() now returns an int.
2023-11-13 16:14:56 +00:00
Serhiy Storchaka 1d75ef6b61
gh-111999: Add signatures and improve docstrings for builtins (GH-112000) 2023-11-13 09:13:49 +02:00
Sam Gross fa84e5fe0a
gh-110481: fix 'unused function' warning for `is_shared_refcnt_dead`. (gh-111974)
Fix 'unused function' warning for `is_shared_refcnt_dead`.

The `is_shared_refcnt_dead` function is only used if `Py_REF_DEBUG` is set.
2023-11-11 08:54:35 +09:00
Serhiy Storchaka 771bd3c94a
Add private _PyUnicode_AsUTF8NoNUL() function (GH-111957)
Like PyUnicode_AsUTF8(), but check for embedded null characters.
2023-11-10 21:31:36 +02:00
Pavel Ovchinnikov 0c61d028be
Improve error message for "float modulo by zero" (#111685) 2023-11-09 11:43:58 -08:00
Irit Katriel 30ec968bef
gh-111354: remove comparisons with enum values, variable reuse, unused imports in genobject.c (#111708) 2023-11-09 10:27:20 +00:00
Sam Gross 31c90d5838
gh-111569: Implement Python critical section API (gh-111571)
Critical sections are helpers to replace the global interpreter lock
with finer grained locking.  They provide similar guarantees to the GIL
and avoid the deadlock risk that plain locking involves.  Critical
sections are implicitly ended whenever the GIL would be released.  They
are resumed when the GIL would be acquired.  Nested critical sections
behave as if the sections were interleaved.
2023-11-08 15:39:29 -07:00
Jelle Zijlstra 0b718e6407
gh-110543: Fix CodeType.replace in presence of comprehensions (#110586) 2023-11-08 13:11:59 -07:00
Victor Stinner 11e83488c5
gh-111089: Revert PyUnicode_AsUTF8() changes (#111833)
* Revert "gh-111089: Use PyUnicode_AsUTF8() in Argument Clinic (#111585)"

This reverts commit d9b606b3d0.

* Revert "gh-111089: Use PyUnicode_AsUTF8() in getargs.c (#111620)"

This reverts commit cde1071b2a.

* Revert "gh-111089: PyUnicode_AsUTF8() now raises on embedded NUL (#111091)"

This reverts commit d731579bfb.

* Revert "gh-111089: Add PyUnicode_AsUTF8() to the limited C API (#111121)"

This reverts commit d8f32be5b6.

* Revert "gh-111089: Use PyUnicode_AsUTF8() in sqlite3 (#111122)"

This reverts commit 37e4e20eaa.
2023-11-07 22:36:13 +00:00
Irit Katriel 2f9cb7e095
gh-81137: deprecate assignment of code object to a function of a mismatched type (#111823) 2023-11-07 18:54:36 +00:00
Serhiy Storchaka f55cb44359
gh-106672: C API: Report indiscriminately ignored errors (GH-106674)
Functions which indiscriminately ignore all errors now report them as
unraisable errors.
2023-11-07 15:58:04 +02:00
Irit Katriel 13405ecffd
gh-79932: raise exception if frame.clear() is called on a suspended frame (#111792) 2023-11-07 08:49:30 +00:00
Irit Katriel c5063fd62a
simplify code to pop exception in frame_setlineno (#111702) 2023-11-06 15:45:33 +00:00
Nikita Sobolev a28a3967ab
gh-111666: Speed up `BaseExceptionGroup.{derive,split,subgroup}` (#111667) 2023-11-04 09:59:46 +00:00
Victor Stinner 20cfab903d
gh-111506: Implement Py_SET_REFCNT() as opaque function in limited C API (#111508)
In the limited C API version 3.13, Py_SET_REFCNT() function is now
implemented as an opaque function call.

Add _Py_SetRefcnt() to the stable ABI.
2023-11-03 18:18:57 +01:00
Tian Gao e0afed7e27
gh-103615: Use local events for opcode tracing (GH-109472)
* Use local monitoring for opcode trace

* Remove f_opcode_trace_set

* Add test for setting f_trace_opcodes after settrace
2023-11-03 16:39:50 +00:00
scoder 24ddaee5ca
gh-106168: Revert the "size before item" setting (#111683)
gh-106168: Update the size only after setting the item, to avoid temporary inconsistencies.
Also remove the "what's new" sentence regarding the size setting since tuples cannot grow after allocation.
2023-11-03 11:02:39 +00:00
Irit Katriel d49aba5a7a
gh-111354: Simplify _PyGen_yf by moving some of its work to the compiler and frame state (#111648) 2023-11-03 10:01:36 +00:00
Mark Shannon 0887b9ce8b
GH-109369 Add vectorcall to `PyLong_Type` (GH-111642) 2023-11-02 15:07:47 +00:00