Commit Graph

8517 Commits

Author SHA1 Message Date
Inada Naoki 322b486010
gh-126024: optimize UTF-8 decoder for short non-ASCII string (#126025) 2024-11-29 19:48:02 +09:00
Sergey B Kirpichev 987311d42e
gh-69639: Add mixed-mode rules for complex arithmetic (C-like) (GH-124829)
"Generally, mixed-mode arithmetic combining real and complex variables should
be performed directly, not by first coercing the real to complex, lest the sign
of zero be rendered uninformative; the same goes for combinations of pure
imaginary quantities with complex variables." (c) Kahan, W: Branch cuts for
complex elementary functions.

This patch implements mixed-mode arithmetic rules, combining real and
complex variables as specified by C standards since C99 (in particular,
there is no special version for the true division with real lhs
operand).  Most C compilers implementing C99+ Annex G have only these
special rules (without support for imaginary type, which is going to be
deprecated in C2y).
2024-11-26 17:57:39 +02:00
Jelle Zijlstra dcf629213b
gh-119180: Add VALUE_WITH_FAKE_GLOBALS format to annotationlib (#124415) 2024-11-26 15:40:13 +00:00
Sergey B Kirpichev f7bb658124
gh-113841: fix possible undefined division by 0 in _Py_c_pow() (GH-127211)
`x**y == 1/x**-y ` thus changing `/=` to `*=` by negating the exponent.
2024-11-23 23:37:37 -08:00
Irit Katriel 4b12a6ff4a
gh-119786: add code object doc, inline locations.md into it (#126832) 2024-11-22 19:27:41 +00:00
Jacob Bower e8bb053941
gh-126091: Always link generator frames when propagating a thrown-in exception through a yield-from chain (#126092)
Always link generator frames when propagating a thrown-in exception through a yield-from chain.
2024-11-21 17:37:49 -06:00
Donghee Na 78a530a578
gh-115999: Add free-threaded specialization for ``TO_BOOL`` (gh-126616) 2024-11-22 07:52:16 +09:00
mpage 09c240f20c
gh-115999: Specialize `LOAD_GLOBAL` in free-threaded builds (#126607)
Enable specialization of LOAD_GLOBAL in free-threaded builds.

Thread-safety of specialization in free-threaded builds is provided by the following:

A critical section is held on both the globals and builtins objects during specialization. This ensures we get an atomic view of both builtins and globals during specialization.
Generation of new keys versions is made atomic in free-threaded builds.
Existing helpers are used to atomically modify the opcode.
Thread-safety of specialized instructions in free-threaded builds is provided by the following:

Relaxed atomics are used when loading and storing dict keys versions. This avoids potential data races as the dict keys versions are read without holding the dictionary's per-object lock in version guards.
Dicts keys objects are passed from keys version guards to the downstream uops. This ensures that we are loading from the correct offset in the keys object. Once a unicode key has been stored in a keys object for a combined dictionary in free-threaded builds, the offset that it is stored in will never be reused for a different key. Once the version guard passes, we know that we are reading from the correct offset.
The dictionary read fast-path is used to read values from the dictionary once we know the correct offset.
2024-11-21 11:22:21 -08:00
Eric Snow 9dabace39d
gh-114940: Add _Py_FOR_EACH_TSTATE_UNLOCKED(), and Friends (gh-127077)
This is a precursor to the actual fix for gh-114940, where we will change these macros to use the new lock.  This change is almost entirely mechanical; the exceptions are the loops in codeobject.c and ceval.c, which now hold the "head" lock.  Note that almost all of the uses of _Py_FOR_EACH_TSTATE_UNLOCKED() here will change to _Py_FOR_EACH_TSTATE_BEGIN() once we add the new per-interpreter lock.
2024-11-21 11:08:38 -07:00
Dino Viehland bf542f8bb9
gh-124470: Fix crash when reading from object instance dictionary while replacing it (#122489)
Delay free a dictionary when replacing it
2024-11-21 10:41:19 -06:00
Sam Gross 3926842117
gh-127020: Make `PyCode_GetCode` thread-safe for free threading (#127043)
Some fields in PyCodeObject are lazily initialized. Use atomics and
critical sections to make their initializations and accesses thread-safe.
2024-11-21 11:00:50 -05:00
Mark Shannon aea0c586d1
GH-127010: Don't lazily track and untrack dicts (GH-127027) 2024-11-20 16:41:20 +00:00
sobolevn 3932e1db53
gh-126980: Fix `bytearray.__buffer__` crash on `PyBUF_{READ,WRITE}` (#126981)
Co-authored-by: Victor Stinner <vstinner@python.org>
2024-11-19 17:44:53 +03:00
Pablo Galindo Salgado 30aeb00d36
gh-126076: Account for relocated objects in tracemalloc (#126077) 2024-11-19 10:35:17 +00:00
Hugo van Kemenade 899fdb213d
Revert "GH-126491: GC: Mark objects reachable from roots before doing cycle collection (GH-126502)" (#126983) 2024-11-19 11:25:09 +02:00
Victor Stinner 84f07c3a4c
gh-126594: Fix typeobject.c wrap_buffer() cast (#126754)
Reject flags smaller than INT_MIN.

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-11-19 09:13:20 +01:00
Mark Shannon b0fcc2c47a
GH-126491: GC: Mark objects reachable from roots before doing cycle collection (GH-126502)
* Mark almost all reachable objects before doing collection phase

* Add stats for objects marked

* Visit new frames before each increment

* Remove lazy dict tracking

* Update docs

* Clearer calculation of work to do.
2024-11-18 14:31:26 +00:00
neonene 8c9c6d3c12
gh-123465: Ensure PyType_FromMetaclass avoids extra strcmp (GH-125460)
use else
2024-11-13 17:09:26 +00:00
Peter Bierma d00878b06a
gh-123619: Add an unstable C API function for enabling deferred reference counting (GH-123635)
Co-authored-by: Sam Gross <colesbury@gmail.com>
2024-11-13 13:27:16 +00:00
Ritvik Pasham a12690ef49
gh-126341: add release check to `__iter__` method of `memoryview` (#126759)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: sobolevn <mail@sobolevn.me>
2024-11-13 11:51:01 +00:00
RUANG (James Roy) 8ff7efb46d
gh-126061: Add PyLong_IsPositive/Zero/Negative() functions (#126065)
Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-11-12 14:18:06 +01:00
Mark Shannon fa40922597
GH-126547: Pre-assign version numbers for a few common classes (GH-126551) 2024-11-08 16:44:44 +00:00
Serhiy Storchaka 061e50f196
gh-122943: Add the varpos parameter in _PyArg_UnpackKeywords (GH-126564)
Remove _PyArg_UnpackKeywordsWithVararg.
Add comments for integer arguments of _PyArg_UnpackKeywords.
2024-11-08 14:23:50 +02:00
Serhiy Storchaka 06a8b0bb5e
gh-122943: Remove the object converter for var-positional parameter (GH-126560) 2024-11-08 05:41:54 +00:00
Serhiy Storchaka 1f777396f5
gh-122943: Rework support of var-positional parameter in Argument Clinic (GH-122945)
Move creation of a tuple for var-positional parameter out of
_PyArg_UnpackKeywordsWithVararg().
Merge _PyArg_UnpackKeywordsWithVararg() with _PyArg_UnpackKeywords().
Add a new parameter in _PyArg_UnpackKeywords().

The "parameters" and "converters" attributes of ParseArgsCodeGen no
longer contain the var-positional parameter. It is now available as the
"varpos" attribute. Optimize code generation for var-positional
parameter and reuse the same generating code for functions with and without
keyword parameters.

Add special converters for var-positional parameter. "tuple" represents it as
a Python tuple and "array" represents it as a continuous array of PyObject*.
"object" is a temporary alias of "tuple".
2024-11-07 23:40:03 +02:00
Michael Droettboom a38e82bd8c
gh-126298: Don't deduplicate slice constants based on equality (#126398)
* gh-126298: Don't deduplicated slice constants based on equality

* NULL check for PySlice_New

* Fix refcounting

* Fix refcounting some more

* Fix refcounting

* Make tests more complete

* Fix tests
2024-11-07 16:39:23 +00:00
Serhiy Storchaka d3840503b0
gh-126303: Fix pickling and copying of os.sched_param objects (GH-126336) 2024-11-05 08:23:17 +02:00
mpage 2e95c5ba3b
gh-115999: Implement thread-local bytecode and enable specialization for `BINARY_OP` (#123926)
Each thread specializes a thread-local copy of the bytecode, created on the first RESUME, in free-threaded builds. All copies of the bytecode for a code object are stored in the co_tlbc array on the code object. Threads reserve a globally unique index identifying its copy of the bytecode in all co_tlbc arrays at thread creation and release the index at thread destruction. The first entry in every co_tlbc array always points to the "main" copy of the bytecode that is stored at the end of the code object. This ensures that no bytecode is copied for programs that do not use threads.

Thread-local bytecode can be disabled at runtime by providing either -X tlbc=0 or PYTHON_TLBC=0. Disabling thread-local bytecode also disables specialization.

Concurrent modifications to the bytecode made by the specializing interpreter and instrumentation use atomics, with specialization taking care not to overwrite an instruction that was instrumented concurrently.
2024-11-04 11:13:32 -08:00
Sergey B Kirpichev 8477951a1c
gh-120026: soft deprecate Py_HUGE_VAL macro (#120027)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2024-11-01 22:04:31 +00:00
Sergey B Kirpichev 8c22eba877
gh-90370: Argument Clinic: avoid temporary tuple creation for varargs (#126064)
Avoid temporary tuple creation when all arguments either positional-only
or vararg.

Objects/setobject.c and Modules/gcmodule.c adapted. This fixes slight
performance regression for set methods, introduced by gh-115112.
2024-10-31 11:37:03 +01:00
Xuanteng Huang 35df4eb959
gh-126072: do not add `None` to `co_consts` if there is no docstring (GH-126101) 2024-10-30 09:01:09 +00:00
Prometheus3375 5527c4051c
gh-116938: Fix `dict.update` docstring and remove erraneous full stop from `dict` documentation (#125421)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2024-10-29 23:00:04 +00:00
Mark Shannon faa3272fb8
GH-125837: Split `LOAD_CONST` into three. (GH-125972)
* Add LOAD_CONST_IMMORTAL opcode

* Add LOAD_SMALL_INT opcode

* Remove RETURN_CONST opcode
2024-10-29 11:15:42 +00:00
Brian Schubert dc76a4ad3c
gh-126012: Add `__class_getitem__` to `memoryview` (#126013)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
2024-10-27 13:36:03 +03:00
Victor Stinner db96327203
gh-121654: Add PyType_Freeze() function (#122457)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-10-25 11:12:48 +02:00
Sam Gross 332356b880
gh-125900: Clean-up logic around immortalization in free-threading (#125901)
* Remove `@suppress_immortalization` decorator
* Make suppression flag per-thread instead of per-interpreter
* Suppress immortalization in `eval()` to avoid refleaks in three tests
  (test_datetime.test_roundtrip, test_logging.test_config8_ok, and
   test_random.test_after_fork).
* frozenset() is constant, but not a singleton. When run multiple times,
  the test could fail due to constant interning.
2024-10-24 18:09:59 -04:00
Shantanu 500f5338a8
gh-123930: Better error for "from imports" when script shadows module (#123929) 2024-10-24 12:11:12 -07:00
Sam Gross 3c4a7fa617
gh-124218: Avoid refcount contention on builtins module (GH-125847)
This replaces `_PyEval_BuiltinsFromGlobals` with
`_PyDict_LoadBuiltinsFromGlobals`, which returns a new reference
instead of a borrowed reference. Internally, the new function uses
per-thread reference counting when possible to avoid contention on the
refcount fields on the builtins module.
2024-10-24 12:44:38 -04:00
Eric Snow 6f26d496d3
gh-125286: Share the Main Refchain With Legacy Interpreters (gh-125709)
They used to be shared, before 3.12.  Returning to sharing them resolves a failure on Py_TRACE_REFS builds.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-10-23 10:10:06 -06:00
Arjun Singh 03f9264ece
fix grammar in comment in dictobject.c (#125822) 2024-10-22 11:48:16 +09:00
Sam Gross 9b0bfba2a2
gh-124218: Use per-thread reference counting for globals and builtins (#125713)
Use per-thread refcounting for the reference from function objects to
the globals and builtins dictionaries.
2024-10-21 12:51:29 -04:00
Tian Gao 5b7a872b26
gh-125590: Allow FrameLocalsProxy to delete and pop keys from extra locals (#125616) 2024-10-21 11:43:08 -04:00
Sam Gross 5989eb7446
gh-125608: Trigger dictionary watchers when inline values change (#125611)
Dictionary watchers on an object's attributes dictionary
(`object.__dict__`) were not triggered when the managed dictionary used
the object's inline values.
2024-10-21 08:23:38 -04:00
Eric Snow 6d93690954
gh-125604: Move _Py_AuditHookEntry, etc. Out of pycore_runtime.h (gh-125605)
This is essentially a cleanup, moving a handful of API declarations to the header files where they fit best, creating new ones when needed.

We do the following:

* add pycore_debug_offsets.h and move _Py_DebugOffsets, etc. there
* inline struct _getargs_runtime_state and struct _gilstate_runtime_state in _PyRuntimeState
* move struct _reftracer_runtime_state to the existing pycore_object_state.h
* add pycore_audit.h and move to it _Py_AuditHookEntry , _PySys_Audit(), and _PySys_ClearAuditHooks
* add audit.h and cpython/audit.h and move the existing audit-related API there
*move the perfmap/trampoline API from cpython/sysmodule.h to cpython/ceval.h, and remove the now-empty cpython/sysmodule.h
2024-10-18 09:26:08 -06:00
Zachary Ware c3164ae3cf
gh-125017: Fix refleak from GH-125636 (GH-125664) 2024-10-17 17:21:32 -05:00
Jelle Zijlstra f203d1cb52
gh-125017: Fix crash on premature access to classmethod/staticmethod annotations (#125636) 2024-10-17 09:45:25 -07:00
Sam Gross 3ea488aac4
gh-124218: Use per-thread refcounts for code objects (#125216)
Use per-thread refcounting for the reference from function objects to
their corresponding code object. This can be a source of contention when
frequently creating nested functions. Deferred refcounting alone isn't a
great fit here because these references are on the heap and may be
modified by other libraries.
2024-10-15 15:06:41 -04:00
Mark Shannon 06ca33020e
GH-125323: Convert DECREF_INPUTS_AND_REUSE_FLOAT into a function that takes PyStackRefs. (GH-125439) 2024-10-14 14:18:57 +01:00
sobolevn 2115d76acc
gh-124787: Fix `TypeAliasType` and incorrect `type_params` (#124795)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2024-10-11 17:39:18 +03:00
Sam Gross b12e99261e
gh-125221: Fix free-threading data race in `object.__reduce_ex__` (#125267) 2024-10-11 13:26:01 +05:30