Commit Graph

8979 Commits

Author SHA1 Message Date
Victor Stinner 75eed5b373
gh-117929: Restore removed PyEval_InitThreads() function (#117931) 2024-04-17 15:01:28 +02:00
Mark Shannon 147cd0581e
GH-117760: Streamline the trashcan mechanism (GH-117763) 2024-04-17 11:08:05 +01:00
Jeff Glass acf69e09c6
gh-115178: Add Counts of UOp Pairs to pystats (GH-115181) 2024-04-16 14:27:18 +01:00
mpage 0d5238373d
gh-117657: Quiet more TSAN warnings due to incorrect modeling of compare/exchange (#117830) 2024-04-15 12:17:55 -04:00
mpage 6e0b327690
gh-117657: Quiet TSAN warning about a data race between `start_the_world()` and `tstate_try_attach()` (#117828)
TSAN erroneously reports a data race between the `_Py_atomic_compare_exchange_int`
on `tstate->state` in `tstate_try_attach()` and the non-atomic load of
`tstate->state` in `start_the_world`. The `_Py_atomic_compare_exchange_int` fails,
but TSAN erroneously treats it as a store.
2024-04-15 12:17:33 -04:00
Sam Gross 4ad8f090cc
gh-117376: Partial implementation of deferred reference counting (#117696)
This marks objects as using deferred refrence counting using the
`ob_gc_bits` field in the free-threaded build and collects those objects
during GC.
2024-04-12 17:36:20 +00:00
Serhiy Storchaka 3a8c1ca7e7
gh-117764: Fix and add signatures for many builtins (GH-117769) 2024-04-12 13:56:41 +03:00
Serhiy Storchaka 2638c28be0
gh-117764: Add signatures for some functions in the sys module (GH-117770)
Use Argument Clinic if possible.
2024-04-12 09:12:48 +00:00
Eric Snow fd259fdabe
gh-76785: Handle Legacy Interpreters Properly (gh-117490)
This is similar to the situation with threading._DummyThread.  The methods (incl. __del__()) of interpreters.Interpreter objects must be careful with interpreters not created by interpreters.create().  The simplest thing to start with is to disable any method that modifies or runs in the interpreter.  As part of this, the runtime keeps track of where an interpreter was created.  We also handle interpreter "refcounts" properly.
2024-04-11 23:23:25 +00:00
Sam Gross 25f6ff5d3e
gh-117649: Raise ImportError for unsupported modules in free-threaded build (#117651)
The free-threaded build does not currently support the combination of
single-phase init modules and non-isolated subinterpreters. Ensure that
`check_multi_interp_extensions` is always `True` for subinterpreters in
the free-threaded build so that importing these modules raises an
`ImportError`.
2024-04-11 15:00:54 -04:00
Eric Snow 993c3cca16
gh-76785: Add More Tests to test_interpreters.test_api (gh-117662)
In addition to the increase test coverage, this is a precursor to sorting out how we handle interpreters created directly via the C-API.
2024-04-10 18:37:01 -06:00
Tian Gao 57183241af
gh-107674: Remove some unnecessary code in instrumentation code (GH-117393) 2024-04-09 09:54:28 +01:00
Sam Gross 1a6594f661
gh-117439: Make refleak checking thread-safe without the GIL (#117469)
This keeps track of the per-thread total reference count operations in
PyThreadState in the free-threaded builds. The count is merged into the
interpreter's total when the thread exits.
2024-04-08 12:11:36 -04:00
mpage df73179048
gh-111926: Make weakrefs thread-safe in free-threaded builds (#117168)
Most mutable data is protected by a striped lock that is keyed on the
referenced object's address. The weakref's hash is protected using the
weakref's per-object lock.
 
Note that this only affects free-threaded builds. Apart from some minor
refactoring, the added code is all either gated by `ifdef`s or is a no-op
(e.g. `Py_BEGIN_CRITICAL_SECTION`).
2024-04-08 10:58:38 -04:00
Brandt Bucher 62aeb0ee69
GH-117512: Allow 64-bit JIT operands on 32-bit platforms (GH-117527) 2024-04-06 08:26:43 -07:00
Erlend E. Aasland 757b62493b
gh-117457: Regen executor cases post PR #117477 (#117559) 2024-04-05 10:13:00 +00:00
Michael Droettboom 0edde64a41
GH-117457: Correct pystats uop "miss" counts (GH-117477) 2024-04-04 15:49:18 -07:00
Dino Viehland 434bc593df
gh-112075: Make _PyDict_LoadGlobal thread safe (#117529)
Make _PyDict_LoadGlobal threadsafe
2024-04-04 12:26:07 -07:00
Irit Katriel 04697bcfaf
gh-117494: extract the Instruction Sequence data structure into a separate file (#117496) 2024-04-04 15:47:26 +00:00
Guido van Rossum 060a96f1a9
gh-116968: Reimplement Tier 2 counters (#117144)
Introduce a unified 16-bit backoff counter type (``_Py_BackoffCounter``),
shared between the Tier 1 adaptive specializer and the Tier 2 optimizer. The
API used for adaptive specialization counters is changed but the behavior is
(supposed to be) identical.

The behavior of the Tier 2 counters is changed:
- There are no longer dynamic thresholds (we never varied these).
- All counters now use the same exponential backoff.
- The counter for ``JUMP_BACKWARD`` starts counting down from 16.
- The ``temperature`` in side exits starts counting down from 64.
2024-04-04 15:03:27 +00:00
Peter Lazorchak 1c43468886
gh-116168: Remove extra `_CHECK_STACK_SPACE` uops (#117242)
This merges all `_CHECK_STACK_SPACE` uops in a trace into a single `_CHECK_STACK_SPACE_OPERAND` uop that checks whether there is enough stack space for all calls included in the entire trace.
2024-04-03 17:14:18 +00:00
Eric Snow 976bcb2379
gh-76785: Raise InterpreterError, Not RuntimeError (gh-117489)
I had meant to switch everything to InterpreterError when I added it a while back.  At the time I missed a few key spots.

As part of this, I've added print-the-exception to _PyXI_InitTypes() and fixed an error case in `_PyStaticType_InitBuiltin().
2024-04-03 10:58:39 -06:00
Eric Snow 65524ab388
gh-76785: Fix a Refleak in _interpreters.new_config() (gh-117491)
This is a follow-up to gh-117170 and gh-117485.
2024-04-03 01:10:26 +00:00
Eric Snow f341d6017d
gh-76785: Add PyInterpreterConfig Helpers (gh-117170)
These helpers make it easier to customize and inspect the config used to initialize interpreters.  This is especially valuable in our tests.  I found inspiration from the PyConfig API for the PyInterpreterConfig dict conversion stuff.  As part of this PR I've also added a bunch of tests.
2024-04-02 20:35:52 +00:00
Guido van Rossum 8eda146e87
Fix successor opcode name printing in Tier 2 DEOPT debug message (#117471) 2024-04-02 18:25:48 +00:00
Mark Shannon c32dc47aca
GH-115776: Embed the values array into the object, for "normal" Python objects. (GH-116115) 2024-04-02 11:59:21 +01:00
Irit Katriel 1d5479b236
gh-117411: move PyFutureFeatures to pycore_symtable.h and make it private (#117412) 2024-04-02 10:34:49 +00:00
Guido van Rossum ddf814db74
Silence compiler warnings in gc.c (#117422) 2024-04-01 16:13:38 +00:00
Steve (Gadget) Barnes 3de09cadde
gh-91565: Replace bugs.python.org links with Devguide/GitHub ones (GH-91568)
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
2024-04-01 13:02:07 +00:00
Sam Gross bfc57d43d8
gh-117303: Don't detach in `PyThreadState_DeleteCurrent()` (#117304)
This fixes a crash in `test_threading.test_reinit_tls_after_fork()` when
running with the GIL disabled. We already properly handle the case where
the thread state is `_Py_THREAD_ATTACHED` in `tstate_delete_common()` --
we just need to remove an assertion.

Keeping the thread attached means that a stop-the-world pause, such as
for a `fork()`, won't commence until we remove our thread state from the
interpreter's linked list. This prevents a crash when the child process
tries to clean up the dead thread states.
2024-03-29 18:58:08 -04:00
Erlend E. Aasland 05e0b67a43
gh-116664: In _warnings.c, make filters_version access thread-safe (#117374)
- assert that the lock is held in already_warned()
- protect 'filters_version' increment in warnings_filters_mutated_impl()
2024-03-29 21:23:28 +01:00
Sam Gross 01bd74eadb
gh-117300: Use stop the world to make `sys._current_frames` and `sys._current_exceptions` thread-safe. (#117301)
This adds a stop the world pause to make the two functions thread-safe
when the GIL is disabled in the free-threaded build.

Additionally, the main test thread may call `sys._current_exceptions()` as
soon as `g_raised.set()` is called. The background thread may not yet reach
the `leave_g.wait()` line.
2024-03-29 15:33:06 -04:00
Sam Gross 19c1dd60c5
gh-117323: Make `cell` thread-safe in free-threaded builds (#117330)
Use critical sections to lock around accesses to cell contents. The critical sections are no-ops in the default (with GIL) build.
2024-03-29 13:35:43 -04:00
Sam Gross f05fb2e65c
gh-112529: Don't untrack tuples or dicts with zero refcount (#117370)
The free-threaded GC sometimes sees objects with zero refcount. This can
happen due to the delay in merging biased reference counting fields,
and, in the future, due to deferred reference counting. We should not
untrack these objects or they will never be collected.

This fixes the refleaks in the free-threaded build.
2024-03-29 13:33:04 -04:00
Erlend E. Aasland ddf95b5f16
gh-116664: Fix unused var warnings in _warnings.c in non-free-threaded builds (#117373)
The warnings were introduced by commit c1712ef06.
2024-03-29 17:26:06 +00:00
Michael Droettboom 26d328b2ba
GH-117121: Add pystats to JIT builds (GH-117346) 2024-03-28 15:23:08 -07:00
Erlend E. Aasland c1712ef066
gh-116664: Make module state Py_SETREF's in _warnings thread-safe (#116959)
Mark the swap operations as critical sections.

Add an internal Py_BEGIN_CRITICAL_SECTION_MUT API that takes a PyMutex
pointer instead of a PyObject pointer.
2024-03-28 15:05:08 +00:00
Sam Gross 8dbfdb2957
gh-110481: Fix biased reference counting queue initialization. (#117271)
The biased reference counting queue must be initialized from the bound
(active) thread because it uses `_Py_ThreadId()` as the key in a hash
table.
2024-03-28 09:28:39 -04:00
Jelle Zijlstra 4c71d51a4b
gh-117266: Fix crashes on user-created AST subclasses (GH-117276)
Fix crashes on user-created AST subclasses
2024-03-28 11:30:31 +01:00
Russell Keith-Magee f006338017
gh-114099: Additions to standard library to support iOS (GH-117052)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Malcolm Smith <smith@chaquo.com>
Co-authored-by: Ned Deily <nad@python.org>
2024-03-28 03:59:33 -04:00
Irit Katriel 262fb911ab
gh-117288: Allocate fewer label IDs in _PyCfg_ToInstructionSequence (#117290) 2024-03-27 17:38:19 +00:00
Irit Katriel 79be75735c
gh-115775: Compiler adds __static_attributes__ field to classes (#115913) 2024-03-26 15:18:17 +00:00
Mark Shannon 8bef34f625
GH-117108: Set the "old space bit" to "visited" for all young objects (#117213)
Change old space bit of young objects from 0 to gcstate->visited_space.
This ensures that any object created *and* collected during cycle GC has the bit set correctly.
2024-03-26 11:11:42 +00:00
Mark Shannon bf82f77957
GH-116422: Tier2 hot/cold splitting (GH-116813)
Splits the "cold" path, deopts and exits, from the "hot" path, reducing the size of most jitted instructions, at the cost of slower exits.
2024-03-26 09:35:11 +00:00
Mark Shannon 23e4f80ce2
A few minor tweaks to get stats working and compiling cleanly. (#117219)
Fixes a compilation error when configured with `--enable-pystats`,
an array size issue, and an unused variable.
2024-03-25 13:43:51 -07:00
Kirill Podoprigora eebea7e515
gh-117176: Fix compiler warning in Python/optimizer_bytecodes.c (GH-117199) 2024-03-24 20:34:55 +02:00
Ken Jin 6c83352bfe
gh-117180: Complete call sequence when trace stack overflow (GH-117184)
---------

Co-authored-by: Peter Lazorchak <lazorchakp@gmail.com>
Co-authored-by: Guido van Rossum <gvanrossum@users.noreply.github.com>
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
2024-03-24 06:19:17 +08:00
Mark Shannon e28477f214
GH-117108: Change the size of the GC increment to about 1% of the total heap size. (GH-117120) 2024-03-22 18:43:25 +00:00
Serhiy Storchaka e2e0b4b4b9
gh-113024: C API: Add PyObject_GenericHash() function (GH-113025) 2024-03-22 20:19:10 +02:00
Eric Snow b3d25df8d3
gh-105716: Fix _PyInterpreterState_IsRunningMain() For Embedders (gh-117140)
When I added _PyInterpreterState_IsRunningMain() and friends last year, I tried to accommodate applications that embed Python but don't call _PyInterpreterState_SetRunningMain() (not that they're expected to).  That mostly worked fine until my recent changes in gh-117049, where the subtleties with the fallback code led to failures; the change ended up breaking test_tools.test_freeze, which exercises a basic embedding situation.

The simplest fix is to drop the fallback code I originally added to _PyInterpreterState_IsRunningMain() (and later to _PyThreadState_IsRunningMain()).  I've kept the fallback in the _xxsubinterpreters module though.  I've also updated Py_FrozenMain() to call _PyInterpreterState_SetRunningMain().
2024-03-21 18:20:20 -06:00
Guido van Rossum 570a82d46a
gh-117045: Add code object to function version cache (#117028)
Changes to the function version cache:

- In addition to the function object, also store the code object,
  and allow the latter to be retrieved even if the function has been evicted.
- Stop assigning new function versions after a critical attribute (e.g. `__code__`)
  has been modified; the version is permanently reset to zero in this case.
- Changes to `__annotations__` are no longer considered critical. (This fixes gh-109998.)

Changes to the Tier 2 optimization machinery:

- If we cannot map a function version to a function, but it is still mapped to a code object,
  we continue projecting the trace.
  The operand of the `_PUSH_FRAME` and `_POP_FRAME` opcodes can be either NULL,
  a function object, or a code object with the lowest bit set.

This allows us to trace through code that calls an ephemeral function,
i.e., a function that may not be alive when we are constructing the executor,
e.g. a generator expression or certain nested functions.
We will lose globals removal inside such functions,
but we can still do other peephole operations
(and even possibly [call inlining](https://github.com/python/cpython/pull/116290),
if we decide to do it), which only need the code object.
As before, if we cannot retrieve the code object from the cache, we stop projecting.
2024-03-21 12:37:41 -07:00
Sam Gross 1f72fb5447
gh-116522: Refactor `_PyThreadState_DeleteExcept` (#117131)
Split `_PyThreadState_DeleteExcept` into two functions:

- `_PyThreadState_RemoveExcept` removes all thread states other than one
  passed as an argument. It returns the removed thread states as a
  linked list.

- `_PyThreadState_DeleteList` deletes those dead thread states. It may
  call destructors, so we want to "start the world" before calling
  `_PyThreadState_DeleteList` to avoid potential deadlocks.
2024-03-21 11:21:02 -07:00
Michael Droettboom 50369e6c34
gh-116996: Add pystats about _Py_uop_analyse_and_optimize (GH-116997) 2024-03-22 01:27:46 +08:00
Eric Snow 617158e078
gh-76785: Drop PyInterpreterID_Type (gh-117101)
I added it quite a while ago as a strategy for managing interpreter lifetimes relative to the PEP 554 (now 734) implementation.  Relatively recently I refactored that implementation to no longer rely on InterpreterID objects.  Thus now I'm removing it.
2024-03-21 17:15:02 +00:00
Victor Stinner 8bea6c411d
gh-115754: Add Py_GetConstant() function (#116883)
Add Py_GetConstant() and Py_GetConstantBorrowed() functions.

In the limited C API version 3.13, getting Py_None, Py_False,
Py_True, Py_Ellipsis and Py_NotImplemented singletons is now
implemented as function calls at the stable ABI level to hide
implementation details. Getting these constants still return borrowed
references.

Add _testlimitedcapi/object.c and test_capi/test_object.py to test
Py_GetConstant() and Py_GetConstantBorrowed() functions.
2024-03-21 16:07:00 +00:00
Eric Snow 5a76d1be8e
gh-105716: Update interp->threads.main After Fork (gh-117049)
I missed this in gh-109921.

We also update Py_Exit() to call _PyInterpreterState_SetNotRunningMain(), if necessary.
2024-03-21 10:06:35 -06:00
Eric Snow bbee57fa8c
gh-76785: Clean Up Interpreter ID Conversions (gh-117048)
Mostly we unify the two different implementations of the conversion code (from PyObject * to int64_t.  We also drop the PyArg_ParseTuple()-style converter function, as well as rename and move PyInterpreterID_LookUp().
2024-03-21 09:56:12 -06:00
Sam Gross e728303532
gh-116522: Stop the world before fork() and during shutdown (#116607)
This changes the free-threaded build to perform a stop-the-world pause
before deleting other thread states when forking and during shutdown.
This fixes some crashes when using multiprocessing and during shutdown
when running with `PYTHON_GIL=0`.

This also changes `PyOS_BeforeFork` to acquire the runtime lock
(i.e., `HEAD_LOCK(&_PyRuntime)`) before forking to ensure that data
protected by the runtime lock (and not just the GIL or stop-the-world)
is in a consistent state before forking.
2024-03-21 10:01:16 -04:00
Mark Shannon 63289b9dfb
GH-117066: Tier 2 optimizer: Don't throw away good traces if we can't optimize them perfectly. (GH-117067) 2024-03-20 18:24:02 +00:00
Brett Simmers 9221ef2d8c
gh-116908: Only write to `_pending_calls.calls_to_do` with atomic operations (#117044)
These writes to `pending->calls_to_do` need to be atomic, because other threads
can read (atomically) from `calls_to_do` without holding `pending->mutex`.
2024-03-20 11:18:26 -04:00
Mark Shannon 15309329b6
GH-108362: Incremental Cycle GC (GH-116206) 2024-03-20 08:54:42 +00:00
Serhiy Storchaka 2d17309cc7
gh-117041: Add "-X gil" in the Python CLI help (GH-117042) 2024-03-19 23:30:25 +02:00
Serhiy Storchaka b85572c47d
gh-90300: Improve the Python CLI help output (GH-115853)
* document equivalent command-line options for all environment variables
* document equivalent environment variables for all command-line options
* reduce the size of variable and option descriptions to minimum
* remove the ending period in single-sentence descriptions

Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2024-03-19 19:26:32 +02:00
Brandt Bucher 2c82592ab4
GH-116017: Put JIT code and data on the same page (GH-116845) 2024-03-19 08:47:28 -07:00
Michael Droettboom 0f278012e8
gh-116808: Fix optimized trace length histogram (GH-116827) 2024-03-19 11:06:43 +00:00
Bogdan Romanyuk a8e93d3dca
gh-115756: make PyCode_GetFirstFree an unstable API (GH-115781) 2024-03-19 09:20:38 +00:00
Guido van Rossum 9c7b3688e6
gh-108716: Cleanup remaining deepfreeze infrastructure (#116919)
Keep Tools/build/deepfreeze.py around (we may repurpose it for deepfreezing non-code objects),
and keep basic "clean" targets that remove the output of former deep-freeze activities,
to keep the build directories of current devs clean.
2024-03-18 11:13:11 -07:00
Guido van Rossum 7e1f38f2de
gh-116916: Remove separate next_func_version counter (#116918)
Somehow we ended up with two separate counter variables tracking "the next function version".
Most likely this was a historical accident where an old branch was updated incorrectly.
This PR merges the two counters into a single one: `interp->func_state.next_version`.
2024-03-18 11:11:10 -07:00
Guido van Rossum 76d0868907
Cleanup tier2 debug output (#116920)
Various tweaks, including a slight refactor of the special cases for `_PUSH_FRAME`/`_POP_FRAME` to show the actual operand emitted.
2024-03-18 11:08:43 -07:00
Victor Stinner 1d95451be1
gh-63207: Use GetSystemTimePreciseAsFileTime() in time.time() (#116822) 2024-03-18 17:13:01 +01:00
Erlend E. Aasland 762f489b31
gh-116664: Ensure thread-safe dict access in _warnings (#116768)
Replace _PyDict_GetItemWithError() with PyDict_GetItemRef().
2024-03-18 09:37:48 +00:00
mpage 33da0e844c
gh-114271: Fix race in `Thread.join()` (#114839)
There is a race between when `Thread._tstate_lock` is released[^1] in `Thread._wait_for_tstate_lock()`
and when `Thread._stop()` asserts[^2] that it is unlocked. Consider the following execution
involving threads A, B, and C:

1. A starts.
2. B joins A, blocking on its `_tstate_lock`.
3. C joins A, blocking on its `_tstate_lock`.
4. A finishes and releases its `_tstate_lock`.
5. B acquires A's `_tstate_lock` in `_wait_for_tstate_lock()`, releases it, but is swapped
   out before calling `_stop()`.
6. C is scheduled, acquires A's `_tstate_lock` in `_wait_for_tstate_lock()` but is swapped
   out before releasing it.
7. B is scheduled, calls `_stop()`, which asserts that A's `_tstate_lock` is not held.
   However, C holds it, so the assertion fails.

The race can be reproduced[^3] by inserting sleeps at the appropriate points in
the threading code. To do so, run the `repro_join_race.py` from the linked repo.

There are two main parts to this PR:

1. `_tstate_lock` is replaced with an event that is attached to `PyThreadState`.
   The event is set by the runtime prior to the thread being cleared (in the same
   place that `_tstate_lock` was released). `Thread.join()` blocks waiting for the
   event to be set.
2. `_PyInterpreterState_WaitForThreads()` provides the ability to wait for all
   non-daemon threads to exit. To do so, an `is_daemon` predicate was added to
   `PyThreadState`. This field is set each time a thread is created. `threading._shutdown()`
   now calls into `_PyInterpreterState_WaitForThreads()` instead of waiting on
   `_tstate_lock`s.

[^1]: 441affc9e7/Lib/threading.py (L1201)
[^2]: 441affc9e7/Lib/threading.py (L1115)
[^3]: 8194653279

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
2024-03-16 13:56:30 +01:00
Donghee Na 20578a1f68
gh-112536: Add TSAN builds on Github Actions (#116872) 2024-03-16 11:10:37 +01:00
Mark Shannon 950667ed07
GH-115802: Reduce the size of _INIT_CALL_PY_EXACT_ARGS. (GH-116856) 2024-03-15 17:16:30 +00:00
Tian Gao 59e30f41ed
gh-116735: Use `MISSING` for `CALL` event if argument is absent (GH-116737) 2024-03-15 14:46:18 +00:00
Mark Shannon 2cf18a4430
GH-116422: Modify a few uops so that they can be supported by tier 2 with hot/cold splitting (GH-116832) 2024-03-15 10:48:00 +00:00
Victor Stinner c432df6d56
gh-111696, PEP 737: Add PyType_GetModuleName() function (#116824)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2024-03-14 18:17:43 +00:00
Mark Shannon 61e54bfcee
GH-116422: Factor out eval breaker checks at end of calls into its own micro-op. (GH-116817) 2024-03-14 16:31:47 +00:00
Victor Stinner 846ad5a26a
gh-88494: Use QueryPerformanceCounter() for time.monotonic() (#116781)
On Windows, time.monotonic() now uses the QueryPerformanceCounter()
clock to have a resolution better than 1 us, instead of the
gGetTickCount64() clock which has a resolution of 15.6 ms.
2024-03-14 16:42:41 +01:00
Serhiy Storchaka 19ac28bd08
gh-90300: Fix undocumented envvars in the Python CLI help (GH-116765) 2024-03-14 12:09:19 +02:00
guangwu 7af4b9f253
Docs: fix spelling of the word 'transferring' (#116641) 2024-03-13 23:53:32 +01:00
Michael Droettboom cef0ec1a3c
gh-116760: Fix pystats for trace attempts (GH-116761)
There are now at least two bytecodes that may attempt to optimize,
JUMP_BACK, and more recently, COLD_EXIT.

Only the JUMP_BACK was counting the attempt in the stats.
This moves that counter to uop_optimize itself so it should
always happen no matter where it is called from.
2024-03-13 22:13:33 +00:00
Serhiy Storchaka 8c6db45ce3
gh-90300: Document equivalent -X options for envvars in the Python CLI help (GH-116756) 2024-03-13 22:59:16 +02:00
Serhiy Storchaka e54bdeab9c
gh-90300: Sort the -X options and some envvars in the Python CLI help (GH-116739) 2024-03-13 21:15:44 +02:00
Serhiy Storchaka 33662d4e01
gh-90300: Fix cmdline.rst (GH-116721)
* Fix the description of the "-b" option.
* Add references to environment variables for "-s" and "-X dev" options.
2024-03-13 15:03:13 +02:00
Ken Jin 617aca9e74
gh-115419: Change default sym to not_null (GH-116562) 2024-03-13 20:57:48 +08:00
Tian Gao 8332e85b2f
gh-116626: Emit `CALL` events for all `INSTRUMENTED_CALL_FUNCTION_EX` (GH-116627) 2024-03-13 08:28:01 +00:00
Tian Gao a53cc3f494
GH-116098: Remove dead frame object creation code (GH-116687) 2024-03-12 23:35:28 +00:00
Sam Gross 5d72b75388
gh-116604: Check for `gcstate->enabled` in _Py_RunGC in free-threaded build (#116663)
This isn't strictly necessary because the implementation of `gc_should_collect`
already checks `gcstate->enabled` in the free-threaded build, but it seems
like a good idea until the common pieces of gc.c and gc_free_threading.c are
refactored out.
2024-03-12 17:12:02 +00:00
Pablo Galindo Salgado 02918aa961
gh-116604: Correctly honor the gc status when calling _Py_RunGC (#116628) 2024-03-12 12:00:49 +00:00
Sam Gross 9f983e00ec
gh-116515: Clear thread-local state before tstate_delete_common() (#116517)
This moves `current_fast_clear()` up so that the current thread state is
`NULL` while running `tstate_delete_common()`.

This doesn't fix any bugs, but it means that we are more consistent that
`_PyThreadState_GET() != NULL` means that the thread is "attached".
2024-03-11 15:14:20 -04:00
Victor Stinner 113053a070
gh-110850: Fix _PyTime_FromSecondsDouble() API (#116606)
Return 0 on success. Set an exception and return -1 on error.

Fix os.timerfd_settime(): properly report exceptions on
_PyTime_FromSecondsDouble() failure.

No longer export _PyTime_FromSecondsDouble().
2024-03-11 16:35:29 +00:00
Brett Simmers 2731913dd5
gh-116167: Allow disabling the GIL with `PYTHON_GIL=0` or `-X gil=0` (#116338)
In free-threaded builds, running with `PYTHON_GIL=0` will now disable the
GIL. Follow-up issues track work to re-enable the GIL when loading an
incompatible extension, and to disable the GIL by default.

In order to support re-enabling the GIL at runtime, all GIL-related data
structures are initialized as usual, and disabling the GIL simply sets a flag
that causes `take_gil()` and `drop_gil()` to return early.
2024-03-11 11:02:58 -04:00
Mark Shannon b6ae6da1bd
GH-116596: Better determination of escaping uops. (GH-116597) 2024-03-11 13:37:48 +00:00
Nikita Sobolev 817fe33a1d
gh-116590: Fix unused `current_thread_holds_gil` function warning (#116591) 2024-03-11 16:25:04 +03:00
Mark Shannon 4e5df2013f
GH-116468: Use constants instead of `oparg` in stack effects when `oparg` is known to be a constant. (GH-116469) 2024-03-11 09:30:15 +00:00
Sam Gross 3cdfdc07a9
gh-108724: Fix _PySemaphore_Wait call during thread deletion (#116483)
In general, when `_PyThreadState_GET()` is non-NULL then the current
thread is "attached", but there is a small window during
`PyThreadState_DeleteCurrent()` where that's not true:
tstate_delete_common() is called when the thread is detached, but before
current_fast_clear().

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2024-03-08 15:26:36 -05:00
Dino Viehland 7db871e4fa
gh-112075: Support freeing object memory via QSBR (#116344)
Free objects with qsbr if shared
2024-03-08 09:56:36 -08:00
Sam Gross cca30230d9
gh-115103: Fix unregistering of QSBR state (#116480)
If a thread blocks while waiting on the `shared->mutex` lock, the array
of QSBR states may be reallocated. The `tstate->qsbr` values before the
lock is acquired may not be the same as the value after the lock is acquired.
2024-03-08 12:39:53 -05:00
Nikita Sobolev fdb2d90a27
gh-116447: Fix possible UB in `arraymodule` and `getargs` (#116459) 2024-03-08 13:49:52 +03:00
Mark Shannon 0003285c8d
GH-113710: Fix optimization of globals using `_CHECK_FUNCTION` (GH-116460) 2024-03-08 09:47:41 +00:00
Sam Gross 834bf57eb7
gh-116396: Pass "detached_state" argument to tstate_set_detached (#116398)
The stop-the-world code was incorrectly setting suspended threads'
states to _Py_THREAD_DETACHED instead of _Py_THREAD_SUSPENDED.
2024-03-07 13:37:43 -05:00
Ken Jin 41457c7fdb
gh-116381: Remove bad specializations, add fail stats (GH-116464)
* Remove bad specializations, add fail stats
2024-03-08 00:21:21 +08:00
Ken Jin 4298d69d4b
gh-116420: Fix unused var compilation warnings (GH-116466)
Fix unused var compilation warnings
2024-03-08 00:19:59 +08:00
Serhiy Storchaka 72d3cc94cd
gh-116437: Use new C API PyDict_Pop() to simplify the code (GH-116438) 2024-03-07 11:21:08 +02:00
Donghee Na 2d4955fcf2
gh-116397: Move the _PyGC_ClearAllFreeLists to the safe point (gh-116414) 2024-03-07 08:29:39 +09:00
Ken Jin 7114cf20c0
gh-116381: Specialize CONTAINS_OP (GH-116385)
* Specialize CONTAINS_OP

* 📜🤖 Added by blurb_it.

* Add PyAPI_FUNC for JIT

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2024-03-07 03:30:11 +08:00
Sam Gross c012c8ab7b
gh-115103: Delay reuse of mimalloc pages that store PyObjects (#115435)
This implements the delayed reuse of mimalloc pages that contain Python
objects in the free-threaded build.

Allocations of the same size class are grouped in data structures called
pages. These are different from operating system pages. For thread-safety, we
want to ensure that memory used to store PyObjects remains valid as long as
there may be concurrent lock-free readers; we want to delay using it for
other size classes, in other heaps, or returning it to the operating system.

When a mimalloc page becomes empty, instead of immediately freeing it, we tag
it with a QSBR goal and insert it into a per-thread state linked list of
pages to be freed. When mimalloc needs a fresh page, we process the queue and
free any still empty pages that are now deemed safe to be freed. Pages
waiting to be freed are still available for allocations of the same size
class and allocating from a page prevent it from being freed. There is
additional logic to handle abandoned pages when threads exit.
2024-03-06 09:42:11 -05:00
Mark Shannon 27858e2a17
GH-113710: Tier 2 optimizer: check the function instead of checking globals. (GH-116410) 2024-03-06 13:12:23 +00:00
Mark Shannon 33c0aa3bb9
GH-115687: Most comparisons create Booleans, so propagate that information (GH-116360)
Most comparisons create booleans
2024-03-06 10:46:42 +00:00
Victor Stinner 2b379968e5
gh-107954: Add PyConfig_MEMBER_BOOL type to PyConfigSpec (#116359)
_PyConfig_AsDict() now returns bool objects for options using the new
PyConfig_MEMBER_BOOL type.

Update tests for these changes.
2024-03-06 09:29:27 +00:00
Guido van Rossum d444dec09a
Fix debug output for optimized executor (#116337)
This adjusts `length` rather than using `length+1`
all over the place.
2024-03-05 10:05:29 -08:00
Mark Shannon 23db9c6227
GH-115685: Split `_TO_BOOL_ALWAYS_TRUE` into micro-ops (GH-116352) 2024-03-05 15:23:08 +00:00
Mark Shannon 0c81ce1360
GH-115819: Eliminate Boolean guards when value is known (GH-116355) 2024-03-05 15:06:00 +00:00
Nikita Sobolev c91bdf86ef
gh-116326: Handler errors correctly in `getwindowsversion` in `sysmodule` (#116339) 2024-03-05 12:31:04 +00:00
Mark Shannon cbf3d38cbe
GH-115685: Optimize `TO_BOOL` and variants based on truthiness of input. (GH-116311) 2024-03-05 11:23:46 +00:00
Nikita Sobolev 207030f552
gh-115320: Refactor `get_hash_info` in `sysmodule.c` not to swallow errors (#115321) 2024-03-04 23:03:59 +03:00
Brandt Bucher ffed8d985b
GH-116134: JIT aarch64-pc-windows-msvc (GH-116130) 2024-03-04 10:16:56 -08:00
Brett Simmers 0adfa8482d
gh-115832: Fix instrumentation version mismatch during interpreter shutdown (#115856)
A previous commit introduced a bug to `interpreter_clear()`: it set
`interp->ceval.instrumentation_version` to 0, without making the corresponding
change to `tstate->eval_breaker` (which holds a thread-local copy of the
version). After this happens, Python code can still run due to object finalizers
during a GC, and the version check in bytecodes.c will see a different result
than the one in instrumentation.c causing an infinite loop.

The fix itself is straightforward: clear `tstate->eval_breaker` when clearing
`interp->ceval.instrumentation_version`.
2024-03-04 11:29:39 -05:00
Donghee Na 2e91578a76
gh-115103: Update refleak checker to trigger _PyMem_ProcessDelayed (gh-116238) 2024-03-03 06:44:16 +09:00
mpage 9e88173d36
gh-114271: Make `_thread.ThreadHandle` thread-safe in free-threaded builds (GH-115190)
Make `_thread.ThreadHandle` thread-safe in free-threaded builds

We protect the mutable state of `ThreadHandle` using a `_PyOnceFlag`.
Concurrent operations (i.e. `join` or `detach`) on `ThreadHandle` block
until it is their turn to execute or an earlier operation succeeds.
Once an operation has been applied successfully all future operations
complete immediately.

The `join()` method is now idempotent. It may be called multiple times
but the underlying OS thread will only be joined once. After `join()`
succeeds, any future calls to `join()` will succeed immediately.

The internal thread handle `detach()` method has been removed.
2024-03-01 13:43:12 -08:00
Ken Jin ff96b81d78
gh-115480: Type propagate _BINARY_OP_ADD_UNICODE (GH-115710) 2024-03-02 03:40:04 +08:00
Tian Gao 7895a61168
gh-116098: Revert "gh-107674: Improve performance of `sys.settrace` (GH-114986)" (GH-116178)
Revert "gh-107674: Improve performance of `sys.settrace` (GH-114986)"

This reverts commit 0a61e23700.
2024-03-01 07:46:33 +01:00
Brett Simmers 339c8e1c13
gh-115999: Disable the specializing adaptive interpreter in free-threaded builds (#116013)
For now, disable all specialization when the GIL might be disabled.
2024-02-29 21:53:32 -05:00
Ken Jin d01886c5c9
gh-115685: Type/values propagate for TO_BOOL in tier 2 (GH-115686) 2024-03-01 06:13:38 +08:00
Guido van Rossum 0656509033
gh-116088: Insert bottom checks after all sym_set_...() calls (#116089)
This changes the `sym_set_...()` functions to return a `bool` which is `false`
when the symbol is `bottom` after the operation.

All calls to such functions now check this result and go to `hit_bottom`,
a special error label that prints a different message and then reports
that it wasn't able to optimize the trace. No executor will be produced
in this case.
2024-02-29 18:55:29 +00:00
Brandt Bucher f0df35eeca
GH-115802: JIT "small" code for Windows (GH-115964) 2024-02-29 08:11:28 -08:00
Guido van Rossum 86e5e063ab
gh-115816: Generate calls to sym_new_const() etc. without _Py_uop prefix (#116077)
This was left behind by GH-115987. Basically a lot of diffs like this:
```
-            res = _Py_uop_sym_new_unknown(ctx);
+            res = sym_new_unknown(ctx);
```
2024-02-29 00:05:53 +00:00
Guido van Rossum 3409bc29c9
gh-115859: Re-enable T2 optimizer pass by default (#116062)
This undoes the *temporary* default disabling of the T2 optimizer pass in gh-115860.

- Add a new test that reproduces Brandt's example from gh-115859; it indeed crashes before gh-116028 with PYTHONUOPSOPTIMIZE=1
- Re-enable the optimizer pass in T2, stop checking PYTHONUOPSOPTIMIZE
- Rename the env var to disable T2 entirely to PYTHON_UOPS_OPTIMIZE (must be explicitly set to 0 to disable)
- Fix skipIf conditions on tests in test_opt.py accordingly
- Export sym_is_bottom() (for debugging)
- Fix various things in the `_BINARY_OP_` specializations in the abstract interpreter:
  - DECREF(temp)
  - out-of-space check after sym_new_const()
  - add sym_matches_type() checks, so even if we somehow reach a binary op with symbolic constants of the wrong type on the stack we won't trigger the type assert
2024-02-28 22:38:01 +00:00
Sam Gross df5212df6c
gh-112529: Simplify PyObject_GC_IsTracked and PyObject_GC_IsFinalized (#114732) 2024-02-28 15:37:59 -05:00
Guido van Rossum e2a3e4b748
gh-115816: Improve internal symbols API in optimizer (#116028)
- Any `sym_set_...` call that attempts to set conflicting information
  cause the symbol to become `bottom` (contradiction).
- All `sym_is...` and similar calls return false or NULL for `bottom`.
- Everything's tested.
- The tests still pass with `PYTHONUOPSOPTIMIZE=1`.
2024-02-28 17:55:56 +00:00
Tian Gao 0a61e23700
gh-107674: Improve performance of `sys.settrace` (GH-114986) 2024-02-28 15:21:42 +00:00
Steve Dower 9578288a3e
gh-116012: Preserve GetLastError() across calls to TlsGetValue on Windows (GH-116014) 2024-02-28 13:58:25 +00:00
Jelle Zijlstra ed4dfd8825
gh-105858: Improve AST node constructors (#105880)
Demonstration:

>>> ast.FunctionDef.__annotations__
{'name': <class 'str'>, 'args': <class 'ast.arguments'>, 'body': list[ast.stmt], 'decorator_list': list[ast.expr], 'returns': ast.expr | None, 'type_comment': str | None, 'type_params': list[ast.type_param]}
>>> ast.FunctionDef()
<stdin>:1: DeprecationWarning: FunctionDef.__init__ missing 1 required positional argument: 'name'. This will become an error in Python 3.15.
<stdin>:1: DeprecationWarning: FunctionDef.__init__ missing 1 required positional argument: 'args'. This will become an error in Python 3.15.
<ast.FunctionDef object at 0x101959460>
>>> node = ast.FunctionDef(name="foo", args=ast.arguments())
>>> node.decorator_list
[]
>>> ast.FunctionDef(whatever="you want", name="x", args=ast.arguments())
<stdin>:1: DeprecationWarning: FunctionDef.__init__ got an unexpected keyword argument 'whatever'. Support for arbitrary keyword arguments is deprecated and will be removed in Python 3.15.
<ast.FunctionDef object at 0x1019581f0>
2024-02-27 18:13:03 -08:00
Mark Shannon 6ecfcfe894
GH-115816: Assorted naming and formatting changes to improve maintainability. (GH-115987)
* Rename _Py_UOpsAbstractInterpContext to _Py_UOpsContext and _Py_UOpsSymType to _Py_UopsSymbol.

* #define shortened form of _Py_uop_... names for improved readability.
2024-02-27 13:25:02 +00:00
Mark Shannon 10fbcd6c5d
GH-115816: Make tier2 optimizer symbols testable, and add a few tests. (GH-115953) 2024-02-27 10:51:26 +00:00
Michael Droettboom b05afdd5ec
gh-115168: Add pystats counter for invalidated executors (GH-115169) 2024-02-26 17:51:47 +00:00
Guido van Rossum c0fdfba7ff
Rename tier 2 redundancy eliminator to optimizer (#115888)
The original name is just too much of a mouthful.
2024-02-26 08:42:53 -08:00
Brandt Bucher 7259480957
GH-115802: JIT "small" code for macOS and Linux (GH-115826) 2024-02-26 08:32:44 -08:00
Sergii K f082a05c67
gh-115914: minor cleanup: simplify filename_obj assignment in PyRun_AnyFileExFlags (gh-115916)
This simplifies the code: less lines, easier to read. Logically equivalent, as any compiler likely already determined.
2024-02-25 12:45:38 -08:00
Ken Jin 3d8fc06d4f
gh-115859: Disable the tier 2 redundancy eliminator by default (GH-115860) 2024-02-23 18:43:52 +00:00
Kirill Podoprigora e4561e0501
gh-115778: Add `tierN` annotation for instruction definitions (#115815)
This replaces the old `TIER_{ONE,TWO}_ONLY` macros. Note that `specialized` implies `tier1`.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-02-23 17:31:57 +00:00
Ken Jin 2ec50b4a66
gh-114058: Improve method information in redundancy eliminator (GH-115848) 2024-02-23 23:41:10 +08:00
Serhiy Storchaka acd6f41ecf
gh-111789: Use PyDict_GetItemRef() in Python/compile.c (GH-112083)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2024-02-23 12:35:27 +01:00
Serhiy Storchaka 2e92ffd7fa
gh-90300: Reformat the Python CLI help output (GH-93415) 2024-02-23 10:27:07 +01:00
Ken Jin a33ffe4785
gh-114058: More robust method handling in redundancy eliminator (GH-115779) 2024-02-23 15:42:03 +08:00
Guido van Rossum 4ee6bdfbaa
gh-115727: Reduce confidence even on 100% predicted jumps (#115748)
The theory is that even if we saw a jump go in the same direction the
last 16 times we got there, we shouldn't be overly confident that it's
still going to go the same way in the future. This PR makes it so that
in the extreme cases, the confidence is multiplied by 0.9 instead of
remaining unchanged. For unpredictable jumps, there is no difference
(still 0.5). For somewhat predictable jumps, we interpolate.
2024-02-22 12:23:48 -08:00
Mark Shannon b348313e7a
GH-115651: Convert `LOAD_MODULE_ATTR` into `LOAD_INLINE_CONST` when the module is itself a constant. (GH-115711) 2024-02-22 14:48:25 +00:00
Irit Katriel 96c1737591
gh-115796: fix exception table construction in _testinternalcapi.assemble_code_object (#115797) 2024-02-22 12:36:44 +00:00
Victor Stinner e4c34f04a1
gh-110850: Cleanup PyTime API: PyTime_t are nanoseconds (#115753)
PyTime_t no longer uses an arbitrary unit, it's always a number of
nanoseconds (64-bit signed integer).

* Rename _PyTime_FromNanosecondsObject() to _PyTime_FromLong().
* Rename _PyTime_AsNanosecondsObject() to _PyTime_AsLong().
* Remove pytime_from_nanoseconds().
* Remove pytime_as_nanoseconds().
* Remove _PyTime_FromNanoseconds().
2024-02-21 11:46:00 +01:00
Victor Stinner 77430b6a32
gh-110850: Replace private _PyTime_MAX with public PyTime_MAX (#115751)
Remove references to the old names _PyTime_MIN
and _PyTime_MAX, now that PyTime_MIN and
PyTime_MAX are public.

Replace also _PyTime_MIN with PyTime_MIN.
2024-02-21 08:11:40 +00:00
Benjamin Peterson 1235e84276
Delete unused sym_clear_flag function. (#115744) 2024-02-20 17:40:48 -08:00
Victor Stinner 145bc2d638
gh-110850: Use public PyTime functions (#115746)
Replace private _PyTime functions with public PyTime functions.

random_seed_time_pid() now reports errors to its caller.
2024-02-20 23:31:30 +00:00
Victor Stinner 52d1477566
gh-110850: Rename internal PyTime C API functions (#115734)
Rename functions:

* _PyTime_GetSystemClock() => _PyTime_TimeUnchecked()
* _PyTime_GetPerfCounter() => _PyTime_PerfCounterUnchecked()
* _PyTime_GetMonotonicClock() => _PyTime_MonotonicUnchecked()
* _PyTime_GetSystemClockWithInfo() => _PyTime_TimeWithInfo()
* _PyTime_GetMonotonicClockWithInfo() => _PyTime_MonotonicWithInfo()
* _PyTime_GetMonotonicClockWithInfo() => _PyTime_MonotonicWithInfo()

Changes:

* Remove "typedef PyTime_t PyTime_t;" which was
  "typedef PyTime_t _PyTime_t;" before a previous rename.
* Update comments of "Unchecked" functions.
* Remove invalid PyTime_Time() comment.
2024-02-20 22:16:37 +00:00
Guido van Rossum 142502ea8d
Tier 2 cleanups and tweaks (#115534)
* Rename `_testinternalcapi.get_{uop,counter}_optimizer` to `new_*_optimizer`
* Use `_PyUOpName()` instead of` _PyOpcode_uop_name[]`
* Add `target` to executor iterator items -- `list(ex)` now returns `(opcode, oparg, target, operand)` quadruples
* Add executor methods `get_opcode()` and `get_oparg()` to get `vmdata.opcode`, `vmdata.oparg`
* Define a helper for printing uops, and unify various places where they are printed
* Add a hack to summarize_stats.py to fix legacy uop names (e.g. `POP_TOP` -> `_POP_TOP`)
* Define helpers in `test_opt.py` for accessing the set or list of opnames of an executor
2024-02-20 20:24:35 +00:00
Sam Gross 520403ed4c
gh-115733: Fix crash involving exhausted list iterator (#115740)
* gh-115733: Fix crash involving exhausted iterator

* Add blurb
2024-02-21 05:18:44 +09:00
Ken Jin 7a8c3ed43a
gh-115735: Fix current executor NULL before _START_EXECUTOR (#115736)
This fixes level 3 or higher lltrace debug output `--with-pydebug` runs.
2024-02-20 18:47:05 +00:00
Sam Gross e3ad6ca56f
gh-115103: Implement delayed free mechanism for free-threaded builds (#115367)
This adds `_PyMem_FreeDelayed()` and supporting functions. The
`_PyMem_FreeDelayed()` function frees memory with the same allocator as
`PyMem_Free()`, but after some delay to ensure that concurrent lock-free
readers have finished.
2024-02-20 13:04:37 -05:00
Victor Stinner d207c7cd5a
gh-110850: Cleanup pycore_time.h includes (#115724)
<pycore_time.h> include is no longer needed to get the PyTime_t type
in internal header files. This type is now provided by <Python.h>
include. Add <pycore_time.h> includes to C files instead.
2024-02-20 16:50:43 +00:00
Sam Gross cc82e33af9
gh-115491: Keep some fields valid across allocations (free-threading) (#115573)
This avoids filling the memory occupied by ob_tid, ob_ref_local, and
ob_ref_shared with debug bytes (e.g., 0xDD) in mimalloc in the
free-threaded build.
2024-02-20 10:36:40 -05:00
Victor Stinner 9af80ec83d
gh-110850: Replace _PyTime_t with PyTime_t (#115719)
Run command:

sed -i -e 's!\<_PyTime_t\>!PyTime_t!g' $(find -name "*.c" -o -name "*.h")
2024-02-20 15:02:27 +00:00
Brett Simmers 0749244d13
gh-112175: Add `eval_breaker` to `PyThreadState` (#115194)
This change adds an `eval_breaker` field to `PyThreadState`. The primary
motivation is for performance in free-threaded builds: with thread-local eval
breakers, we can stop a specific thread (e.g., for an async exception) without
interrupting other threads.

The source of truth for the global instrumentation version is stored in the
`instrumentation_version` field in PyInterpreterState. Threads usually read the
version from their local `eval_breaker`, where it continues to be colocated
with the eval breaker bits.
2024-02-20 09:57:48 -05:00
Victor Stinner d24bed5ba0
gh-110850: PyTime_Time() return 0 on success (GH-115713)
Thanks!
2024-02-20 14:35:41 +01:00
Ken Jin dcba21f905
gh-115687: Split up guards from COMPARE_OP (GH-115688) 2024-02-20 11:30:49 +00:00
Mark Shannon 626c414995
GH-115457: Support splitting and replication of micro ops. (GH-115558) 2024-02-20 10:50:59 +00:00
Mark Shannon 7b21403ccd
GH-112354: Initial implementation of warm up on exits and trace-stitching (GH-114142) 2024-02-20 09:39:55 +00:00
wookie184 664965a1c1
gh-96497: Mangle name before symtable lookup in 'symtable_extend_namedexpr_scope' (GH-96561) 2024-02-17 12:06:31 +00:00
Sam Gross 5903190727
gh-115103: Implement delayed memory reclamation (QSBR) (#115180)
This adds a safe memory reclamation scheme based on FreeBSD's "GUS" and
quiescent state based reclamation (QSBR). The API provides a mechanism
for callers to detect when it is safe to free memory that may be
concurrently accessed by readers.
2024-02-16 15:25:19 -05:00
Peter Lazorchak 13addd2bbd
gh-115480: Type / constant propagation for float binary uops (GH-115550)
Co-authored-by: Ken Jin <kenjin@python.org>
2024-02-17 02:02:48 +08:00
Sam Gross b24c9161a6
gh-112529: Make the GC scheduling thread-safe (#114880)
The GC keeps track of the number of allocations (less deallocations)
since the last GC. This buffers the count in thread-local state and uses
atomic operations to modify the per-interpreter count. The thread-local
buffering avoids contention on shared state.

A consequence is that the GC scheduling is not as precise, so
"test_sneaky_frame_object" is skipped because it requires that the GC be
run exactly after allocating a frame object.
2024-02-16 11:22:27 -05:00
Ken Jin f92857a930
gh-115480: Minor fixups in int constant propagation (GH-115507) 2024-02-16 22:59:43 +08:00
Nikita Sobolev 20eaf4d5df
gh-115503: Fix `run_presite` error handling (#115504) 2024-02-16 08:33:17 +03:00
Donghee Na 321d13fd2b
gh-111968: Split _Py_dictkeys_freelist out of _Py_dict_freelist (gh-115505) 2024-02-16 01:01:36 +00:00
Dino Viehland 454d7963e3
gh-113743: Use per-interpreter locks for types (#115541)
Move type-lock to per-interpreter lock to avoid heavy contention in interpreters test
2024-02-15 16:28:31 -08:00
Dino Viehland ae460d450a
gh-113743: Make the MRO cache thread-safe in free-threaded builds (#113930)
Makes _PyType_Lookup thread safe, including:
    Thread safety of the underlying cache.
    Make mutation of mro and type members thread safe
    Also _PyType_GetMRO and _PyType_GetBases are currently returning borrowed references which aren't safe.
2024-02-15 10:54:57 -08:00
Irit Katriel f42e112fd8
gh-115420: Fix translation of exception hander targets by _testinternalcapi.optimize_cfg. (#115425) 2024-02-15 14:32:52 +00:00
Irit Katriel 3a9e67a9fd
gh-115376: fix segfault in _testinternalcapi.compiler_codegen on bad input (#115379) 2024-02-15 14:32:21 +00:00
Victor Stinner 94f1334e52
gh-115124: Use _PyObject_ASSERT() in gc.c (#115125)
Replace assert() with _PyObject_ASSERT() in gc.c to dump the object
when an assertion fails.
2024-02-15 14:29:42 +00:00
Irit Katriel 732faf17a6
gh-115347: avoid emitting redundant NOP for the docstring with -OO (#115494) 2024-02-15 14:20:19 +00:00
David Hewitt 9e3729bbd7
gh-114626: add PyCFunctionFast and PyCFunctionFastWithKeywords (GH-114627)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-02-15 11:05:20 +01:00
Ken Jin 4ebf8fbdab
gh-115480: Type and constant propagation for int BINARY_OPs (GH-115478) 2024-02-15 14:02:18 +08:00
Eric Snow 468430189d
gh-115482: Assume the Main Interpreter is Always Running "main" (gh-115484)
This is a temporary fix to unblock embedders that do not call Py_Main().

_PyInterpreterState_IsRunningMain() will always return true for the main interpreter, even in corner cases where it technically should not. The (future) full solution will do the right thing in those corner cases.
2024-02-14 16:07:22 -07:00
Donghee Na a2d4281415
gh-112087: Make __sizeof__ and listiter_{len, next} to be threadsafe (gh-114843) 2024-02-15 02:00:50 +09:00
Donghee Na f15795c9a0
gh-111968: Rename freelist related struct names to Eric's suggestion (gh-115329) 2024-02-14 00:32:51 +00:00
Eric Snow 514b1c91b8
gh-76785: Improved Subinterpreters Compatibility with 3.12 (gh-115424)
For the most part, these changes make is substantially easier to backport subinterpreter-related code to 3.12, especially the related modules (e.g. _xxsubinterpreters). The main motivation is to support releasing a PyPI package with the 3.13 capabilities compiled for 3.12.

A lot of the changes here involve either hiding details behind macros/functions or splitting up some files.
2024-02-13 14:56:49 -07:00
Mark Shannon 681778c56a
GH-113710: Improve `_SET_IP` and `_CHECK_VALIDITY` (GH-115248) 2024-02-13 16:28:19 +00:00
Mark Shannon f9f6156c5a
GH-113710: Backedge counter improvements. (GH-115166) 2024-02-13 14:16:37 +00:00
Ken Jin 7cce857622
gh-114058: Foundations of the Tier2 redundancy eliminator (GH-115085)
---------

Co-authored-by: Mark Shannon <9448417+markshannon@users.noreply.github.com>
Co-authored-by: Jules <57632293+JuliaPoo@users.noreply.github.com>
Co-authored-by: Guido van Rossum <gvanrossum@users.noreply.github.com>
2024-02-13 21:24:48 +08:00
Petr Viktorin 879f4546bf
gh-110850: Add PyTime_t C API (GH-115215)
* gh-110850: Add PyTime_t C API

Add PyTime_t API:

* PyTime_t type.
* PyTime_MIN and PyTime_MAX constants.
* PyTime_AsSecondsDouble(), PyTime_Monotonic(),
  PyTime_PerfCounter() and PyTime_GetSystemClock() functions.

Co-authored-by: Victor Stinner <vstinner@python.org>
2024-02-12 18:13:10 +01:00
Mark Shannon 8144661017
GH-113710: Fix updating of dict version tag and add watched dict stats (GH-115221) 2024-02-12 16:07:38 +00:00
Brandt Bucher 235cacff81
GH-114695: Add `sys._clear_internal_caches` (GH-115152) 2024-02-12 09:04:36 +00:00
Serhiy Storchaka d9d6909697
gh-115011: Improve support of __index__() in setters of members with unsigned integer type (GH-115029)
Setters for members with an unsigned integer type now support
the same range of valid values for objects that has a __index__()
method as for int.

Previously, Py_T_UINT, Py_T_ULONG and Py_T_ULLONG did not support
objects that has a __index__() method larger than LONG_MAX.

Py_T_ULLONG did not support negative ints. Now it supports them and
emits a RuntimeWarning.
2024-02-11 12:45:58 +02:00
Serhiy Storchaka e2c4038924
gh-76763: Make chr() always raising ValueError for out-of-range values (GH-114882)
Previously it raised OverflowError for very large or very small values.
2024-02-10 12:21:35 +02:00
Donghee Na d4d5bae147
gh-111968: Refactor _PyXXX_Fini to integrate with _PyObject_ClearFreeLists (gh-114899) 2024-02-10 00:57:04 +00:00
Sam Gross a3af3cb4f4
gh-110481: Implement inter-thread queue for biased reference counting (#114824)
Biased reference counting maintains two refcount fields in each object:
`ob_ref_local` and `ob_ref_shared`. The true refcount is the sum of these two
fields. In some cases, when refcounting operations are split across threads,
the ob_ref_shared field can be negative (although the total refcount must be
at least zero). In this case, the thread that decremented the refcount
requests that the owning thread give up ownership and merge the refcount
fields.
2024-02-09 17:08:32 -05:00
Shantanu 17689e3c41
gh-107944: Improve error message for getargs with bad keyword arguments (#114792) 2024-02-08 01:04:41 -08:00
Sam Gross ef3ceab09d
gh-112066: Use `PyDict_SetDefaultRef` in place of `PyDict_SetDefault`. (#112211)
This changes a number of internal usages of `PyDict_SetDefault` to use `PyDict_SetDefaultRef`.

Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-02-07 13:43:18 -05:00
Carl Meyer fedbf77191
gh-114828: Fix __class__ in class-scope inlined comprehensions (#115139) 2024-02-07 16:56:16 +00:00
Mark Shannon 8a3c499ffe
GH-108362: Revert "GH-108362: Incremental GC implementation (GH-108038)" (#115132)
Revert "GH-108362: Incremental GC implementation (GH-108038)"

This reverts commit 36518e69d7.
2024-02-07 12:38:34 +00:00
Sam Gross b6228b521b
gh-115035: Mark ThreadHandles as non-joinable earlier after forking (#115042)
This marks dead ThreadHandles as non-joinable earlier in
`PyOS_AfterFork_Child()` before we execute any Python code. The handles
are stored in a global linked list in `_PyRuntimeState` because `fork()`
affects the entire process.
2024-02-06 14:45:04 -05:00
mpage c32bae5290
gh-114944: Fix race between `_PyParkingLot_Park` and `_PyParkingLot_UnparkAll` when handling interrupts (#114945)
Fix race between `_PyParkingLot_Park` and `_PyParkingLot_UnparkAll` when handling interrupts

There is a potential race when `_PyParkingLot_UnparkAll` is executing in
one thread and another thread is unblocked because of an interrupt in
`_PyParkingLot_Park`. Consider the following scenario:

1. Thread T0 is blocked[^1] in `_PyParkingLot_Park` on address `A`.
2. Thread T1 executes `_PyParkingLot_UnparkAll` on address `A`. It
   finds the `wait_entry` for `T0` and unlinks[^2] its list node.
3. Immediately after (2), T0 is woken up due to an interrupt. It
   then segfaults trying to unlink[^3] the node that was previously
   unlinked in (2).

To fix this we mark each waiter as unparking before releasing the bucket
lock. `_PyParkingLot_Park` will wait to handle the coming wakeup, and not
attempt to unlink the node, when this field is set. `_PyParkingLot_Unpark`
does this already, presumably to handle this case.
2024-02-05 13:48:37 -08:00
Mark Shannon 36518e69d7
GH-108362: Incremental GC implementation (GH-108038) 2024-02-05 18:28:51 +00:00
Serhiy Storchaka d466052ad4
gh-114388: Fix an error in GH-114391 (GH-115000) 2024-02-04 17:06:22 +00:00
Serhiy Storchaka 3ddc515255
gh-114388: Fix warnings when assign an unsigned integer member (GH-114391)
* Fix a RuntimeWarning emitted when assign an integer-like value that
  is not an instance of int to an attribute that corresponds to a C
  struct member of type T_UINT and T_ULONG.
* Fix a double RuntimeWarning emitted when assign a negative integer value
  to an attribute that corresponds to a C struct member of type T_UINT.
2024-02-04 17:32:25 +02:00
Jason Zhang efc489021c
gh-111417: Remove unused code block in math.trunc() and round() (GH-111454)
_PyObject_LookupSpecial() now ensures that the type is ready.
2024-02-03 17:11:10 +02:00
Andrew Rogers b3f0b698da
gh-104530: Enable native Win32 condition variables by default (GH-104531) 2024-02-02 13:50:51 +00:00
Mark Shannon 0e71a295e9
GH-113710: Add a "globals to constants" pass (GH-114592)
Converts specializations of `LOAD_GLOBAL` into constants during tier 2 optimization.
2024-02-02 12:14:34 +00:00
Irit Katriel 2091fb2a85
gh-107901: make compiler inline basic blocks with no line number and no fallthrough (#114750) 2024-02-02 11:26:31 +00:00
Donghee Na 13907968d7
gh-111968: Use per-thread freelists for dict in free-threading (gh-114323) 2024-02-01 20:53:53 +00:00
Sam Gross 587d480203
gh-112529: Remove PyGC_Head from object pre-header in free-threaded build (#114564)
* gh-112529: Remove PyGC_Head from object pre-header in free-threaded build

This avoids allocating space for PyGC_Head in the free-threaded build.
The GC implementation for free-threaded CPython does not use the
PyGC_Head structure.

 * The trashcan mechanism uses the `ob_tid` field instead of `_gc_prev`
   in the free-threaded build.
 * The GDB libpython.py file now determines the offset of the managed
   dict field based on whether the running process is a free-threaded
   build. Those are identified by the `ob_ref_local` field in PyObject.
 * Fixes `_PySys_GetSizeOf()` which incorrectly incorrectly included the
   size of `PyGC_Head` in the size of static `PyTypeObject`.
2024-02-01 12:29:19 -08:00
Mark Shannon e66d0399cc
GH-114806. Don't specialize calls to classes with metaclasses. (GH-114870) 2024-02-01 19:39:32 +00:00
Anders Kaseorg 84e0e32184
Remove unused Py_XDECREF from _PyFrame_ClearExceptCode (GH-106158)
frame->frame_obj was set to NULL a few lines earlier.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2024-02-01 18:26:22 +02:00
Sam Gross e6d6d5dcc0
gh-114746: Avoid quadratic behavior in free-threaded GC (GH-114817)
The free-threaded build's GC implementation is non-generational, but was
scheduled as if it were collecting a young generation leading to
quadratic behavior. This increases the minimum threshold and scales it
to the number of live objects as we do for the old generation in the
default build.

Note that the scheduling is still not thread-safe without the GIL. Those
changes will come in later PRs.

A few tests, like "test_sneaky_frame_object" rely on prompt scheduling
of the GC. For now, to keep that test passing, we disable the scaled
threshold after calls like `gc.set_threshold(1, 0, 0)`.
2024-02-01 10:26:23 +01:00
Matt Prodani e5e186609f
gh-112606: Use pthread_cond_timedwait_relative_np() in parking_lot.c when available (#112616)
Add a configure define for HAVE_PTHREAD_COND_TIMEDWAIT_RELATIVE_NP and
replaces pthread_cond_timedwait() with pthread_cond_timedwait_relative_np()
for relative time when supported in semaphore waiting logic.
2024-01-30 22:22:17 +01:00
Victor Stinner 58f883b91b
gh-103323: Remove current_fast_get() unused parameter (#114593)
The current_fast_get() static inline function doesn't use its
'runtime' parameter, so just remove it.
2024-01-30 11:47:58 +01:00
Erlend E. Aasland 8612230c1c
gh-114569: Use PyMem_* APIs for non-PyObjects in compiler (#114587) 2024-01-30 00:04:34 +01:00
Nikita Sobolev 1ac1b2f953
gh-114685: Fix incorrect use of PyBUF_READ in import.c (GH-114686) 2024-01-29 11:37:06 +02:00
Brandt Bucher f6d9e5926b
GH-113464: Add a JIT backend for tier 2 (GH-113465)
Add an option (--enable-experimental-jit for configure-based builds
or --experimental-jit for PCbuild-based ones) to build an
*experimental* just-in-time compiler, based on copy-and-patch (https://fredrikbk.com/publications/copy-and-patch.pdf).

See Tools/jit/README.md for more information on how to install the required build-time tooling.
2024-01-28 18:48:48 -08:00
Neil Schemenauer 7a7bce5a0a
gh-113055: Use pointer for interp->obmalloc state (gh-113412)
For interpreters that share state with the main interpreter, this points
to the same static memory structure.  For interpreters with their own
obmalloc state, it is heap allocated.  Add free_obmalloc_arenas() which
will free the obmalloc arenas and radix tree structures for interpreters
with their own obmalloc state.

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2024-01-26 19:38:14 -08:00
Donghee Na 699779256e
gh-111968: Unify freelist naming schema to Eric's suggestion (gh-114581) 2024-01-27 00:25:16 +09:00
Irit Katriel ac5e53e150
gh-107901: compiler replaces POP_BLOCK instruction by NOPs before optimisations (#114530) 2024-01-25 20:06:48 +00:00
Sam Gross b52fc70d1a
gh-112529: Implement GC for free-threaded builds (#114262)
* gh-112529: Implement GC for free-threaded builds

This implements a mark and sweep GC for the free-threaded builds of
CPython. The implementation relies on mimalloc to find GC tracked
objects (i.e., "containers").
2024-01-25 10:27:36 -08:00
Irit Katriel 0315941441
gh-114265: remove i_loc_propagated, jump threading does not consider line numbers anymore (#114535) 2024-01-25 12:54:19 +00:00
Michael Droettboom ea3cd0498c
gh-114312: Collect stats for unlikely events (GH-114493) 2024-01-25 11:10:51 +00:00
Kirill Podoprigora 191531f352
Update outdated comment in ``Python/bytecodes.c`` (#114522) 2024-01-24 09:14:15 -08:00
Mark Shannon 981d172f7f
GH-112354: `END_FOR` instruction to only pop one value. (GH-114247)
* Compiler emits END_FOR; POP_TOP instead of END_FOR. To support tier 2 side exits in loops.
2024-01-24 15:10:17 +00:00
Mark Shannon 384429d1c0
GH-113710: Add a tier 2 peephole optimization pass. (GH-114487)
* Convert _LOAD_CONST to inline versions

* Remove PEP 523 checks
2024-01-24 12:08:31 +00:00
Sam Gross 441affc9e7
gh-111964: Implement stop-the-world pauses (gh-112471)
The `--disable-gil` builds occasionally need to pause all but one thread.  Some
examples include:

* Cyclic garbage collection, where this is often called a "stop the world event"
* Before calling `fork()`, to ensure a consistent state for internal data structures
* During interpreter shutdown, to ensure that daemon threads aren't accessing Python objects

This adds the following functions to implement global and per-interpreter pauses:

* `_PyEval_StopTheWorldAll()` and `_PyEval_StartTheWorldAll()` (for the global runtime)
* `_PyEval_StopTheWorld()` and `_PyEval_StartTheWorld()` (per-interpreter)

(The function names may change.)

These functions are no-ops outside of the `--disable-gil` build.
2024-01-23 11:08:23 -07:00
Irit Katriel ed30a3c337
gh-114083: apply optimization of LOAD_CONST instructions to the whole CFG before optimize_basic_block. (#114408) 2024-01-22 17:12:06 +00:00
Guido van Rossum 2f2ddabd1a
gh-113102: Fix typo in INSTRUMENTED_RESUME (GH-114349) 2024-01-22 11:56:28 +00:00
Nikita Sobolev 38768e4cdd
gh-114384: Align sys.set_asyncgen_hooks signature in docs to reflect implementation (#114385) 2024-01-21 13:49:49 +01:00
Sam Gross 1d6d5e854c
gh-112529: Use GC heaps for GC allocations in free-threaded builds (gh-114157)
* gh-112529: Use GC heaps for GC allocations in free-threaded builds

The free-threaded build's garbage collector implementation will need to
find GC objects by traversing mimalloc heaps. This hooks up the
allocation calls with the correct heaps by using a thread-local
"current_obj_heap" variable.

* Refactor out setting heap based on type
2024-01-21 01:14:45 +09:00
Irit Katriel 7e49f27b41
gh-114265: move line number propagation before cfg optimization, remove guarantee_lineno_for_exits (#114267) 2024-01-19 14:49:26 +00:00
Donghee Na 7fa511ba57
gh-111968: Use per-thread freelists for generator in free-threading (gh-114189) 2024-01-18 18:15:00 +00:00
Mark Shannon 2ff072f21f
Delete unused macro (GH-114238) 2024-01-18 15:49:50 +00:00
solya0x c86571e4c9
Update copyright years to 2024. (GH-113608)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2024-01-16 21:54:05 +01:00
Donghee Na 867f59f234
gh-111968: Use per-thread freelists for PyContext in free-threading (gh-114122) 2024-01-16 16:14:56 +00:00
Serhiy Storchaka d2d8332f71
gh-113626: Add allow_code parameter in marshal functions (GH-113648)
Passing allow_code=False prevents serialization and de-serialization of
code objects which is incompatible between Python versions.
2024-01-16 18:05:15 +02:00
Mark Shannon 17b73ab99e
GH-113655: Lower the C recursion limit on various platforms (GH-113944) 2024-01-16 09:32:01 +00:00
Donghee Na 3eae76554b
gh-111968: Use per-thread slice_cache in free-threading (gh-113972) 2024-01-16 00:38:57 +09:00
Mark Shannon ac10947ba7
GH-112354: `_GUARD_IS_TRUE_POP` side-exits to target the next instruction, not themselves. (GH-114078) 2024-01-15 11:41:06 +00:00
Ken Jin ac92527c08
gh-113710: Add types to the interpreter DSL (#113711)
Co-authored-by: Jules <57632293+JuliaPoo@users.noreply.github.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2024-01-13 01:30:27 +08:00
Irit Katriel 8aa0088ea2
gh-107901: duplicate blocks with no lineno that have an eval break and multiple predecessors (#113950) 2024-01-12 15:38:09 +00:00
Brandt Bucher 30e6cbdba2
GH-113860: Get rid of `_PyUOpExecutorObject` (GH-113954) 2024-01-12 11:58:23 +00:00
Donghee Na 2e7577b622
gh-111968: Use per-thread freelists for tuple in free-threading (gh-113921) 2024-01-12 03:46:28 +09:00
Serhiy Storchaka 8717f7b495
gh-113845: Fix a compiler warning in Python/suggestions.c (GH-113949) 2024-01-11 20:31:24 +02:00
Mark Shannon 55824d01f8
GH-113853: Guarantee forward progress in executors (GH-113854) 2024-01-11 18:20:42 +00:00
Irit Katriel 0d8fec79ca
gh-107901: jump leaving an exception handler doesn't need an eval break check (#113943) 2024-01-11 14:27:41 +00:00
Peter Lazorchak f653caa5a8
gh-89811: Check for valid tp_version_tag in specializer (GH-113558) 2024-01-11 13:33:05 +08:00
Donghee Na f728f7242c
gh-111968: Use per-thread freelists for float in free-threading (gh-113886) 2024-01-10 15:47:13 +00:00