Commit Graph

282 Commits

Author SHA1 Message Date
Tian Gao 0d9148823d
gh-118414: Fix assertion in YIELD_VALUE when tracing lines or instrs (#118683) 2024-05-06 21:22:59 -07:00
Mark Shannon 616b745b89
GH-115709: Invalidate executors when a local variable is changed via frame.f_locals (#118639)
Also fix unrelated assert in debug Tier2/JIT builds.
2024-05-06 21:21:06 +00:00
Mark Shannon 1ab6356ebe
GH-118095: Use broader specializations of CALL in tier 1, for better tier 2 support of calls. (GH-118322)
* Add CALL_PY_GENERAL, CALL_BOUND_METHOD_GENERAL and call CALL_NON_PY_GENERAL specializations.

* Remove CALL_PY_WITH_DEFAULTS specialization

* Use CALL_NON_PY_GENERAL in more cases when otherwise failing to specialize
2024-05-04 12:11:11 +01:00
Mark Shannon da2cfc4cb6
GH-113464: Remove the extra jump via `_SIDE_EXIT` in `_EXIT_TRACE` (GH-118545) 2024-05-04 08:50:24 +01:00
Tian Gao 9c14ed0618
gh-107674: Improve performance of `sys.settrace` (GH-117133)
* Check tracing in RESUME_CHECK

* Only change to RESUME_CHECK if not tracing
2024-05-03 19:49:24 +01:00
Mark Shannon 72867c962c
GH-118095: Unify the behavior of tier 2 FOR_ITER branch micro-ops (GH-118420)
* Target _FOR_ITER_TIER_TWO at POP_TOP following the matching END_FOR

* Modify _GUARD_NOT_EXHAUSTED_RANGE, _GUARD_NOT_EXHAUSTED_LIST and _GUARD_NOT_EXHAUSTED_TUPLE so that they also target the POP_TOP following the matching END_FOR
2024-05-02 16:17:59 +01:00
Mark Shannon 67bba9dd0f
GH-117442: Check eval-breaker at start (rather than end) of tier 2 loops (GH-118482) 2024-05-02 13:10:31 +01:00
Mark Shannon f6fab21721
GH-118095: Make invalidating and clearing executors memory safe (GH-118459) 2024-05-01 11:34:50 +01:00
Guido van Rossum 7d83f7bcc4
gh-118335: Configure Tier 2 interpreter at build time (#118339)
The code for Tier 2 is now only compiled when configured
with `--enable-experimental-jit[=yes|interpreter]`.

We drop support for `PYTHON_UOPS` and -`Xuops`,
but you can disable the interpreter or JIT
at runtime by setting `PYTHON_JIT=0`.
You can also build it without enabling it by default
using `--enable-experimental-jit=yes-off`;
enable with `PYTHON_JIT=1`.

On Windows, the `build.bat` script supports
`--experimental-jit`, `--experimental-jit-off`,
`--experimental-interpreter`.

In the C code, `_Py_JIT` is defined as before
when the JIT is enabled; the new variable
`_Py_TIER2` is defined when the JIT *or* the
interpreter is enabled. It is actually a bitmask:
1: JIT; 2: default-off; 4: interpreter.
2024-04-30 18:26:34 -07:00
Ken Jin 7fabcc727d
gh-117657: Don't specialize RESUME_CHECK when specialization is disabled (GH-118349) 2024-04-30 21:51:59 +00:00
Dino Viehland 4a1cf66c5c
gh-117657: Fix small issues with instrumentation and TSAN (#118064)
Small TSAN fixups for instrumentation
2024-04-30 11:38:05 -07:00
Mark Shannon 5b05d452cd
GH-118095: Add tier 2 support for YIELD_VALUE (GH-118380) 2024-04-30 11:33:13 +01:00
Mark Shannon ab6eda0ee5
GH-118095: Allow a variant of RESUME_CHECK in tier 2 (GH-118286) 2024-04-29 07:54:05 +01:00
Mark Shannon 3e06c7f719
GH-118095: Add dynamic exit support and FOR_ITER_GEN support to tier 2 (GH-118279) 2024-04-26 18:08:50 +01:00
Mark Shannon f180b31e76
GH-118095: Handle `RETURN_GENERATOR` in tier 2 (GH-118180) 2024-04-25 11:32:47 +01:00
Mark Shannon 83235f7791
GH-115419: Move setting the instruction pointer to error exit stubs (GH-118088) 2024-04-24 14:41:30 +01:00
Mark Shannon 77cd0428b6
GH-118095: Convert DEOPT_IFs on likely side exits to EXIT_IFs (GH-118106)
Covert DEOPT_IFs on likely side exits to EXIT_IFs
2024-04-24 14:37:55 +01:00
Mark Shannon a6647d16ab
GH-115480: Reduce guard strength for binary ops when type of one operand is known already (GH-118050) 2024-04-22 13:34:06 +01:00
Dino Viehland 8b541c017e
gh-112075: Make instance attributes stored in inline "dict" thread safe (#114742)
Make instance attributes stored in inline "dict" thread safe on free-threaded builds
2024-04-21 22:57:05 -07:00
Dino Viehland 07525c9a85
gh-116818: Make `sys.settrace`, `sys.setprofile`, and monitoring thread-safe (#116775)
Makes sys.settrace, sys.setprofile, and monitoring generally thread-safe.

Mostly uses a stop-the-world approach and synchronization around the code object's _co_instrumentation_version.  There may be a little bit of extra synchronization around the monitoring data that's required to be TSAN clean.
2024-04-19 14:47:42 -07:00
Mark Shannon 7e6fa5fced
GH-116202: Incorporate invalidation check into _START_EXECUTOR. (GH-118044) 2024-04-19 09:26:42 +01:00
Dino Viehland 434bc593df
gh-112075: Make _PyDict_LoadGlobal thread safe (#117529)
Make _PyDict_LoadGlobal threadsafe
2024-04-04 12:26:07 -07:00
Guido van Rossum 060a96f1a9
gh-116968: Reimplement Tier 2 counters (#117144)
Introduce a unified 16-bit backoff counter type (``_Py_BackoffCounter``),
shared between the Tier 1 adaptive specializer and the Tier 2 optimizer. The
API used for adaptive specialization counters is changed but the behavior is
(supposed to be) identical.

The behavior of the Tier 2 counters is changed:
- There are no longer dynamic thresholds (we never varied these).
- All counters now use the same exponential backoff.
- The counter for ``JUMP_BACKWARD`` starts counting down from 16.
- The ``temperature`` in side exits starts counting down from 64.
2024-04-04 15:03:27 +00:00
Peter Lazorchak 1c43468886
gh-116168: Remove extra `_CHECK_STACK_SPACE` uops (#117242)
This merges all `_CHECK_STACK_SPACE` uops in a trace into a single `_CHECK_STACK_SPACE_OPERAND` uop that checks whether there is enough stack space for all calls included in the entire trace.
2024-04-03 17:14:18 +00:00
Mark Shannon c32dc47aca
GH-115776: Embed the values array into the object, for "normal" Python objects. (GH-116115) 2024-04-02 11:59:21 +01:00
Sam Gross 19c1dd60c5
gh-117323: Make `cell` thread-safe in free-threaded builds (#117330)
Use critical sections to lock around accesses to cell contents. The critical sections are no-ops in the default (with GIL) build.
2024-03-29 13:35:43 -04:00
Mark Shannon bf82f77957
GH-116422: Tier2 hot/cold splitting (GH-116813)
Splits the "cold" path, deopts and exits, from the "hot" path, reducing the size of most jitted instructions, at the cost of slower exits.
2024-03-26 09:35:11 +00:00
Mark Shannon 950667ed07
GH-115802: Reduce the size of _INIT_CALL_PY_EXACT_ARGS. (GH-116856) 2024-03-15 17:16:30 +00:00
Tian Gao 59e30f41ed
gh-116735: Use `MISSING` for `CALL` event if argument is absent (GH-116737) 2024-03-15 14:46:18 +00:00
Mark Shannon 2cf18a4430
GH-116422: Modify a few uops so that they can be supported by tier 2 with hot/cold splitting (GH-116832) 2024-03-15 10:48:00 +00:00
Mark Shannon 61e54bfcee
GH-116422: Factor out eval breaker checks at end of calls into its own micro-op. (GH-116817) 2024-03-14 16:31:47 +00:00
Michael Droettboom cef0ec1a3c
gh-116760: Fix pystats for trace attempts (GH-116761)
There are now at least two bytecodes that may attempt to optimize,
JUMP_BACK, and more recently, COLD_EXIT.

Only the JUMP_BACK was counting the attempt in the stats.
This moves that counter to uop_optimize itself so it should
always happen no matter where it is called from.
2024-03-13 22:13:33 +00:00
Tian Gao 8332e85b2f
gh-116626: Emit `CALL` events for all `INSTRUMENTED_CALL_FUNCTION_EX` (GH-116627) 2024-03-13 08:28:01 +00:00
Mark Shannon 4e5df2013f
GH-116468: Use constants instead of `oparg` in stack effects when `oparg` is known to be a constant. (GH-116469) 2024-03-11 09:30:15 +00:00
Mark Shannon 0003285c8d
GH-113710: Fix optimization of globals using `_CHECK_FUNCTION` (GH-116460) 2024-03-08 09:47:41 +00:00
Ken Jin 41457c7fdb
gh-116381: Remove bad specializations, add fail stats (GH-116464)
* Remove bad specializations, add fail stats
2024-03-08 00:21:21 +08:00
Serhiy Storchaka 72d3cc94cd
gh-116437: Use new C API PyDict_Pop() to simplify the code (GH-116438) 2024-03-07 11:21:08 +02:00
Ken Jin 7114cf20c0
gh-116381: Specialize CONTAINS_OP (GH-116385)
* Specialize CONTAINS_OP

* 📜🤖 Added by blurb_it.

* Add PyAPI_FUNC for JIT

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2024-03-07 03:30:11 +08:00
Mark Shannon 27858e2a17
GH-113710: Tier 2 optimizer: check the function instead of checking globals. (GH-116410) 2024-03-06 13:12:23 +00:00
Mark Shannon 23db9c6227
GH-115685: Split `_TO_BOOL_ALWAYS_TRUE` into micro-ops (GH-116352) 2024-03-05 15:23:08 +00:00
Tian Gao 7895a61168
gh-116098: Revert "gh-107674: Improve performance of `sys.settrace` (GH-114986)" (GH-116178)
Revert "gh-107674: Improve performance of `sys.settrace` (GH-114986)"

This reverts commit 0a61e23700.
2024-03-01 07:46:33 +01:00
Ken Jin d01886c5c9
gh-115685: Type/values propagate for TO_BOOL in tier 2 (GH-115686) 2024-03-01 06:13:38 +08:00
Brandt Bucher f0df35eeca
GH-115802: JIT "small" code for Windows (GH-115964) 2024-02-29 08:11:28 -08:00
Tian Gao 0a61e23700
gh-107674: Improve performance of `sys.settrace` (GH-114986) 2024-02-28 15:21:42 +00:00
Kirill Podoprigora e4561e0501
gh-115778: Add `tierN` annotation for instruction definitions (#115815)
This replaces the old `TIER_{ONE,TWO}_ONLY` macros. Note that `specialized` implies `tier1`.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-02-23 17:31:57 +00:00
Mark Shannon b348313e7a
GH-115651: Convert `LOAD_MODULE_ATTR` into `LOAD_INLINE_CONST` when the module is itself a constant. (GH-115711) 2024-02-22 14:48:25 +00:00
Sam Gross 520403ed4c
gh-115733: Fix crash involving exhausted list iterator (#115740)
* gh-115733: Fix crash involving exhausted iterator

* Add blurb
2024-02-21 05:18:44 +09:00
Brett Simmers 0749244d13
gh-112175: Add `eval_breaker` to `PyThreadState` (#115194)
This change adds an `eval_breaker` field to `PyThreadState`. The primary
motivation is for performance in free-threaded builds: with thread-local eval
breakers, we can stop a specific thread (e.g., for an async exception) without
interrupting other threads.

The source of truth for the global instrumentation version is stored in the
`instrumentation_version` field in PyInterpreterState. Threads usually read the
version from their local `eval_breaker`, where it continues to be colocated
with the eval breaker bits.
2024-02-20 09:57:48 -05:00
Ken Jin dcba21f905
gh-115687: Split up guards from COMPARE_OP (GH-115688) 2024-02-20 11:30:49 +00:00
Mark Shannon 626c414995
GH-115457: Support splitting and replication of micro ops. (GH-115558) 2024-02-20 10:50:59 +00:00