d24d1c986d
gh-121404: split compiler_nameop into a codegen part and a compiler part ( #123398 )
2024-08-27 16:57:31 +01:00
f49a91648a
GH-117759: Document incremental GC (GH-123266)
...
* Update what's new
* Update gc module docs and fix inconsistency in gc.get_objects
2024-08-27 15:23:39 +01:00
da4302699f
gh-121404: compiler_visit_* --> codegen_visit_* ( #123382 )
2024-08-27 11:32:24 +00:00
54a05a4600
GH-123232: Factor BINARY_SLICE and STORE_SLICE to handle stats properly for tier 2. (GH-123381)
2024-08-27 10:49:39 +01:00
86f06cb3fb
Remove comment from pystate created in 2003 ( #123259 )
2024-08-24 20:36:42 +00:00
556e855684
gh-117376: Make `Py_DECREF` a macro in ceval.c in free-threaded build ( #122975 )
...
`Py_DECREF` and `PyStackRef_CLOSE` are now implemented as macros in the
free-threaded build in ceval.c. There are two motivations;
* MSVC has problems inlining functions in ceval.c in the PGO build.
* We will want to mark escaping calls in order to spill the stack
pointer in ceval.c and we will want to do this around `_Py_Dealloc`
(or `_Py_MergeZeroLocalRefcount` or `_Py_DecRefShared`), not around
the entire `Py_DECREF` or `PyStackRef_CLOSE` call.
2024-08-23 15:36:14 -04:00
67f2c84bff
gh-123205: `Python/bytecodes.c`: Fix compiler warning ( #123206 )
...
Fix MSVC warning "conversion from '__int64' to 'int'"
2024-08-23 15:35:25 -04:00
bf1b5d323b
gh-121404: more compiler_* -> codegen_*, class_body and comprehensions ( #123262 )
2024-08-23 19:53:15 +01:00
7cd3aa42f0
GH-122298: Restore printing of GC stats (GH-123261)
2024-08-23 16:39:08 +01:00
0b0f7befad
GH-123232: Fix "not specialized" stats (GH-123236)
2024-08-23 10:46:03 +01:00
5fce482c9a
gh-121404: compiler_annassign --> codegen_annassign ( #123245 )
2024-08-23 10:39:42 +01:00
5d3201fe3f
GH-123040: Specialize shadowed `LOAD_ATTR`. (GH-123219)
2024-08-23 10:22:35 +01:00
297f2e093e
gh-123083: Fix a potential use-after-free in ``STORE_ATTR_WITH_HINT`` (gh-123092)
2024-08-22 23:49:09 +09:00
4abc1c1456
gh-121404: remove redundant c_nestlevel. more compiler abstractions. more macro usage consistency ( #123225 )
2024-08-22 15:08:16 +01:00
a3d8c0542e
GH-123197: Only count an instruction as deferred if it hasn't deopted first. (GH-123222)
...
Only count an instruction as deferred if hasn't deopted first.
2024-08-22 14:17:10 +01:00
427b106162
GH-118093: Specialize calls to non-vectorcall classes as `CALL_NON_PY_GENERAL` (GH-123212)
...
Specialize classes without vectorcall as CALL_NON_PY_GENERAL
2024-08-22 11:50:55 +01:00
ec89620e5e
gh-123142: Fix too wide source locations in tracebacks of exceptions from broken iterables in comprehensions ( #123173 )
2024-08-21 19:12:05 +01:00
a4fd7aa4a6
GH-115776: Allow any fixed sized object to have inline values (GH-123192)
2024-08-21 15:52:04 +01:00
4b7c4880a0
gh-121404: split fblock handling into compiler_* and codegen_* parts ( #123199 )
2024-08-21 15:48:56 +01:00
7b26c4d1e3
GH-123197: Increment correct stat for CALL_KW (GH-123200)
2024-08-21 12:52:28 +01:00
1eba8bae92
GH-123185: Check for `NULL` after calling `_PyEvalFramePushAndInit` (GH-123194)
2024-08-21 12:44:56 +01:00
bb1d30336e
GH-118093: Make `CALL_ALLOC_AND_ENTER_INIT` suitable for tier 2. (GH-123140)
...
* Convert CALL_ALLOC_AND_ENTER_INIT to micro-ops such that tier 2 supports it
* Allow inexact arguments for CALL_ALLOC_AND_ENTER_INIT.
2024-08-20 16:52:58 +01:00
bffed80230
gh-123048: Fix missing source location in pattern matching code ( #123167 )
2024-08-20 11:39:41 +01:00
b6d0a40a42
gh-121404: rename functions to use codegen_* prefix. Use macros more consistently. ( #123139 )
2024-08-19 14:39:53 +00:00
e077b201f4
gh-121404: pass metadata to compiler_enter_scope (optionally) so that codegen functions don't need to set it on the code unit ( #123078 )
2024-08-19 11:49:42 +01:00
c13e7d98fb
GH-118093: Specialize `CALL_KW` (GH-123006)
2024-08-16 17:11:24 +01:00
e2f2dc708e
gh-121404: rename compiler_addop* to codegen_addop*, and replace direct usages by the macros ( #123043 )
2024-08-16 15:57:32 +01:00
d7a3df9150
Add debug offsets for free threaded builds ( #123041 )
2024-08-15 18:42:41 +00:00
e001027188
gh-117139: Garbage collector support for deferred refcounting ( #122956 )
...
The free-threaded GC now visits interpreter stacks to keep objects
that use deferred reference counting alive.
Interpreter frames are zero initialized in the free-threaded GC so
that the GC doesn't see garbage data. This is a temporary measure
until stack spilling around escaping calls is implemented.
Co-authored-by: Ken Jin <kenjin@python.org>
2024-08-15 16:09:11 +00:00
b6cb435ac0
gh-121404: split compiler's push/pop_inlined_comprehension_state into codegen and compiler parts ( #123021 )
2024-08-15 11:08:54 +00:00
f84754b705
GH-118093: Turn some DEOPT_IFs into EXIT_IFs (GH-122998)
2024-08-14 07:54:42 -07:00
eec7bdaf01
GH-120024: Remove `CHECK_EVAL_BREAKER` macro. (GH-122968)
...
* Factor some instructions into micro-ops to isolate CHECK_EVAL_BREAKER for escape analysis
* Eliminate CHECK_EVAL_BREAKER macro
2024-08-14 12:04:05 +01:00
05fc4d758a
gh-122985: add SYMBOL_TO_SCOPE macro in symtable ( #122986 )
2024-08-14 06:17:04 +01:00
ee1b8ce26e
gh-122907: Fix Builds Without HAVE_DYNAMIC_LOADING Set (gh-122952)
...
As of 529a160
2024-08-13 14:44:57 -06:00
7a65439b93
GH-122390: Replace `_Py_GetbaseOpcode` with `_Py_GetBaseCodeUnit` (GH-122942)
2024-08-13 14:22:57 +01:00
fe23f8ed97
GH-122821: Simplify compilation of while statements to ensure consistency of offsets for sys.monitoring (GH-122934)
2024-08-13 10:25:44 +01:00
9621a7d017
GH-118093: Handle some polymorphism before requiring progress in tier two (GH-122843)
2024-08-12 12:39:31 -07:00
ab094d1b2b
gh-117139: Replace _PyList_FromArraySteal with stack ref variant ( #122830 )
...
This replaces `_PyList_FromArraySteal` with `_PyList_FromStackRefSteal`.
It's functionally equivalent, but takes a `_PyStackRef` array instead of
an array of `PyObject` pointers.
Co-authored-by: Ken Jin <kenjin@python.org>
2024-08-12 14:49:49 -04:00
736fe4d23e
gh-117139: Fix a few `_PyStackRef` related bugs ( #122831 )
...
`BUILD_SET` should use a borrow instead of a steal. The cleanup in `_DO_CALL`
`CONVERSION_FAILED` was incorrect.
Co-authored-by: Ken Jin <kenjin@python.org>
2024-08-12 14:49:33 -04:00
1795d6ceba
GH-122869: Add missing tier two optimizer cases (GH-122936)
2024-08-12 10:35:52 -07:00
1069190bad
gh-122860: Remove unused macro `_Py_atomic_load_relaxed_int32` ( #122861 )
2024-08-11 22:45:02 +05:30
2d9d3a9f53
gh-122697: Fix free-threading memory leaks at shutdown ( #122703 )
...
We were not properly accounting for interpreter memory leaks at
shutdown and had two sources of leaks:
* Objects that use deferred reference counting and were reachable via
static types outlive the final GC. We now disable deferred reference
counting on all objects if we are calling the GC due to interpreter
shutdown.
* `_PyMem_FreeDelayed` did not properly check for interpreter shutdown
so we had some memory blocks that were enqueued to be freed, but
never actually freed.
* `_PyType_FinalizeIdPool` wasn't called at interpreter shutdown.
2024-08-08 12:48:17 -04:00
833eb106f5
gh-122562: fix dump_symtable for ste_free and ste_child_free removal ( #122825 )
2024-08-08 14:03:45 +01:00
81c739e2dc
GH-120024: Tidy up case generator code a bit. (GH-122780)
2024-08-08 10:57:59 +01:00
57d7c3e78f
gh-122247: Move instruction instrumentation sanity check after tracing check ( #122251 )
2024-08-07 21:30:14 -07:00
540fcc62f5
gh-118814: Fix the TypeVar constructor when name is passed by keyword (GH-122664)
...
Fix _PyArg_UnpackKeywordsWithVararg for the case when argument for
positional-or-keyword parameter is passed by keyword.
There was only one such case in the stdlib -- the TypeVar constructor.
2024-08-07 23:30:10 +03:00
3e753c689a
gh-118926: Spill deferred references to stack in cases generator ( #122748 )
...
This automatically spills the results from `_PyStackRef_FromPyObjectNew`
to the in-memory stack so that the deferred references are visible to
the GC before we make any possibly escaping call.
Co-authored-by: Ken Jin <kenjin@python.org>
2024-08-07 13:23:53 -04:00
61a8bf2853
gh-122759: Remove `assert` from `RERAISE` error handling ( #122760 )
2024-08-07 17:25:25 +03:00
674a50ef2f
gh-117139: Fix an incorrect borrow in bytecodes.c ( #122318 )
...
`_PyDict_SetItem_Take2` steals both the key (i.e., `sub`) and the value.
2024-08-07 19:06:19 +05:30
4767a6e31c
gh-122728: Fix SystemError in PyEval_GetLocals() ( #122735 )
...
Fix PyEval_GetLocals() to avoid SystemError ("bad argument to
internal function"). Don't redefine the 'ret' variable in the if
block.
Add an unit test on PyEval_GetLocals().
2024-08-06 23:01:44 +02:00
dc09301067
gh-122417: Implement per-thread heap type refcounts ( #122418 )
...
The free-threaded build partially stores heap type reference counts in
distributed manner in per-thread arrays. This avoids reference count
contention when creating or destroying instances.
Co-authored-by: Ken Jin <kenjin@python.org>
2024-08-06 14:36:57 -04:00
4c31791848
GH-120024: Move three more escaping calls out of conditional statements (GH-122734)
2024-08-06 14:14:52 +01:00
a8be8fc6c4
GH-120024: Refactor code a bit so that escaping calls can be wrapped in spill code in code generator (GH-122693)
2024-08-06 08:40:39 +01:00
e74680b718
gh-122595: Add more error checks in the compiler (GH-122596)
2024-08-06 08:59:44 +03:00
5bd72912a1
GH-122616: Simplify LOAD_ATTR_WITH_HINT and STORE_ATTR_WITH_HINT (GH-122620)
2024-08-05 16:27:48 +01:00
efcd65cd84
gh-122313: Clean up deep recursion guarding code in the compiler (GH-122640)
...
Add ENTER_RECURSIVE and LEAVE_RECURSIVE macros in ast.c, ast_opt.c and
symtable.c. Remove VISIT_QUIT macro in symtable.c.
The current recursion depth counter only needs to be updated during
normal execution -- all functions should just return an error code
if an error occurs.
2024-08-03 12:45:45 +03:00
7aca84e557
GH-117224: Move the body of a few large-ish micro-ops into helper functions (GH-122601)
2024-08-02 16:31:17 +01:00
498376d7a7
gh-122445: populate only modified fields in __static_attributes__ ( #122446 )
2024-08-02 15:40:42 +01:00
9fc1c992d6
gh-122334: Fix crash when importing ssl after re-initialization ( #122481 )
...
* Fix crash when importing ssl after re-initialization
2024-08-02 19:06:20 +05:30
03b88522f5
gh-122188: Remove _imp.pyc_magic_number (GH-122503)
...
_imp.pyc_magic_number_token should be enough.
2024-08-02 13:12:19 +03:00
8234419c32
gh-122562: Remove ste_free and ste_child_free from symtable ( #122563 )
2024-08-01 16:28:25 -07:00
df13a1821a
GH-118095: Add tier two support for BINARY_SUBSCR_GETITEM (GH-120793)
2024-08-01 16:19:05 -07:00
fda6bd842a
Replace PyObject_Del with PyObject_Free ( #122453 )
...
PyObject_Del() is just a alias to PyObject_Free() kept for backward
compatibility. Use directly PyObject_Free() instead.
2024-08-01 14:12:33 +02:00
a9d56e38a0
GH-122155: Track local variables between pops and pushes in cases generator (GH-122286)
2024-08-01 09:27:26 +01:00
46f5a4f9e1
Fix typos in docs, error messages and comments ( #122502 )
...
Signed-off-by: jianghuyiyuan <shuangcui@live.com>
2024-08-01 00:26:09 +00:00
af0a00f022
gh-122188: Move magic number to its own file ( #122243 )
...
* gh-122188: Move magic number to its own file
* Add versionadded directive
* Do work in C
* Integrate launcher.c
* Make _pyc_magic_number private
* Remove metadata
* Move sys.implementation -> _imp
* Modernize comment
* Move _RAW_MAGIC_NUMBER to the C side as well
* _pyc_magic_number -> pyc_magic_number
* Remove unused import
* Update docs
* Apply suggestions from code review
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* Fix typo in tests
---------
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2024-07-30 15:31:05 -04:00
2b163aa9e7
gh-117657: Avoid race in `PAUSE_ADAPTIVE_COUNTER` in free-threaded build ( #122190 )
...
The adaptive counter doesn't do anything currently in the free-threaded
build and TSan reports a data race due to concurrent modifications to
the counter.
2024-07-30 13:53:47 -04:00
7797182b78
GH-118093: Improve handling of short and mid-loop traces (GH-122252)
2024-07-29 14:49:17 -07:00
15d4cd0967
GH-116090: Fire RAISE events from _FOR_ITER_TIER_TWO (GH-122413)
2024-07-29 12:17:47 -07:00
89fa05fdce
gh-122234: Add DECREFs to error paths ( #122406 )
...
Co-Authored-By: Kirill Podoprigora <kirill.bast9@mail.ru>
2024-07-29 16:10:25 +00:00
169e7138ab
gh-122234: fix accuracy issues for sum() ( #122236 )
...
* Use compensated summation for complex sums with floating-point items.
This amends #121176 .
* sum() specializations for floats and complexes now use
PyLong_AsDouble() instead of PyLong_AsLongAndOverflow() and
compensated summation as well.
2024-07-29 05:56:40 +02:00
ae192262ad
gh-119180: Add evaluate functions for type params and type aliases ( #122212 )
2024-07-27 17:24:10 +00:00
1ca99ed240
Manually override bytecode definition in optimizer, to avoid build error (GH-122316)
2024-07-26 18:38:52 +01:00
c557ae97d6
gh-122201: Lock mutex when setting handling_thread to NULL ( #122204 )
...
In the free-threaded build, we need to lock pending->mutex when clearing
the handling_thread in order not to race with a concurrent
make_pending_calls in the same interpreter.
2024-07-26 13:06:07 -04:00
64857d849f
GH-122294: Burn in the addresses of side exits (GH-122295)
2024-07-26 09:40:15 -07:00
bc94cf7e25
gh-122245: move checks for writes and shadowing of __debug__ to symtable ( #122246 )
2024-07-26 14:39:56 +01:00
95a73917cd
GH-122029: Break INSTRUMENTED_CALL into micro-ops, so that its behavior is consistent with CALL (GH-122177)
2024-07-26 14:35:57 +01:00
afb0aa6ed2
GH-121131: Clean up and fix some instrumented instructions. (GH-121132)
...
* Add support for 'prev_instr' to code generator and refactor some INSTRUMENTED instructions
2024-07-26 12:24:12 +01:00
d9efa45d74
GH-118093: Add tier two support for BINARY_OP_INPLACE_ADD_UNICODE (GH-122253)
2024-07-25 14:45:07 -07:00
5f6001130f
GH-118093: Add tier two support for LOAD_ATTR_PROPERTY (GH-122283)
2024-07-25 10:45:28 -07:00
5e686ff57d
GH-122034: Add StackRef variants of type checks to reduce the number of PyStackRef_AsPyObjectBorrow calls (GH-122037)
2024-07-25 18:32:43 +01:00
2e14a52cce
GH-122160: Remove BUILD_CONST_KEY_MAP opcode. (GH-122164)
2024-07-25 16:24:29 +01:00
9ac606080a
gh-121404: extract compiler_lookup_arg out of compiler_make_closure ( #122181 )
2024-07-24 17:22:18 +01:00
794546fd53
GH-118093: Remove invalidated executors from side exits (GH-121885)
2024-07-24 09:16:30 -07:00
e91ef13861
gh-122029: Log call events in sys.setprofile when it's a method with c function (GH-122072)
...
Log call events in sys.setprofile when it is a method with a C function.
2024-07-23 15:25:26 -07:00
7b7b90d1ce
gh-119180: Add `annotationlib` module to support PEP 649 ( #119891 )
...
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2024-07-23 21:16:50 +00:00
2762c6cc5e
gh-121637: Syntax error for optimized-away incorrect await ( #121656 )
...
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-07-22 14:12:43 -07:00
d5a12b4440
gh-121404: move calculation of module start location from compiler_body up to compiler_codegen ( #122127 )
2024-07-22 17:48:30 +01:00
5716cc3529
gh-100240: Use a consistent implementation for freelists ( #121934 )
...
This combines and updates our freelist handling to use a consistent
implementation. Objects in the freelist are linked together using the
first word of memory block.
If configured with freelists disabled, these operations are essentially
no-ops.
2024-07-22 12:08:27 -04:00
e059aa6b01
gh-120973: Fix thread-safety issues with `threading.local` ( #121655 )
...
This is a small refactoring to the current design that allows us to
avoid manually iterating over threads.
This should also fix gh-118490.
2024-07-19 13:22:02 -04:00
1a0c7b9ba4
gh-121905: Consistently use "floating-point" instead of "floating point" (GH-121907)
2024-07-19 08:06:02 +00:00
7b36b67b1e
GH-118093: Add tier two support to several instructions (GH-121884)
2024-07-18 14:24:58 -07:00
3eacfc1a4d
GH-121784: Generate an error during code gen if a variable is marked `unused`, but is used and thus cached in a prior uop. ( #121788 )
...
* Reject uop definitions that declare values as 'unused' that are already cached by prior uops
* Track which variables are defined and only load from memory when needed
* Support explicit `flush` in macro definitions.
* Make sure stack is flushed in where needed.
2024-07-18 12:49:24 +01:00
169324c27a
GH-120024: Use pointer for stack pointer (GH-121923)
2024-07-18 12:47:21 +01:00
81fd625b5c
gh-121621: Move asyncio_running_loop to private struct ( #121939 )
...
This avoids changing the ABI and keeps the field in the private struct.
2024-07-17 15:21:24 -07:00
f036a463db
GH-121583: Remove dependency from pystats.h to internal header file (GH-121587)
...
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
2024-07-16 15:38:29 -07:00
e65cb4c6f0
gh-118934: Make PyEval_GetLocals return borrowed reference ( #119769 )
...
Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
2024-07-16 12:17:47 -07:00
69c68de43a
gh-121621: Move asyncio running loop to thread state (GH-121695)
2024-07-17 01:09:58 +08:00
8549559f38
gh-120317: Lock around global state in the tokenize module ( #120318 )
...
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2024-07-16 11:35:57 +02:00
2b1b68939b
gh-121814: Only check f_trace_opcodes if Python frame exists ( #121818 )
...
Co-authored-by: Matt Wozniski <godlygeek@gmail.com>
2024-07-15 16:11:02 -07:00
Irit Katriel
Mark Shannon
Anthony Shaw
Sam Gross
Kirill Podoprigora
Donghee Na
Brandt Bucher
Pablo Galindo Salgado
Eric Snow
Tian Gao
Serhiy Storchaka
sobolevn
Victor Stinner
neonene
Jelle Zijlstra
jianghuyiyuan
Michael Droettboom
Petr Viktorin
Sergey B Kirpichev
mpage
Ken Jin
Lysandros Nikolaou