Commit Graph

9013 Commits

Author SHA1 Message Date
mpage e059aa6b01
gh-120973: Fix thread-safety issues with `threading.local` (#121655)
This is a small refactoring to the current design that allows us to
avoid manually iterating over threads.

This should also fix gh-118490.
2024-07-19 13:22:02 -04:00
Serhiy Storchaka 1a0c7b9ba4
gh-121905: Consistently use "floating-point" instead of "floating point" (GH-121907) 2024-07-19 08:06:02 +00:00
Brandt Bucher 7b36b67b1e
GH-118093: Add tier two support to several instructions (GH-121884) 2024-07-18 14:24:58 -07:00
Mark Shannon 3eacfc1a4d
GH-121784: Generate an error during code gen if a variable is marked `unused`, but is used and thus cached in a prior uop. (#121788)
* Reject uop definitions that declare values as 'unused' that are already cached by prior uops

* Track which variables are defined and only load from memory when needed

* Support explicit `flush` in macro definitions. 

* Make sure stack is flushed in where needed.
2024-07-18 12:49:24 +01:00
Mark Shannon 169324c27a
GH-120024: Use pointer for stack pointer (GH-121923) 2024-07-18 12:47:21 +01:00
Sam Gross 81fd625b5c
gh-121621: Move asyncio_running_loop to private struct (#121939)
This avoids changing the ABI and keeps the field in the private struct.
2024-07-17 15:21:24 -07:00
Michael Droettboom f036a463db
GH-121583: Remove dependency from pystats.h to internal header file (GH-121587)
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
2024-07-16 15:38:29 -07:00
Tian Gao e65cb4c6f0
gh-118934: Make PyEval_GetLocals return borrowed reference (#119769)
Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
2024-07-16 12:17:47 -07:00
Ken Jin 69c68de43a
gh-121621: Move asyncio running loop to thread state (GH-121695) 2024-07-17 01:09:58 +08:00
Lysandros Nikolaou 8549559f38
gh-120317: Lock around global state in the tokenize module (#120318)
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2024-07-16 11:35:57 +02:00
Tian Gao 2b1b68939b
gh-121814: Only check f_trace_opcodes if Python frame exists (#121818)
Co-authored-by: Matt Wozniski <godlygeek@gmail.com>
2024-07-15 16:11:02 -07:00
Sam Gross d23be3947c
gh-121794: Don't set `ob_tid` to zero in fast-path dealloc (#121799)
We should maintain the invariant that a zero `ob_tid` implies the
refcount fields are merged.

* Move the assignment in `_Py_MergeZeroLocalRefcount` to immediately
  before the refcount merge.
* Update `_PyTrash_thread_destroy_chain` to set `ob_ref_shared` to
  `_Py_REF_MERGED` when setting `ob_tid` to zero.

Also check this invariant with assertions in the GC in debug builds.
That uncovered a bug when running out of memory during GC.
2024-07-15 17:50:10 -04:00
Ken Jin e904300882
gh-121546: Disable contextvar caching on free-threading build (GH-121740) 2024-07-16 02:57:58 +08:00
Eric Snow 985dd8e17b
gh-118297: Make Sure All Pending Calls Run in _Py_FinishPendingCalls() (gh-118298) 2024-07-15 12:44:20 -06:00
Hood Chatham cae1526716
gh-121698 Emscripten: Use updated WebAssembly type reflection proposal (GH-121699) 2024-07-14 11:25:09 +02:00
Gregor 178e44de8f
gh-121657: Display correct error message for yield from outside of a function (GH-121680)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-07-13 17:14:39 +02:00
Serhiy Storchaka 18015451d0
gh-121153: Fix some errors with use of _PyLong_CompactValue() (GH-121154)
* The result has type Py_ssize_t, not intptr_t.
* Type cast between unsigned and signdet integer types should be explicit.
* Downcasting should be explicit.
* Fix integer overflow check in sum().
2024-07-13 13:40:44 +03:00
Jelle Zijlstra 58e8cf2bb6
gh-121332: Make AST node constructor check _attributes instead of hardcoding attributes (#121334) 2024-07-11 14:34:53 +00:00
Irit Katriel 6557af6698
gh-121554: remove unnecessary internal functions in compile.c (#121555)
Co-authored-by: Erlend E. Aasland <erlend@python.org>
2024-07-10 23:48:37 +01:00
Irit Katriel 0177a34335
gh-121404: remove some accesses to compiler internals from codegen functions (#121538) 2024-07-10 17:09:45 +01:00
Irit Katriel cced22c760
gh-121547: deduplicate the code of const_cache update functions (#121548) 2024-07-10 10:34:54 +00:00
Eric Snow 15d48aea02
gh-121110: Fix Extension Module Tests Under Py_TRACE_REFS Builds (gh-121503)
The change in gh-118157 (b2cd54a) should have also updated clear_singlephase_extension() but didn't.  We fix that here.  Note that clear_singlephase_extension() (AKA _PyImport_ClearExtension()) is only used in tests.
2024-07-08 15:10:00 -06:00
Irit Katriel 2be37ec8e2
gh-121404: remove direct accesses to u_private from codegen functions (#121500) 2024-07-08 21:02:01 +01:00
Sam Gross 1d3cf79a50
gh-121368: Fix seq lock memory ordering in _PyType_Lookup (#121388)
The `_PySeqLock_EndRead` function needs an acquire fence to ensure that
the load of the sequence happens after any loads within the read side
critical section. The missing fence can trigger bugs on macOS arm64.

Additionally, we need a release fence in `_PySeqLock_LockWrite` to
ensure that the sequence update is visible before any modifications to
the cache entry.
2024-07-08 14:52:07 -04:00
Mark Shannon 8ad6067bd4
GH-121012: Set index to -1 when list iterators become exhausted in tier 2 (GH-121483) 2024-07-08 14:20:13 +01:00
Michael Droettboom d69529d31c
gh-121338: Remove #pragma optimize (#121340) 2024-07-08 08:48:42 -04:00
Sergey B Kirpichev d4faa7bd32
gh-121149: improve accuracy of builtin sum() for complex inputs (gh-121176) 2024-07-05 10:01:05 -05:00
Josh Brobst db39bc42f9
gh-121390: tracemalloc: Fix tracebacks memory leak (#121391)
The tracemalloc_tracebacks hash table has traceback keys and NULL
values, but its destructors do not reflect this -- key_destroy_func is
NULL while value_destroy_func is raw_free. Swap these to free the
traceback keys instead.
2024-07-05 08:39:48 +02:00
Irit Katriel 67a05de17c
gh-121272: move async for/with validation from compiler to symtable (#121361) 2024-07-04 14:47:21 +01:00
Irit Katriel 19d1e43e43
gh-121352: use _Py_SourceLocation in symtable (#121353) 2024-07-04 11:28:44 +01:00
Bénédikt Tran 9728ead361
gh-121141: add support for `copy.replace` to AST nodes (#121162) 2024-07-03 20:10:54 -07:00
Irit Katriel 93156880ef
gh-121272: set ste_coroutine during symtable construction (#121297)
compiler no longer modifies the symtable after this.
2024-07-03 10:18:34 +01:00
Sam Gross 8e8d202f55
gh-117139: Add _PyTuple_FromStackRefSteal and use it (#121244)
Avoids the extra conversion from stack refs to PyObjects.
2024-07-02 12:30:14 -04:00
Irit Katriel 1ac273224a
gh-121272: move __future__ import validation from compiler to symtable (#121273) 2024-07-02 16:22:08 +00:00
Brandt Bucher 33903c53db
GH-116017: Get rid of _COLD_EXITs (GH-120960) 2024-07-01 13:17:40 -07:00
Victor Stinner a0b8b342c5
gh-121199: Use _Py__has_attribute() in timemodule.c (#121203)
Use the _Py__has_attribute() macro in timemodule.c and
bootstrap_hash.c to fix a build error on old GCC versions (GCC 4.8.5
on s390x).
2024-07-01 08:49:33 +00:00
Kirill Podoprigora 1a84bdc237
gh-121163: Add "all" as an valid alias for "always" in warnings.simplefilter() (#121164)
Add support for ``all`` as an valid alias for ``always`` in ``warnings.simplefilter()``
and ``warnings.filterswarnings()``.
2024-06-30 19:48:00 +02:00
Ken Jin e6543daf12
gh-117139: Fix a few wrong steals in bytecodes.c (GH-121127)
Fix a few wrong steals in bytecodes.c
2024-06-29 02:14:48 +08:00
Donghee Na 1a2e7a7475
gh-120837: Update _Py_DumpExtensionModules to be async-signal-safe (gh-121051) 2024-06-28 06:46:46 +09:00
Nadeshiko Manju 223c03a43c
gh-121082: Fix build failure when the developer use `--enable-pystats` arguments in configuration command after #118450 (#121083)
Signed-off-by: Manjusaka <me@manjusaka.me>
Co-authored-by: Ken Jin <kenjin4096@gmail.com>
2024-06-27 19:35:25 +08:00
Victor Stinner 12af8ec864
gh-121040: Use __attribute__((fallthrough)) (#121044)
Fix warnings when using -Wimplicit-fallthrough compiler flag.

Annotate explicitly "fall through" switch cases with a new
_Py_FALLTHROUGH macro which uses __attribute__((fallthrough)) if
available. Replace "fall through" comments with _Py_FALLTHROUGH.

Add _Py__has_attribute() macro. No longer define __has_attribute()
macro if it's not defined. Move also _Py__has_builtin() at the top
of pyport.h.

Co-Authored-By: Nikita Sobolev <mail@sobolevn.me>
2024-06-27 09:58:44 +00:00
Eric Snow 4be1f37b20
gh-113433: Automatically Clean Up Subinterpreters in Py_Finalize() (gh-121060)
This change makes things a little less painful for some users.  It also fixes a failing assert (gh-120765), by making sure all subinterpreters are destroyed before the main interpreter.  As part of that, we make sure Py_Finalize() always runs with the main interpreter active.
2024-06-26 21:17:26 +00:00
Ken Jin 22b0de2755
gh-117139: Convert the evaluation stack to stack refs (#118450)
This PR sets up tagged pointers for CPython.

The general idea is to create a separate struct _PyStackRef for everything on the evaluation stack to store the bits. This forces the C compiler to warn us if we try to cast things or pull things out of the struct directly.

Only for free threading: We tag the low bit if something is deferred - that means we skip incref and decref operations on it. This behavior may change in the future if Mark's plans to defer all objects in the interpreter loop pans out.

This implies a strict stack reference discipline is required. ALL incref and decref operations on stackrefs must use the stackref variants. It is unsafe to untag something then do normal incref/decref ops on it.

The new incref and decref variants are called dup and close. They mimic a "handle" API operating on these stackrefs.

Please read Include/internal/pycore_stackref.h for more information!

---------

Co-authored-by: Mark Shannon <9448417+markshannon@users.noreply.github.com>
2024-06-27 03:10:43 +08:00
Victor Stinner 9e4a81f00f
gh-120642: Move private PyCode APIs to the internal C API (#120643)
* Move _Py_CODEUNIT and related functions to pycore_code.h.
* Move _Py_BackoffCounter to pycore_backoff.h.
* Move Include/cpython/optimizer.h content to pycore_optimizer.h.
* Remove Include/cpython/optimizer.h.
* Remove PyUnstable_Replace_Executor().

Rename functions:

* PyUnstable_GetExecutor() => _Py_GetExecutor()
* PyUnstable_GetOptimizer() => _Py_GetOptimizer()
* PyUnstable_SetOptimizer() => _Py_SetTier2Optimizer()
* PyUnstable_Optimizer_NewCounter() => _PyOptimizer_NewCounter()
* PyUnstable_Optimizer_NewUOpOptimizer() => _PyOptimizer_NewUOpOptimizer()
2024-06-26 13:54:03 +02:00
Eric Snow a905721b9c
gh-120838: Add _PyThreadState_WHENCE_FINI (gh-121010)
We also add _PyThreadState_NewBound() and drop _PyThreadState_SetWhence().

This change only affects internal API.
2024-06-25 14:35:12 -06:00
Mark Shannon 8f5a01707f
GH-120982: Add stack check assertions to generated interpreter code (GH-120992) 2024-06-25 16:42:29 +01:00
Jelle Zijlstra 42b2c9d78d
gh-120108: Fix deepcopying of AST trees with .parent attributes (#120114) 2024-06-25 08:12:11 -07:00
Steve Dower e731554337
Fixes loop variables to be the same types as their limit (GH-120958) 2024-06-24 17:11:47 +01:00
Brandt Bucher a47abdb45d
GH-117062: Make _JUMP_TO_TOP a general absolute jump (GH-120854) 2024-06-24 08:35:10 -07:00
Irit Katriel 65a12c559c
gh-120834: fix type of *_iframe field in _PyGenObject_HEAD declaration (#120835) 2024-06-24 10:23:38 +01:00