Commit Graph

14501 Commits

Author SHA1 Message Date
Sam Gross 5610860840
gh-126688: Reinit import lock after fork (#126692)
The PyMutex implementation supports unlocking after fork because we
clear the list of waiters in parking_lot.c. This doesn't work as well
for _PyRecursiveMutex because on some systems, such as SerenityOS, the
thread id is not preserved across fork().
2024-11-12 15:53:58 -05:00
Eric Snow 73cf069099
gh-76785: Improved Subinterpreters Compatibility with 3.12 (2/2) (gh-126707)
These changes makes it easier to backport the _interpreters, _interpqueues, and _interpchannels modules to Python 3.12.

This involves the following:

* add the _PyXI_GET_STATE() and _PyXI_GET_GLOBAL_STATE() macros
* add _PyXIData_lookup_context_t and _PyXIData_GetLookupContext()
* add _Py_xi_state_init() and _Py_xi_state_fini()
2024-11-12 10:41:51 -07:00
Serhiy Storchaka 6b2a19681e
gh-95382: Use cache for indentations in the JSON encoder (GH-118636) 2024-11-12 17:19:15 +00:00
Hugo van Kemenade 91f4908798
gh-126133: Only use start year in PSF copyright, remove end years (#126236) 2024-11-12 15:59:19 +02:00
RUANG (James Roy) 8ff7efb46d
gh-126061: Add PyLong_IsPositive/Zero/Negative() functions (#126065)
Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-11-12 14:18:06 +01:00
Sayandip Dutta abb90ba46c
gh-125916: Allow functools.reduce() 'initial' to be a keyword argument (#125917) 2024-11-12 13:11:58 +00:00
Bénédikt Tran 6e3bb8a913
gh-126595: fix a crash when calling `itertools.count(sys.maxsize)` (#126617) 2024-11-12 14:10:10 +01:00
Kumar Aditya 37c57dfad1
gh-126405: fix use-after-free in `_asyncio.Future.remove_done_callback` (#126733) 2024-11-12 18:01:34 +05:30
Eric Snow a6d48e8f83
gh-76785: Improved Subinterpreters Compatibility with 3.12 (1/2) (gh-126704)
These changes makes it easier to backport the _interpreters, _interpqueues, and _interpchannels modules to Python 3.12.

This involves the following:

* rename several structs and typedefs
* add several typedefs
* stop using the PyThreadState.state field directly in parking_lot.c
2024-11-11 15:58:46 -07:00
Eric Snow b697d8c48e
gh-76785: Minor Cleanup of Exception-related Cross-interpreter State (gh-126602)
This change makes it easier to backport the _interpreters, _interpqueues, and _interpchannels modules to Python 3.12.
2024-11-11 14:49:41 -07:00
sobolevn 9fc2808eaf
gh-126654: Fix crash in several functions in `_interpreters` module (#126678) 2024-11-11 17:05:56 +05:30
Kumar Aditya b19d12f447
remove minor redundant code from `_asyncio` (#126578) 2024-11-08 13:17:03 +00:00
Kumar Aditya a3e8e7bbc3
GH-107803: use circular double linked list for tasks in `_asyncio` (#126577) 2024-11-08 12:51:11 +00:00
Serhiy Storchaka 061e50f196
gh-122943: Add the varpos parameter in _PyArg_UnpackKeywords (GH-126564)
Remove _PyArg_UnpackKeywordsWithVararg.
Add comments for integer arguments of _PyArg_UnpackKeywords.
2024-11-08 14:23:50 +02:00
Serhiy Storchaka 06a8b0bb5e
gh-122943: Remove the object converter for var-positional parameter (GH-126560) 2024-11-08 05:41:54 +00:00
Serhiy Storchaka 1f777396f5
gh-122943: Rework support of var-positional parameter in Argument Clinic (GH-122945)
Move creation of a tuple for var-positional parameter out of
_PyArg_UnpackKeywordsWithVararg().
Merge _PyArg_UnpackKeywordsWithVararg() with _PyArg_UnpackKeywords().
Add a new parameter in _PyArg_UnpackKeywords().

The "parameters" and "converters" attributes of ParseArgsCodeGen no
longer contain the var-positional parameter. It is now available as the
"varpos" attribute. Optimize code generation for var-positional
parameter and reuse the same generating code for functions with and without
keyword parameters.

Add special converters for var-positional parameter. "tuple" represents it as
a Python tuple and "array" represents it as a continuous array of PyObject*.
"object" is a temporary alias of "tuple".
2024-11-07 23:40:03 +02:00
Eric Snow 9357fdcaf0
gh-76785: Minor Cleanup of "Cross-interpreter" Code (gh-126457)
The primary objective here is to allow some later changes to be cleaner. Mostly this involves renaming things and moving a few things around.

* CrossInterpreterData -> XIData
* crossinterpdatafunc -> xidatafunc
* split out pycore_crossinterp_data_registry.h
* add _PyXIData_lookup_t
2024-11-07 09:32:42 -07:00
Gregory P. Smith d46d3f2ec7
Cleanup multiprocessing comment and unusual import error message (#126532)
Define constants as constants rather than calling `list(range(2))`.
Explain which values must remain in sync via comments.
2024-11-07 00:06:14 -08:00
Serhiy Storchaka 223d3dc554
gh-125631: Enable setting persistent_id and persistent_load of pickler and unpickler (GH-125752)
pickle.Pickler.persistent_id and pickle.Unpickler.persistent_load can
again be overridden as instance attributes.
2024-11-07 08:53:02 +02:00
Victor Stinner a1c57bcfd2
gh-126461: Fix _Unpickler_ReadFromFile() error handling (#126485)
Handle _Unpickler_SetStringInput() failure.
2024-11-06 14:24:46 +01:00
Victor Stinner b1c4ffc205
gh-126455: Disallow _ssl.SSLSocket instantiation (#126481)
Prevent creation of incomplete/invalid _ssl.SSLSocket objects when
created directly.
2024-11-06 11:59:39 +01:00
Victor Stinner 09d7083962
gh-126433: Change channel_info.count to int64_t (#126447)
Fix compiler warnings on 32-bit Windows: change channel_info.count
type from Py_ssize_t to int64_t in _interpchannelsmodule.c.
2024-11-06 15:50:17 +05:30
Victor Stinner 0b67ce930a
gh-126433: Fix compiler warnings on 32-bit Windows (#126444) 2024-11-05 16:05:13 +01:00
sobolevn 75872605aa
gh-126425: Refactor `_lsprof_Profiler_enable` (#126426)
- Explicit memory management for `None` objects (since we still try to treat immortal objects as regular objects)
- Respect possible errors of `sys.monitoring.register_callback` call
2024-11-05 15:23:24 +03:00
Lukas Geiger 3d4fda2165
gh-119793: Prefer `map(..., strict=True)` over starmap/zip in examples (#126407) 2024-11-05 08:53:32 +01:00
Serhiy Storchaka d3840503b0
gh-126303: Fix pickling and copying of os.sched_param objects (GH-126336) 2024-11-05 08:23:17 +02:00
mpage 2e95c5ba3b
gh-115999: Implement thread-local bytecode and enable specialization for `BINARY_OP` (#123926)
Each thread specializes a thread-local copy of the bytecode, created on the first RESUME, in free-threaded builds. All copies of the bytecode for a code object are stored in the co_tlbc array on the code object. Threads reserve a globally unique index identifying its copy of the bytecode in all co_tlbc arrays at thread creation and release the index at thread destruction. The first entry in every co_tlbc array always points to the "main" copy of the bytecode that is stored at the end of the code object. This ensures that no bytecode is copied for programs that do not use threads.

Thread-local bytecode can be disabled at runtime by providing either -X tlbc=0 or PYTHON_TLBC=0. Disabling thread-local bytecode also disables specialization.

Concurrent modifications to the bytecode made by the specializing interpreter and instrumentation use atomics, with specialization taking care not to overwrite an instruction that was instrumented concurrently.
2024-11-04 11:13:32 -08:00
sobolevn c806cd5af6
gh-126220: Adapt `_lsprof` to Argument Clinic (#126233)
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-11-04 19:18:21 +03:00
Bénédikt Tran 19d9358437
gh-126313: Fix a crash in curses.napms() due to incorrect error handling (GH-126351) 2024-11-03 15:08:34 +00:00
Nico-Posada f032f6ba8f
gh-126138: Fix use-after-free in `_asyncio.Task` by evil `__getattribute__` (#126305)
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
2024-11-02 13:16:00 +05:30
Sergey B Kirpichev 8477951a1c
gh-120026: soft deprecate Py_HUGE_VAL macro (#120027)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2024-11-01 22:04:31 +00:00
sobolevn 28b148fb32
gh-126220: Fix crash on calls to `_lsprof.Profiler` methods with 0 args (backportable) (#126271)
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-11-02 00:53:29 +03:00
Cody Maloney 72dd4714f9
gh-120754: _io Ensure stat cache is cleared on fd change (#125166)
Performed an audit of `fileio.c` and `_pyio` and made sure anytime the
fd changes the stat result, if set, is also cleared/changed.

There's one case where it's not cleared, if code would clear it in
__init__, keep the memory allocated and just do another fstat with the
existing memory.
2024-11-01 22:50:49 +01:00
Sergey B Kirpichev 51ef54abc4
gh-125916: Adapt functools.reduce() to Argument Clinic (#125999) 2024-11-01 21:15:39 +01:00
Bénédikt Tran 0e8665554b
gh-126080: fix UAF on `task->task_context` in `task_call_step_soon` due to an evil `loop.__getattribute__` (#126120) 2024-10-31 10:14:47 -07:00
Sergey B Kirpichev 3275cb1953
gh-101123: Adapt vararg functions in the math module to Argument Clinic (#126235)
This implicitly fixes the math.hypot signature, which was previously
incomprehensible to inspect.signature().
2024-10-31 14:37:47 +00:00
Peter Bierma 01415213d7
gh-126223: Propagate unicode errors in `_interpreters.create()` (#126224)
Co-authored-by: sobolevn <mail@sobolevn.me>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-10-31 17:14:37 +03:00
Sergey B Kirpichev 8c22eba877
gh-90370: Argument Clinic: avoid temporary tuple creation for varargs (#126064)
Avoid temporary tuple creation when all arguments either positional-only
or vararg.

Objects/setobject.c and Modules/gcmodule.c adapted. This fixes slight
performance regression for set methods, introduced by gh-115112.
2024-10-31 11:37:03 +01:00
Nico-Posada d07dcce693
gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (#126103) 2024-10-31 10:47:57 +03:00
sobolevn a64a1c9206
gh-126106: Fix `NULL` possible derefrence in `Modules/_ssl.c` (#126111)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-10-29 12:03:18 +03:00
Sergey B Kirpichev 9b14083497
Align functools.reduce() docstring with PEP-257 (#126045)
Yak-shave in preparation for Argument Clinic adaption in gh-125999.
2024-10-29 08:08:08 +00:00
Bénédikt Tran 19e93e2e26
gh-126035: add missing whitespace to *Py_EnterRecursiveCall() messages (#126036) 2024-10-27 22:55:48 +01:00
Bénédikt Tran ed5059eeb1
gh-125966: fix use-after-free on `fut->fut_callback0` due to an evil callback's `__eq__` in asyncio (#125967) 2024-10-27 22:40:10 +05:30
Bénédikt Tran f819d4301d
gh-125984: fix use-after-free on `fut->fut_{callback,context}0` due to an evil `loop.__getattribute__` (#126003) 2024-10-27 20:34:43 +05:30
Bénédikt Tran f6cc7c8bd0
gh-94512: Fix forced arg format in posixmodule.c clinic code (#122516) 2024-10-26 23:40:31 +02:00
Bénédikt Tran c5b99f5c2c
gh-125969: fix OOB in `future_schedule_callbacks` due to an evil `call_soon` (#125970)
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
2024-10-25 23:45:09 +05:30
Kumar Aditya cae853e3b4
GH-125789: fix `fut._callbacks` to always return a copy of callbacks (#125922)
Fix `asyncio.Future._callbacks` to always return a copy of the internal list of callbacks to avoid mutation from user code affecting the internal state.
2024-10-25 18:19:30 +05:30
Victor Stinner ebcc578dff
gh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_STR) (#125583)
Replace PyUnicode_FromStringAndSize(NULL, 0)
with Py_GetConstant(Py_CONSTANT_EMPTY_STR).
2024-10-25 11:14:52 +02:00
Victor Stinner db96327203
gh-121654: Add PyType_Freeze() function (#122457)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-10-25 11:12:48 +02:00
Sam Gross 332356b880
gh-125900: Clean-up logic around immortalization in free-threading (#125901)
* Remove `@suppress_immortalization` decorator
* Make suppression flag per-thread instead of per-interpreter
* Suppress immortalization in `eval()` to avoid refleaks in three tests
  (test_datetime.test_roundtrip, test_logging.test_config8_ok, and
   test_random.test_after_fork).
* frozenset() is constant, but not a singleton. When run multiple times,
  the test could fail due to constant interning.
2024-10-24 18:09:59 -04:00