Commit Graph

14488 Commits

Author SHA1 Message Date
Serhiy Storchaka 061e50f196
gh-122943: Add the varpos parameter in _PyArg_UnpackKeywords (GH-126564)
Remove _PyArg_UnpackKeywordsWithVararg.
Add comments for integer arguments of _PyArg_UnpackKeywords.
2024-11-08 14:23:50 +02:00
Serhiy Storchaka 06a8b0bb5e
gh-122943: Remove the object converter for var-positional parameter (GH-126560) 2024-11-08 05:41:54 +00:00
Serhiy Storchaka 1f777396f5
gh-122943: Rework support of var-positional parameter in Argument Clinic (GH-122945)
Move creation of a tuple for var-positional parameter out of
_PyArg_UnpackKeywordsWithVararg().
Merge _PyArg_UnpackKeywordsWithVararg() with _PyArg_UnpackKeywords().
Add a new parameter in _PyArg_UnpackKeywords().

The "parameters" and "converters" attributes of ParseArgsCodeGen no
longer contain the var-positional parameter. It is now available as the
"varpos" attribute. Optimize code generation for var-positional
parameter and reuse the same generating code for functions with and without
keyword parameters.

Add special converters for var-positional parameter. "tuple" represents it as
a Python tuple and "array" represents it as a continuous array of PyObject*.
"object" is a temporary alias of "tuple".
2024-11-07 23:40:03 +02:00
Eric Snow 9357fdcaf0
gh-76785: Minor Cleanup of "Cross-interpreter" Code (gh-126457)
The primary objective here is to allow some later changes to be cleaner. Mostly this involves renaming things and moving a few things around.

* CrossInterpreterData -> XIData
* crossinterpdatafunc -> xidatafunc
* split out pycore_crossinterp_data_registry.h
* add _PyXIData_lookup_t
2024-11-07 09:32:42 -07:00
Gregory P. Smith d46d3f2ec7
Cleanup multiprocessing comment and unusual import error message (#126532)
Define constants as constants rather than calling `list(range(2))`.
Explain which values must remain in sync via comments.
2024-11-07 00:06:14 -08:00
Serhiy Storchaka 223d3dc554
gh-125631: Enable setting persistent_id and persistent_load of pickler and unpickler (GH-125752)
pickle.Pickler.persistent_id and pickle.Unpickler.persistent_load can
again be overridden as instance attributes.
2024-11-07 08:53:02 +02:00
Victor Stinner a1c57bcfd2
gh-126461: Fix _Unpickler_ReadFromFile() error handling (#126485)
Handle _Unpickler_SetStringInput() failure.
2024-11-06 14:24:46 +01:00
Victor Stinner b1c4ffc205
gh-126455: Disallow _ssl.SSLSocket instantiation (#126481)
Prevent creation of incomplete/invalid _ssl.SSLSocket objects when
created directly.
2024-11-06 11:59:39 +01:00
Victor Stinner 09d7083962
gh-126433: Change channel_info.count to int64_t (#126447)
Fix compiler warnings on 32-bit Windows: change channel_info.count
type from Py_ssize_t to int64_t in _interpchannelsmodule.c.
2024-11-06 15:50:17 +05:30
Victor Stinner 0b67ce930a
gh-126433: Fix compiler warnings on 32-bit Windows (#126444) 2024-11-05 16:05:13 +01:00
sobolevn 75872605aa
gh-126425: Refactor `_lsprof_Profiler_enable` (#126426)
- Explicit memory management for `None` objects (since we still try to treat immortal objects as regular objects)
- Respect possible errors of `sys.monitoring.register_callback` call
2024-11-05 15:23:24 +03:00
Lukas Geiger 3d4fda2165
gh-119793: Prefer `map(..., strict=True)` over starmap/zip in examples (#126407) 2024-11-05 08:53:32 +01:00
Serhiy Storchaka d3840503b0
gh-126303: Fix pickling and copying of os.sched_param objects (GH-126336) 2024-11-05 08:23:17 +02:00
mpage 2e95c5ba3b
gh-115999: Implement thread-local bytecode and enable specialization for `BINARY_OP` (#123926)
Each thread specializes a thread-local copy of the bytecode, created on the first RESUME, in free-threaded builds. All copies of the bytecode for a code object are stored in the co_tlbc array on the code object. Threads reserve a globally unique index identifying its copy of the bytecode in all co_tlbc arrays at thread creation and release the index at thread destruction. The first entry in every co_tlbc array always points to the "main" copy of the bytecode that is stored at the end of the code object. This ensures that no bytecode is copied for programs that do not use threads.

Thread-local bytecode can be disabled at runtime by providing either -X tlbc=0 or PYTHON_TLBC=0. Disabling thread-local bytecode also disables specialization.

Concurrent modifications to the bytecode made by the specializing interpreter and instrumentation use atomics, with specialization taking care not to overwrite an instruction that was instrumented concurrently.
2024-11-04 11:13:32 -08:00
sobolevn c806cd5af6
gh-126220: Adapt `_lsprof` to Argument Clinic (#126233)
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-11-04 19:18:21 +03:00
Bénédikt Tran 19d9358437
gh-126313: Fix a crash in curses.napms() due to incorrect error handling (GH-126351) 2024-11-03 15:08:34 +00:00
Nico-Posada f032f6ba8f
gh-126138: Fix use-after-free in `_asyncio.Task` by evil `__getattribute__` (#126305)
Co-authored-by: Carol Willing <carolcode@willingconsulting.com>
2024-11-02 13:16:00 +05:30
Sergey B Kirpichev 8477951a1c
gh-120026: soft deprecate Py_HUGE_VAL macro (#120027)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2024-11-01 22:04:31 +00:00
sobolevn 28b148fb32
gh-126220: Fix crash on calls to `_lsprof.Profiler` methods with 0 args (backportable) (#126271)
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2024-11-02 00:53:29 +03:00
Cody Maloney 72dd4714f9
gh-120754: _io Ensure stat cache is cleared on fd change (#125166)
Performed an audit of `fileio.c` and `_pyio` and made sure anytime the
fd changes the stat result, if set, is also cleared/changed.

There's one case where it's not cleared, if code would clear it in
__init__, keep the memory allocated and just do another fstat with the
existing memory.
2024-11-01 22:50:49 +01:00
Sergey B Kirpichev 51ef54abc4
gh-125916: Adapt functools.reduce() to Argument Clinic (#125999) 2024-11-01 21:15:39 +01:00
Bénédikt Tran 0e8665554b
gh-126080: fix UAF on `task->task_context` in `task_call_step_soon` due to an evil `loop.__getattribute__` (#126120) 2024-10-31 10:14:47 -07:00
Sergey B Kirpichev 3275cb1953
gh-101123: Adapt vararg functions in the math module to Argument Clinic (#126235)
This implicitly fixes the math.hypot signature, which was previously
incomprehensible to inspect.signature().
2024-10-31 14:37:47 +00:00
Peter Bierma 01415213d7
gh-126223: Propagate unicode errors in `_interpreters.create()` (#126224)
Co-authored-by: sobolevn <mail@sobolevn.me>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-10-31 17:14:37 +03:00
Sergey B Kirpichev 8c22eba877
gh-90370: Argument Clinic: avoid temporary tuple creation for varargs (#126064)
Avoid temporary tuple creation when all arguments either positional-only
or vararg.

Objects/setobject.c and Modules/gcmodule.c adapted. This fixes slight
performance regression for set methods, introduced by gh-115112.
2024-10-31 11:37:03 +01:00
Nico-Posada d07dcce693
gh-126083: Fix a reference leak in `asyncio.Task` when reinitializing with new non-`None` context (#126103) 2024-10-31 10:47:57 +03:00
sobolevn a64a1c9206
gh-126106: Fix `NULL` possible derefrence in `Modules/_ssl.c` (#126111)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-10-29 12:03:18 +03:00
Sergey B Kirpichev 9b14083497
Align functools.reduce() docstring with PEP-257 (#126045)
Yak-shave in preparation for Argument Clinic adaption in gh-125999.
2024-10-29 08:08:08 +00:00
Bénédikt Tran 19e93e2e26
gh-126035: add missing whitespace to *Py_EnterRecursiveCall() messages (#126036) 2024-10-27 22:55:48 +01:00
Bénédikt Tran ed5059eeb1
gh-125966: fix use-after-free on `fut->fut_callback0` due to an evil callback's `__eq__` in asyncio (#125967) 2024-10-27 22:40:10 +05:30
Bénédikt Tran f819d4301d
gh-125984: fix use-after-free on `fut->fut_{callback,context}0` due to an evil `loop.__getattribute__` (#126003) 2024-10-27 20:34:43 +05:30
Bénédikt Tran f6cc7c8bd0
gh-94512: Fix forced arg format in posixmodule.c clinic code (#122516) 2024-10-26 23:40:31 +02:00
Bénédikt Tran c5b99f5c2c
gh-125969: fix OOB in `future_schedule_callbacks` due to an evil `call_soon` (#125970)
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
2024-10-25 23:45:09 +05:30
Kumar Aditya cae853e3b4
GH-125789: fix `fut._callbacks` to always return a copy of callbacks (#125922)
Fix `asyncio.Future._callbacks` to always return a copy of the internal list of callbacks to avoid mutation from user code affecting the internal state.
2024-10-25 18:19:30 +05:30
Victor Stinner ebcc578dff
gh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_STR) (#125583)
Replace PyUnicode_FromStringAndSize(NULL, 0)
with Py_GetConstant(Py_CONSTANT_EMPTY_STR).
2024-10-25 11:14:52 +02:00
Victor Stinner db96327203
gh-121654: Add PyType_Freeze() function (#122457)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-10-25 11:12:48 +02:00
Sam Gross 332356b880
gh-125900: Clean-up logic around immortalization in free-threading (#125901)
* Remove `@suppress_immortalization` decorator
* Make suppression flag per-thread instead of per-interpreter
* Suppress immortalization in `eval()` to avoid refleaks in three tests
  (test_datetime.test_roundtrip, test_logging.test_config8_ok, and
   test_random.test_after_fork).
* frozenset() is constant, but not a singleton. When run multiple times,
  the test could fail due to constant interning.
2024-10-24 18:09:59 -04:00
Eric Snow 4848b0b92c
gh-125716: Use A Global Mutex When Initializing Global State For The _interpqueues Module (gh-125803)
This includes a drive-by cleanup in _queues_init() and _queues_fini().

This change also applies to the _interpchannels module.
2024-10-21 15:49:58 -06:00
Eric Snow 44f841f01a
gh-125716: Raise an Exception If _globals_init() Fails In the _interpqueues Module (gh-125802)
The fix applies to the _interpchannels module as well.

I've also included a drive-by typo fix for _interpqueues.
2024-10-21 19:39:07 +00:00
Serhiy Storchaka dcc4fb2c90
gh-124969: Make locale.nl_langinfo(locale.ALT_DIGITS) returning a string again (GH-125774)
This is a follow up of GH-124974. Only Glibc needed a fix.
Now the returned value is a string consisting of semicolon-separated
symbols on all Posix platforms.
2024-10-21 18:54:12 +00:00
Peter Bierma 4c53b25775
gh-124984: Enhance `ssl` thread safety (#124993)
Make SSL objects thread safe in Free Theaded build by
using critical sections.

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2024-10-19 23:16:36 +02:00
Serhiy Storchaka a7443a1735
gh-52551: Use wcsftime() to implement time.strftime() on Windows (GH-125658) 2024-10-19 11:29:51 +03:00
Eric Snow 6d93690954
gh-125604: Move _Py_AuditHookEntry, etc. Out of pycore_runtime.h (gh-125605)
This is essentially a cleanup, moving a handful of API declarations to the header files where they fit best, creating new ones when needed.

We do the following:

* add pycore_debug_offsets.h and move _Py_DebugOffsets, etc. there
* inline struct _getargs_runtime_state and struct _gilstate_runtime_state in _PyRuntimeState
* move struct _reftracer_runtime_state to the existing pycore_object_state.h
* add pycore_audit.h and move to it _Py_AuditHookEntry , _PySys_Audit(), and _PySys_ClearAuditHooks
* add audit.h and cpython/audit.h and move the existing audit-related API there
*move the perfmap/trampoline API from cpython/sysmodule.h to cpython/ceval.h, and remove the now-empty cpython/sysmodule.h
2024-10-18 09:26:08 -06:00
Eric Snow 7cf2dbc3cb
gh-125667: Statically Initialize the Arg Converter Data Values in _interpqueuesmodule.c (gh-125668) 2024-10-17 23:49:15 +00:00
chrysn b454662921
gh-118986: expose `socket.IPV6_RECVERR` (#118987) 2024-10-17 20:49:12 +03:00
Serhiy Storchaka ad3eac1963
gh-52551: Fix encoding issues in strftime() (GH-125193)
Fix time.strftime(), the strftime() method and formatting of the
datetime classes datetime, date and time.

* Characters not encodable in the current locale are now acceptable in
  the format string.
* Surrogate pairs and sequence of surrogatescape-encoded bytes are no
  longer recombinated.
* Embedded null character no longer terminates the format string.

This fixes also gh-78662 and gh-124531.
2024-10-17 15:46:59 +00:00
Jonathan Protzenko 528bbab96f
GH-99108: Make vectorized versions of Blake2 available on x86, too (#125244)
Accomplished by updating HACL* vendored code from hacl-star/hacl-star@a6a09496d9 to hacl-star/hacl-star@315a9e491d

Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Zachary Ware <zach@python.org>
2024-10-17 15:08:43 +00:00
Kirill Podoprigora bee112a94d
gh-124872: Replace enter/exit events with "switched" (#125532)
Users want to know when the current context switches to a different
context object.  Right now this happens when and only when a context
is entered or exited, so the enter and exit events are synonymous with
"switched".  However, if the changes proposed for gh-99633 are
implemented, the current context will also switch for reasons other
than context enter or exit.  Since users actually care about context
switches and not enter or exit, replace the enter and exit events with
a single switched event.

The former exit event was emitted just before exiting the context.
The new switched event is emitted after the context is exited to match
the semantics users expect of an event with a past-tense name.  If
users need the ability to clean up before the switch takes effect,
another event type can be added in the future.  It is not added here
because YAGNI.

I skipped 0 in the enum as a matter of practice.  Skipping 0 makes it
easier to troubleshoot when code forgets to set zeroed memory, and it
aligns with best practices for other tools (e.g.,
https://protobuf.dev/programming-guides/dos-donts/#unspecified-enum).

Co-authored-by: Richard Hansen <rhansen@rhansen.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
2024-10-16 13:53:21 +02:00
Mikhail Efimov aac89b54c5
gh-125206: Bug in ctypes with old libffi is fixed (#125322)
Workaround for old libffi versions is added.
Module ctypes now supports C11 double complex only with libffi >= 3.3.0.

Co-authored-by: Sergey B Kirpichev <skirpichev@gmail.com>
2024-10-15 16:17:10 +00:00
Kirill Podoprigora d3c82b9cce
gh-125512: Revert "gh-124872: Replace enter/exit events with "switched" (#124776)" (#125513) 2024-10-15 17:42:16 +03:00