Commit Graph

5023 Commits

Author SHA1 Message Date
Victor Stinner 828451dfde
gh-111545: Add Py_HashPointer() function (#112096)
* Implement _Py_HashPointerRaw() as a static inline function.
* Add Py_HashPointer() tests to test_capi.test_hash.
* Keep _Py_HashPointer() function as an alias to Py_HashPointer().
2023-12-06 15:09:22 +01:00
Donghee Na 5aa317e4ca
gh-112535: Add comment for ppc32/64 registers (gh-112746) 2023-12-05 19:44:19 +09:00
Donghee Na d824512059
gh-112535: Update _Py_ThreadId() to support PowerPC (gh-112624) 2023-12-05 18:03:32 +09:00
Serhiy Storchaka da6760bdf5
gh-65210: Add const qualifiers in PyArg_VaParseTupleAndKeywords() (GH-105958)
Change the declaration of the keywords parameter in functions
PyArg_ParseTupleAndKeywords() and PyArg_VaParseTupleAndKeywords() from `char **`
to `char * const *` in C and `const char * const *` in C++.

It makes these functions compatible with argument of type `const char * const *`,
`const char **` or `char * const *` in C++ and `char * const *` in C
without explicit type cast.

Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
2023-12-04 13:14:56 +02:00
Victor Stinner a74902a14c
gh-106550: Fix sign conversion in pycore_code.h (#112613)
Fix sign conversion in pycore_code.h: use unsigned integers and cast
explicitly when needed.
2023-12-04 11:42:58 +01:00
Victor Stinner 1f2a676785
gh-106560: Fix redundant declarations in Include/ (#112611)
Don't declare PyBool_Type, PyLong_Type and PySys_Audit() twice, but
only once.

Compiler warnings seen by building Python with gcc -Wredundant-decls.
2023-12-03 12:16:31 +01:00
Victor Stinner 5c5022b862
gh-112567: Add _PyTimeFraction C API (#112568)
Use a fraction internally in the _PyTime API to reduce the risk of
integer overflow: simplify the fraction using Greatest Common
Divisor (GCD). The fraction API is used by time functions:
perf_counter(), monotonic() and process_time().

For example, QueryPerformanceFrequency() usually returns 10 MHz on
Windows 10 and newer. The fraction SEC_TO_NS / frequency =
1_000_000_000 / 10_000_000 can be simplified to 100 / 1.

* Add _PyTimeFraction type.
* Add functions:

  * _PyTimeFraction_Set()
  * _PyTimeFraction_Mul()
  * _PyTimeFraction_Resolution()

* No longer check "numer * denom <= _PyTime_MAX" in
  _PyTimeFraction_Set(). _PyTimeFraction_Mul() uses _PyTime_Mul()
  which handles integer overflow.
2023-12-01 19:50:10 +01:00
Victor Stinner 05a370abd6
gh-112567: Add _Py_GetTicksPerSecond() function (#112587)
* Move _PyRuntimeState.time to _posixstate.ticks_per_second and
  time_module_state.ticks_per_second.
* Add time_module_state.clocks_per_second.
* Rename _PyTime_GetClockWithInfo() to py_clock().
* Rename _PyTime_GetProcessTimeWithInfo() to py_process_time().
* Add process_time_times() helper function, called by
  py_process_time().
* os.times() is now always built: no longer rely on HAVE_TIMES.
2023-12-01 17:05:56 +01:00
Victor Stinner a9073564ee
gh-110481: Fix typo in Py_SET_REFCNT() (#112595) 2023-12-01 15:54:40 +00:00
Victor Stinner 5f6ac2d88a
gh-110481: Fix Py_SET_REFCNT() integer overflow (#112174)
If Py_NOGIL is defined and Py_SET_REFCNT() is called with a reference
count larger than UINT32_MAX, make the object immortal.

Set _Py_IMMORTAL_REFCNT constant type to Py_ssize_t to fix the
following compiler warning:

Include/internal/pycore_global_objects_fini_generated.h:14:24:
warning: comparison of integers of different signs: 'Py_ssize_t'
(aka 'long') and 'unsigned int' [-Wsign-compare]

    if (Py_REFCNT(obj) < _Py_IMMORTAL_REFCNT) {
        ~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~
2023-12-01 15:50:16 +01:00
Pablo Galindo Salgado a73aa48e6b
gh-112367: Only free perf trampoline arenas at shutdown (#112368)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2023-12-01 13:20:51 +00:00
Irit Katriel 07ebd46f9e
gh-112519: Make it possible to specify instruction flags for pseudo instructions in bytecodes.c (#112520) 2023-11-30 11:03:30 +00:00
Kirill Podoprigora 0785c68559
gh-111972: Make Unicode name C APIcapsule initialization thread-safe (#112249) 2023-11-30 11:12:49 +01:00
Guido van Rossum e723700190
Rename ...Uop... to ...UOp... (uppercase O) for consistency (#112327)
* Rename _PyUopExecute to _PyUOpExecute (uppercase O) for consistency
* Also rename _PyUopName and _PyUOp_Replacements, and some output strings
2023-11-28 17:10:11 -08:00
Grant Ramsay e954ac7205
gh-63284: Add support for TLS-PSK (pre-shared key) to the ssl module (#103181)
Add support for TLS-PSK (pre-shared key) to the ssl module.

---------

Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
2023-11-27 04:01:44 +00:00
Eric Snow 9e56eedd01
gh-76785: Return an "excinfo" Object From Interpreter.run() (gh-111573) 2023-11-23 00:55:00 +00:00
Eric Snow 790db85c77
gh-76785: Add _PyType_GetModuleName() to the Internal C-API (gh-112323)
The new function corresponds to the existing (public) PyType_GetName() and PyType_GetQualName().
2023-11-22 15:03:33 -07:00
Thomas Wouters 1c5279b9d6 Post 3.13.0a2 2023-11-22 22:24:48 +01:00
Thomas Wouters 9c4347ef8b Python 3.13.0a2 2023-11-22 12:20:24 +01:00
Guido van Rossum 8deb8bc2e5
gh-112287: Speed up Tier 2 (uop) interpreter a little (#112286)
This makes the Tier 2 interpreter a little faster.
I calculated by about 3%,
though I hesitate to claim an exact number.

This starts by doubling the trace size limit (to 512),
making it more likely that loops fit in a trace.

The rest of the approach is to only load
`oparg` and `operand` in cases that use them.
The code generator know when these are used.

For `oparg`, it will conditionally emit
```
oparg = CURRENT_OPARG();
```
at the top of the case block.
(The `oparg` variable may be referenced multiple times
by the instructions code block, so it must be in a variable.)

For `operand`, it will use `CURRENT_OPERAND()` directly
instead of referencing the `operand` variable,
which no longer exists.
(There is only one place where this will be used.)
2023-11-20 11:25:32 -08:00
Guido van Rossum 1995955173
gh-106529: Make FOR_ITER a viable uop (#112134)
This uses the new mechanism whereby certain uops
are replaced by others during translation,
using the `_PyUop_Replacements` table.
We further special-case the `_FOR_ITER_TIER_TWO` uop
to update the deoptimization target to point
just past the corresponding `END_FOR` opcode.

Two tiny code cleanups are also part of this PR.
2023-11-20 10:08:53 -08:00
Hugo van Kemenade 3b3ec0d77f
gh-111863: Rename `Py_NOGIL` to `Py_GIL_DISABLED` (#111864)
Rename Py_NOGIL to Py_GIL_DISABLED
2023-11-20 15:52:00 +02:00
Donghee Na 7c9f2677fb
gh-111926: Update _PyWeakref_IS_DEAD to be thread-safe (gh-112267) 2023-11-20 07:36:45 +09:00
Guido van Rossum be0bd54c6b
gh-106529: Cleanups split off gh-112134 (#112214)
- Double max trace size to 256
- Add a dependency on executor_cases.c.h for ceval.o
- Mark `_SPECIALIZE_UNPACK_SEQUENCE` as `TIER_ONE_ONLY`
- Add debug output back showing the optimized trace
- Bunch of cleanups to Tools/cases_generator/
2023-11-17 11:49:42 -08:00
Sam Gross 446f18a911
gh-111956: Add thread-safe one-time initialization. (gh-111960) 2023-11-16 12:19:54 -07:00
Victor Stinner bd89bca9e2
gh-111798: Use lower Py_C_RECURSION_LIMIT in debug mode (#112124)
* Run again test_ast_recursion_limit() on WASI platform.
* Add _testinternalcapi.get_c_recursion_remaining().
* Fix test_ast and test_sys_settrace: test_ast_recursion_limit() and
  test_trace_unpack_long_sequence() now adjust the maximum recursion
  depth depending on the the remaining C recursion.
2023-11-16 13:52:33 +00:00
Victor Stinner 7e2308aaa2
gh-112026: Add again _PyThreadState_UncheckedGet() function (#112121)
Add again the private _PyThreadState_UncheckedGet() function as an
alias to the new public PyThreadState_GetUnchecked() function.
2023-11-15 19:15:03 +01:00
Victor Stinner bd2f1485b0
gh-112026: Restore removed _PyDict_GetItemStringWithError() (#112119)
Restore the removed _PyDict_GetItemStringWithError() function. It is
used by numpy.
2023-11-15 17:10:06 +00:00
Victor Stinner 58469244be
gh-112026: Restore removed private C API (#112115)
Restore removed private C API functions, macros and structures which
have no simple replacement for now:

* _PyDict_GetItem_KnownHash()
* _PyDict_NewPresized()
* _PyHASH_BITS
* _PyHASH_IMAG
* _PyHASH_INF
* _PyHASH_MODULUS
* _PyHASH_MULTIPLIER
* _PyLong_Copy()
* _PyLong_FromDigits()
* _PyLong_New()
* _PyLong_Sign()
* _PyObject_CallMethodId()
* _PyObject_CallMethodNoArgs()
* _PyObject_CallMethodOneArg()
* _PyObject_CallOneArg()
* _PyObject_EXTRA_INIT
* _PyObject_FastCallDict()
* _PyObject_GetAttrId()
* _PyObject_Vectorcall()
* _PyObject_VectorcallMethod()
* _PyStack_AsDict()
* _PyThread_CurrentFrames()
* _PyUnicodeWriter structure
* _PyUnicodeWriter_Dealloc()
* _PyUnicodeWriter_Finish()
* _PyUnicodeWriter_Init()
* _PyUnicodeWriter_Prepare()
* _PyUnicodeWriter_PrepareKind()
* _PyUnicodeWriter_WriteASCIIString()
* _PyUnicodeWriter_WriteChar()
* _PyUnicodeWriter_WriteLatin1String()
* _PyUnicodeWriter_WriteStr()
* _PyUnicodeWriter_WriteSubstring()
* _PyUnicode_AsString()
* _PyUnicode_FromId()
* _PyVectorcall_Function()
* _Py_HashDouble()
* _Py_HashPointer()
* _Py_IDENTIFIER()
* _Py_c_abs()
* _Py_c_diff()
* _Py_c_neg()
* _Py_c_pow()
* _Py_c_prod()
* _Py_c_quot()
* _Py_c_sum()
* _Py_static_string()
* _Py_static_string_init()
2023-11-15 16:38:31 +00:00
Victor Stinner b338ffa4bc
gh-112026: Add again <unistd.h> include in Python.h (#112046)
Add again <ctype.h> and <unistd.h> includes in Python.h, but don't
include them in the limited C API version 3.13 and newer.
2023-11-15 16:59:32 +01:00
Mark Shannon 4bbb367ba6
GH-111848: Set the IP when de-optimizing (GH-112065)
* Replace jumps with deopts in tier 2

* Fewer special cases of uop names

* Add target field to uop IR

* Remove more redundant SET_IP and _CHECK_VALIDITY micro-ops

* Extend whitelist of non-escaping API functions.
2023-11-15 15:48:58 +00:00
Victor Stinner 62802b6228
gh-111545: Add Include/cpython/pyhash.h header file (#112063)
Move non-limited C API to a new Include/cpython/pyhash.h header file.
2023-11-15 01:19:20 +01:00
Mark Shannon a519b87958
GH-111848: Convert remaining jumps to deopts into tier 2 code. (GH-112045) 2023-11-14 15:30:33 +00:00
Victor Stinner 4f04172c92
gh-111262: Add PyDict_Pop() function (#112028)
_PyDict_Pop_KnownHash(): remove the default value and the return type
becomes an int.

Co-authored-by: Stefan Behnel <stefan_ml@behnel.de>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
2023-11-14 12:51:00 +00:00
Irit Katriel 36aab34fab
gh-107149: make new opcode util functions private rather than public and unstable (#112042) 2023-11-14 00:31:02 +00:00
Victor Stinner babb787047
gh-111138: Add PyList_Extend() and PyList_Clear() functions (#111862)
* Split list_extend() into two sub-functions: list_extend_fast() and
  list_extend_iter().
* list_inplace_concat() no longer has to call Py_DECREF() on the
  list_extend() result, since list_extend() now returns an int.
2023-11-13 16:14:56 +00:00
Serhiy Storchaka 771bd3c94a
Add private _PyUnicode_AsUTF8NoNUL() function (GH-111957)
Like PyUnicode_AsUTF8(), but check for embedded null characters.
2023-11-10 21:31:36 +02:00
Sam Gross 289af86122
gh-111569: Fix critical sections test on WebAssembly (GH-111897)
This adds a macro `Py_CAN_START_THREADS` that corresponds to the Python
function `test.support.threading_helper.can_start_thread()`. WASI and
some Emscripten builds do not have a working pthread implementation.

This macro is used to guard the critical sections C API tests that
require a working threads implementation.
2023-11-09 15:37:11 -08:00
Samuel Thibault 0802fd6c8e
gh-81925: Implement native thread ids for kFreeBSD (#111761)
---------

Co-authored-by: Antoine Pitrou <antoine@python.org>
2023-11-09 21:02:30 +01:00
Mark Shannon 34a03e951b
GH-111843: Tier 2 exponential backoff (GH-111850) 2023-11-09 13:49:51 +00:00
Mark Shannon 25c4956488
GH-109369: Exit tier 2 if executor is invalid (GH-111657) 2023-11-09 11:19:51 +00:00
Irit Katriel 30ec968bef
gh-111354: remove comparisons with enum values, variable reuse, unused imports in genobject.c (#111708) 2023-11-09 10:27:20 +00:00
Sam Gross 31c90d5838
gh-111569: Implement Python critical section API (gh-111571)
Critical sections are helpers to replace the global interpreter lock
with finer grained locking.  They provide similar guarantees to the GIL
and avoid the deadlock risk that plain locking involves.  Critical
sections are implicitly ended whenever the GIL would be released.  They
are resumed when the GIL would be acquired.  Nested critical sections
behave as if the sections were interleaved.
2023-11-08 15:39:29 -07:00
Mark Shannon 06efb60264
GH-111848: Tidy up tier 2 handling of FOR_ITER specialization by using DEOPT_IF instead of jumps. (GH-111849) 2023-11-08 13:31:55 +00:00
Victor Stinner 11e83488c5
gh-111089: Revert PyUnicode_AsUTF8() changes (#111833)
* Revert "gh-111089: Use PyUnicode_AsUTF8() in Argument Clinic (#111585)"

This reverts commit d9b606b3d0.

* Revert "gh-111089: Use PyUnicode_AsUTF8() in getargs.c (#111620)"

This reverts commit cde1071b2a.

* Revert "gh-111089: PyUnicode_AsUTF8() now raises on embedded NUL (#111091)"

This reverts commit d731579bfb.

* Revert "gh-111089: Add PyUnicode_AsUTF8() to the limited C API (#111121)"

This reverts commit d8f32be5b6.

* Revert "gh-111089: Use PyUnicode_AsUTF8() in sqlite3 (#111122)"

This reverts commit 37e4e20eaa.
2023-11-07 22:36:13 +00:00
Mark Shannon 931f4438c9
GH-111485: Allow arbitrary annotations on instructions and micro-ops. (GH-111697) 2023-11-07 09:42:39 +00:00
Brandt Bucher 3e99c9cbf6
GH-111485: Make BEFORE_WITH a uop (GH-111812) 2023-11-06 16:42:49 -08:00
Eric Snow d4426e8d00
gh-76785: Move _Py_excinfo Functions Out of the Internal C-API (gh-111715)
I added _Py_excinfo to the internal API (and added its functions in Python/errors.c) in gh-111530 (9322ce9).  Since then I've had a nagging sense that I should have added the type and functions in its own PR.  While I do plan on using _Py_excinfo outside crossinterp.c very soon (see gh-111572/gh-111573), I'd still feel more comfortable if the _Py_excinfo stuff went in as its own PR.  Hence, here we are.

(FWIW, I may combine that with gh-111572, which I may, in turn, combine with gh-111573.  We'll see.)
2023-11-06 11:09:22 -07:00
Eric Snow 836e0a75d5
gh-111698: Restrict Py_mod_multiple_interpreters to 3.12+ Under Py_LIMITED_API (gh-111707)
This should have been done in gh-104148.

(A similar fix has already be done for that slot's value macros, and backported to 3.12.  See gh-110968.)
2023-11-06 09:34:57 -07:00
Antoine Pitrou 0e9c364f4a
GH-110829: Ensure Thread.join() joins the OS thread (#110848)
Joining a thread now ensures the underlying OS thread has exited. This is required for safer fork() in multi-threaded processes.

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-11-04 13:59:24 +00:00