Move private functions to the internal C API (pycore_dict.h):
* _PyDictView_Intersect()
* _PyDictView_New()
* _PyDict_ContainsId()
* _PyDict_DelItemId()
* _PyDict_DelItem_KnownHash()
* _PyDict_GetItemIdWithError()
* _PyDict_GetItem_KnownHash()
* _PyDict_HasSplitTable()
* _PyDict_NewPresized()
* _PyDict_Next()
* _PyDict_Pop()
* _PyDict_SetItemId()
* _PyDict_SetItem_KnownHash()
* _PyDict_SizeOf()
No longer export most of these functions.
Move also the _PyDictViewObject structure to the internal C API.
Move dict_getitem_knownhash() function from _testcapi to the
_testinternalcapi extension. Update test_capi.test_dict for this
change.
Move private _PyEval functions to the internal C API
(pycore_ceval.h):
* _PyEval_GetBuiltin()
* _PyEval_GetBuiltinId()
* _PyEval_GetSwitchInterval()
* _PyEval_MakePendingCalls()
* _PyEval_SetProfile()
* _PyEval_SetSwitchInterval()
* _PyEval_SetTrace()
No longer export most of these functions.
106320: Remove private float C API functions
Remove private C API functions:
* _Py_parse_inf_or_nan()
* _Py_string_to_number_with_underscores()
Move these functions to the internal C API and no longer export them.
No longer export _PyCompile_AstOptimize() internal C API function.
Change comment style to "// comment" and add comment explaining why
other functions have to be exported.
No longer export _PyUnicode_FromId() internal C API function.
Change comment style to "// comment" and add comment explaining why
other functions have to be exported.
Update Tools/build/generate_token.py to update Include/internal/pycore_token.h
comments.
Remove the internal _PyDict_GetItemStringWithError() function. It can
now be replaced with the new public PyDict_ContainsString() and
PyDict_GetItemStringRef() functions.
getargs.c now now uses a strong reference for current_arg.
find_keyword() returns a strong reference.
No longer export these 5 internal C API functions:
* _PyArena_AddPyObject()
* _PyArena_Free()
* _PyArena_Malloc()
* _PyArena_New()
* _Py_FatalRefcountErrorFunc()
Change comment style to "// comment" and add comment explaining why
other functions have to be exported.
No longer export these 2 internal C API functions:
* _PyTime_AsNanoseconds()
* _PyTime_GetSystemClockWithInfo()
Change comment style to "// comment" and add comment explaining why
other functions have to be exported.
The _socket extension uses _PyCapsule_SetTraverse() to visit and clear
the socket type in the garbage collector. So the _socket.socket type
can be cleared in some corner cases when it wasn't possible before.
Remove private _PyDict_GetItemStringWithError() function of the
public C API: the new PyDict_GetItemStringRef() can be used instead.
* Move private _PyDict_GetItemStringWithError() to the internal C API.
* _testcapi get_code_extra_index() uses PyDict_GetItemStringRef().
Avoid using private functions in _testcapi which tests the public C
API.
* Add missing includes.
* Remove unused includes.
* Update old include/symbol names to newer names.
* Mention at least one included symbol.
* Sort includes.
* Update Tools/cases_generator/generate_cases.py used to generated
pycore_opcode_metadata.h.
* Update Parser/asdl_c.py used to generate pycore_ast.h.
* Cleanup also includes in _testcapimodule.c and _testinternalcapi.c.
* pycore_intrinsics.h does nothing if included twice
(add #ifndef and #define).
* Update Tools/cases_generator/generate_cases.py to generate the
Py_BUILD_CORE test.
* _bz2, _lzma, _opcode and zlib extensions now define the
Py_BUILD_CORE_MODULE macro to use internal headers
(pycore_code.h, pycore_intrinsics.h and pycore_blocks_output_buffer.h).
This finishes the work begun in gh-107760. When, while projecting a superblock, we encounter a call to a short, simple function, the superblock will now enter the function using `_PUSH_FRAME`, continue through it, and leave it using `_POP_FRAME`, and then continue through the original code. Multiple frame pushes and pops are even possible. It is also possible to stop appending to the superblock in the middle of a called function, when running out of space or encountering an unsupported bytecode.
* Split `CALL_PY_EXACT_ARGS` into uops
This is only the first step for doing `CALL` in Tier 2.
The next step involves tracing into the called code object and back.
After that we'll have to do the remaining `CALL` specialization.
Finally we'll have to deal with `KW_NAMES`.
Note: this moves setting `frame->return_offset` directly in front of
`DISPATCH_INLINED()`, to make it easier to move it into `_PUSH_FRAME`.
Introducing a new file, stacking.py, that takes over several responsibilities related to symbolic evaluation of push/pop operations, with more generality.
The linked list of objects was a global variable, which broke isolation between interpreters, causing crashes. To solve this, we've moved the linked list to each interpreter.
There's no need to use a dummy uop to skip unused cache entries. The macro syntax lets you write `unused/1` instead.
Similarly, move `unused/5` from op `_LOAD_ATTR_INSTANCE_VALUE` to macro `LOAD_ATTR_INSTANCE_VALUE`.
This fixes a crasher due to a race condition, triggered infrequently when two isolated (own GIL) subinterpreters simultaneously initialize their sys or builtins modules. The crash happened due the combination of the "detached" thread state we were using and the "last holder" logic we use for the GIL. It turns out it's tricky to use the same thread state for different threads. Who could have guessed?
We solve the problem by eliminating the one object we were still sharing between interpreters. We replace it with a low-level hashtable, using the "raw" allocator to avoid tying it to the main interpreter.
We also remove the accommodations for "detached" thread states, which were a dubious idea to start with.
The _xxsubinterpreters module should not rely on internal API. Some of the functions it uses were recently moved there however. Here we move them back (and expose them properly).
We tried this before with a dict and for all interned strings. That ran into problems due to interpreter isolation. However, exclusively using a per-interpreter cache caused some inconsistency that can eliminate the benefit of interning. Here we circle back to using a global cache, but only for statically allocated strings. We also use a more-basic _Py_hashtable_t for that global cache instead of a dict.
Ideally we would only have the global cache, but the optional isolation of each interpreter's allocator means that a non-static string object must not outlive its interpreter. Thus we would have to store a copy of each such interned string in the global cache, tied to the main interpreter.
Fix warnings C4100 in Py_UNUSED() when Python is built with "cl /W4".
Example with this function included by Python.h:
static inline unsigned int
PyUnicode_IS_READY(PyObject* Py_UNUSED(op))
{ return 1; }
Without this change, building a C program with "cl /W4" which just
includes Python.h emits the warning:
Include\cpython/unicodeobject.h(199):
warning C4100: '_unused_op': unreferenced formal parameter
This change fix this warning.
No longer export these 5 internal C API variables:
* _PyBufferWrapper_Type
* _PyImport_FrozenBootstrap
* _PyImport_FrozenStdlib
* _PyImport_FrozenTest
* _Py_SwappedOp
Fix the definition of these internal functions, replace PyAPI_DATA()
with PyAPI_FUNC():
* _PyImport_ClearExtension()
* _PyObject_IsFreed()
* _PyThreadState_GetCurrent()
No longer export these 2 internal C API functions:
* _PyEval_SignalAsyncExc()
* _PyEval_SignalReceived()
Add also comments explaining why some internal functions have to be
exported, and update existing comments.
Move PyInterpreterConfig structure and associated macros from
initconfig.h to pylifecycle.h: it's not related to the Python
Initialization Configuration.
Declare the following 3 PyUnstable functions in
Include/cpython/pyframe.h rather than Include/cpython/frameobject.h,
so they are now provided by the standard "#include <Python.h>".
Move private _PyMem functions to the internal C API (pycore_pymem.h):
* _PyMem_GetCurrentAllocatorName()
* _PyMem_RawStrdup()
* _PyMem_RawWcsdup()
* _PyMem_Strdup()
No longer export these functions.
Move pymem_getallocatorsname() function from _testcapi to _testinternalcapi,
since the API moved to the internal C API.
Move private _PyDict functions to the internal C API (pycore_dict.h):
* _PyDict_Contains_KnownHash()
* _PyDict_DebugMallocStats()
* _PyDict_DelItemIf()
* _PyDict_GetItemWithError()
* _PyDict_HasOnlyStringKeys()
* _PyDict_MaybeUntrack()
* _PyDict_MergeEx()
No longer export these functions.
Move private _PyObject and private _PyType functions to the internal
C API (pycore_object.h):
* _PyObject_GetMethod()
* _PyObject_IsAbstract()
* _PyObject_NextNotImplemented()
* _PyType_CalculateMetaclass()
* _PyType_GetDocFromInternalDoc()
* _PyType_GetTextSignatureFromInternalDoc()
No longer export these functions.
Move private _PyBytes functions to the internal C API
(pycore_bytesobject.h):
* _PyBytes_DecodeEscape()
* _PyBytes_FormatEx()
* _PyBytes_FromHex()
* _PyBytes_Join()
No longer export these functions.
Move private debug _PyObject functions to the internal C API
(pycore_object.h):
* _PyDebugAllocatorStats()
* _PyObject_CheckConsistency()
* _PyObject_DebugTypeStats()
* _PyObject_IsFreed()
No longer export most of these functions, except of
_PyObject_IsFreed().
Move test functions using _PyObject_IsFreed() from _testcapi to
_testinternalcapi. check_pyobject_is_freed() test no longer catch
_testcapi.error: the tested function cannot raise _testcapi.error.
Move private _PyModule API to the internal C API
(pycore_moduleobject.h):
* _PyModule_Clear()
* _PyModule_ClearDict()
* _PyModuleSpec_IsInitializing()
* _PyModule_IsExtension()
No longer export these functions.
Rename private C API constants:
* Rename PY_MONITORING_UNGROUPED_EVENTS to _PY_MONITORING_UNGROUPED_EVENTS
* Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS
Move the private _PyInterpreterID C API to the internal C API: add a
new pycore_interp_id.h header file.
Remove Include/interpreteridobject.h and
Include/cpython/interpreteridobject.h header files.
Move private _PyGen API to internal C API:
* _PyAsyncGenAThrow_Type
* _PyAsyncGenWrappedValue_Type
* _PyCoroWrapper_Type
* _PyGen_FetchStopIterationValue()
* _PyGen_Finalize()
* _PyGen_SetStopIterationValue()
No longer these symbols, except of the ones by the _asyncio shared
extensions.
* No longer export most private _PyHash symbols, only export the ones
which are needed by shared extensions.
* Modules/_xxtestfuzz/fuzzer.c now uses the internal C API.
Remove private _PyUnicode_AsString() alias to PyUnicode_AsUTF8(). It
was kept for backward compatibility with Python 3.0 - 3.2.
The PyUnicode_AsUTF8() is available since Python
3.3. The PyUnicode_AsUTF8String() function can be used to keep
compatibility with Python 3.2 and older.
* Add PyDict_GetItemRef() and PyDict_GetItemStringRef() functions.
Add these functions to the stable ABI version 3.13.
* Add unit tests on the PyDict C API in test_capi.
By turning `assert(kwnames == NULL)` into a macro that is not in the "forbidden" list, many instructions that formerly were skipped because they contained such an assert (but no other mention of `kwnames`) are now supported in Tier 2. This covers 10 instructions in total (all specializations of `CALL` that invoke some C code):
- `CALL_NO_KW_TYPE_1`
- `CALL_NO_KW_STR_1`
- `CALL_NO_KW_TUPLE_1`
- `CALL_NO_KW_BUILTIN_O`
- `CALL_NO_KW_BUILTIN_FAST`
- `CALL_NO_KW_LEN`
- `CALL_NO_KW_ISINSTANCE`
- `CALL_NO_KW_METHOD_DESCRIPTOR_O`
- `CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS`
- `CALL_NO_KW_METHOD_DESCRIPTOR_FAST`
This moves EXIT_TRACE, SAVE_IP, JUMP_TO_TOP, and
_POP_JUMP_IF_{FALSE,TRUE} from ceval.c to bytecodes.c.
They are no less special than before, but this way
they are discoverable o the copy-and-patch tooling.
* Convert PyObject_DelAttr() and PyObject_DelAttrString() macros to
functions.
* Add PyObject_DelAttr() and PyObject_DelAttrString() functions to
the stable ABI.
* Replace PyObject_SetAttr(obj, name, NULL) with
PyObject_DelAttr(obj, name).
Remove private pylifecycle.h functions: move them to the internal C
API ( pycore_atexit.h, pycore_pylifecycle.h and pycore_signal.h). No
longer export most of these functions.
Move _testcapi.test_atexit() to _testinternalcapi.
Remove private _PyUnicode_TransformDecimalAndSpaceToASCII() and other
private _PyUnicode C API functions: move them to the internal C API
(pycore_unicodeobject.h). No longer most of these functions.
Replace _testcapi.unicode_transformdecimalandspacetoascii() with
_testinternal._PyUnicode_TransformDecimalAndSpaceToASCII().
* Remove private _PyImport C API functions: move them to the internal
C API (pycore_import.h).
* No longer export most of these private functions.
* _testcapi avoids private _PyImport_GetModuleAttrString().
Remove more private _PyUnicode C API functions:
move them to the internal C API (pycore_unicodeobject.h).
No longer export most pycore_unicodeobject.h functions.
- Tweak uops debugging output
- Fix the bug from gh-106290
- Rename `SET_IP` to `SAVE_IP` (per https://github.com/faster-cpython/ideas/issues/558)
- Add a `SAVE_IP` uop at the start of the trace (ditto)
- Allow `unbound_local_error`; this gives us uops for `LOAD_FAST_CHECK`, `LOAD_CLOSURE`, and `DELETE_FAST`
- Longer traces
- Support `STORE_FAST_LOAD_FAST`, `STORE_FAST_STORE_FAST`
- Add deps on pycore_uops.h to Makefile(.pre.in)
Remove the following functions from the C API, move them to the internal C
API: add a new pycore_modsupport.h internal header file:
* PyModule_CreateInitialized()
* _PyArg_NoKwnames()
* _Py_VaBuildStack()
No longer export these functions.
Remove private _PyThreadState and _PyInterpreterState C API
functions: move them to the internal C API (pycore_pystate.h and
pycore_interp.h). Don't export most of these functions anymore, but
still export functions used by tests.
Remove _PyThreadState_Prealloc() and _PyThreadState_Init() from the C
API, but keep it in the stable API.
* Remove private _PyTraceMalloc C API functions: move them to the
internal C API.
* Don't export most of these functions anymore, but still export
_PyTraceMalloc_GetTraceback() used by tests.
* Rename Include/tracemalloc.h to Include/cpython/tracemalloc.h
Remove the "cpython/pytime.h" header file: it only contained private
functions. Move functions to the internal pycore_time.h header file.
Move tests from _testcapi to _testinternalcapi. Rename also test
methods to have the same name than tested C functions.
No longer export these functions:
* _PyTime_Add()
* _PyTime_As100Nanoseconds()
* _PyTime_FromMicrosecondsClamp()
* _PyTime_FromTimespec()
* _PyTime_FromTimeval()
* _PyTime_GetPerfCounterWithInfo()
* _PyTime_MulDiv()
Victor Stinner
Irit Katriel
Serhiy Storchaka
Pablo Galindo Salgado
Guido van Rossum
Mark Shannon
brandonardenwalli
Ken Jin
Finn Womack
Carl Meyer
Brandt Bucher
Eric Snow
Ivin Lee
Dong-hee Na
Inada Naoki