* Add PyDict_GetItemRef() and PyDict_GetItemStringRef() functions.
Add these functions to the stable ABI version 3.13.
* Add unit tests on the PyDict C API in test_capi.
Schedule the removal of C API global configuration variables in
Python 3.14. Announce the removal to help C extension maintainers to
upgrade their code.
PyTuple_SET_ITEM() and PyList_SET_ITEM() now check the index argument
with an assertion if Python is built in debug mode or is built with
assertions.
* list_extend() and _PyList_AppendTakeRef() now set the list size
before calling PyList_SET_ITEM().
* PyStructSequence_GetItem() and PyStructSequence_SetItem() now check
the index argument: must be lesser than REAL_SIZE(op).
* PyStructSequence_GET_ITEM() and PyStructSequence_SET_ITEM() are now
aliases to PyStructSequence_GetItem() and
PyStructSequence_SetItem().
Use a note:: tag so that these dict and object API deficiencies show up clearly.
A caution:: tag was considered, but our current python docs rendering doesn't do much with that (no box or color change). warning:: seemed too extreme. note looks good.
* Add tests on PyImport_AddModuleRef(), PyImport_AddModule() and
PyImport_AddModuleObject().
* pythonrun.c: Replace Py_XNewRef(PyImport_AddModule(name)) with
PyImport_AddModuleRef(name).
_PyUnicode_ToLowercase(), _PyUnicode_ToUppercase(),
_PyUnicode_ToTitlecase() are no longer deprecated in the
documentation. It's no longer needed since they now use Py_UCS4 type,
rather than the deprecated Py_UNICODE type.
Remove functions in the C API:
* PyEval_AcquireLock()
* PyEval_ReleaseLock()
* PyEval_InitThreads()
* PyEval_ThreadsInitialized()
But keep these functions in the stable ABI.
Mention "make regen-limited-abi" in "make regen-all".
Remove the following old functions to configure the Python
initialization, deprecated in Python 3.11:
* PySys_AddWarnOptionUnicode()
* PySys_AddWarnOption()
* PySys_AddXOption()
* PySys_HasWarnOptions()
* PySys_SetArgvEx()
* PySys_SetArgv()
* PySys_SetPath()
* Py_SetPath()
* Py_SetProgramName()
* Py_SetPythonHome()
* Py_SetStandardStreamEncoding()
* _Py_SetProgramFullPath()
Most of these functions are kept in the stable ABI, except:
* Py_SetStandardStreamEncoding()
* _Py_SetProgramFullPath()
Update Doc/extending/embedding.rst and Doc/extending/extending.rst to
use the new PyConfig API.
_testembed.c:
* check_stdio_details() now sets stdio_encoding and stdio_errors
of PyConfig.
* Add definitions of functions removed from the API but kept in the
stable ABI.
* test_init_from_config() and test_init_read_set() now use
PyConfig_SetString() instead of PyConfig_SetBytesString().
Remove _Py_ClearStandardStreamEncoding() internal function.
Deprecate the old Py_UNICODE and PY_UNICODE_TYPE types in the C API:
use wchar_t instead.
Replace Py_UNICODE with wchar_t in multiple C files.
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
* Support for conversion specifiers o (octal) and X (uppercase hexadecimal).
* Support for length modifiers j (intmax_t) and t (ptrdiff_t).
* Length modifiers are now applied to all integer conversions.
* Support for wchar_t C strings (%ls and %lV).
* Support for variable width and precision (*).
* Support for flag - (left alignment).
* Uncomment builtin removal in pairindextypes
* Use new-style index directive ('builtin') - C API
* Use new-style index directive ('builtin') - Extending
* Use new-style index directive ('builtin') - Library
* Use new-style index directive ('builtin') - Reference
* Use new-style index directive ('builtin') - Tutorial
* Uncomment object removal in pairindextypes
* Use new-style index directive ('object') - C API
* Use new-style index directive ('object') - Library
* Use new-style index directive ('object') - Reference
* Use new-style index directive ('object') - Tutorial
* Use new-style index directive ('module') - C API
* Use new-style index directive ('module') - Library
* Use new-style index directive ('module') - Reference
* Use new-style index directive ('module') - Tutorial
* Uncomment module removal in pairindextypes
* Use new-style index directive ('module') - C API
* Use new-style index directive ('module') - Library
* Use new-style index directive ('module') - Reference
* Make sure that the current exception is always normalized.
* Remove redundant type and traceback fields for the current exception.
* Add new API functions: PyErr_GetRaisedException, PyErr_SetRaisedException
* Add new API functions: PyException_GetArgs, PyException_SetArgs
We don't need direct C APIs to get at a bigint representation of PyLong but we
do want the few people who need to understand how.
Additional Author: CAM-Gerlach
The Py_CLEAR(), Py_SETREF() and Py_XSETREF() macros now only evaluate
their arguments once. If an argument has side effects, these side
effects are no longer duplicated.
Use temporary variables to avoid duplicating side effects of macro
arguments. If available, use _Py_TYPEOF() to avoid type punning.
Otherwise, use memcpy() for the assignment to prevent a
miscompilation with strict aliasing caused by type punning.
Add _Py_TYPEOF() macro: __typeof__() on GCC and clang.
Add test_py_clear() and test_py_setref() unit tests to _testcapi.
* Add API to allow extensions to set callback function on creation and destruction of PyCodeObject
Co-authored-by: Ye11ow-Flash <janshah@cs.stonybrook.edu>
These slots are marked "should be treated as read-only" in the
table at the start of the document. That doesn't say anything about
setting them in the static struct.
`tp_bases` docs did say that it should be ``NULL`` (TIL!). If you
ignore that, seemingly nothing bad happens. However, some slots
may not be inherited, depending on which sub-slot structs are present.
(FWIW, NumPy sets tp_bases and is affected by the quirk -- though to
be fair, its DUAL_INHERIT code probably predates tp_bases docs, and
also the result happens to be benign.)
This patch makes things explicit.
It also makes the summary table legend easier to scan.
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
The ``structmember.h`` header is deprecated, though it continues to be available
and there are no plans to remove it. There are no deprecation warnings. Old code
can stay unchanged (unless the extra include and non-namespaced macros bother
you greatly). Specifically, no uses in CPython are updated -- that would just be
unnecessary churn.
The ``structmember.h`` header is deprecated, though it continues to be
available and there are no plans to remove it.
Its contents are now available just by including ``Python.h``,
with a ``Py`` prefix added if it was missing:
- `PyMemberDef`, `PyMember_GetOne` and`PyMember_SetOne`
- Type macros like `Py_T_INT`, `Py_T_DOUBLE`, etc.
(previously ``T_INT``, ``T_DOUBLE``, etc.)
- The flags `Py_READONLY` (previously ``READONLY``) and
`Py_AUDIT_READ` (previously all uppercase)
Several items are not exposed from ``Python.h``:
- `T_OBJECT` (use `Py_T_OBJECT_EX`)
- `T_NONE` (previously undocumented, and pretty quirky)
- The macro ``WRITE_RESTRICTED`` which does nothing.
- The macros ``RESTRICTED`` and ``READ_RESTRICTED``, equivalents of
`Py_AUDIT_READ`.
- In some configurations, ``<stddef.h>`` is not included from ``Python.h``.
It should be included manually when using ``offsetof()``.
The deprecated header continues to provide its original
contents under the original names.
Your old code can stay unchanged, unless the extra include and non-namespaced
macros bother you greatly.
There is discussion on the issue to rename `T_PYSSIZET` to `PY_T_SSIZE` or
similar. I chose not to do that -- users will probably copy/paste that with any
spelling, and not renaming it makes migration docs simpler.
Co-Authored-By: Alexander Belopolsky <abalkin@users.noreply.github.com>
Co-Authored-By: Matthias Braun <MatzeB@users.noreply.github.com>
The Py_CLEAR(), Py_SETREF() and Py_XSETREF() macros now only evaluate
their argument once. If an argument has side effects, these side
effects are no longer duplicated.
Add test_py_clear() and test_py_setref() unit tests to _testcapi.
Add PyFrame_GetVar() and PyFrame_GetVarString() functions to get a
frame variable by its name.
Move PyFrameObject C API tests from test_capi to test_frame.
(see https://github.com/python/cpython/issues/98608)
This change does the following:
1. change the argument to a new `_PyInterpreterConfig` struct
2. rename the function to `_Py_NewInterpreterFromConfig()`, inspired by `Py_InitializeFromConfig()` (takes a `_PyInterpreterConfig` instead of `isolated_subinterpreter`)
3. split up the boolean `isolated_subinterpreter` into the corresponding multiple granular settings
* allow_fork
* allow_subprocess
* allow_threads
4. add `PyInterpreterState.feature_flags` to store those settings
5. add a function for checking if a feature is enabled on an opaque `PyInterpreterState *`
6. drop `PyConfig._isolated_interpreter`
The existing default (see `Py_NewInterpeter()` and `Py_Initialize*()`) allows fork, subprocess, and threads and the optional "isolated" interpreter (see the `_xxsubinterpreters` module) disables all three. None of that changes here; the defaults are preserved.
Note that the given `_PyInterpreterConfig` will not be used outside `_Py_NewInterpreterFromConfig()`, nor preserved. This contrasts with how `PyConfig` is currently preserved, used, and even modified outside `Py_InitializeFromConfig()`. I'd rather just avoid that mess from the start for `_PyInterpreterConfig`. We can preserve it later if we find an actual need.
This change allows us to follow up with a number of improvements (e.g. stop disallowing subprocess and support disallowing exec instead).
(Note that this PR adds "private" symbols. We'll probably make them public, and add docs, in a separate change.)
The definition of obj in the `Py_buffer` struct is as a PyObject*
ec091bd47e/Include/pybuffer.h (L22)
PyMemoryView_GET_BASE returns `.obj` - thus its return type
should be a PyObject* (or at least a void*). It definitely
doesn't return `Py_buffer`
It had to live as a global outside of PyConfig for stable ABI reasons in
the pre-3.12 backports.
This removes the `_Py_global_config_int_max_str_digits` and gets rid of
the equivalent field in the internal `struct _is PyInterpreterState` as
code can just use the existing nested config struct within that.
Adds tests to verify unique settings and configs in subinterpreters.
⚠️⚠️ Note for reviewers, hackers and fellow systems/low-level/compiler engineers ⚠️⚠️
If you have a lot of experience with this kind of shenanigans and want to improve the **first** version, **please make a PR against my branch** or **reach out by email** or **suggest code changes directly on GitHub**.
If you have any **refinements or optimizations** please, wait until the first version is merged before starting hacking or proposing those so we can keep this PR productive.
* gh-93503: Add APIs to set profiling and tracing functions in all threads in the C-API
* Use a separate API
* Fix NEWS entry
* Add locks around the loop
* Document ignoring exceptions
* Use the new APIs in the sys module
* Update docs
An unrecognized format character in PyUnicode_FromFormat() and
PyUnicode_FromFormatV() now sets a SystemError.
In previous versions it caused all the rest of the format string to be
copied as-is to the result string, and any extra arguments discarded.
* Store tp_weaklist on the interpreter state for static builtin types.
* Factor out _PyStaticType_GET_WEAKREFS_LISTPTR().
* Add _PyStaticType_ClearWeakRefs().
* Add a comment about how _PyStaticType_ClearWeakRefs() loops.
* Document the change.
* Update Doc/whatsnew/3.12.rst
* Fix a typo.
This checks the bases of of a type created using the FromSpec
API to inherit the bases metaclasses. The metaclass's alloc
function will be called as is done in `tp_new` for classes
created in Python.
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@protonmail.com>
* Remove outdated notes from functions that aren't in the Limited API
Nowadays everything that *is* in the Limited API has a note added
automatically.
These notes could mislead people to think that these functions
could never be added to the limited API. Remove them.
* Also remove forgotten note on tp_vectorcall_offset not being finalized
Added a new stable API function ``PyType_FromMetaclass``, which mirrors
the behavior of ``PyType_FromModuleAndSpec`` except that it takes an
additional metaclass argument. This is, e.g., useful for language
binding tools that need to store additional information in the type
object.
Python now always use the ``%zu`` and ``%zd`` printf formats to
format a size_t or Py_ssize_t number. Building Python 3.12 requires a
C11 compiler, so these printf formats are now always supported.
* PyObject_Print() and _PyObject_Dump() now use the printf %zd format
to display an object reference count.
* Update PY_FORMAT_SIZE_T comment.
* Remove outdated notes about the %zd format in PyBytes_FromFormat()
and PyUnicode_FromFormat() documentations.
* configure no longer checks for the %zd format and no longer defines
PY_FORMAT_SIZE_T macro in pyconfig.h.
* pymacconfig.h no longer undefines PY_FORMAT_SIZE_T: macOS 10.4 is
no longer supported. Python 3.12 now requires macOS 10.6 (Snow
Leopard) or newer.
Update documentation of PyUnicode_DecodeFSDefault(),
PyUnicode_DecodeFSDefaultAndSize() and PyUnicode_EncodeFSDefault():
they now use the filesystem encoding and error handler of PyConfig,
Py_FileSystemDefaultEncoding and Py_FileSystemDefaultEncodeErrors
variables are no longer used.
Use _Py_CAST() and _Py_STATIC_CAST() in macros wrapping static inline
functions of unicodeobject.h.
Change also the kind type from unsigned int to int: same parameter
type than PyUnicode_FromKindAndData().
The limited API version 3.11 no longer casts arguments to expected
types.
* Remove redundant footnote ref: the footnote has been removed
* Fix footnote ref to match footnote
* Convert footnotes into reST footnotes: will error if missing
Add the -P command line option and the PYTHONSAFEPATH environment
variable to not prepend a potentially unsafe path to sys.path.
* Add sys.flags.safe_path flag.
* Add PyConfig.safe_path member.
* Programs/_bootstrap_python.c uses config.safe_path=0.
* Update subprocess._optim_args_from_interpreter_flags() to handle
the -P command line option.
* Modules/getpath.py sets safe_path to 1 if a "._pth" file is
present.
Convert the following macros to static inline functions:
* PyByteArray_AS_STRING()
* PyByteArray_GET_SIZE()
* PyBytes_AS_STRING()
* PyBytes_GET_SIZE()
Limited C API version 3.11 no longer casts arguments.
Add _PyBytes_CAST() and _PyByteArray_CAST() macros.
Py_REFCNT(), Py_TYPE(), Py_SIZE() and Py_IS_TYPE() functions argument
type is now "PyObject*", rather than "const PyObject*".
* Replace also "const PyObject*" with "PyObject*" in functions:
* _Py_strhex_impl()
* _Py_strhex_with_sep()
* _Py_strhex_bytes_with_sep()
* Remove _PyObject_CAST_CONST() and _PyVarObject_CAST_CONST() macros.
* Py_IS_TYPE() can now use Py_TYPE() in its implementation.
* Revert "bpo-46850: Move _PyInterpreterState_SetEvalFrameFunc() to internal C API (GH-32054)"
This reverts commit f877b40e3f.
* Revert "bpo-46850: Move _PyEval_EvalFrameDefault() to internal C API (GH-32052)"
This reverts commit b9a5522dd9.
Move the private _PyFrameEvalFunction type, and private
_PyInterpreterState_GetEvalFrameFunc() and
_PyInterpreterState_SetEvalFrameFunc() functions to the internal C
API. The _PyFrameEvalFunction callback function type now uses the
_PyInterpreterFrame type which is part of the internal C API.
Update the _PyFrameEvalFunction documentation.
* `PyFrame_FastToLocalsWithError` and `PyFrame_LocalsToFast` are no longer called during profile and tracing.
(Contributed by Fabio Zadrozny)
* Make accesses to a frame's `f_locals` safe from C code, not relying on calls to `PyFrame_FastToLocals` or `PyFrame_LocalsToFast`.
* Document new `PyFrame_GetLocals` C-API function.
Add new functions to pack and unpack C double (serialize and
deserialize):
* PyFloat_Pack2(), PyFloat_Pack4(), PyFloat_Pack8()
* PyFloat_Unpack2(), PyFloat_Unpack4(), PyFloat_Unpack8()
Document these functions and add unit tests.
Rename private functions and move them from the internal C API
to the public C API:
* _PyFloat_Pack2() => PyFloat_Pack2()
* _PyFloat_Pack4() => PyFloat_Pack4()
* _PyFloat_Pack8() => PyFloat_Pack8()
* _PyFloat_Unpack2() => PyFloat_Unpack2()
* _PyFloat_Unpack4() => PyFloat_Unpack4()
* _PyFloat_Unpack8() => PyFloat_Unpack8()
Replace the "unsigned char*" type with "char*" which is more common
and easy to use.
* C API docs: move PyErr_SetImportErrorSubclass docs
It was in the section about warnings, but it makes more sense to
put it with PyErr_SetImportError.
* C API docs: document closeit argument to PyRun_AnyFileExFlags
It was already documented for PyRun_SimpleFileExFlags.
* textual fixes to unicode docs
* Move paragraph about tp_dealloc into tp_dealloc section
* __aiter__ returns an async iterator, not an awaitable
This reduces the size of the data segment by **300 KB** of the executable because if the modules are deep-frozen then the marshalled frozen data just wastes space. This was inspired by comment by @gvanrossum in https://github.com/python/cpython/pull/29118#issuecomment-958521863. Note: There is a new option `--deepfreeze-only` in `freeze_modules.py` to change this behavior, it is on be default to save disk space.
```console
# du -s ./python before
27892 ./python
# du -s ./python after
27524 ./python
```
Automerge-Triggered-By: GH:ericsnowcurrently
When Python is embedded in other applications, it is not easy to determine which version of Python is being used. This change exposes the Python version as part of the API data. Tools like Austin (https://github.com/P403n1x87/austin) can benefit from this data when targeting applications like uWSGI, as the Python version can then be inferred systematically by looking at the exported symbols rather than relying on unreliable pattern matching or other hacks (like remote code execution etc...).
Automerge-Triggered-By: GH:pablogsal
The getpath.py file is frozen at build time and executed as code over a namespace. It is never imported, nor is it meant to be importable or reusable. However, it should be easier to read, modify, and patch than the previous code.
This commit attempts to preserve every previously tested quirk, but these may be changed in the future to better align platforms.
Rename PyConfig.no_debug_ranges to PyConfig.code_debug_ranges and
invert the value.
Document -X no_debug_ranges and PYTHONNODEBUGRANGES env var in
PyConfig.code_debug_ranges documentation.
It is now considered a historical accident that e.g. `for` loops and the `iter()` built-in function do not require the iterators they work with to define `__iter__`, only `__next__`.
Objects that support garbage collection ("container" objects) should
call PyObject_GC_UnTrack() from their destructors before clearing any
fields which may point to other "container" objects.
Add PyThreadState_EnterTracing() and PyThreadState_LeaveTracing()
functions to the limited C API to suspend and resume tracing and
profiling.
Add an unit test on the PyThreadState C API to _testcapi.
Add also internal _PyThreadState_DisableTracing() and
_PyThreadState_ResetTracing().
Replace old names when they refer to actual versions of macOS.
Keep historical names in references to older versions.
Co-authored-by: Patrick Reader <_@pxeger.com>
Add the Py_ALWAYS_INLINE macro to ask the compiler to always inline a
static inline function. The compiler can ignore it and decides to not
inline the function.
Convert the Py_TYPE() and Py_SIZE() macros to static inline
functions. The Py_SET_TYPE() and Py_SET_SIZE() functions must now be
used to set an object type and size.
Fix PyAiter_Check to only check for the `__anext__` presense (not for
`__aiter__`). Rename `PyAiter_Check()` to `PyAIter_Check()`,
`PyObject_GetAiter()` -> `PyObject_GetAIter()`.
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Frozen modules must be added to several files in order to work properly. Before this change this had to be done manually. Here we add a tool to generate the relevant lines in those files instead. This helps us avoid mistakes and omissions.
https://bugs.python.org/issue45019
* Remove code that checks Py_TPFLAGS_HAVE_VERSION_TAG
The field is always present in the type struct, as explained
in the added comment.
* Remove Py_TPFLAGS_HAVE_AM_SEND
The flag is not needed, and since it was added in 3.10 it can be removed now.
Heap types with the Py_TPFLAGS_IMMUTABLETYPE flag can now inherit the
PEP 590 vectorcall protocol. Previously, this was only possible for static types.
Co-authored-by: Victor Stinner <vstinner@python.org>
I tried to be relatively thorough and give lots of links.
One reason is that this wasn't deprecated very long; also it seems people running into this tend to not be familiar with similar APIs.
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
Py_RunMain() now resets PyImport_Inittab to its initial value at
exit. It must be possible to call PyImport_AppendInittab() or
PyImport_ExtendInittab() at each Python initialization.
Serhiy Storchaka
Victor Stinner
Eric Snow
Jelle Zijlstra
F3eQnxN3RriK
Gregory P. Smith
Irit Katriel
TATHAGATA ROY
Petr Viktorin
Inada Naoki
Brandt Bucher
Mark Shannon
gsallam
Adam Turner
Jurica Bradarić
Barry Warsaw
Erlend E. Aasland
Brett Simmers
Hugo van Kemenade
Nikita Sobolev
Julien Palard
Jacob Bower
Carl Meyer
Tanner Firl
Yeojin Kim
Oleg Iarygin
Furkan Onder
Max Bachmann
Peter Jiping Xie
Semen Zhydenko
David Lechner
Alexander Shadchin
ram vikram singh
Itamar Ostricher
mpage
Rafael Fontenelle
Steve Dower
Wenzel Jakob
Johnny11502
Pablo Galindo Salgado
wim glenn
QuakeIV
da-woods
Eric Wieser
Brett Cannon
C.A.M. Gerlach
adphrost
ov2k
Ken Jin
Pamela Fox
JustAnotherArchivist
Dong-hee Na
Ned Batchelder
Max Bachmann
Sebastian Berg
Charlie Zhao
Matt Wozniski
Robert Howlett
Ken Jin
Erlend Egeberg Aasland
Kumar Aditya
Christian Heimes
Piotr Fusik
Julian Gilbey
David Hewitt
Raymond Hettinger
Matti Picus
Georg Brandl
Gabriele N. Tornetta
Sam Gross
Christian Clauss
Konstantin Popov
Yury Selivanov
Sunny Bean
Hai Shi
Ammar Askar
Srinivas Reddy Thatiparthy (శ్రీనివాస్ రెడ్డి తాటిపర్తి)
Mark Dickinson