* expand on What's New entry for PEP 667 (including porting notes)
* define 'optimized scope' as a glossary term
* cover comprehensions and generator expressions in locals() docs
* review all mentions of "locals" in documentation (updating if needed)
* review all mentions of "f_locals" in documentation (updating if needed)
This is *not* sufficient for the final 3.13 release, but it will do for beta 1:
- What's new entry
- Updated changelog entry (news blurb)
- Mention the proxy for f_globals in the datamodel and Python frame object docs
This doesn't have any C API details (what's new refers to the PEP).
This PR adds the ability to enable the GIL if it was disabled at
interpreter startup, and modifies the multi-phase module initialization
path to enable the GIL when loading a module, unless that module's spec
includes a slot indicating it can run safely without the GIL.
PEP 703 called the constant for the slot `Py_mod_gil_not_used`; I went
with `Py_MOD_GIL_NOT_USED` for consistency with gh-104148.
A warning will be issued up to once per interpreter for the first
GIL-using module that is loaded. If `-v` is given, a shorter message
will be printed to stderr every time a GIL-using module is loaded
(including the first one that issues a warning).
Add "Raw" variant of PyTime functions:
* PyTime_MonotonicRaw()
* PyTime_PerfCounterRaw()
* PyTime_TimeRaw()
Changes:
* Add documentation and tests. Tests release the GIL while calling
raw clock functions.
* py_get_system_clock() and py_get_monotonic_clock() now check that
the GIL is hold by the caller if raise_exc is non-zero.
* Reimplement "Unchecked" functions with raw clock functions.
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Add Py_GetConstant() and Py_GetConstantBorrowed() functions.
In the limited C API version 3.13, getting Py_None, Py_False,
Py_True, Py_Ellipsis and Py_NotImplemented singletons is now
implemented as function calls at the stable ABI level to hide
implementation details. Getting these constants still return borrowed
references.
Add _testlimitedcapi/object.c and test_capi/test_object.py to test
Py_GetConstant() and Py_GetConstantBorrowed() functions.
This expands the examples to cover both realistic use cases for the API.
I noticed thing in the test that could be done better so I added those as well: We need to guarantee that all bytes of the result are overwritten and that too many are not written. Tests now pre-fills the result with data in order to ensure that.
Co-authored-by: Steve Dower <steve.dower@microsoft.com>
The `PyDict_SetDefaultRef` function is similar to `PyDict_SetDefault`,
but returns a strong reference through the optional `**result` pointer
instead of a borrowed reference.
Co-authored-by: Petr Viktorin <encukou@gmail.com>
The new `PyList_GetItemRef` is similar to `PyList_GetItem`, but returns
a strong reference instead of a borrowed reference. Additionally, if the
passed "list" object is not a list, the function sets a `TypeError`
instead of calling `PyErr_BadInternalCall()`.
`PyComplex_RealAsDouble()`/`PyComplex_ImagAsDouble` now try to convert
an object to a `complex` instance using its `__complex__()` method
before falling back to the ``__float__()`` method.
PyComplex_ImagAsDouble() also will not silently return 0.0 for
non-complex types anymore. Instead we try to call PyFloat_AsDouble()
and return 0.0 only if this call is successful.
* Implement _Py_HashPointerRaw() as a static inline function.
* Add Py_HashPointer() tests to test_capi.test_hash.
* Keep _Py_HashPointer() function as an alias to Py_HashPointer().
In the slot typedefs table, the parameter of `destructor`
and the first parameter of `traverseproc` should both be
`PyObject *` rather than `void *`.
Same for `inquiry`.
Change the declaration of the keywords parameter in functions
PyArg_ParseTupleAndKeywords() and PyArg_VaParseTupleAndKeywords() from `char **`
to `char * const *` in C and `const char * const *` in C++.
It makes these functions compatible with argument of type `const char * const *`,
`const char **` or `char * const *` in C++ and `char * const *` in C
without explicit type cast.
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
If Py_NOGIL is defined and Py_SET_REFCNT() is called with a reference
count larger than UINT32_MAX, make the object immortal.
Set _Py_IMMORTAL_REFCNT constant type to Py_ssize_t to fix the
following compiler warning:
Include/internal/pycore_global_objects_fini_generated.h:14:24:
warning: comparison of integers of different signs: 'Py_ssize_t'
(aka 'long') and 'unsigned int' [-Wsign-compare]
if (Py_REFCNT(obj) < _Py_IMMORTAL_REFCNT) {
~~~~~~~~~~~~~~ ^ ~~~~~~~~~~~~~~~~~~~
* Fix Sphinx warning in library/xml.rst
Direct use of the pyexpat module is deprecated, but this is how to check the version for security purposes
* Fix Sphinx warning in library/importlib.resources.rst
* Use italics for parameters
* Link to the exception
* Fix Sphinx warning in library/gzip.rst
* Document message and header defect base classes to fix Sphinx warning in library/email.headerregistry.rst
* Restore feed_eof() doc to fix Sphinx warning in library/asyncio-stream.rst
* Fix Sphinx warning in extending/newtypes.rst
* Fix Sphinx warning in c-api/set.rst
On stdtypes.rst, set and frozenset are documented together and the frozenset has the working refs
_PyDict_Pop_KnownHash(): remove the default value and the return type
becomes an int.
Co-authored-by: Stefan Behnel <stefan_ml@behnel.de>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
* Split list_extend() into two sub-functions: list_extend_fast() and
list_extend_iter().
* list_inplace_concat() no longer has to call Py_DECREF() on the
list_extend() result, since list_extend() now returns an int.
* Revert "gh-111089: Use PyUnicode_AsUTF8() in Argument Clinic (#111585)"
This reverts commit d9b606b3d0.
* Revert "gh-111089: Use PyUnicode_AsUTF8() in getargs.c (#111620)"
This reverts commit cde1071b2a.
* Revert "gh-111089: PyUnicode_AsUTF8() now raises on embedded NUL (#111091)"
This reverts commit d731579bfb.
* Revert "gh-111089: Add PyUnicode_AsUTF8() to the limited C API (#111121)"
This reverts commit d8f32be5b6.
* Revert "gh-111089: Use PyUnicode_AsUTF8() in sqlite3 (#111122)"
This reverts commit 37e4e20eaa.
* Add mimalloc v2.12
Modified src/alloc.c to remove include of alloc-override.c and not
compile new handler.
Did not include the following files:
- include/mimalloc-new-delete.h
- include/mimalloc-override.h
- src/alloc-override-osx.c
- src/alloc-override.c
- src/static.c
- src/region.c
mimalloc is thread safe and shares a single heap across all runtimes,
therefore finalization and getting global allocated blocks across all
runtimes is different.
* mimalloc: minimal changes for use in Python:
- remove debug spam for freeing large allocations
- use same bytes (0xDD) for freed allocations in CPython and mimalloc
This is important for the test_capi debug memory tests
* Don't export mimalloc symbol in libpython.
* Enable mimalloc as Python allocator option.
* Add mimalloc MIT license.
* Log mimalloc in Lib/test/pythoninfo.py.
* Document new mimalloc support.
* Use macro defs for exports as done in:
https://github.com/python/cpython/pull/31164/
Co-authored-by: Sam Gross <colesbury@gmail.com>
Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
* PyUnicode_AsUTF8() now raises an exception if the string contains
embedded null characters.
* Update related C API tests (test_capi.test_unicode).
* type_new_set_doc() uses PyUnicode_AsUTF8AndSize() to silently
truncate doc containing null bytes.
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Clarify the size of arenas
From 3.10.0 alpha 7, the pymalloc allocator uses arenas with a fixed size of 1
MiB on 64-bit platforms instead of 256 KiB on 32-bit platforms.
It already mostly worked, except in the case when invalid keyword
argument with non-ASCII name was passed to function with non-ASCII
parameter names. Then it crashed in the debug mode.
* gh-110803: Reorganize docs for what PyType_Slot doesn't cover
- Cover the offset fields first
- Mention the old alternative for MANAGED flags, which is needed
to support older Pythons
- De-emphasize the internal flags: use an inline list.
- Add a note to PyMemberDef saying what to do with it
* Remove an older draft...
sys.audit() now has assertions to check that the event argument is
not NULL and that the format argument does not use the "N" format.
Add tests on PySys_AuditTuple().
Add PyThreadState_GetUnchecked() function: similar to
PyThreadState_Get(), but don't issue a fatal error if it is NULL. The
caller is responsible to check if the result is NULL. Previously,
this function was private and known as _PyThreadState_UncheckedGet().
Make PyObject_VisitManagedDict() and PyObject_ClearManagedDict()
functions public in Python 3.13 C API.
* Rename _PyObject_VisitManagedDict() to PyObject_VisitManagedDict().
* Rename _PyObject_ClearManagedDict() to PyObject_ClearManagedDict().
* Document these functions.
Document PyMODINIT_FUNC macro.
Remove links to PyAPI_FUNC() and PyAPI_DATA() macros since they are
not documented. These macros should only be used to define the Python
C API. They should not be used outside Python code base.
* Express functions which take argument as a C string in terms of
functions which take Python object.
* Use "note" directive for PyMapping_HasKey() and
PyMapping_HasKeyString() notes.
Python built with "configure --with-trace-refs" (tracing references)
is now ABI compatible with Python release build and debug build.
Moreover, it now also supports the Limited API.
Change Py_TRACE_REFS build:
* Remove _PyObject_EXTRA_INIT macro.
* The PyObject structure no longer has two extra members (_ob_prev
and _ob_next).
* Use a hash table (_Py_hashtable_t) to trace references (all
objects): PyInterpreterState.object_state.refchain.
* Py_TRACE_REFS build is now ABI compatible with release build and
debug build.
* Limited C API extensions can now be built with Py_TRACE_REFS:
xxlimited, xxlimited_35, _testclinic_limited.
* No longer rename PyModule_Create2() and PyModule_FromDefAndSpec2()
functions to PyModule_Create2TraceRefs() and
PyModule_FromDefAndSpec2TraceRefs().
* _Py_PrintReferenceAddresses() is now called before
finalize_interp_delete() which deletes the refchain hash table.
* test_tracemalloc find_trace() now also filters by size to ignore
the memory allocated by _PyRefchain_Trace().
Test changes for Py_TRACE_REFS:
* Add test.support.Py_TRACE_REFS constant.
* Add test_sys.test_getobjects() to test sys.getobjects() function.
* test_exceptions skips test_recursion_normalizing_with_no_memory()
and test_memory_error_in_PyErr_PrintEx() if Python is built with
Py_TRACE_REFS.
* test_repl skips test_no_memory().
* test_capi skisp test_set_nomemory().
PEP 683 (immortal objects) revealed some ways in which the Python documentation has been unnecessarily coupled to the implementation details of reference counts. In the end users should focus on reference ownership, including taking references and releasing them, rather than on how many reference counts an object has.
This change updates the documentation to reflect that perspective. It also updates the docs relative to immortal objects in a handful of places.
Declare the following functions as macros, since they are actually
macros. It avoids a warning on "TYPE" or "macro" argument.
* PyMem_New()
* PyMem_Resize()
* PyModule_AddIntMacro()
* PyModule_AddStringMacro()
* PyObject_GC_New()
* PyObject_GC_NewVar()
* PyObject_New()
* PyObject_NewVar()
Add C standard C types to nitpick_ignore in Doc/conf.py:
* int64_t
* uint64_t
* uintptr_t
No longer ignore non existing "__int" type in nitpick_ignore.
Update Doc/tools/.nitignore
The documentation implies that PyVectorcall_Function() was available in Python 3.8.
This is half-true - it was available under a different name. I think it's clearer to set
the "version added" to 3.9.
* Add PyDict_GetItemRef() and PyDict_GetItemStringRef() functions.
Add these functions to the stable ABI version 3.13.
* Add unit tests on the PyDict C API in test_capi.
Schedule the removal of C API global configuration variables in
Python 3.14. Announce the removal to help C extension maintainers to
upgrade their code.
PyTuple_SET_ITEM() and PyList_SET_ITEM() now check the index argument
with an assertion if Python is built in debug mode or is built with
assertions.
* list_extend() and _PyList_AppendTakeRef() now set the list size
before calling PyList_SET_ITEM().
* PyStructSequence_GetItem() and PyStructSequence_SetItem() now check
the index argument: must be lesser than REAL_SIZE(op).
* PyStructSequence_GET_ITEM() and PyStructSequence_SET_ITEM() are now
aliases to PyStructSequence_GetItem() and
PyStructSequence_SetItem().
Use a note:: tag so that these dict and object API deficiencies show up clearly.
A caution:: tag was considered, but our current python docs rendering doesn't do much with that (no box or color change). warning:: seemed too extreme. note looks good.
Alyssa Coghlan
Lysandros Nikolaou
Serhiy Storchaka
Hugo van Kemenade
Petr Viktorin
Gregory P. Smith
Guido van Rossum
Pablo Galindo Salgado
Irit Katriel
Brett Simmers
Victor Stinner
Xie Yanbo
Savannah Ostrowski
Jelle Zijlstra
Steve Dower
Bogdan Romanyuk
Sergey B Kirpichev
bssyousefi
David Hewitt
qqwqqw689
Skip Montanaro
Nikita Sobolev
Erlend E. Aasland
Sam Gross
patenaud
DerSchinken
AN Long
Alex Waygood
Inada Naoki
Christopher Chavez
Rune Tynan
Amioplk
Hugo van Kemenade
lefp
Dino Viehland
Anthony Shaw
Mienxiu
Łukasz Langa
Ezio Melotti
Donghee Na
Heinz-Alexander Fuetterer
xzmeng
Carl Meyer
Eric Snow
da-woods
wulmer
F3eQnxN3RriK