Add the functions PyObject_GC_IsTracked and PyObject_GC_IsFinalized to the public API to allow to query if Python objects are being currently tracked or have been already finalized by the garbage collector respectively.
If Py_AddPendingCall() is called in a subinterpreter, the function is
now scheduled to be called from the subinterpreter, rather than being
called from the main interpreter.
Each subinterpreter now has its own list of scheduled calls.
* Move pending and eval_breaker fields from _PyRuntimeState.ceval
to PyInterpreterState.ceval.
* new_interpreter() now calls _PyEval_InitThreads() to create
pending calls lock.
* Fix Py_AddPendingCall() for subinterpreters. It now calls
_PyThreadState_GET() which works in a subinterpreter if the
caller holds the GIL, and only falls back on
PyGILState_GetThisThreadState() if _PyThreadState_GET()
returns NULL.
Extension modules: m_traverse, m_clear and m_free functions of
PyModuleDef are no longer called if the module state was requested
but is not allocated yet. This is the case immediately after the
module is created and before the module is executed (Py_mod_exec
function). More precisely, these functions are not called if m_size is
greater than 0 and the module state (as returned by
PyModule_GetState()) is NULL.
Extension modules without module state (m_size <= 0) are not affected.
Co-Authored-By: Petr Viktorin <encukou@gmail.com>
* Rename _PyInterpreterState_Get() to PyInterpreterState_Get() and
move it the limited C API.
* Add _PyInterpreterState_Get() alias to PyInterpreterState_Get() for
backward compatibility with Python 3.8.
* sys.settrace(), sys.setprofile() and _lsprof.Profiler.enable() now
properly report PySys_Audit() error if "sys.setprofile" or
"sys.settrace" audit event is denied.
* Add _PyEval_SetProfile() and _PyEval_SetTrace() function: similar
to PyEval_SetProfile() and PyEval_SetTrace() but take a tstate
parameter and return -1 on error.
* Add _PyObject_FastCallTstate() function.
PyInterpreterState.eval_frame function now requires a tstate (Python
thread state) parameter.
Add private functions to the C API to get and set the frame
evaluation function:
* Add tstate parameter to _PyFrameEvalFunction function type.
* Add _PyInterpreterState_GetEvalFrameFunc() and
_PyInterpreterState_SetEvalFrameFunc() functions.
* Add tstate parameter to _PyEval_EvalFrameDefault().
The Py_FatalError() function is replaced with a macro which logs
automatically the name of the current function, unless the
Py_LIMITED_API macro is defined.
Changes:
* Add _Py_FatalErrorFunc() function.
* Remove the function name from the message of Py_FatalError() calls
which included the function name.
* Update tests.
Reflecting changes to the code, removed weakref.ReferenceError from weakref.rst and exceptions.rst.
Issue submitter provided evidence that the `weakref.ReferenceError` alias for `ReferenceError` was removed from the code in 2007. Working with @gvanrossum at PyCascades CPython sprint we looked at the code and confirmed that `weakref.ReferenceError` was no longer in `weakref.py`.
Based on that analysis I removed references `weakref.ReferenceError` from the two documents where it was still being referenced: `weakref.rst` and `exceptions.rst`.
https://bugs.python.org/issue38374
From the source for `PyUnicode_Decode`, the implementation is:
```
if (encoding == NULL) {
return PyUnicode_DecodeUTF8Stateful(s, size, errors, NULL);
}
```
which is pretty clearly not defaulting to ASCII.
---
I assume this needs neither a news entry nor bpo link.
* Add backcompat defines and move non-limited API declaration to cpython/
This partially reverts commit 2ff58a24e8
which added PyObject_CallNoArgs to the 3.9+ stable ABI. This should not
be done; there are enough other call APIs in the stable ABI to choose from.
* Adjust documentation
Mark all newly public functions as added in 3.9.
Add a note about the 3.8 provisional names.
Add notes on public API.
* Put PyObject_CallNoArgs back in the limited API
* Rename PyObject_FastCallDict to PyObject_VectorcallDict
PyThreadState.on_delete is a callback used to notify Python when a
thread completes. _thread._set_sentinel() function creates a lock
which is released when the thread completes. It sets on_delete
callback to the internal release_sentinel() function. This lock is
known as Threading._tstate_lock in the threading module.
The release_sentinel() function uses the Python C API. The problem is
that on_delete is called late in the Python finalization, when the C
API is no longer fully working.
The PyThreadState_Clear() function now calls the
PyThreadState.on_delete callback. Previously, that happened in
PyThreadState_Delete().
The release_sentinel() function is now called when the C API is still
fully working.
Replace a few Py_FatalError() calls if tstate is NULL with
assert(tstate != NULL) in ceval.c.
PyEval_AcquireThread(), PyEval_ReleaseThread() and
PyEval_RestoreThread() must never be called with a NULL tstate.
Some of the *SetItem methods in the C API steal a reference to the
given value. This annotates the better behaved ones to assure the
reader that these are not the ones with the inconsistent behaviour.
* 📜🤖 Added by blurb_it.
* make docs consistent with signature
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
The added parentheses around the PyIter_Next assignment suppress the following warning which gcc throws without:
```
warning: using the result of an assignment as a condition without parentheses [-Wparentheses]
```
The other change is a typo fix
The C-API docs are a bit sparse on the interplay between C `fork()` and the CPython runtime. This change adds some more information on the subject.
https://bugs.python.org/issue38816
CC @encukou
I'm also adding Petr Viktorin as contributor for vectorcall in the "what's new" section.
https://bugs.python.org/issue36974
Automerge-Triggered-By: @encukou
Automerge-Triggered-By: @encukou
bpo-3605, bpo-38733: Optimize _PyErr_Occurred(): remove "tstate ==
NULL" test.
Py_FatalError() no longer calls PyErr_Occurred() if called without
holding the GIL. So PyErr_Occurred() no longer has to support
tstate==NULL case.
_Py_CheckFunctionResult(): use directly _PyErr_Occurred() to avoid
explicit "!= NULL" test.
Additional note: the `method_check_args` function in `Objects/descrobject.c` is written in such a way that it applies to all kinds of descriptors. In particular, a future re-implementation of `wrapper_descriptor` could use that code.
CC @vstinner @encukou
https://bugs.python.org/issue37645
Automerge-Triggered-By: @encukou
Provide Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() as
regular functions for the limited API. Previously, there were defined
as macros, but these macros didn't work with the limited API which
cannot access PyThreadState.recursion_depth field.
Remove _Py_CheckRecursionLimit from the stable ABI.
Add Include/cpython/ceval.h header file.
For now, we'll rely on the fact that the config structures aren't covered by the stable ABI.
We may revisit this in the future if we further explore the idea of offering a stable embedding API.
(cherry picked from commit bdace21b76)
Fix warnings options priority: PyConfig.warnoptions has the highest
priority, as stated in the PEP 587.
* Document options order in PyConfig.warnoptions documentation.
* Make PyWideStringList_INIT macro private: replace "Py" prefix
with "_Py".
* test_embed: add test_init_warnoptions().
Add a new struct_size field to PyPreConfig and PyConfig structures to
allow to modify these structures in the future without breaking the
backward compatibility.
* Replace private _config_version field with public struct_size field
in PyPreConfig and PyConfig.
* Public PyPreConfig_InitIsolatedConfig() and
PyPreConfig_InitPythonConfig()
return type becomes PyStatus, instead of void.
* Internal _PyConfig_InitCompatConfig(),
_PyPreConfig_InitCompatConfig(), _PyPreConfig_InitFromConfig(),
_PyPreConfig_InitFromPreConfig() return type becomes PyStatus,
instead of void.
* Remove _Py_CONFIG_VERSION
* Update the Initialization Configuration documentation.
* Add test_embed.test_init_setpath_config(): test Py_SetPath()
with PyConfig.
* test_init_setpath() and test_init_setpythonhome() no longer call
Py_SetProgramName(), but use the default program name.
* _PyPathConfig: isolated, site_import and base_executable
fields are now only available on Windows.
* If executable is set explicitly in the configuration, ignore
calculated base_executable: _PyConfig_InitPathConfig() copies
executable to base_executable.
* Complete path config documentation.
Py_SetPath() now sets sys.executable to the program full path
(Py_GetProgramFullPath()), rather than to the program name
(Py_GetProgramName()).
Fix also memory leaks in pathconfig_set_from_config().
* Add a note to the PyModule_AddObject docs.
* Correct example usages of PyModule_AddObject.
* Whitespace.
* Clean up wording.
* 📜🤖 Added by blurb_it.
* First code review.
* Add < 0 in the tests with PyModule_AddObject
- drop TargetScopeError in favour of raising SyntaxError directly
as per the updated PEP 572
- comprehension iteration variables are explicitly local, but
named expression targets in comprehensions are nonlocal or
global. Raise SyntaxError as specified in PEP 572
- named expression targets in the outermost iterable of a
comprehension have an ambiguous target scope. Avoid resolving
that question now by raising SyntaxError. PEP 572
originally required this only for cases where the bound name
conflicts with the iteration variable in the comprehension,
but CPython can't easily restrict the exception to that case
(as it doesn't know the target variable names when visiting
the outermost iterator expression)
There are plenty of legitimate scripts in the tree that begin with a
`#!`, but also a few that seem to be marked executable by mistake.
Found them with this command -- it gets executable files known to Git,
filters to the ones that don't start with a `#!`, and then unmarks
them as executable:
$ git ls-files --stage \
| perl -lane 'print $F[3] if (!/^100644/)' \
| while read f; do
head -c2 "$f" | grep -qxF '#!' \
|| chmod a-x "$f"; \
done
Looking at the list by hand confirms that we didn't sweep up any
files that should have the executable bit after all. In particular
* The `.psd` files are images from Photoshop.
* The `.bat` files sure look like things that can be run.
But we have lots of other `.bat` files, and they don't have
this bit set, so it must not be needed for them.
Automerge-Triggered-By: @benjaminp
The fact that keyword names are strings is now part of the vectorcall and `METH_FASTCALL` protocols. The biggest concrete change is that `_PyStack_UnpackDict` now checks that and raises `TypeError` if not.
CC @markshannon @vstinner
https://bugs.python.org/issue37540
* Remove a vague statement in documentation
* Remove another vague sentence
A sentence starting with "So it should be possible..." shouldn't be in the docs either.
Co-Authored-By: Kyle Stanley <aeros167@gmail.com>
* Include the removal of the previous line
Co-Authored-By: Kyle Stanley <aeros167@gmail.com>
* Remove an extra space
* Rename PyImport_Cleanup() to _PyImport_Cleanup() and move it to the
internal C API. Add 'tstate' parameters.
* Remove documentation of _PyImport_Init(), PyImport_Cleanup(),
_PyImport_Fini(). All three were documented as "For internal use
only.".
Add a new public PyObject_CallNoArgs() function to the C API: call a
callable Python object without any arguments.
It is the most efficient way to call a callback without any argument.
On x86-64, for example, PyObject_CallFunctionObjArgs(func, NULL)
allocates 960 bytes on the stack per call, whereas
PyObject_CallNoArgs(func) only allocates 624 bytes per call.
It is excluded from stable ABI 3.8.
Replace private _PyObject_CallNoArg() with public
PyObject_CallNoArgs() in C extensions: _asyncio, _datetime,
_elementtree, _pickle, _tkinter and readline.
* Update PyCompilerFlags structure documentation.
* Document the new cf_feature_version field in the Changes in the C
API section of the What's New in Python 3.8 doc.
It is now allowed to add new fields at the end of the PyTypeObject struct without having to allocate a dedicated compatibility flag in tp_flags.
This will reduce the risk of running out of bits in the 32-bit tp_flags value.
* bpo-36540: Documentation for PEP570 - Python positional only arguments
* fixup! bpo-36540: Documentation for PEP570 - Python positional only arguments
* Update reference for compound statements
* Apply suggestions from Carol
Co-Authored-By: Carol Willing <carolcode@willingconsulting.com>
* Update Doc/tutorial/controlflow.rst
Co-Authored-By: Carol Willing <carolcode@willingconsulting.com>
* Add extra bullet point and minor edits
bpo-34626: Document creating heap types from the C-API
Add missing descriptions of PEP384's PyType_Spec and PyType_Slot,
along with some introductory prose.
Add new sys.unraisablehook() function which can be overridden to
control how "unraisable exceptions" are handled. It is called when an
exception has occurred but there is no way for Python to handle it.
For example, when a destructor raises an exception or during garbage
collection (gc.collect()).
Changes:
* Add an internal UnraisableHookArgs type used to pass arguments to
sys.unraisablehook.
* Add _PyErr_WriteUnraisableDefaultHook().
* The default hook now ignores exception on writing the traceback.
* test_sys now uses unittest.main() to automatically discover tests:
remove test_main().
* Add _PyErr_Init().
* Fix PyErr_WriteUnraisable(): hold a strong reference to sys.stderr
while using it
PyEval_AcquireLock() and PyEval_AcquireThread() now
terminate the current thread if called while the interpreter is
finalizing, making them consistent with PyEval_RestoreThread(),
Py_END_ALLOW_THREADS, and PyGILState_Ensure().
Calling these function from a thread when the runtime is finalizing will terminate
the thread, even if the thread was not created by Python. Users can use
_Py_IsFinalizing or sys.is_finalizing to check if the interpreter is in the process of
being finalized before calling this function to avoid unwanted termination.
Modify CLEANBYTE, DEADDYTE and FORBIDDENBYTE constants: use 0xCD,
0xDD and 0xFD, rather than 0xCB, 0xBB and 0xFB, to use the same byte
patterns than Windows CRT debug malloc() and free().
I have added documentation for `PyInterpreterState_Main()`.
I chose to place it under Advanced Debugger Support together with similar functions like `PyInterpreterState_Head()`, `PyInterpreterState_Next(`), and `PyInterpreterState_ThreadHead()` .
https://bugs.python.org/issue36157
Use UTF-8 as the system encoding on VxWorks.
The main reason are:
1. The locale is frequently misconfigured.
2. Missing some functions to deal with locale in VxWorks C library.
Pablo Galindo
Victor Stinner
Dong-hee Na
Zackery Spytz
Daniel Hahler
Serhiy Storchaka
Inada Naoki
Julien Danjou
Hai Shi
Brandt Bucher
Roger Hurwitz
Eric Wieser
Petr Viktorin
Joannah Nanjekye
Keith Erskine
Gurupad Hegde
cocoatomo
William Ayd
Sergey Fedoseev
Steve Dower
Terry Jan Reedy
Alex
Aveheuzed
Eric Snow
Jeroen Demeyer
Victor Stinner
Ammar Askar
Eddie Elizondo
t k
Julien Palard
Matti Picus
Christian Heimes
Ezio Melotti
Kyle Stanley
Jeroen Demeyer
Min ho Kim
Nick Coghlan
Greg Price
Joseph Fox-Rabinovitz
sgal
Antoine Pitrou
Xtreak
Matěj Cepl
Steve Dower
Stéphane Wirtel
Edison A
toonarmycaptain
Daniel Hahler
Andre Delfino
Emmanuel Arias
pxinwr
Gregory P. Smith