Move the following private functions and structures to
pycore_modsupport.h internal C API:
* _PyArg_BadArgument()
* _PyArg_CheckPositional()
* _PyArg_NoKeywords()
* _PyArg_NoPositional()
* _PyArg_ParseStack()
* _PyArg_ParseStackAndKeywords()
* _PyArg_Parser structure
* _PyArg_UnpackKeywords()
* _PyArg_UnpackKeywordsWithVararg()
* _PyArg_UnpackStack()
* _Py_ANY_VARARGS()
Changes:
* Python/getargs.h now includes pycore_modsupport.h to export
functions.
* clinic.py now adds pycore_modsupport.h when one of these functions
is used.
* Add pycore_modsupport.h includes when a C extension uses one of
these functions.
* Define Py_BUILD_CORE_MODULE in C extensions which now include
directly or indirectly (via code generated by Argument Clinic)
pycore_modsupport.h:
* _csv
* _curses_panel
* _dbm
* _gdbm
* _multiprocessing.posixshmem
* _sqlite.row
* _statistics
* grp
* resource
* syslog
* _testcapi: bad_get() no longer uses METH_FASTCALL calling
convention but METH_VARARGS. Replace _PyArg_UnpackStack() with
PyArg_ParseTuple().
* _testcapi: add PYTESTCAPI_NEED_INTERNAL_API macro which is defined
by _testcapi sub-modules which need the internal C API
(pycore_modsupport.h): exceptions.c, float.c, vectorcall.c,
watchers.c.
* Remove Include/cpython/modsupport.h header file.
Include/modsupport.h no longer includes the removed header file.
* Fix mypy clinic.py
* Remove unused <locale.h> includes.
* Remove unused <fcntl.h> include in traceback.h.
* Remove redundant <assert.h> and <stddef.h> includes. They are already
included by "Python.h".
* Remove <object.h> include in faulthandler.c. Python.h already includes it.
* Add missing <stdbool.h> in pycore_pythread.h if HAVE_PTHREAD_STUBS
is defined.
* Fix also warnings in pthread_stubs.h: don't redefine macros if they
are already defined, like the __NEED_pthread_t macro.
Move the private _PyLong_Sign() and _PyLong_NumBits() functions
to the internal C API (pycore_long.h).
Modules/_testcapi/long.c now uses the internal C API.
Argument Clinic now only includes pycore_gc.h if PyGC_Head is needed,
and only includes pycore_runtime.h if _Py_ID() is needed.
* Add 'condition' optional argument to Clinic.add_include().
* deprecate_keyword_use() includes pycore_runtime.h when using
the _PyID() function.
* Fix rendering of includes: comments start at the column 35.
* Mark PC/clinic/_wmimodule.cpp.h and
"Objects/stringlib/clinic/*.h.h" header files as generated in
.gitattributes.
Effects:
* 42 header files generated by AC no longer include the internal C
API, instead of 4 header files before. For example,
Modules/clinic/_abc.c.h no longer includes the internal C API.
* Fix _testclinic_depr.c.h: it now always includes pycore_runtime.h
to get _Py_ID().
Remove _PyErr_ChainExceptions(), _PyErr_ChainExceptions1() and
_PyErr_SetFromPyStatus() functions from the public C API.
* Move the private _PyErr_ChainExceptions() and
_PyErr_ChainExceptions1() function to the internal C API
(pycore_pyerrors.h).
* Move the private _PyErr_SetFromPyStatus() to the internal C API
(pycore_initconfig.h).
* No longer export the _PyErr_ChainExceptions() function.
* Move run_in_subinterp_with_config() from _testcapi to
_testinternalcapi.
- Add param docstrings
- Link to os.SEEK_* constants
- Mention the return value in the initial paragraph
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Functions like PyErr_SetFromErrno() and SetFromWindowsErr() should be
called immediately after using the C API which sets errno or the Windows
error code.
Move these private functions to the internal C API
(pycore_abstract.h):
* _Py_convert_optional_to_ssize_t()
* _PyNumber_Index()
Argument Clinic now emits #include "pycore_abstract.h" when these
functions are used.
The parser of the c-analyzer tool now uses a list of files which use
the limited C API, rather than a list of files using the internal C
API.
Move private functions to the internal C API (pycore_sysmodule.h):
* _PySys_GetAttr()
* _PySys_GetSizeOf()
No longer export most of these functions.
Fix also a typo in Include/cpython/optimizer.h: add a missing space.
Clearly document the supported seek() operations:
- Rewind to the start of the stream
- Restore a previous stream position (given by tell())
- Fast-forward to the end of the stream
Move private _PyBytes functions to the internal C API
(pycore_bytesobject.h):
* _PyBytes_DecodeEscape()
* _PyBytes_FormatEx()
* _PyBytes_FromHex()
* _PyBytes_Join()
No longer export these functions.
Remove private pylifecycle.h functions: move them to the internal C
API ( pycore_atexit.h, pycore_pylifecycle.h and pycore_signal.h). No
longer export most of these functions.
Move _testcapi.test_atexit() to _testinternalcapi.
Remove the following private functions of the C API:
* _PyCodecInfo_GetIncrementalDecoder()
* _PyCodecInfo_GetIncrementalEncoder()
* _PyCodec_DecodeText()
* _PyCodec_EncodeText()
* _PyCodec_Forget()
* _PyCodec_Lookup()
* _PyCodec_LookupTextEncoding()
Move these functions to a new pycore_codecs.h internal header file.
These functions are no longer exported.
Remove the following private functions from the public C API:
* _Py_CheckFunctionResult()
* _PyObject_CallMethod()
* _PyObject_CallMethodId()
* _PyObject_CallMethodIdNoArgs()
* _PyObject_CallMethodIdObjArgs()
* _PyObject_CallMethodIdOneArg()
* _PyObject_MakeTpCall()
* _PyObject_VectorcallMethodId()
* _PyStack_AsDict()
Move these functions to the internal C API (pycore_call.h).
No longer export the following functions:
* _PyObject_Call()
* _PyObject_CallMethod()
* _PyObject_CallMethodId()
* _PyObject_CallMethodIdObjArgs()
* _PyObject_Call_Prepend()
* _PyObject_FastCallDictTstate()
* _PyStack_AsDict()
The following functions are still exported for stdlib shared
extensions:
* _Py_CheckFunctionResult()
* _PyObject_MakeTpCall()
Mark the following internal functions as extern:
* _PyStack_UnpackDict()
* _PyStack_UnpackDict_Free()
* _PyStack_UnpackDict_FreeNoDecRef()
Deprecate the old Py_UNICODE and PY_UNICODE_TYPE types in the C API:
use wchar_t instead.
Replace Py_UNICODE with wchar_t in multiple C files.
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
When preparing the _io extension module for isolation, many methods were
adapted to Argument Clinic. Some of these used the '*args: object'
signature, which is incorrect. These are now corrected to an exact
signature, and marked unused, since they are stub methods.
- Replace query with parameter in bufferediobase_unsupported()
- Replace query with parameter in iobase_unsupported()
- Hide delegate: Add method wrapper for _PyIOBase_check_seekable
- Hide delegate: Add method wraper for _PyIOBase_check_readable
- Hide delegate: Add method wraper for _PyIOBase_check_writable
- Replace query with parameter in _PyIOBase_check_seekable()
- Replace query with parameter in _PyIOBase_check_readable()
- Replace query with parameter in _PyIOBase_check_writable()
Aside from sys and builtins, _io is the only core builtin module that hasn't been ported to multi-phase init. We may do so later (e.g. gh-101948), but in the meantime we must at least take care of the module's static types properly. (This came up while working on gh-101660.)
https://github.com/python/cpython/issues/94673
Add `MS_WINDOWS_DESKTOP`, `MS_WINDOWS_APPS`, `MS_WINDOWS_SYSTEM` and `MS_WINDOWS_GAMES` preprocessor definitions to allow switching off functionality missing from particular API partitions ("partitions" are used in Windows to identify overlapping subsets of APIs).
CPython only officially supports `MS_WINDOWS_DESKTOP` and `MS_WINDOWS_SYSTEM` (APPS is included by normal desktop builds, but APPS without DESKTOP is not covered). Other configurations are a convenience for people building their own runtimes.
`MS_WINDOWS_GAMES` is for the Xbox subset of the Windows API, which is also available on client OS, but is restricted compared to `MS_WINDOWS_DESKTOP`. These restrictions may change over time, as they relate to the build headers rather than the OS support, and so we assume that Xbox builds will use the latest available version of the GDK.
builtins and extension module functions and methods that expect boolean values for parameters now accept any Python object rather than just a bool or int type. This is more consistent with how native Python code itself behaves.
The implementation of __sizeof__() methods using _PyObject_SIZE() now
use an unsigned type (size_t) to compute the size, rather than a signed
type (Py_ssize_t).
Cast explicitly signed (Py_ssize_t) values to unsigned type
(Py_ssize_t).
Fix potential race condition in code patterns:
* Replace "Py_DECREF(var); var = new;" with "Py_SETREF(var, new);"
* Replace "Py_XDECREF(var); var = new;" with "Py_XSETREF(var, new);"
* Replace "Py_CLEAR(var); var = new;" with "Py_XSETREF(var, new);"
Other changes:
* Replace "old = var; var = new; Py_DECREF(var)"
with "Py_SETREF(var, new);"
* Replace "old = var; var = new; Py_XDECREF(var)"
with "Py_XSETREF(var, new);"
* And remove the "old" variable.
We only statically initialize for core code and builtin modules. Extension modules still create
the tuple at runtime. We'll solve that part of interpreter isolation separately.
This change includes generated code. The non-generated changes are in:
* Tools/clinic/clinic.py
* Python/getargs.c
* Include/cpython/modsupport.h
* Makefile.pre.in (re-generate global strings after running clinic)
* very minor tweaks to Modules/_codecsmodule.c and Python/Python-tokenize.c
All other changes are generated code (clinic, global strings).
`TextIOWrapper.__init__()` called `os.device_encoding(file.fileno())` if fileno is 0-2 and encoding=None.
But it is very rarely works, and never documented behavior.
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code. It is still used in a number of non-builtin stdlib modules.
The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime. A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).
https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.
The core of the change is in:
* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers
I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings. That check is added to the PR CI config.
The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()). This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.
The following are not changed (yet):
* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init
https://bugs.python.org/issue46541
* Move _PyObject_CallNoArgs() to pycore_call.h (internal C API).
* _ssl, _sqlite and _testcapi extensions now call the public
PyObject_CallNoArgs() function, rather than _PyObject_CallNoArgs().
* _lsprof extension is now built with Py_BUILD_CORE_MODULE macro
defined to get access to internal _PyObject_CallNoArgs().
Fix typo in the private _PyObject_CallNoArg() function name: rename
it to _PyObject_CallNoArgs() to be consistent with the public
function PyObject_CallNoArgs().
Serhiy Storchaka
Furkan Onder
Tamás Hegedűs
Victor Stinner
Erlend E. Aasland
Brandt Bucher
Inada Naoki
Kumar Aditya
Eric Snow
Erlend E. Aasland
Max Bachmann
JosephSBoyle
Irit Katriel
Partha P. Mukherjee
Paul Moore
Zackery Spytz
Shantanu
Aivars Kalvāns
fikotta
Christian Heimes
slateny
Benjamin Peterson