Py_REFCNT(), Py_TYPE(), Py_SIZE() and Py_IS_TYPE() functions argument
type is now "PyObject*", rather than "const PyObject*".
* Replace also "const PyObject*" with "PyObject*" in functions:
* _Py_strhex_impl()
* _Py_strhex_with_sep()
* _Py_strhex_bytes_with_sep()
* Remove _PyObject_CAST_CONST() and _PyVarObject_CAST_CONST() macros.
* Py_IS_TYPE() can now use Py_TYPE() in its implementation.
Copying and pickling instances of subclasses of builtin types
bytearray, set, frozenset, collections.OrderedDict, collections.deque,
weakref.WeakSet, and datetime.tzinfo now copies and pickles instance attributes
implemented as slots.
Move forward declarations of Python C API types to a new pytypedefs.h
header file to solve interdependency issues between header files.
pytypedefs.h contains forward declarations of the following types:
* PyCodeObject
* PyFrameObject
* PyGetSetDef
* PyInterpreterState
* PyLongObject
* PyMemberDef
* PyMethodDef
* PyModuleDef
* PyObject
* PyThreadState
* PyTypeObject
When a static inline function is wrapped by a macro which casts its
arguments to the expected type, there is no need that the function
has a different name than the macro. Use the same name for the macro
and the function to avoid confusion.
Rename _PyUnicode_get_wstr_length() to PyUnicode_WSTR_LENGTH().
Don't rename static inline _Py_NewRef() and _Py_XNewRef() functions,
since the C API exports Py_NewRef() and Py_XNewRef() functions as
regular functions. The name cannot be reused in this case.
* Never change types' cached keys. It could invalidate inline attribute objects.
* Lazily create object dictionaries.
* Update specialization of LOAD/STORE_ATTR.
* Don't update shared keys version for deletion of value.
* Update gdb support to handle instance values.
* Rename SPLIT_KEYS opcodes to INSTANCE_VALUE.
Convert the Py_TYPE() and Py_SIZE() macros to static inline
functions. The Py_SET_TYPE() and Py_SET_SIZE() functions must now be
used to set an object type and size.
* Remove code that checks Py_TPFLAGS_HAVE_VERSION_TAG
The field is always present in the type struct, as explained
in the added comment.
* Remove Py_TPFLAGS_HAVE_AM_SEND
The flag is not needed, and since it was added in 3.10 it can be removed now.
Convert the Py_TYPE() and Py_SIZE() macros to static inline
functions. The Py_SET_TYPE() and Py_SET_SIZE() functions must now be
used to set an object type and size.
Add a new Py_TPFLAGS_DISALLOW_INSTANTIATION type flag to disallow
creating type instances: set tp_new to NULL and don't create the
"__new__" key in the type dictionary.
The flag is set automatically on static types if tp_base is NULL or
&PyBaseObject_Type and tp_new is NULL.
Use the flag on the following types:
* _curses.ncurses_version type
* _curses_panel.panel
* _tkinter.Tcl_Obj
* _tkinter.tkapp
* _tkinter.tktimertoken
* _xxsubinterpretersmodule.ChannelID
* sys.flags type
* sys.getwindowsversion() type
* sys.version_info type
Update MyStr example in the C API documentation to use
Py_TPFLAGS_DISALLOW_INSTANTIATION.
Add _PyStructSequence_InitType() function to create a structseq type
with the Py_TPFLAGS_DISALLOW_INSTANTIATION flag set.
type_new() calls _PyType_CheckConsistency() at exit.
* Add Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING, add to all relevant standard builtin classes.
* Set relevant flags on collections.abc.Sequence and Mapping.
* Use flags in MATCH_SEQUENCE and MATCH_MAPPING opcodes.
* Inherit Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING.
* Add NEWS
* Remove interpreter-state map_abc and seq_abc fields.
Introduce Py_TPFLAGS_IMMUTABLETYPE flag for immutable type objects, and
modify PyType_Ready() to set it for static types.
Co-authored-by: Victor Stinner <vstinner@python.org>
Add the Py_Is(x, y) function to test if the 'x' object is the 'y'
object, the same as "x is y" in Python. Add also the Py_IsNone(),
Py_IsTrue(), Py_IsFalse() functions to test if an object is,
respectively, the None singleton, the True singleton or the False
singleton.
The limited C API is now supported if Python is built in debug mode
(if the Py_DEBUG macro is defined). In the limited C API, the
Py_INCREF() and Py_DECREF() functions are now implemented as opaque
function calls, rather than accessing directly the PyObject.ob_refcnt
member, if Python is built in debug mode and the Py_LIMITED_API macro
targets Python 3.10 or newer. It became possible to support the
limited C API in debug mode because the PyObject structure is the
same in release and debug mode since Python 3.8 (see bpo-36465).
The limited C API is still not supported in the --with-trace-refs
special build (Py_TRACE_REFS macro).
This change partically reverts
commit ad3252bad9
and the commit fe2978b3b9.
Many third party C extension modules rely on the ability of using
Py_TYPE() to set an object type: "Py_TYPE(obj) = type;" or to set an
object type using: "Py_SIZE(obj) = size;".
Convert Py_REFCNT() and Py_SIZE() macros to static inline functions.
They cannot be used as l-value anymore: use Py_SET_REFCNT() and
Py_SET_SIZE() to set an object reference count and size.
Replace &Py_SIZE(self) with &((PyVarObject*)self)->ob_size
in arraymodule.c.
This change is backward incompatible on purpose, to prepare the C API
for an opaque PyObject structure.
Module C state is now accessible from C-defined heap type methods (PEP 573).
Patch by Marcel Plch and Petr Viktorin.
Co-authored-by: Marcel Plch <mplch@redhat.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
PyType_HasFeature() now always calls PyType_GetFlags() to hide
implementation details. Previously, it accessed directly the
PyTypeObject.tp_flags member when the limited C API was not used.
Add fast inlined version _PyType_HasFeature() and _PyType_IS_GC()
for object.c and typeobject.c.
* Add backcompat defines and move non-limited API declaration to cpython/
This partially reverts commit 2ff58a24e8
which added PyObject_CallNoArgs to the 3.9+ stable ABI. This should not
be done; there are enough other call APIs in the stable ABI to choose from.
* Adjust documentation
Mark all newly public functions as added in 3.9.
Add a note about the 3.8 provisional names.
Add notes on public API.
* Put PyObject_CallNoArgs back in the limited API
* Rename PyObject_FastCallDict to PyObject_VectorcallDict
In the limited C API, PyObject_INIT() and PyObject_INIT_VAR() are now
defined as aliases to PyObject_Init() and PyObject_InitVar() to make
their implementation opaque. It avoids to leak implementation details
in the limited C API.
Exclude the following functions from the limited C API, move them
from object.h to cpython/object.h:
* _Py_NewReference()
* _Py_ForgetReference()
* _PyTraceMalloc_NewReference()
* _Py_GetRefTotal()
Exclude trashcan mechanism from the limited C API: it requires access to
PyTypeObject and PyThreadState structure fields, whereas these structures
are opaque in the limited C API.
The trashcan mechanism never worked with the limited C API. Move it
from object.h to cpython/object.h.
_Py_NewReference() becomes a regular opaque function, rather than a
static inline function in the C API (object.h), to better hide
implementation details.
Move _Py_tracemalloc_config from public pymem.h to internal
pycore_pymem.h header.
Make _Py_AddToAllObjects() private.
* Remove _Py_INC_REFTOTAL and _Py_DEC_REFTOTAL macros: modify
directly _Py_RefTotal.
* _Py_ForgetReference() is no longer defined if the Py_TRACE_REFS
macro is not defined.
* Remove _Py_NewReference() implementation from object.c:
unify the two implementations in object.h inline function.
* Fix Py_TRACE_REFS build: _Py_INC_TPALLOCS() macro has been removed.
Move the following functions from the public C API to the internal C
API:
* _PyDebug_PrintTotalRefs(),
* _Py_PrintReferenceAddresses()
* _Py_PrintReferences()
Filename and line numbers are not needed when Py_REF_DEBUG are not
defined.
The static inline _Py_DECREF() function was introduced by
commit 2aaf0c1204.
It is now allowed to add new fields at the end of the PyTypeObject struct without having to allocate a dedicated compatibility flag in tp_flags.
This will reduce the risk of running out of bits in the 32-bit tp_flags value.
Add new trashcan macros to deal with a double deallocation that could occur when the `tp_dealloc` of a subclass calls the `tp_dealloc` of a base class and that base class uses the trashcan mechanism.
Patch by Jeroen Demeyer.
Release build and debug build are now ABI compatible: the Py_DEBUG
define no longer implies Py_TRACE_REFS define which introduces the
only ABI incompatibility.
A new "./configure --with-trace-refs" build option is now required to
get Py_TRACE_REFS define which adds sys.getobjects() function and
PYTHONDUMPREFS environment variable.
Changes:
* Add ./configure --with-trace-refs
* Py_DEBUG no longer implies Py_TRACE_REFS
Fix the following clang warning:
Include/cpython/pystate.h:217:3: warning: redefinition of
typedef 'PyThreadState' is a C11 feature [-Wtypedef-redefinition]
* Move object.h code surrounded by "#ifndef Py_LIMITED_API"
to a new Include/cpython/object.h header file.
* "typedef struct _typeobject PyTypeObject;" is now always defined
in object.h, but if Py_LIMITED_API is not defined,
Include/cpython/object.h also defines the structure.
* Complete the printfunc comment to mention Py_LIMITED_API define.
* Add _PyObject_ASSERT_FROM() and _PyObject_ASSERT_FAILED_MSG()
macros.
* PyObject_GC_Track() now calls _PyObject_ASSERT_FAILED_MSG(),
instead of Py_FatalError(), if the object is already tracked, to
dump more information on error.
* _PyObject_GC_TRACK() no longer checks if the object is already
tracked at runtime, use an assertion instead for best performances;
PyObject_GC_Track() still checks at runtime.
* pycore_object.h now includes pycore_pystate.h.
* Convert _PyObject_GC_TRACK() and _PyObject_GC_UNTRACK() macros to
inline functions.
Configuring python with ./configure --with-pydebug CFLAGS="-D COUNT_ALLOCS -O0"
makes "make smelly" fail as some symbols were being exported without the "Py_" or
"_Py" prefixes.
_PyTraceMalloc_NewReference() is now called by _Py_NewReference(), so
move its definition to object.h. Moreover, define it even if
Py_LIMITED_API is defined, since _Py_NewReference() is also exposed
even if Py_LIMITED_API is defined.
Changes:
* Add _PyObject_AssertFailed() function.
* Add _PyObject_ASSERT() and _PyObject_ASSERT_WITH_MSG() macros.
* gc_decref(): replace assert() with _PyObject_ASSERT_WITH_MSG() to
dump the faulty object if the assertion fails.
_PyObject_AssertFailed() calls:
* _PyMem_DumpTraceback(): try to log the traceback where the object
memory has been allocated if tracemalloc is enabled.
* _PyObject_Dump(): log repr(obj).
* Py_FatalError(): log the current Python traceback.
_PyObject_AssertFailed() uses _PyObject_IsFreed() heuristic to check
if the object memory has been freed by a debug hook on Python memory
allocators.
Initial patch written by David Malcolm.
Co-Authored-By: David Malcolm <dmalcolm@redhat.com>
* Add Py_STATIC_INLINE() macro to declare a "static inline" function.
If the compiler supports it, try to always inline the function even if no
optimization level was specified.
* Modify pydtrace.h to use Py_STATIC_INLINE() when WITH_DTRACE is
not defined.
* Add an unit test on Py_DECREF() to make sure that
_Py_NegativeRefcount() reports the correct filename.
* Modify object.h to ensure that pymem.h is included,
to get _Py_tracemalloc_config variable.
* Move _PyTraceMalloc_XXX() functions to tracemalloc.h,
they need PyObject type. Break circular dependency between pymem.h
and object.h.
tracemalloc now tries to update the traceback when an object is
reused from a "free list" (optimization for faster object creation,
used by the builtin list type for example).
Changes:
* Add _PyTraceMalloc_NewReference() function which tries to update
the Python traceback of a Python object.
* _Py_NewReference() now calls _PyTraceMalloc_NewReference().
* Add an unit test.
_PyObject_Dump() now uses an heuristic to check if the object memory
has been freed: log "<freed object>" in that case.
The heuristic rely on the debug hooks on Python memory allocators
which fills the memory with DEADBYTE (0xDB) when memory is
deallocated. Use PYTHONMALLOC=debug to always enable these debug
hooks.
AttributeError was raised always when attribute is not found.
This commit skip raising AttributeError when `tp_getattro` is `PyObject_GenericGetAttr`.
It makes hasattr() and getattr() about 4x faster when attribute is not found.
Py_Main() now handles two more -X options:
* -X showrefcount: new _PyCoreConfig.show_ref_count field
* -X showalloccount: new _PyCoreConfig.show_alloc_count field
PR #1638, for bpo-28411, causes problems in some (very) edge cases. Until that gets sorted out, we're reverting the merge. PR #3506, a fix on top of #1638, is also getting reverted.
* Drop warnoptions from PyInterpreterState.
* Drop xoptions from PyInterpreterState.
* Don't set warnoptions and _xoptions again.
* Decref after adding to sys.__dict__.
* Drop an unused macro.
* Check sys.xoptions *before* we delete it.
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals
Other globals are excluded (see globals.txt and check-c-globals.py).
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals
Other globals are excluded (see globals.txt and check-c-globals.py).
Issue #27350: `dict` implementation is changed like PyPy. It is more compact
and preserves insertion order.
_PyDict_Dummy() function has been removed.
Disable test_gdb: python-gdb.py is not updated yet to the new structure of
compact dictionaries (issue #28023).
Patch written by INADA Naoki.
Victor Stinner
Serhiy Storchaka
Raymond Hettinger
Matti Picus
Mark Shannon
Christian Clauss
Hai Shi
Petr Viktorin
Pablo Galindo
Erlend Egeberg Aasland
Brandt Bucher
Vladimir Matveev
Dong-hee Na
Jimmy Yang
Andy Lester
Hansraj Das
Antoine Pitrou
Jeroen Demeyer
Victor Stinner
Dmitry Marakasov
INADA Naoki
Victor Stinner
luzpaz
stratakis
Eric Snow
Xiang Zhang
Benjamin Peterson