Commit Graph

968 Commits

Author SHA1 Message Date
Dong-hee Na e2d78baed3
bpo-47250: Fix refleak from object.__getstate__() (GH-32403)
Co-authored-by: Brandt Bucher <brandt@python.org>
2022-04-08 08:27:00 +09:00
Brandt Bucher ef6a482b02
bpo-47177: Replace `f_lasti` with `prev_instr` (GH-32208) 2022-04-07 12:31:01 -07:00
Serhiy Storchaka 884eba3c76
bpo-26579: Add object.__getstate__(). (GH-2821)
Copying and pickling instances of subclasses of builtin types
bytearray, set, frozenset, collections.OrderedDict, collections.deque,
weakref.WeakSet, and datetime.tzinfo now copies and pickles instance attributes
implemented as slots.
2022-04-06 20:00:14 +03:00
Victor Stinner c14d7e4b81
bpo-47164: Add _PyASCIIObject_CAST() macro (GH-32191)
Add macros to cast objects to PyASCIIObject*, PyCompactUnicodeObject*
and PyUnicodeObject*: _PyASCIIObject_CAST(),
_PyCompactUnicodeObject_CAST() and _PyUnicodeObject_CAST(). Using
these new macros make the code more readable and check their argument
with: assert(PyUnicode_Check(op)).

Remove redundant assert(PyUnicode_Check(op)) in macros using directly
or indirectly these new CAST macros.

Replacing existing casts with these macros.
2022-03-31 09:59:27 +02:00
Brandt Bucher 2bde6827ea
bpo-46841: Quicken code in-place (GH-31888)
* Moves the bytecode to the end of the corresponding PyCodeObject, and quickens it in-place.

* Removes the almost-always-unused co_varnames, co_freevars, and co_cellvars member caches

* _PyOpcode_Deopt is a new mapping from all opcodes to their un-quickened forms.

* _PyOpcode_InlineCacheEntries is renamed to _PyOpcode_Caches

* _Py_IncrementCountAndMaybeQuicken is renamed to _PyCode_Warmup

* _Py_Quicken is renamed to _PyCode_Quicken

* _co_quickened is renamed to _co_code_adaptive (and is now a read-only memoryview).

* Do not emit unused nonzero opargs anymore in the compiler.
2022-03-21 11:11:17 +00:00
Ken Jin 602024e6e1
bpo-46921: Vectorcall support for `super()` (GH-31687)
Co-Authored-By: Dong-hee Na <donghee.na@python.org>
2022-03-06 14:21:28 +08:00
Victor Stinner 87af12bff3
bpo-46836: Rename InterpreterFrame to _PyInterpreterFrame (GH-31583)
Rename also struct _interpreter_frame to struct _PyInterpreterFrame.

Reduce risk of name conflicts if a project includes pycore_frame.h.
2022-02-25 16:22:00 +01:00
Victor Stinner 042f31da55
bpo-45459: C API uses type names rather than structure names (GH-31528)
Thanks to the new pytypedefs.h, it becomes to use type names like
PyObject rather like structure names like "struct _object".
2022-02-24 17:51:59 +01:00
Jelle Zijlstra 0a222db2bc
fix __bool__ docstring (GH-31301) 2022-02-21 07:42:27 -08:00
Eric Snow 12360aa159
bpo-46541: Discover the global strings. (gh-31346)
Instead of manually enumerating the global strings in generate_global_objects.py, we extrapolate the list from usage of _Py_ID() and _Py_STR() in the source files.

This is partly inspired by gh-31261.

https://bugs.python.org/issue46541
2022-02-14 17:36:51 -07:00
Petr Viktorin 204946986f
bpo-46613: Add PyType_GetModuleByDef to the public API (GH-31081)
* Make PyType_GetModuleByDef public (remove underscore)

Co-authored-by: Victor Stinner <vstinner@python.org>
2022-02-11 17:22:11 +01:00
Eric Snow 81c72044a1
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928)
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code.  It is still used in a number of non-builtin stdlib modules.

The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime.  A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).

https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.

The core of the change is in:

* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers

I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings.  That check is added to the PR CI config.

The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()).  This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.

The following are not changed (yet):

* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init

https://bugs.python.org/issue46541
2022-02-08 13:39:07 -07:00
Petr Viktorin 0ef0853012
bpo-46433: _PyType_GetModuleByDef: handle static types in MRO (GH-30696)
Automerge-Triggered-By: GH:encukou
2022-02-02 07:57:51 -08:00
Kumar Aditya b9ebde8db7
bpo-46564: do not create frame object for super object (GH-31002) 2022-02-01 21:40:46 +08:00
Victor Stinner 18ea973c21
bpo-40170: Remove PyHeapType_GET_MEMBERS() macro (GH-30942)
Remove the PyHeapType_GET_MEMBERS() macro. It was exposed in the
public C API by mistake, it must only be used by Python internally.
Use the PyTypeObject.tp_members member instead.

Rename PyHeapType_GET_MEMBERS() to _PyHeapType_GET_MEMBERS() and move
it to the internal C API.
2022-01-28 00:39:52 +01:00
Victor Stinner af32b3ef1f
bpo-40170: PyType_SUPPORTS_WEAKREFS() becomes a regular function (GH-30938)
Convert the PyType_SUPPORTS_WEAKREFS() macro to a regular function.
It no longer access the PyTypeObject.tp_weaklistoffset member
directly.

Add _PyType_SUPPORTS_WEAKREFS() static inline functions, used
internally by Python for best performance.
2022-01-27 03:00:55 +01:00
Victor Stinner a1444f4358
bpo-46417: Fix _PyStaticType_Dealloc() (GH-30810)
_PyStaticType_Dealloc() now only calls PyObject_ClearWeakRefs()
if the call is not going to fail.
2022-01-23 00:32:05 +01:00
Victor Stinner 1626bf4ac7
bpo-46417: Clear Unicode static types at exit (GH-30806)
Add _PyUnicode_FiniTypes() function, called by
finalize_interp_types(). It clears these static types:

* EncodingMapType
* PyFieldNameIter_Type
* PyFormatterIter_Type

_PyStaticType_Dealloc() now does nothing if tp_subclasses
is not NULL.
2022-01-22 22:55:39 +01:00
Victor Stinner 3a4c15bb98
bpo-46417: Cleanup typeobject.c code (GH-30795)
* Add comment to recurse_down_subclasses() explaining why it's safe
  to use a borrowed reference to tp_subclasses.
* remove_all_subclasses() no longer accept NULL cases
* type_set_bases() now relies on the fact that new_bases is not NULL.
* type_dealloc_common() avoids PyErr_Fetch/PyErr_Restore if tp_bases
  is NULL.
* remove_all_subclasses() makes sure that no exception is raised.
* Don't test at runtime if tp_mro only contains types: rely on
  _PyType_CAST() assertion for that.
* _PyStaticType_Dealloc() no longer clears tp_subclasses which is
  already NULL.
* mro_hierarchy() avoids calling _PyType_GetSubclasses() if
  tp_subclasses is NULL.

Coding style:

* Use Py_NewRef().
* Add braces and move variable declarations to the first variable
  assignement.
* Rename a few variables and parameters to use better names.
2022-01-22 18:56:11 +01:00
Victor Stinner 2d03b73cc9
bpo-46417: remove_subclass() clears tp_subclasses (GH-30793)
The remove_subclass() function now deletes the dictionary when
removing the last subclass (if the dictionary becomes empty) to save
memory: set PyTypeObject.tp_subclasses to NULL. remove_subclass() is
called when a type is deallocated.

_PyType_GetSubclasses() no longer holds a reference to tp_subclasses:
its loop cannot modify tp_subclasses.
2022-01-22 16:53:30 +01:00
Victor Stinner f1c6ae3270
bpo-46417: Fix race condition on setting type __bases__ (GH-30788)
Fix a race condition on setting a type __bases__ attribute: the
internal function add_subclass() now gets the
PyTypeObject.tp_subclasses member after calling PyWeakref_NewRef()
which can trigger a garbage collection which can indirectly modify
PyTypeObject.tp_subclasses.
2022-01-22 15:08:42 +01:00
Victor Stinner 8ee07dda13
bpo-46417: Add _PyType_GetSubclasses() function (GH-30761)
Add a new _PyType_GetSubclasses() function to get type's subclasses.

_PyType_GetSubclasses(type) returns a list which holds strong
refererences to subclasses. It is safer than iterating on
type->tp_subclasses which yields weak references and can be modified
in the loop.

_PyType_GetSubclasses(type) now holds a reference to the tp_subclasses
dict while creating the list of subclasses.

set_collection_flag_recursive() of _abc.c now uses
_PyType_GetSubclasses().
2022-01-21 23:29:10 +01:00
Victor Stinner bc67f189fd
bpo-46417: Add _PyType_CAST() macro (GH-30760)
In debug mode, the macro makes sure that its argument is a type using
an assertion.
2022-01-21 21:39:01 +01:00
Victor Stinner fda8886498
bpo-46417: Revert remove_subclass() change (GH-30750)
remove_subclass() doesn't clear the tp_subclasses dict if the dict
becomes empty.
2022-01-21 16:45:14 +01:00
Victor Stinner 595225e86d
bpo-46417: Py_Finalize() clears static types (GH-30743)
Add _PyTypes_FiniTypes() best-effort function to clear static types:
don't deallocate a type if it still has subclasses.

remove_subclass() now sets tp_subclasses to NULL when removing the
last subclass.
2022-01-21 13:06:34 +01:00
Victor Stinner e9e3eab0b8
bpo-46417: Finalize structseq types at exit (GH-30645)
Add _PyStructSequence_FiniType() and _PyStaticType_Dealloc()
functions to finalize a structseq static type in Py_Finalize().
Currrently, these functions do nothing if Python is built in release
mode.

Clear static types:

* AsyncGenHooksType: sys.set_asyncgen_hooks()
* FlagsType: sys.flags
* FloatInfoType: sys.float_info
* Hash_InfoType: sys.hash_info
* Int_InfoType: sys.int_info
* ThreadInfoType: sys.thread_info
* UnraisableHookArgsType: sys.unraisablehook
* VersionInfoType: sys.version
* WindowsVersionType: sys.getwindowsversion()
2022-01-21 01:42:25 +01:00
Victor Stinner 35d6540c90
bpo-46006: Revert "bpo-40521: Per-interpreter interned strings (GH-20085)" (GH-30422)
This reverts commit ea251806b8.

Keep "assert(interned == NULL);" in _PyUnicode_Fini(), but only for
the main interpreter.

Keep _PyUnicode_ClearInterned() changes avoiding the creation of a
temporary Python list object.
2022-01-06 08:53:44 +01:00
Raymond Hettinger 29ea68bd1d
Revert "bpo-46131: add fastpath for PyFloat_Check() (GH-30200)" (GH-30208)
This reverts commit 2ef06d4125.
2021-12-19 15:05:46 -06:00
Matti Picus 2ef06d4125
bpo-46131: add fastpath for PyFloat_Check() (#30200) 2021-12-19 14:24:30 -06:00
Eric Snow c8749b5783
bpo-46008: Make runtime-global object/type lifecycle functions and state consistent. (gh-29998)
This change is strictly renames and moving code around.  It helps in the following ways:

* ensures type-related init functions focus strictly on one of the three aspects (state, objects, types)
* passes in PyInterpreterState * to all those functions, simplifying work on moving types/objects/state to the interpreter
* consistent naming conventions help make what's going on more clear
* keeping API related to a type in the corresponding header file makes it more obvious where to look for it

https://bugs.python.org/issue46008
2021-12-09 12:59:26 -07:00
Mark Shannon 8319114fee
bpo-45947: Place dict and values pointer at fixed (negative) offset just before GC header. (GH-29879)
* Place __dict__ immediately before GC header for plain Python objects.

* Fix up lazy dict creation logic to use managed dict pointers.

* Manage values pointer, placing them directly before managed dict pointers.

* Convert hint-based load/store attr specialization target managed dict classes.

* Specialize LOAD_METHOD for managed dict objects.

* Remove unsafe _PyObject_GC_Calloc function.

* Remove unsafe _PyObject_GC_Malloc() function.

* Add comment explaning use of Py_TPFLAGS_MANAGED_DICT.
2021-12-07 16:02:53 +00:00
Mark Shannon 135cabd328
bpo-44525: Copy free variables in bytecode to allow calls to inner functions to be specialized (GH-29595)
* Make internal APIs that take PyFrameConstructor take a PyFunctionObject instead.

* Add reference to function to frame, borrow references to builtins and globals.

* Add COPY_FREE_VARS instruction to allow specialization of calls to inner functions.
2021-11-23 09:53:24 +00:00
Mark Dickinson 9127520729
bpo-45392: Update the docstring of the 'type' built-in (GH-29439) 2021-11-06 20:08:44 +01:00
Itamar Ostricher 2c045bd567
bpo-45697: Use PyObject_TypeCheck in type_call (GH-29392) 2021-11-04 12:39:50 +02:00
Petr Viktorin 8a310dd5f4
bpo-45315: PyType_FromSpec: Copy spec->name and have the type own the memory for its name (GH-29103) 2021-10-21 11:46:20 +02:00
Mark Shannon a8b9350964
bpo-45340: Don't create object dictionaries unless actually needed (GH-28802)
* Never change types' cached keys. It could invalidate inline attribute objects.

* Lazily create object dictionaries.

* Update specialization of LOAD/STORE_ATTR.

* Don't update shared keys version for deletion of value.

* Update gdb support to handle instance values.

* Rename SPLIT_KEYS opcodes to INSTANCE_VALUE.
2021-10-13 14:19:34 +01:00
Victor Stinner 7cdc2a0f4b
pycore_pystate.h no longer redefines PyThreadState_GET() (GH-28921)
Redefining the PyThreadState_GET() macro in pycore_pystate.h is
useless since it doesn't affect files not including it. Either use
_PyThreadState_GET() directly, or don't use pycore_pystate.h internal
C API. For example, the _testcapi extension don't use the internal C
API, but use the public PyThreadState_Get() function instead.

Replace PyThreadState_Get() with _PyThreadState_GET(). The
_PyThreadState_GET() macro is more efficient than PyThreadState_Get()
and PyThreadState_GET() function calls which call fail with a fatal
Python error.

posixmodule.c and _ctypes extension now include <windows.h> before
pycore header files (like pycore_call.h).

_PyTraceback_Add() now uses _PyErr_Fetch()/_PyErr_Restore() instead
of PyErr_Fetch()/PyErr_Restore().

The _decimal and _xxsubinterpreters extensions are now built with the
Py_BUILD_CORE_MODULE macro defined to get access to the internal C
API.
2021-10-13 14:09:13 +02:00
Victor Stinner d943d19172
bpo-45439: Move _PyObject_CallNoArgs() to pycore_call.h (GH-28895)
* Move _PyObject_CallNoArgs() to pycore_call.h (internal C API).
* _ssl, _sqlite and _testcapi extensions now call the public
  PyObject_CallNoArgs() function, rather than _PyObject_CallNoArgs().
* _lsprof extension is now built with Py_BUILD_CORE_MODULE macro
  defined to get access to internal _PyObject_CallNoArgs().
2021-10-12 08:38:19 +02:00
Victor Stinner ce3489cfdb
bpo-45439: Rename _PyObject_CallNoArg() to _PyObject_CallNoArgs() (GH-28891)
Fix typo in the private _PyObject_CallNoArg() function name: rename
it to _PyObject_CallNoArgs() to be consistent with the public
function PyObject_CallNoArgs().
2021-10-12 00:42:23 +02:00
Raymond Hettinger 94b462686b
bpo-45198: __set_name__ documentation not clear about its usage with non-descriptor classes (GH-28439) 2021-09-18 01:49:43 -05:00
Mark Shannon f9242d50b1
bpo-44990: Change layout of evaluation frames. "Layout B" (GH-27933)
Places the locals between the specials and stack. This is the more "natural" layout for a C struct, makes the code simpler and gives a slight speedup (~1%)
2021-08-25 13:44:20 +01:00
Hai Shi 3e2c643ae0
bpo-42035: Add PyType_GetQualName() to get a type's qualified name. (GH-27551) 2021-08-17 15:39:34 +02:00
Mark Shannon 1a511dc92d
bpo-44914: Maintain invariants of type version tags. (GH-27773)
* Do not invalidate type versions unnecessarily.
2021-08-16 12:21:34 +01:00
Mark Shannon cee67fa661
bpo-44821: Eagerly assign __dict__ for new objects. (GH-27589) 2021-08-04 16:41:14 +01:00
Hai Shi a390ebea17
bpo-42035: Add a PyType_GetName() to get type's short name. (GH-23903) 2021-07-29 09:57:02 +02:00
Mark Shannon ae0a2b7562
bpo-44590: Lazily allocate frame objects (GH-27077)
* Convert "specials" array to InterpreterFrame struct, adding f_lasti, f_state and other non-debug FrameObject fields to it.

* Refactor, calls pushing the call to the interpreter upward toward _PyEval_Vector.

* Compute f_back when on thread stack, only filling in value when frame object outlives stack invocation.

* Move ownership of InterpreterFrame in generator from frame object to generator object.

* Do not create frame objects for Python calls.

* Do not create frame objects for generators.
2021-07-26 11:22:16 +01:00
Petr Viktorin a4760cc32d
bpo-42747: Remove Py_TPFLAGS_HAVE_AM_SEND and make Py_TPFLAGS_HAVE_VERSION_TAG no-op (GH-27260)
* Remove code that checks Py_TPFLAGS_HAVE_VERSION_TAG
    
    The field is always present in the type struct, as explained
    in the added comment.

* Remove Py_TPFLAGS_HAVE_AM_SEND
    
    The flag is not needed, and since it was added in 3.10 it can be removed now.
2021-07-23 06:21:11 -07:00
Serhiy Storchaka 0fd27375ca
bpo-44654: Refactor and clean up the union type implementation (GH-27196) 2021-07-17 22:44:10 +03:00
T. Wouters 074e7659f2
bpo-44184: Apply GH-26274 to the non-GC-type branch of subtype_dealloc (GH-27165)
The non-GC-type branch of subtype_dealloc is using the type of an object after freeing in the same unsafe way as GH-26274 fixes. (I believe the old news entry covers this change well enough.)

https://bugs.python.org/issue44184
2021-07-16 00:40:57 +02:00
Erlend Egeberg Aasland a3739b207a
bpo-43908: Immutable types inherit vectorcall (GH-27001)
Heap types with the Py_TPFLAGS_IMMUTABLETYPE flag can now inherit the
PEP 590 vectorcall protocol.  Previously, this was only possible for static types.

Co-authored-by: Victor Stinner <vstinner@python.org>
2021-07-08 12:48:01 +02:00