Commit Graph

614 Commits

Author SHA1 Message Date
Jelle Zijlstra b9dce3aec4
gh-104549: Set __module__ on TypeAliasType (#104550) 2023-05-18 15:56:15 -07:00
Jelle Zijlstra 24d8b88420
gh-103763: Implement PEP 695 (#103764)
This implements PEP 695, Type Parameter Syntax. It adds support for:

- Generic functions (def func[T](): ...)
- Generic classes (class X[T](): ...)
- Type aliases (type X = ...)
- New scoping when the new syntax is used within a class body
- Compiler and interpreter changes to support the new syntax and scoping rules 

Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
Co-authored-by: Eric Traut <eric@traut.com>
Co-authored-by: Larry Hastings <larry@hastings.org>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-15 20:36:23 -07:00
Jelle Zijlstra 04f6733275
gh-102500: Implement PEP 688 (#102521)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-05-04 07:59:46 -07:00
Itamar Ostricher fdcb49c36b
gh-104066: Improve performance of hasattr for module objects (#104063) 2023-05-04 08:50:26 -06:00
Itamar Ostricher 8d34031068
gh-104078: Improve performance of PyObject_HasAttrString (#104079) 2023-05-03 00:20:00 -07:00
Eric Snow fdd878650d
gh-94673: Properly Initialize and Finalize Static Builtin Types for Each Interpreter (gh-104072)
Until now, we haven't been initializing nor finalizing the per-interpreter state properly.
2023-05-01 19:36:00 -06:00
Eric Snow 424a785a07
gh-94673: Fix _PyTypes_InitTypes() and get_type_attr_as_size() (gh-103961)
This change has two small parts:

1. a follow-up to gh-103940 with one case I missed
2. adding a missing return that I noticed while working on related code
2023-04-27 18:28:51 -06:00
Eric Snow d2e2e53f73
gh-94673: Ensure Builtin Static Types are Readied Properly (gh-103940)
There were cases where we do unnecessary work for builtin static types. This also simplifies some work necessary for a per-interpreter GIL.
2023-04-27 16:19:43 -06:00
Eric Snow df3173d28e
gh-101659: Isolate "obmalloc" State to Each Interpreter (gh-101660)
This is strictly about moving the "obmalloc" runtime state from
`_PyRuntimeState` to `PyInterpreterState`.  Doing so improves isolation
between interpreters, specifically most of the memory (incl. objects)
allocated for each interpreter's use.  This is important for a
per-interpreter GIL, but such isolation is valuable even without it.

FWIW, a per-interpreter obmalloc is the proverbial
canary-in-the-coalmine when it comes to the isolation of objects between
interpreters.  Any object that leaks (unintentionally) to another
interpreter is highly likely to cause a crash (on debug builds at
least).  That's a useful thing to know, relative to interpreter
isolation.
2023-04-24 17:23:57 -06:00
Alex Gaynor 543009347e
gh-103712: Increase the length of the type name in AttributeError messages (#103713) 2023-04-24 08:23:08 -06:00
Eddie Elizondo ea2c001650
gh-84436: Implement Immortal Objects (gh-19474)
This is the implementation of PEP683

Motivation:

The PR introduces the ability to immortalize instances in CPython which bypasses reference counting. Tagging objects as immortal allows up to skip certain operations when we know that the object will be around for the entire execution of the runtime.

Note that this by itself will bring a performance regression to the runtime due to the extra reference count checks. However, this brings the ability of having truly immutable objects that are useful in other contexts such as immutable data sharing between sub-interpreters.
2023-04-22 13:39:37 -06:00
AN Long d83faf7f1b
gh-103092: Isolate winreg (#103250) 2023-04-17 12:30:48 -06:00
Mark Shannon 411b169281
GH-103082: Implementation of PEP 669: Low Impact Monitoring for CPython (GH-103083)
* The majority of the monitoring code is in instrumentation.c

* The new instrumentation bytecodes are in bytecodes.c

* legacy_tracing.c adapts the new API to the old sys.setrace and sys.setprofile APIs
2023-04-12 12:04:55 +01:00
Nikita Sobolev 059bb04245
gh-102213: Revert "gh-102213: Optimize the performance of `__getattr__` (GH-102248)" (GH-103332)
This reverts commit aa0a73d1bc.
2023-04-07 17:22:36 +08:00
Eric Snow 743687434c
gh-102304: Move the Total Refcount to PyInterpreterState (gh-102545)
Moving it valuable with a per-interpreter GIL.  However, it is also useful without one, since it allows us to identify refleaks within a single interpreter or where references are escaping an interpreter.  This becomes more important as we move the obmalloc state to PyInterpreterState.

https://github.com/python/cpython/issues/102304
2023-03-21 11:46:09 -06:00
Eric Snow ad77d16a62
gh-102304: Move _Py_RefTotal to _PyRuntimeState (gh-102543)
The essentially eliminates the global variable, with the associated benefits. This is also a precursor to isolating this bit of state to PyInterpreterState.

Folks that currently read _Py_RefTotal directly would have to start using _Py_GetGlobalRefTotal() instead.

https://github.com/python/cpython/issues/102304
2023-03-20 10:03:04 -06:00
wangxiang-hz aa0a73d1bc
gh-102213: Optimize the performance of `__getattr__` (GH-102248)
When __getattr__ is defined, python with try to find an attribute using _PyObject_GenericGetAttrWithDict
find nothing is reasonable so we don't need an exception, it will hurt performance.
2023-03-11 19:11:37 +08:00
Eric Snow cbb0aa71d0
gh-102304: Consolidate Direct Usage of _Py_RefTotal (gh-102514)
This simplifies further changes to _Py_RefTotal (e.g. make it atomic or move it to PyInterpreterState).

https://github.com/python/cpython/issues/102304
2023-03-08 12:03:50 -07:00
Irit Katriel 11a2c6ce51
gh-102192: Replace PyErr_Fetch/Restore etc by more efficient alternatives (in Objects/) (#102218) 2023-03-08 17:03:18 +00:00
Dong-hee Na 5b946d3719
gh-101430: Update tracemalloc to handle presize properly. (gh-101745) 2023-02-10 08:30:03 +09:00
Mark Shannon feec49c407
GH-101578: Normalize the current exception (GH-101607)
* Make sure that the current exception is always normalized.

* Remove redundant type and traceback fields for the current exception.

* Add new API functions: PyErr_GetRaisedException, PyErr_SetRaisedException

* Add new API functions: PyException_GetArgs, PyException_SetArgs
2023-02-08 09:31:12 +00:00
Eric Snow 7b20a0f55a
gh-59956: Allow the "Trashcan" Mechanism to Work Without a Thread State (gh-101209)
We've factored out a struct from the two PyThreadState fields. This accomplishes two things:

* make it clear that the trashcan-related code doesn't need any other parts of PyThreadState
* allows us to use the trashcan mechanism even when there isn't a "current" thread state

We still expect the caller to hold the GIL.

https://github.com/python/cpython/issues/59956
2023-01-23 08:30:20 -07:00
Pieter Eendebak 7fc7909677
gh-92216: improve performance of `hasattr` for type objects (GH-99979) 2022-12-23 23:23:36 +08:00
yonillasky 432117cd1f
gh-99540: Constant hash for _PyNone_Type to aid reproducibility (GH-99541)
Needed for ASLR builds of Python.
2022-12-16 12:36:13 -08:00
Victor Stinner 74d5f61ebd
gh-99845: Clean up _PyObject_VAR_SIZE() usage (#99847)
* code_sizeof() now uses an unsigned type (size_t) to compute the result.
* Fix _PyObject_ComputedDictPointer(): cast _PyObject_VAR_SIZE() to
  Py_ssize_t, rather than long: it's a different type on 64-bit Windows.
* Clarify that _PyObject_VAR_SIZE() uses an unsigned type (size_t).
2022-11-29 12:15:21 +01:00
Eric Snow 3c57971a2d
gh-81057: Move Globals in Core Code to _PyRuntimeState (gh-99496)
This is the first of several changes to consolidate non-object globals in core code.

https://github.com/python/cpython/issues/81057
2022-11-15 09:45:11 -07:00
Victor Stinner 3a1dde8f29
gh-99300: Use Py_NewRef() in Objects/ directory (#99354)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in C files of the Objects/ directory.
2022-11-10 23:58:07 +01:00
MonadChains c60b3b3cca
gh-98421: Clean Up PyObject_Print (GH-98422)
Work on test coverage for `PyObject_Print` made it clear that some lines can't get executed.
Simplify the function by excluding the checks for non-string types.
Also eliminate creating a temporary bytes object.
2022-10-20 15:59:10 +02:00
philg314 b9634ac776
gh-96352: Set AttributeError context in _PyObject_GenericGetAttrWithDict (#96353) 2022-09-08 12:12:14 +01:00
Kumar Aditya 4e4bfffe2d
GH-90699: use statically allocated interned strings in typeobject's slotdefs (GH-94706) 2022-09-07 15:02:08 -07:00
Mark Shannon 3ef3c6306d
GH-95707: Fix uses of `Py_TPFLAGS_MANAGED_DICT` (GH-95854)
* Make sure that tp_dictoffset is correct with Py_TPFLAGS_MANAGED_DICT is set.

* Avoid traversing managed dict twice when subclassing class with Py_TPFLAGS_MANAGED_DICT set.
2022-08-15 12:29:27 +01:00
Mark Shannon 8d37c62c2a
GH-92678: Document that you shouldn't be doing your own dictionary offset calculations. (GH-95598)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Stanley <46876382+slateny@users.noreply.github.com>
2022-08-09 14:26:37 +01:00
Mark Shannon de388c0a7b
GH-95245: Store object values and dict pointers in single tagged pointer. (GH-95278) 2022-08-01 14:34:54 +01:00
Eric Snow 4a1dd73431
gh-94673: Add _PyStaticType_InitBuiltin() (#95152)
This is the first of several precursors to storing tp_subclasses (and tp_weaklist) on the interpreter state for static builtin types.

We do the following:

* add `_PyStaticType_InitBuiltin()`
* add `_Py_TPFLAGS_STATIC_BUILTIN`
* set it on all static builtin types in `_PyStaticType_InitBuiltin()`
* shuffle some code around to be able to use _PyStaticType_InitBuiltin()
    * rename `_PyStructSequence_InitType()` to `_PyStructSequence_InitBuiltinWithFlags()`
    * add `_PyStructSequence_InitBuiltin()`.
2022-07-25 12:47:31 -06:00
Victor Stinner 27b9894033
gh-93937, C API: Move PyFrame_GetBack() to Python.h (#93938)
Move the follow functions and type from frameobject.h to pyframe.h,
so the standard <Python.h> provide frame getter functions:

* PyFrame_Check()
* PyFrame_GetBack()
* PyFrame_GetBuiltins()
* PyFrame_GetGenerator()
* PyFrame_GetGlobals()
* PyFrame_GetLasti()
* PyFrame_GetLocals()
* PyFrame_Type

Remove #include "frameobject.h" from many C files. It's no longer
needed.
2022-06-19 12:02:33 +02:00
Victor Stinner 71d8775fee
gh-93202: Always use %zd printf formatter (#93201)
Python now always use the ``%zu`` and ``%zd`` printf formats to
format a size_t or Py_ssize_t number. Building Python 3.12 requires a
C11 compiler, so these printf formats are now always supported.

* PyObject_Print() and _PyObject_Dump() now use the printf %zd format
  to display an object reference count.
* Update PY_FORMAT_SIZE_T comment.
* Remove outdated notes about the %zd format in PyBytes_FromFormat()
  and PyUnicode_FromFormat() documentations.
* configure no longer checks for the %zd format and no longer defines
  PY_FORMAT_SIZE_T macro in pyconfig.h.
* pymacconfig.h no longer undefines PY_FORMAT_SIZE_T: macOS 10.4 is
  no longer supported. Python 3.12 now requires macOS 10.6 (Snow
  Leopard) or newer.
2022-05-25 14:21:36 +02:00
Kumar Aditya c5f5f978ca
GH-92955: fix memory leak in code object lines and positions iterators (gh-92956) 2022-05-19 22:55:22 +09:00
Kumar Aditya d923fdf54b
GH-92804: Fix memory leak in memoryview iterator (gh-92805) 2022-05-14 23:24:20 +09:00
Victor Stinner ffcc7cd57f
gh-89653: PEP 670: Convert pycore_gc.h macros to functions (#92649)
Convert the following macros to static inline functions:

* _Py_AS_GC()
* _PyGCHead_FINALIZED(), _PyGCHead_SET_FINALIZED()
* _PyGCHead_NEXT(), _PyGCHead_SET_NEXT()
* _PyGCHead_PREV(), _PyGCHead_SET_PREV()
* _PyGC_FINALIZED(), _PyGC_SET_FINALIZED()
* _PyObject_GC_IS_TRACKED()
* _PyObject_GC_MAY_BE_TRACKED()

Add a macro wrapping the _PyObject_GC_IS_TRACKED() function to cast
the argument to PyObject*.
2022-05-11 13:37:18 +02:00
Géry Ogam a95138b2c5
bpo-43857: Improve the AttributeError message when deleting a missing attribute (#25424)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-05-05 06:37:26 -07:00
Victor Stinner d716a0dfe2
Use static inline function Py_EnterRecursiveCall() (#91988)
Currently, calling Py_EnterRecursiveCall() and
Py_LeaveRecursiveCall() may use a function call or a static inline
function call, depending if the internal pycore_ceval.h header file
is included or not. Use a different name for the static inline
function to ensure that the static inline function is always used in
Python internals for best performance. Similar approach than
PyThreadState_GET() (function call) and _PyThreadState_GET() (static
inline function).

* Rename _Py_EnterRecursiveCall() to _Py_EnterRecursiveCallTstate()
* Rename _Py_LeaveRecursiveCall() to _Py_LeaveRecursiveCallTstate()
* pycore_ceval.h: Rename Py_EnterRecursiveCall() to
  _Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() and
  _Py_LeaveRecursiveCall()
2022-05-04 13:30:23 +02:00
Victor Stinner 364ed94092
gh-89373: _Py_Dealloc() checks tp_dealloc exception (#32357)
If Python is built in debug mode, _Py_Dealloc() now ensures that the
tp_dealloc function leaves the current exception unchanged.
2022-04-21 23:04:01 +02:00
Dong-hee Na f571c26fc1
gh-91632: Fix generic_alias_iterator to be finalized at exit. (GH-91727) 2022-04-20 23:10:41 +09:00
Dennis Sweeney da6c78584b
gh-90667: Add specializations of Py_DECREF when types are known (GH-30872) 2022-04-19 19:02:19 +01:00
Kumar Aditya 8c54c3dacc
gh-91576: Speed up iteration of strings (#91574) 2022-04-18 07:18:27 -07:00
Victor Stinner 2217462bda
bpo-45786: Remove _PyFrame_Fini() and _PyFrame_DebugMallocStats() (GH-31874)
Remove private empty _PyFrame_Fini() and _PyFrame_DebugMallocStats()
functions.
2022-03-18 11:16:55 +01:00
Pablo Galindo Salgado 3b3be05a16
bpo-46940: Don't override existing AttributeError suggestion information (GH-31710)
When an exception is created in a nested call to PyObject_GetAttr, any
external calls will override the context information of the
AttributeError that we have already placed in the most internal call.
This will cause the suggestions we create to nor work properly as the
attribute name and object that we will be using are the incorrect ones.

To avoid this, we need to check first if these attributes are already
set and bail out if that's the case.
2022-03-07 12:23:11 +00:00
Victor Stinner ad56919c5e
bpo-46857: Fix refleak in OSError INIT_ALIAS() (GH-31594)
_Py_GetRefTotal() no longer decrements _PySet_Dummy refcount.
2022-02-27 00:28:24 +01:00
Victor Stinner 4657bf7016
bpo-1635741: Fix winreg reference leaks (GH-31560)
Clear also the PyHKEY_Type static type at exit.
2022-02-25 12:34:00 +01:00
Eric Snow 81c72044a1
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928)
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code.  It is still used in a number of non-builtin stdlib modules.

The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime.  A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).

https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.

The core of the change is in:

* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers

I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings.  That check is added to the PR CI config.

The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()).  This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.

The following are not changed (yet):

* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init

https://bugs.python.org/issue46541
2022-02-08 13:39:07 -07:00