Commit Graph

8053 Commits

Author SHA1 Message Date
Eric Snow 87e7cb09e4
gh-105699: Fix an Interned Strings Crasher (gh-106930)
A static (process-global) str object must only have its "interned" state cleared when no longer interned in any interpreters.  They are the only ones that can be shared by interpreters so we don't have to worry about any other str objects.

We trigger clearing the state with the main interpreter, since no other interpreters may exist at that point and _PyUnicode_ClearInterned() is only called during interpreter finalization.

We do not address here the fact that a string will only be interned in the first interpreter that interns it.  In any subsequent interpreters str.state.interned is already set so _PyUnicode_InternInPlace() will skip it.  That needs to be addressed separately from fixing the crasher.
2023-07-21 08:32:42 -06:00
Serhiy Storchaka e1c295e3da
gh-106719: Fix __annotations__ getter and setter in the type and module types (GH-106720)
No longer suppress arbitrary errors. Simplify the code.
2023-07-18 08:56:58 +03:00
Dennis Sweeney ab86426a34
gh-105235: Prevent reading outside buffer during mmap.find() (#105252)
* Add a special case for s[-m:] == p in _PyBytes_Find

* Add tests for _PyBytes_Find

* Make sure that start <= end in mmap.find
2023-07-12 22:50:45 -04:00
Serhiy Storchaka be1b968dc1
gh-106521: Remove _PyObject_LookupAttr() function (GH-106642) 2023-07-12 08:57:10 +03:00
Serhiy Storchaka 4bf43710d1
gh-106307: C API: Add PyMapping_GetOptionalItem() function (GH-106308)
Also add PyMapping_GetOptionalItemString() function.
2023-07-11 23:04:12 +03:00
Serhiy Storchaka 579aa89e68
gh-106521: Add PyObject_GetOptionalAttr() function (GH-106522)
It is a new name of former _PyObject_LookupAttr().

Add also PyObject_GetOptionalAttrString().
2023-07-11 22:13:27 +03:00
Jelle Zijlstra 945d3cbf2e
gh-106403: Restore weakref support for TypeVar and friends (#106418) 2023-07-11 08:43:24 -07:00
Victor Stinner 1f2921b72c
gh-106572: Convert PyObject_DelAttr() to a function (#106611)
* Convert PyObject_DelAttr() and PyObject_DelAttrString() macros to
  functions.
* Add PyObject_DelAttr() and PyObject_DelAttrString() functions to
  the stable ABI.
* Replace PyObject_SetAttr(obj, name, NULL) with
  PyObject_DelAttr(obj, name).
2023-07-11 11:38:22 +02:00
Eric Snow a840806d33
gh-105227: Add PyType_GetDict() (GH-105747)
This compensates for static builtin types having `tp_dict` set to `NULL`.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2023-07-10 18:41:02 +02:00
Hugo van Kemenade 34c14147a2
gh-106487: Allow the 'count' argument of `str.replace` to be a keyword (#106488) 2023-07-10 12:52:36 +03:00
Victor Stinner ee46cb6aa9
gh-105927: PyWeakref_GetRef() returns 1 on success (#106561)
PyWeakref_GetRef() now returns 1 on success, and return 0 if the
reference is dead.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-07-09 15:50:26 +00:00
Serhiy Storchaka 93d292c2b3
gh-106303: Use _PyObject_LookupAttr() instead of PyObject_GetAttr() (GH-106304)
It simplifies and speed up the code.
2023-07-09 15:27:03 +03:00
Carl Meyer 104d7b760f
gh-105340: include hidden fast-locals in locals() (#105715)
* gh-105340: include hidden fast-locals in locals()
2023-07-05 17:05:02 -06:00
Dong-hee Na 217f47d6e5
gh-96844: Improve error message of list.remove (gh-106455) 2023-07-06 07:19:49 +09:00
Victor Stinner c5afc97fc2
gh-106320: Remove private _PyErr C API functions (#106356)
Remove private _PyErr C API functions: move them to the internal
C API (pycore_pyerrors.h).
2023-07-03 10:48:50 +00:00
Victor Stinner 35963da40f
gh-106320: Create pycore_modsupport.h header file (#106355)
Remove the following functions from the C API, move them to the internal C
API: add a new pycore_modsupport.h internal header file:

* PyModule_CreateInitialized()
* _PyArg_NoKwnames()
* _Py_VaBuildStack()

No longer export these functions.
2023-07-03 09:39:11 +00:00
Victor Stinner c38c66687f
gh-106320: Add pycore_complexobject.h header file (#106339)
Add internal pycore_complexobject.h header file.

Move _Py_c_xxx() functions and _PyComplex_FormatAdvancedWriter()
function to this new header file.
2023-07-02 21:19:59 +00:00
Victor Stinner bc7eb17084
gh-106320: Use _PyInterpreterState_GET() (#106336)
Replace PyInterpreterState_Get() with inlined
_PyInterpreterState_GET().
2023-07-02 16:37:37 +00:00
Inada Naoki d5bd32fb48
gh-104922: remove PY_SSIZE_T_CLEAN (#106315) 2023-07-02 15:07:46 +09:00
Victor Stinner 18b1fdebe0
gh-106320: Remove _PyInterpreterState_Get() alias (#106321)
Replace calls to the (removed) slow _PyInterpreterState_Get() with
fast inlined _PyInterpreterState_GET() function.
2023-07-01 23:44:07 +00:00
Md Sadman Chowdhury 822db860ea
Fix duplicate word typos in comments (#106225) 2023-07-01 12:47:14 -07:00
Nikita Sobolev eb7d6e7ad8
gh-105486: Change the `repr` of `ParamSpec` list of args in `GenericAlias` (#105488) 2023-07-01 00:04:50 +00:00
Victor Stinner 1a2652ceaa
Cleanup clear_static_tp_subclasses() (#106276)
Only iterate on the dictionary if Python is built with assertions:
subclass is only needed when Python is built with assertions.
2023-06-30 10:39:55 +00:00
Victor Stinner 2efdd2a14e
gh-106023: Remove _PyObject_FastCall() function (#106265) 2023-06-30 11:46:43 +02:00
Victor Stinner 0b51463862
Remove private _PyCodec_Lookup() function (#106269)
Remove the following private functions of the C API:

* _PyCodecInfo_GetIncrementalDecoder()
* _PyCodecInfo_GetIncrementalEncoder()
* _PyCodec_DecodeText()
* _PyCodec_EncodeText()
* _PyCodec_Forget()
* _PyCodec_Lookup()
* _PyCodec_LookupTextEncoding()

Move these functions to a new pycore_codecs.h internal header file.

These functions are no longer exported.
2023-06-30 09:34:01 +00:00
Victor Stinner e17420db5d
gh-106023: Rename _PyObject_FastCallDictTstate() (#106264)
Rename _PyObject_FastCallDictTstate() to
_PyObject_VectorcallDictTstate().
2023-06-30 07:53:42 +00:00
Inada Naoki 77ddc9a7b1
fix typos (#106247)
Most typos are in comments, but two typos are in docstring.
2023-06-30 13:00:22 +09:00
Victor Stinner 8c5f74fc89
gh-106023: Update code using _PyObject_FastCall() (#106257)
Replace _PyObject_FastCall() calls with PyObject_Vectorcall().
2023-06-30 01:05:01 +00:00
Serhiy Storchaka 3c70d467c1
Fix possible refleak in CodeType.replace() (GH-106243)
A reference to c_code was leaked if PySys_Audit() failed.
2023-06-29 17:22:15 +03:00
Serhiy Storchaka 08c08d21b0
gh-106033: Get rid of PyDict_GetItem in _PyFunction_FromConstructor (GH-106044) 2023-06-29 12:31:08 +03:00
Victor Stinner 3f8483cad2
gh-106168: PyTuple_SET_ITEM() now checks the index (#106164)
PyTuple_SET_ITEM() and PyList_SET_ITEM() now check the index argument
with an assertion if Python is built in debug mode or is built with
assertions.

* list_extend() and _PyList_AppendTakeRef() now set the list size
  before calling PyList_SET_ITEM().
* PyStructSequence_GetItem() and PyStructSequence_SetItem() now check
  the index argument: must be lesser than REAL_SIZE(op).
* PyStructSequence_GET_ITEM() and PyStructSequence_SET_ITEM() are now
  aliases to PyStructSequence_GetItem() and
  PyStructSequence_SetItem().
2023-06-28 03:45:57 +02:00
Victor Stinner 84caa3324a
gh-106084: Remove _PyObject_CallMethod() function (#106159)
Remove the following private functions from the public C API:

* _Py_CheckFunctionResult()
* _PyObject_CallMethod()
* _PyObject_CallMethodId()
* _PyObject_CallMethodIdNoArgs()
* _PyObject_CallMethodIdObjArgs()
* _PyObject_CallMethodIdOneArg()
* _PyObject_MakeTpCall()
* _PyObject_VectorcallMethodId()
* _PyStack_AsDict()

Move these functions to the internal C API (pycore_call.h).

No longer export the following functions:

* _PyObject_Call()
* _PyObject_CallMethod()
* _PyObject_CallMethodId()
* _PyObject_CallMethodIdObjArgs()
* _PyObject_Call_Prepend()
* _PyObject_FastCallDictTstate()
* _PyStack_AsDict()

The following functions are still exported for stdlib shared
extensions:

* _Py_CheckFunctionResult()
* _PyObject_MakeTpCall()

Mark the following internal functions as extern:

* _PyStack_UnpackDict()
* _PyStack_UnpackDict_Free()
* _PyStack_UnpackDict_FreeNoDecRef()
2023-06-28 01:34:37 +02:00
Victor Stinner e8e59ee474
gh-106084: Remove _PyObject_RealIsInstance() function (#106106)
Remove the following functions from the public C API:

* _PyObject_RealIsInstance()
* _PyObject_RealIsSubclass()
* _Py_add_one_to_index_F()
* _Py_add_one_to_index_C()

Move _PyObject_RealIsInstance() and _PyObject_RealIsSubclass() to the
internal C API (pycore_abstract.h) and no longer export their symbols
(in libpython).

Make _Py_add_one_to_index_F() and _Py_add_one_to_index_C() functions
static: no longer export them.
2023-06-26 13:38:41 +00:00
Victor Stinner 6200aaf296
gh-106084: Remove _PyObject_HasLen() function (#106103)
Remove _PyObject_HasLen() and _PySequence_IterSearch() functions from
the public C API: move them to the internal C API
(pycore_abstract.h).

No longer export these symbols (in libpython).

Remove also unused pycore_initconfig.h include in typeobject.c.
2023-06-26 13:02:43 +00:00
Victor Stinner c075a1974b
gh-105927: Deprecate PyWeakref_GetObject() function (#106006)
Deprecate PyWeakref_GetObject() and PyWeakref_GET_OBJECT() functions.
2023-06-26 12:10:53 +02:00
Victor Stinner 7ca871634e
gh-106084: Remove _PySequence_BytesToCharpArray() function (#106088)
Remove private _PySequence_BytesToCharpArray() and
_Py_FreeCharPArray() functions from the public C API: move these
functions from Objects/abstract.c to Modules/_posixsubprocess.c.
2023-06-26 08:30:59 +02:00
Victor Stinner 00e75a3372
gh-106084: Remove old PyObject call aliases (#106085)
Remove old aliases which were kept backwards compatibility with
Python 3.8:

* _PyObject_CallMethodNoArgs()
* _PyObject_CallMethodOneArg()
* _PyObject_CallOneArg()
* _PyObject_FastCallDict()
* _PyObject_Vectorcall()
* _PyObject_VectorcallMethod()
* _PyVectorcall_Function()

Update code which used these aliases to use new names.
2023-06-26 08:08:12 +02:00
Irit Katriel d8ca5a11bc
gh-105730: support more callables in ExceptionGroup.split() and subgroup() (#106035) 2023-06-23 19:47:47 +01:00
Serhiy Storchaka 1d33d53780
gh-106033: Get rid of new occurrences of PyDict_GetItem and PyObject_HasAttr (GH-106034)
These functions are broken by design because they discard any exceptions raised
inside, including MemoryError and KeyboardInterrupt.  They should not be
used in new code.
2023-06-23 20:10:32 +03:00
Victor Stinner 46a3190fcf
gh-105927: Avoid calling PyWeakref_GET_OBJECT() (#105997)
* Replace PyWeakref_GET_OBJECT() with _PyWeakref_GET_REF().
* _sqlite/blob.c now holds a strong reference to the blob object
  while calling close_blob().
* _xidregistry_find_type() now holds a strong reference to registered
  while using it.
2023-06-22 22:31:31 +02:00
Mark Shannon 04492cbc9a
GH-91095: Specialize calls to normal Python classes. (GH-99331) 2023-06-22 09:48:19 +01:00
Victor Stinner 4328dc6465
gh-105927: finalize_modules_clear_weaklist() uses _PyWeakref_GET_REF() (#105971)
finalize_modules_clear_weaklist() now holds a strong reference to the
module longer than before: replace PyWeakref_GET_OBJECT() with
_PyWeakref_GET_REF().
2023-06-21 21:50:20 +02:00
Victor Stinner fc32522b08
gh-105927: type_from_ref() uses _PyWeakref_GET_REF() (#105963)
type_from_ref() now returns a strong reference to the type, instead
of a borrowed reference: replace PyWeakref_GET_OBJECT() with
_PyWeakref_GET_REF().
2023-06-21 16:35:58 +02:00
Victor Stinner 9c44656feb
gh-105927: Add PyWeakref_GetRef() function (#105932)
Add tests on PyWeakref_NewRef(), PyWeakref_GetObject(),
PyWeakref_GET_OBJECT() and PyWeakref_GetRef().
2023-06-21 11:40:09 +02:00
Victor Stinner cb388c9a85
gh-105927: Add _PyWeakref_GET_REF() internal function (#105929)
Add new pycore_weakref.h internal header file.
2023-06-20 08:52:40 +02:00
Victor Stinner 7f97c8e367
gh-105927: Refactor weakrefobject.c (#105928)
* Rename proxy_checkref() to proxy_check_ref().
* proxy_check_ref() now checks the object, not the proxy.
* Most functions take PyObject* instead of PyWeakReference*.
* Remove redundant calls to PyWeakref_GET_OBJECT().
2023-06-19 23:31:17 +00:00
Mark Shannon 581619941e
GH-104584: Assorted fixes for the optimizer API. (GH-105683)
* Add test for long loops

* Clear ENTER_EXECUTOR when deopting code objects.
2023-06-19 10:32:20 +01:00
Mark Shannon 7199584ac8
GH-100987: Allow objects other than code objects as the "executable" of an internal frame. (GH-105727)
* Add table describing possible executable classes for out-of-process debuggers.

* Remove shim code object creation code as it is no longer needed.

* Make lltrace a bit more robust w.r.t. non-standard frames.
2023-06-14 13:46:37 +01:00
Petr Viktorin 2b90796be6
gh-103968: PyType_FromMetaclass: Allow metaclasses with tp_new=NULL (GH-105386) 2023-06-12 17:45:49 +02:00
Erlend E. Aasland 555be81026
gh-105375: Improve error handling in PyUnicode_BuildEncodingMap() (#105491)
Bail on first error to prevent exceptions from possibly being overwritten.
2023-06-11 21:29:19 +02:00
Alex Waygood 8e755923c9
Miscellaneous improvements to the typing docs (#105529)
Mostly, these are changes so that we use shorter sentences and shorter paragraphs. In particular, I've tried to make the first sentence introducing each object in the typing API short and declarative.
2023-06-09 15:08:57 +00:00
Eric Snow 68dfa49627
gh-100227: Lock Around Modification of the Global Allocators State (gh-105516)
The risk of a race with this state is relatively low, but we play it safe anyway. We do avoid using the lock in performance-sensitive cases where the risk of a race is very, very low.
2023-06-08 14:06:54 -06:00
Dong-hee Na 058b960535
gh-103906: Remove immortal refcounting in compile/marshal.c (gh-103922) 2023-06-05 22:38:36 +09:00
Gregory P. Smith 418befd75d
gh-98963: Restore the ability to have a dict-less property. (#105262)
Ignore doc string assignment failures in `property` as has been the
behavior of all past Python releases.
2023-06-05 03:18:15 +00:00
Eric Snow e6373c0d8b
gh-101524: Only Use Public C-API in the _xxsubinterpreters Module (gh-105258)
The _xxsubinterpreters module was meant to only use public API.  Some internal C-API usage snuck in over the last few years (e.g. gh-28969).  This fixes that.
2023-06-02 22:52:33 +00:00
Mark Shannon 4bfa01b9d9
GH-104584: Plugin optimizer API (GH-105100) 2023-06-02 11:46:18 +01:00
Mark Shannon 601ae09f0c
GH-105162: Account for `INSTRUMENTED_RESUME` in gen.close/throw. (GH-105187) 2023-06-02 10:39:38 +01:00
Inada Naoki 37498fc950
gh-85275: Remove old buffer APIs (#105137)
They are now abi-only.

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-06-02 01:12:40 +00:00
Victor Stinner ef300937c2
gh-92536: Remove PyUnicode_READY() calls (#105210)
Since Python 3.12, PyUnicode_READY() does nothing and always
returns 0.
2023-06-02 01:33:17 +02:00
Victor Stinner cbb9ba844f
gh-92536: Argument Clinic no longer emits PyUnicode_READY() (#105208)
Since Python 3.12, PyUnicode_READY() does nothing and always
returns 0.

Argument Clinic now also checks for .cpp files (PC/_wmimodule.cpp).
2023-06-02 01:31:58 +02:00
Eric Snow 146939306a
gh-104614: Make Sure ob_type is Always Set Correctly by PyType_Ready() (gh-105122)
When I added the relevant condition to type_ready_set_bases() in gh-103912, I had missed that the function also sets tp_base and ob_type (if necessary).  That led to problems for third-party static types.

We fix that here, by making those extra operations distinct and by adjusting the condition to be more specific.
2023-06-01 22:28:31 +00:00
Irit Katriel 60f8117d0c
gh-105140: remove unused arg of _PyErr_ChainStackItem (#105141) 2023-06-01 20:57:28 +00:00
Victor Stinner 27f9491c60
gh-105107: Remove PyCFunction_Call() function (#105181)
* Keep the function in the stable ABI.
* Add unit tests on PyCFunction_Call() since it remains supported in
  the stable ABI.
2023-06-01 11:25:55 +02:00
Victor Stinner 7d07e5891d
gh-105156: Cleanup usage of old Py_UNICODE type (#105158)
* refcounts.dat:

  * Remove Py_UNICODE functions.
  * Replace Py_UNICODE argument type with wchar_t.

* _PyUnicode_ToLowercase(), _PyUnicode_ToUppercase(),
  _PyUnicode_ToTitlecase() are no longer deprecated in comments.
  It's no longer needed since they now use Py_UCS4 type, rather than
  the deprecated Py_UNICODE type.
* gdb: Remove unused char_width() method.
2023-06-01 07:18:09 +00:00
Victor Stinner 8ed705c083
gh-105156: Deprecate the old Py_UNICODE type in C API (#105157)
Deprecate the old Py_UNICODE and PY_UNICODE_TYPE types in the C API:
use wchar_t instead.

Replace Py_UNICODE with wchar_t in multiple C files.

Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2023-06-01 08:56:35 +02:00
Victor Stinner 579c41c102
gh-105107: Remove PyEval_CallFunction() function (#105108)
Remove 4 functions from the C API, deprecated in Python 3.9:

* PyEval_CallObjectWithKeywords()
* PyEval_CallObject()
* PyEval_CallFunction()
* PyEval_CallMethod()

Keep 3 functions in the stable ABI:

* PyEval_CallObjectWithKeywords()
* PyEval_CallFunction()
* PyEval_CallMethod()
2023-05-31 11:17:06 +00:00
Inada Naoki adccff3b3f
gh-104922: Make `PY_SSIZE_T_CLEAN` not mandatory again (#105051) 2023-05-31 18:38:55 +09:00
Eric Snow 7be667dfaf
gh-105020: Share tp_bases and tp_mro Between Interpreters For All Static Builtin Types (gh-105115)
In gh-103912 we added tp_bases and tp_mro to each PyInterpreterState.types.builtins entry.  However, doing so ignored the fact that both PyTypeObject fields are public API, and not documented as internal (as opposed to tp_subclasses).  We address that here by reverting back to shared objects, making them immortal in the process.
2023-05-31 00:13:35 +00:00
Irit Katriel b7aadb4583
gh-105071: add PyUnstable_Exc_PrepReraiseStar to expose except* implementation in the unstable API (#105072) 2023-05-30 15:03:36 +01:00
Inada Naoki e92ac0a741
Fix compiler warning in unicodeobject.c (#105050) 2023-05-29 17:31:03 +09:00
Jelle Zijlstra 060277d96b
gh-103921: Document PEP 695 (#104642)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-26 10:48:17 -07:00
Jelle Zijlstra 6e1eccdcce
gh-104955: Fix __release_buffer__ signature (#104956)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-05-26 05:44:23 -07:00
Kumar Aditya 7fc542c88d
GH-89091: raise `RuntimeWarning` for unawaited async generator methods (#104611) 2023-05-26 16:53:29 +05:30
Jelle Zijlstra fe77a99fc8
gh-104879: Fix TypeAliasType.__module__ in exec() (#104881) 2023-05-24 11:48:13 -07:00
Nicolas Tessore d56c933992
gh-104770: Let generator.close() return value (#104771)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2023-05-23 13:51:56 -07:00
Serhiy Storchaka f3466bc040
gh-98836: Extend PyUnicode_FromFormat() (GH-98838)
* Support for conversion specifiers o (octal) and X (uppercase hexadecimal).
* Support for length modifiers j (intmax_t) and t (ptrdiff_t).
* Length modifiers are now applied to all integer conversions.
* Support for wchar_t C strings (%ls and %lV).
* Support for variable width and precision (*).
* Support for flag - (left alignment).
2023-05-22 00:32:39 +03:00
Mark Shannon 93923793f6
GH-101291: Add low level, unstable API for pylong (GH-101685)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2023-05-21 14:45:48 +01:00
Dong-hee Na b9fcfa6024
gh-104717: Add comment about manual loop unrolling (gh-104718) 2023-05-21 21:08:28 +09:00
Jelle Zijlstra 8f1f3b9abd
gh-104600: Make type.__type_params__ writable (#104634)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-19 09:04:47 -07:00
Jelle Zijlstra 3fadd7d585
gh-104600: Make function.__type_params__ writable (#104601) 2023-05-18 16:45:37 -07:00
Jelle Zijlstra b9dce3aec4
gh-104549: Set __module__ on TypeAliasType (#104550) 2023-05-18 15:56:15 -07:00
Jelle Zijlstra 26931944dd
typing: Add more tests for TypeVar (#104571)
During the PEP 695 implementation at one point I made
TypeVar.__name__ return garbage, and all of test_typing passed.
So I decided to add a few more tests. In the process I discovered
a minor incompatibility from the C implementation of TypeVar:
empty constraints were returned as None instead of an empty tuple.
2023-05-17 06:08:21 -07:00
Brandt Bucher b4a9747923
GH-103906: Remove immortal refcounting in the interpreter (GH-103909) 2023-05-16 14:36:02 -07:00
Jelle Zijlstra 24d8b88420
gh-103763: Implement PEP 695 (#103764)
This implements PEP 695, Type Parameter Syntax. It adds support for:

- Generic functions (def func[T](): ...)
- Generic classes (class X[T](): ...)
- Type aliases (type X = ...)
- New scoping when the new syntax is used within a class body
- Compiler and interpreter changes to support the new syntax and scoping rules 

Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
Co-authored-by: Eric Traut <eric@traut.com>
Co-authored-by: Larry Hastings <larry@hastings.org>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-15 20:36:23 -07:00
Charles Machalow 7d2deafb73
gh-104454: Fix refleak in AttributeError_reduce (#104455)
* Fix the reference leak introduced by https://github.com/python/cpython/issues/103333

Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
2023-05-13 13:45:36 -07:00
Brandt Bucher a10b026f0f
GH-94841: Fix usage of Py_ALWAYS_INLINE (GH-104409) 2023-05-12 14:26:54 -07:00
Charles Machalow 79b17f2cf0
gh-103333: Pickle the keyword attributes of AttributeError (#103352)
* Pickle the `name` and `args` attributes of AttributeError when present.

Co-authored-by: Gregory P. Smith <greg@krypto.org>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-12 13:33:23 -07:00
Kumar Aditya a052be4c0d
GH-104371: check return value of calling `mv.release` (#104417) 2023-05-12 18:27:06 +05:30
Mark Shannon 45f5aa8fc7
GH-103082: Filter LINE events in VM, to simplify tool implementation. (GH-104387)
When monitoring LINE events, instrument all instructions that can have a predecessor on a different line.
Then check that the a new line has been hit in the instrumentation code.
This brings the behavior closer to that of 3.11, simplifying implementation and porting of tools.
2023-05-12 12:21:20 +01:00
Brandt Bucher a781484c8e
Fix refleak in `super_descr_get` (#104408) 2023-05-12 12:41:27 +05:30
Jelle Zijlstra a0a98ddb31
gh-104371: Fix calls to `__release_buffer__` while an exception is active (#104378)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-05-12 05:22:40 +00:00
Carl Meyer 77262458fe
gh-87729: improve hit rate of LOAD_SUPER_ATTR specialization (#104270) 2023-05-11 08:08:13 -06:00
penguin_wwy 373bca0cc5
GH-102181: Improve specialization stats for SEND (GH-102182) 2023-05-10 22:40:59 +00:00
Sebastian Berg 7a3b03509e
gh-104263: Rely on Py_NAN and introduce Py_INFINITY (GH-104202)
This PR removes `_Py_dg_stdnan` and `_Py_dg_infinity` in favour of
using the standard `NAN` and `INFINITY` macros provided by C99.
This change has the side-effect of fixing a bug on MIPS where the
hard-coded value used by `_Py_dg_stdnan` gave a signalling NaN
rather than a quiet NaN.
---------

Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2023-05-10 17:44:52 +01:00
Eric Snow b8f7ab5783
gh-104252: Immortalize Py_EMPTY_KEYS (gh-104253)
This was missed in gh-19474.  It matters for with a per-interpreter GIL since PyDictKeysObject.dk_refcnt breaks isolation and leads to races.
2023-05-10 07:28:40 -06:00
Carl Meyer c3b595e73e
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 11:02:14 -06:00
Jelle Zijlstra 405eacc1b8
gh-104223: Fix issues with inheriting from buffer classes (#104227)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-05-08 09:52:41 -07:00
John Belmonte 69621d1b09
gh-104018: remove unused format "z" handling in string formatfloat() (#104107)
This is a cleanup overlooked in PR #104033.
2023-05-07 10:11:42 +05:30
Eric Snow fff193bbfe
gh-99113: Add a check for Py_MOD_PER_INTERPRETER_GIL_SUPPORTED (gh-104206)
Py_MOD_PER_INTERPRETER_GIL_SUPPORTED is a new supported value for Py_mod_multiple_interpreters, added in gh-104205.
2023-05-06 21:57:35 +00:00
Eric Snow a9c6e0618f
gh-99113: Add Py_MOD_PER_INTERPRETER_GIL_SUPPORTED (gh-104205)
Here we are doing no more than adding the value for Py_mod_multiple_interpreters and using it for stdlib modules.  We will start checking for it in gh-104206 (once PyInterpreterState.ceval.own_gil is added in gh-104204).
2023-05-05 21:11:27 +00:00
Eric Snow 1c420e138f
gh-104108: Add the Py_mod_multiple_interpreters Module Def Slot (gh-104148)
I'll be adding a value to indicate support for per-interpreter GIL in gh-99114.
2023-05-05 14:04:55 -06:00
Mark Shannon a0df9ee8fc
GH-96803: Add three C-API functions to make _PyInterpreterFrame less opaque for users of PEP 523. (GH-96849) 2023-05-05 17:53:07 +01:00
Brandt Bucher 7d35c3121a
GH-103899: Provide a hint when accidentally calling a module (GH-103900) 2023-05-04 15:07:42 -07:00
Jelle Zijlstra 04f6733275
gh-102500: Implement PEP 688 (#102521)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-05-04 07:59:46 -07:00
Itamar Ostricher fdcb49c36b
gh-104066: Improve performance of hasattr for module objects (#104063) 2023-05-04 08:50:26 -06:00
Petr Viktorin cd9a56c2b0
gh-103509: PEP 697 -- Limited C API for Extending Opaque Types (GH-103511)
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-04 09:56:53 +02:00
Petr Viktorin 524a7f77fd
gh-103968: Deprecate creating heap types whose metaclass has custom tp_new. (GH-103972)
(That's a mouthful of an edge case!)

Co-authored-by: Barney Gale <barney.gale@gmail.com>
2023-05-03 15:17:14 +02:00
Itamar Ostricher 8d34031068
gh-104078: Improve performance of PyObject_HasAttrString (#104079) 2023-05-03 00:20:00 -07:00
Tim Hoffmann fdb3ef8c0f
gh-82012: Deprecate bitwise inversion (~) of bool (#103487)
The bitwise inversion operator on bool returns the bitwise inversion of the
underlying int value; i.e. `~True == -2` such that `bool(~True) == True`.

It's a common pitfall that users mistake `~` as negation operator and actually
want `not`. Supporting `~` is an artifact of bool inheriting from int. Since there
is no real use-case for the current behavior, let's deprecate `~` on bool and
later raise an error. This removes a potential source errors for users.

Full reasoning: https://github.com/python/cpython/issues/82012#issuecomment-1258705971

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2023-05-03 00:00:42 -07:00
Eric Snow de64e75616
gh-94673: More Per-Interpreter Fields for Builtin Static Types (gh-103912)
his involves moving tp_dict, tp_bases, and tp_mro to PyInterpreterState, in the same way we did for tp_subclasses.  Those three fields are effectively const for builtin static types (unlike tp_subclasses).  In theory we only need to make their values immortal, along with their contents.  However, that isn't such a simple proposition.  (See gh-103823.)  In the meantime the simplest solution is to move the fields into the interpreter.

One alternative is to statically allocate the values, but that's its own can of worms.
2023-05-02 21:30:03 -06:00
Eric Snow f73abf8e03
gh-94673: Hide Objects in PyTypeObject Behind Accessors (gh-104074)
This makes it much cleaner to move more PyTypeObject fields to PyInterpreterState.
2023-05-01 20:34:43 -06:00
Eric Snow fdd878650d
gh-94673: Properly Initialize and Finalize Static Builtin Types for Each Interpreter (gh-104072)
Until now, we haven't been initializing nor finalizing the per-interpreter state properly.
2023-05-01 19:36:00 -06:00
Eric Snow 59bc36aacd
gh-84436: Immortalize in _PyStructSequence_InitBuiltinWithFlags() (gh-104054)
This also does some cleanup.
2023-05-01 15:08:34 -06:00
John Belmonte 3ed8c88290
gh-104018: disallow "z" format specifier in %-format of byte strings (GH-104033)
PEP-0682 specified that %-formatting would not support the "z" specifier,
but it was unintentionally allowed for bytes. This PR makes use of the "z"
flag an error for %-formatting in a bytestring.

Issue: #104018

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-05-01 20:47:14 +01:00
sunmy2019 59c27fa5cb
gh-102213: Optimize the performance of `__getattr__` (GH-103761)
Co-authored-by: Kirill <80244920+Eclips4@users.noreply.github.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
Co-authored-by: Xiang Wang <34048878+wangxiang-hz@users.noreply.github.com>
2023-05-01 18:10:35 +08:00
Ken Jin ed95e8cbd4
gh-98003: Inline call frames for CALL_FUNCTION_EX (GH-98004) 2023-04-30 21:08:26 +08:00
Eric Snow 424a785a07
gh-94673: Fix _PyTypes_InitTypes() and get_type_attr_as_size() (gh-103961)
This change has two small parts:

1. a follow-up to gh-103940 with one case I missed
2. adding a missing return that I noticed while working on related code
2023-04-27 18:28:51 -06:00
Eric Snow d2e2e53f73
gh-94673: Ensure Builtin Static Types are Readied Properly (gh-103940)
There were cases where we do unnecessary work for builtin static types. This also simplifies some work necessary for a per-interpreter GIL.
2023-04-27 16:19:43 -06:00
Irit Katriel 63842bd907
gh-103590: do not wrap a single exception raised from a try-except* (#103665) 2023-04-27 12:52:15 +01:00
Carl Meyer ef25febcf2
gh-87729: specialize LOAD_SUPER_ATTR_METHOD (#103809) 2023-04-25 17:45:51 +00:00
sunmy2019 0acea96dad
gh-103826: fix unused variable warning introduced in gh-102343 (#103825) 2023-04-25 15:31:04 +05:30
Eric Snow df3173d28e
gh-101659: Isolate "obmalloc" State to Each Interpreter (gh-101660)
This is strictly about moving the "obmalloc" runtime state from
`_PyRuntimeState` to `PyInterpreterState`.  Doing so improves isolation
between interpreters, specifically most of the memory (incl. objects)
allocated for each interpreter's use.  This is important for a
per-interpreter GIL, but such isolation is valuable even without it.

FWIW, a per-interpreter obmalloc is the proverbial
canary-in-the-coalmine when it comes to the isolation of objects between
interpreters.  Any object that leaks (unintentionally) to another
interpreter is highly likely to cause a crash (on debug builds at
least).  That's a useful thing to know, relative to interpreter
isolation.
2023-04-24 17:23:57 -06:00
Eric Snow 209a0a7655
gh-95795: Move types.next_version_tag to PyInterpreterState (gh-102343)
Core static types will continue to use the global value.  All other types
will use the per-interpreter value.  They all share the same range, where
the global types use values < 2^16 and each interpreter uses values
higher than that.
2023-04-24 22:30:13 +00:00
Carl Meyer 0dc8b50d33
gh-87729: add LOAD_SUPER_ATTR instruction for faster super() (#103497)
This speeds up `super()` (by around 85%, for a simple one-level
`super().meth()` microbenchmark) by avoiding allocation of a new
single-use `super()` object on each use.
2023-04-24 22:22:14 +00:00
Nikita Sobolev 58b6be3791
gh-99184: Bypass instance attribute access in `repr` of `weakref.ref` (#99244) 2023-04-24 12:57:36 -06:00
Brett Simmers b7f4811c88
gh-103091: Add PyUnstable_Type_AssignVersionTag (#103095) 2023-04-24 10:07:47 -06:00
Alex Gaynor 543009347e
gh-103712: Increase the length of the type name in AttributeError messages (#103713) 2023-04-24 08:23:08 -06:00
Eddie Elizondo ea2c001650
gh-84436: Implement Immortal Objects (gh-19474)
This is the implementation of PEP683

Motivation:

The PR introduces the ability to immortalize instances in CPython which bypasses reference counting. Tagging objects as immortal allows up to skip certain operations when we know that the object will be around for the entire execution of the runtime.

Note that this by itself will bring a performance regression to the runtime due to the extra reference count checks. However, this brings the ability of having truly immutable objects that are useful in other contexts such as immutable data sharing between sub-interpreters.
2023-04-22 13:39:37 -06:00
Eric Wieser 3d2a46845b
gh-83791: Raise TypeError for len(memoryview_0d) (#18463)
Changes the behaviour of `len` on a zero-dimensional `memoryview` to raise `TypeError`. Previously, `len` would return `1`.
2023-04-22 17:32:47 +01:00
AN Long d83faf7f1b
gh-103092: Isolate winreg (#103250) 2023-04-17 12:30:48 -06:00
Mark Shannon 411b169281
GH-103082: Implementation of PEP 669: Low Impact Monitoring for CPython (GH-103083)
* The majority of the monitoring code is in instrumentation.c

* The new instrumentation bytecodes are in bytecodes.c

* legacy_tracing.c adapts the new API to the old sys.setrace and sys.setprofile APIs
2023-04-12 12:04:55 +01:00
Nikita Sobolev 50b4b15984
gh-87864: Use correct function definition syntax in the docs (#103312) 2023-04-11 16:50:25 +03:00
Irit Katriel 55c99d97e1
gh-77757: replace exception wrapping by PEP-678 notes in typeobject's __set_name__ (#103402) 2023-04-11 11:53:06 +01:00
Nikita Sobolev 059bb04245
gh-102213: Revert "gh-102213: Optimize the performance of `__getattr__` (GH-102248)" (GH-103332)
This reverts commit aa0a73d1bc.
2023-04-07 17:22:36 +08:00
Nikita Sobolev 2a721258a1
gh-101865: Deprecate `co_lnotab` from code objects as per PEP 626 (#101866)
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
2023-04-03 17:35:04 +02:00
Brandt Bucher 121057aa36
GH-89987: Shrink the BINARY_SUBSCR caches (GH-103022) 2023-03-29 15:53:30 -07:00
Eric Snow ba65a065cf
gh-100227: Move the Dict of Interned Strings to PyInterpreterState (gh-102339)
We can revisit the options for keeping it global later, if desired.  For now the approach seems quite complex, so we've gone with the simpler isolation solution in the meantime.

https://github.com/python/cpython/issues/100227
2023-03-28 12:52:28 -06:00
Eric Snow 89e67ada69
gh-100227: Revert gh-102925 "gh-100227: Make the Global Interned Dict Safe for Isolated Interpreters" (gh-103063)
This reverts commit 87be8d9.

This approach to keeping the interned strings safe is turning out to be too complex for my taste (due to obmalloc isolation). For now I'm going with the simpler solution, making the dict per-interpreter. We can revisit that later if we want a sharing solution.
2023-03-27 16:53:05 -06:00
Nikita Sobolev 2cdc5189a6
gh-102941: Fix "‘subobj’ may be used uninitialized in this function" warning in `bytes_methods.c` (#102942) 2023-03-26 17:05:06 -07:00
Brandt Bucher 0444ae2487
GH-100982: Break up COMPARE_AND_BRANCH (GH-102801) 2023-03-23 15:25:09 -07:00
Eric Snow 87be8d9522
gh-100227: Make the Global Interned Dict Safe for Isolated Interpreters (gh-102925)
This is effectively two changes.  The first (the bulk of the change) is where we add _Py_AddToGlobalDict() (and _PyRuntime.cached_objects.main_tstate, etc.).  The second (much smaller) change is where we update PyUnicode_InternInPlace() to use _Py_AddToGlobalDict() instead of calling PyDict_SetDefault() directly.

Basically, _Py_AddToGlobalDict() is a wrapper around PyDict_SetDefault() that should be used whenever we need to add a value to a runtime-global dict object (in the few cases where we are leaving the container global rather than moving it to PyInterpreterState, e.g. the interned strings dict).  _Py_AddToGlobalDict() does all the necessary work to make sure the target global dict is shared safely between isolated interpreters.  This is especially important as we move the obmalloc state to each interpreter (gh-101660), as well as, potentially, the GIL (PEP 684).

https://github.com/python/cpython/issues/100227
2023-03-22 18:30:04 -06:00
Irit Katriel 3468c768ce
gh-102859: Remove JUMP_IF_FALSE_OR_POP and JUMP_IF_TRUE_OR_POP (#102870) 2023-03-22 18:10:48 +00:00
Mark Shannon 7559f5fda9
GH-101291: Rearrange the size bits in PyLongObject (GH-102464)
* Eliminate all remaining uses of Py_SIZE and Py_SET_SIZE on PyLongObject, adding asserts.

* Change layout of size/sign bits in longobject to support future addition of immortal ints and tagged medium ints.

* Add functions to hide some internals of long object, and for setting sign and digit count.

* Replace uses of IS_MEDIUM_VALUE macro with _PyLong_IsCompact().
2023-03-22 14:49:51 +00:00
Irit Katriel 76350e85eb
gh-102406: replace exception chaining by PEP-678 notes in codecs (#102407) 2023-03-21 21:36:31 +00:00
Eric Snow 743687434c
gh-102304: Move the Total Refcount to PyInterpreterState (gh-102545)
Moving it valuable with a per-interpreter GIL.  However, it is also useful without one, since it allows us to identify refleaks within a single interpreter or where references are escaping an interpreter.  This becomes more important as we move the obmalloc state to PyInterpreterState.

https://github.com/python/cpython/issues/102304
2023-03-21 11:46:09 -06:00
Eric Snow ad77d16a62
gh-102304: Move _Py_RefTotal to _PyRuntimeState (gh-102543)
The essentially eliminates the global variable, with the associated benefits. This is also a precursor to isolating this bit of state to PyInterpreterState.

Folks that currently read _Py_RefTotal directly would have to start using _Py_GetGlobalRefTotal() instead.

https://github.com/python/cpython/issues/102304
2023-03-20 10:03:04 -06:00
Inada Naoki 65fb7c4055
gh-102701: Fix overflow in dictobject.c (GH-102750) 2023-03-17 22:39:09 +09:00
Irit Katriel 51d693c584
gh-102594: PyErr_SetObject adds note to exception raised on normalization error (#102675) 2023-03-16 10:16:01 +00:00
Kumar Aditya 3d872a74c8
GH-100227: cleanup initialization of global interned dict (#102682) 2023-03-14 14:22:21 +05:30
Thomas Krennwallner ced13c96a4
gh-79940: add introspection API for asynchronous generators to `inspect` module (#11590) 2023-03-11 18:49:40 +05:30
wangxiang-hz aa0a73d1bc
gh-102213: Optimize the performance of `__getattr__` (GH-102248)
When __getattr__ is defined, python with try to find an attribute using _PyObject_GenericGetAttrWithDict
find nothing is reasonable so we don't need an exception, it will hurt performance.
2023-03-11 19:11:37 +08:00
Max Bachmann c6858d1e7f
gh-102255: Improve build support for Windows API partitions (GH-102256)
Add `MS_WINDOWS_DESKTOP`, `MS_WINDOWS_APPS`, `MS_WINDOWS_SYSTEM` and `MS_WINDOWS_GAMES` preprocessor definitions to allow switching off functionality missing from particular API partitions ("partitions" are used in Windows to identify overlapping subsets of APIs).
CPython only officially supports `MS_WINDOWS_DESKTOP` and `MS_WINDOWS_SYSTEM` (APPS is included by normal desktop builds, but APPS without DESKTOP is not covered). Other configurations are a convenience for people building their own runtimes.
`MS_WINDOWS_GAMES` is for the Xbox subset of the Windows API, which is also available on client OS, but is restricted compared to `MS_WINDOWS_DESKTOP`. These restrictions may change over time, as they relate to the build headers rather than the OS support, and so we assume that Xbox builds will use the latest available version of the GDK.
2023-03-09 21:09:12 +00:00
Eric Snow b45d14b886
gh-100227: Move dict_state.global_version to PyInterpreterState (gh-102338)
https://github.com/python/cpython/issues/100227
2023-03-09 08:16:30 -07:00
Eric Snow 5e5acd291f
gh-100227: Move next_keys_version to PyInterpreterState (gh-102335)
https://github.com/python/cpython/issues/100227
2023-03-08 18:04:16 -07:00
Eric Snow 66ff374d4f
gh-100227: Move func_state.next_version to PyInterpreterState (gh-102334)
https://github.com/python/cpython/issues/100227
2023-03-08 15:56:36 -07:00
Eric Snow cbb0aa71d0
gh-102304: Consolidate Direct Usage of _Py_RefTotal (gh-102514)
This simplifies further changes to _Py_RefTotal (e.g. make it atomic or move it to PyInterpreterState).

https://github.com/python/cpython/issues/102304
2023-03-08 12:03:50 -07:00
Irit Katriel 11a2c6ce51
gh-102192: Replace PyErr_Fetch/Restore etc by more efficient alternatives (in Objects/) (#102218) 2023-03-08 17:03:18 +00:00
Kumar Aditya e499680100
fix typo in async generator code field name `ag_code` (#102448) 2023-03-08 09:09:50 +05:30
Carl Meyer 1e703a4733
gh-102381: don't call watcher callback with dead object (#102382)
Co-authored-by: T. Wouters <thomas@python.org>
2023-03-07 17:10:58 -07:00
Eric Snow 8606697f49
gh-90110: Fix the c-analyzer Tool (#102483)
Some incompatible changes had gone in, and the "ignore" lists weren't properly undated. This change fixes that. It's necessary prior to enabling test_check_c_globals, which I hope to do soon.

Note that this does include moving last_resort_memory_error to PyInterpreterState.

https://github.com/python/cpython/issues/90110
2023-03-06 19:40:09 -07:00
Raj 705487c655
gh-101892: Fix `SystemError` when a callable iterator call exhausts the iterator (#101896)
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
2023-03-04 19:51:29 +05:30
Irit Katriel 71db5dbcd7
gh-102371: move _Py_Mangle from compile.c to symtable.c (#102372) 2023-03-02 18:38:22 +00:00
Michael K 73250000ac
Fix typos in documentation and comments (GH-102374)
Found some duplicate `to`s in the documentation and some code comments and fixed them.


[Misc/NEWS.d/3.12.0a1.rst](ed55c69ebd/Misc/NEWS.d/3.12.0a1.rst) also contains two duplicate `to`s, but I wasn't sure if it's ok to touch that file.  Looks auto generated.  I'm happy to amend the PR if requested. :)

Automerge-Triggered-By: GH:AlexWaygood
2023-03-02 07:26:49 -08:00
Petr Viktorin 6b2d7c0ddb
gh-101101: Unstable C API tier (PEP 689) (GH-101102) 2023-02-28 09:31:01 +01:00
Sergey B Kirpichev 4624987b29
gh-101825: Clarify that as_integer_ratio() output is always normalized (#101843)
Make docstrings for `as_integer_ratio` consistent across types, and document that
the returned pair is always normalized (coprime integers, with positive denominator).

---------

Co-authored-by: Owain Davies <116417456+OTheDev@users.noreply.github.com>
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2023-02-27 19:11:28 +00:00
Jelle Zijlstra 8d0f09b1be
gh-101765: unicodeobject: use Py_XDECREF correctly (#102283) 2023-02-26 14:45:37 -08:00
Jelle Zijlstra d71edbd1b7
gh-101765: Fix refcount issues in list and unicode pickling (#102265)
Followup from #101769.
2023-02-25 16:01:58 -08:00
Ionite 54dfa14c5a
gh-101765: Fix SystemError / segmentation fault in iter `__reduce__` when internal access of `builtins.__dict__` exhausts the iterator (#101769) 2023-02-24 15:02:04 -08:00
Mark Shannon 22b8d77b98
GH-100719: Remove redundant `gi_code` field from generator object. (GH-100749) 2023-02-23 10:19:01 +00:00
Irit Katriel 572223f9ce
Revert "bpo-46978: Correct docstrings for in-place builtin operators #31802) (#102146)
Revert "bpo-46978: Correct docstrings for in-place builtin operators (#31802)"

This reverts commit 128379b8cd.
2023-02-23 10:17:44 +00:00
Steve Dower a99eb5cd99
gh-101907: Stop using `_Py_OPCODE` and `_Py_OPARG` macros (GH-101912)
* gh-101907: Removes use of non-standard C++ extension from Include/cpython/code.h

* Make cases_generator correct on Windows
2023-02-20 14:56:48 +00:00
Furkan Onder 61f1e67c6f
GH-84783: Make the slice object hashable (GH-101264) 2023-02-18 18:22:02 -06:00
Nicko van Someren 128379b8cd
bpo-46978: Correct docstrings for in-place builtin operators (#31802) 2023-02-18 18:44:41 +00:00
Eric Snow b2fc549278
gh-101758: Clean Up Uses of Import State (gh-101919)
This change is almost entirely moving code around and hiding import state behind internal API.  We introduce no changes to behavior, nor to non-internal API.  (Since there was already going to be a lot of churn, I took this as an opportunity to re-organize import.c into topically-grouped sections of code.)  The motivation is to simplify a number of upcoming changes.

Specific changes:

* move existing import-related code to import.c, wherever possible
* add internal API for interacting with import state (both global and per-interpreter)
* use only API outside of import.c (to limit churn there when changing the location, etc.)
* consolidate the import-related state of PyInterpreterState into a single struct field (this changes layout slightly)
* add macros for import state in import.c (to simplify changing the location)
* group code in import.c into sections
*remove _PyState_AddModule()

https://github.com/python/cpython/issues/101758
2023-02-15 15:32:31 -07:00
Mark Shannon 160f2fe2b9
GH-87849: Simplify stack effect of SEND and specialize it for generators and coroutines. (GH-101788) 2023-02-13 11:24:55 +00:00
Nikita Sobolev 34c50ceb1e
gh-101747: Fix refleak in new `OrderedDict` repr (GH-101748) 2023-02-09 21:00:58 -06:00
Dong-hee Na 5b946d3719
gh-101430: Update tracemalloc to handle presize properly. (gh-101745) 2023-02-10 08:30:03 +09:00
Kumar Aditya d9de079248
GH-101696: invalidate type version tag in `_PyStaticType_Dealloc` (#101697) 2023-02-08 23:32:15 +05:30
Mark Shannon feec49c407
GH-101578: Normalize the current exception (GH-101607)
* Make sure that the current exception is always normalized.

* Remove redundant type and traceback fields for the current exception.

* Add new API functions: PyErr_GetRaisedException, PyErr_SetRaisedException

* Add new API functions: PyException_GetArgs, PyException_SetArgs
2023-02-08 09:31:12 +00:00
Nikita Sobolev 790ff6bc6a
gh-101446: Change `repr` of `collections.OrderedDict` (#101661) 2023-02-07 20:01:10 -06:00
penguin_wwy 753fc8a5d6
gh-101632: Add the new RETURN_CONST opcode (#101633) 2023-02-07 22:32:21 +00:00
Matthieu Dartiailh ae62bddaf8
gh-101072: support default and kw default in PyEval_EvalCodeEx for 3.11+ (#101127)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2023-02-07 10:34:21 +01:00
Mark Dickinson 39017e04b5
gh-101266: Fix __sizeof__ for subclasses of int (#101394)
Fix the behaviour of the `__sizeof__` method (and hence the results returned by `sys.getsizeof`) for subclasses of `int`. Previously, `int` subclasses gave identical results to the `int` base class, ignoring the presence of the instance dictionary.

<!-- gh-issue-number: gh-101266 -->
* Issue: gh-101266
<!-- /gh-issue-number -->
2023-02-05 10:02:53 +00:00
Erlend E. Aasland 2753cf2ed6
gh-101409: Improve generated clinic code for self type checks (#101411) 2023-01-31 21:42:03 +01:00
Mark Shannon c1b1f51cd1
GH-101291: Refactor the `PyLongObject` struct into object header and PyLongValue struct. (GH-101292) 2023-01-30 10:03:04 +00:00
Mark Shannon f02fa64bf2
GH-100762: Don't call `gen.throw()` in `gen.close()`, unless necessary. (GH-101013)
* Store exception stack depth in YIELD_VALUE's oparg and use it avoid expensive gen.throw() in gen.close() where possible.
2023-01-24 17:25:37 +00:00
Eric Snow 7b20a0f55a
gh-59956: Allow the "Trashcan" Mechanism to Work Without a Thread State (gh-101209)
We've factored out a struct from the two PyThreadState fields. This accomplishes two things:

* make it clear that the trashcan-related code doesn't need any other parts of PyThreadState
* allows us to use the trashcan mechanism even when there isn't a "current" thread state

We still expect the caller to hold the GIL.

https://github.com/python/cpython/issues/59956
2023-01-23 08:30:20 -07:00
Pieter Eendebak f63f525e16
gh-100726: Optimize construction of range object for medium sized integers (#100810)
Use C long arithmetic instead of PyLong arithmetic to compute the range length, where possible.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2023-01-21 19:33:08 +00:00
Mark Dickinson 401fdf9c85
gh-101037: Fix potential memory underallocation for zeros of int subtypes (#101038)
This PR fixes object allocation in long_subtype_new to ensure that there's at least one digit in all cases, and makes sure that the value of that digit is copied over from the source long.

Needs backport to 3.11, but not any further: the change to require at least one digit was only introduced for Python 3.11.

Fixes #101037.
2023-01-21 10:23:59 +00:00
Mark Shannon 7b14c2ef19
GH-100982: Add `COMPARE_AND_BRANCH` instruction (GH-100983) 2023-01-16 12:35:21 +00:00
Nikita Sobolev b1a74a182d
gh-101056: Fix memory leak in `formatfloat()` in `bytesobject.c` (#101057) 2023-01-16 16:16:07 +05:30
Steve Dower b5d4347950
gh-86682: Adds sys._getframemodulename as an alternative to using _getframe (GH-99520)
Also updates calls in collections, doctest, enum, and typing modules to use _getframemodulename first when available.
2023-01-13 11:31:06 +00:00
Raymond Hettinger 94fc7706b7
GH-100942: Fix incorrect cast in property_copy(). (#100965) 2023-01-12 16:13:56 -06:00
Neil Schemenauer c549fcccbb
GH-81381: Add longer comment _PyType_AllocNoTrack() (GH-100954)
The details on the "nitems+1" expression is a bit subtle so add a longer
comment about it.
2023-01-12 10:03:50 -08:00
Brandt Bucher f07daaf4f7
GH-100117: Make `co_lines` more efficient (GH-100447) 2023-01-10 10:56:53 +00:00
Brandt Bucher 61762b9387
GH-100126: Skip incomplete frames in more places (GH-100613) 2023-01-09 12:20:04 -08:00
Irit Katriel 15c44789bb
gh-100758: Refactor initialisation of frame headers into a single function (_PyFrame_Initialize) (GH-100759) 2023-01-06 14:55:56 +00:00
Mark Shannon 78068126a1
GH-99005: More intrinsics (GH-100774)
* Remove UNARY_POSITIVE, LIST_TO_TUPLE and ASYNC_GEN_WRAP, replacing them with intrinsics.
2023-01-06 14:47:57 +00:00
Mark Shannon 15aecf8dd7
GH-100719: Remove the `co_nplaincellvars` field from code objects. (GH-100721) 2023-01-04 15:41:39 +00:00
Irit Katriel c31e356a10
gh-100720: refactor calculation of number of frame slots for a code object into the new function _PyFrame_NumSlotsForCodeObject (#100722) 2023-01-04 13:37:06 +00:00
L. A. F. Pereira e6d4440782
gh-100146: Steal references from stack when building a list (#100147)
When executing the BUILD_LIST opcode, steal the references from the stack,
in a manner similar to the BUILD_TUPLE opcode.  Implement this by offloading
the logic to a new private API, _PyList_FromArraySteal(), that works similarly
to _PyTuple_FromArraySteal().

This way, instead of performing multiple stack pointer adjustments while the
list is being initialized, the stack is adjusted only once and a fast memory
copy operation is performed in one fell swoop.
2023-01-03 10:49:49 -08:00
Ionite d7e7f79ca7
gh-100637: Fix int and bool __sizeof__ calculation to include the 1 element ob_digit array for 0 and False (#100663)
Fixes behaviour where int (and subtypes like bool) __sizeof__ under-reports true size as it did not take into account the size 1 `ob_digit` array for the zero int.

Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2023-01-02 21:11:49 +00:00
Géry Ogam ba1342ce99
gh-87980: Fix the error message for disallowed __weakref__ slots (#25362)
Fix the error message for disallowed `__weakref__` slots.
2023-01-01 21:11:33 +05:30
Pieter Eendebak b3da698952
gh-94603: micro optimize list.pop (gh-94604) 2022-12-27 19:55:54 +09:00
Shantanu 3e46f9fe05
gh-100268: Add is_integer method to int (#100439)
This improves the lives of type annotation users of `float` - which type checkers implicitly treat as `int|float` because that is what most code actually wants. Before this change a `.is_integer()` method could not be assumed to exist on things annotated as `: float` due to the method not existing on both types.
2022-12-23 18:30:27 -08:00
Sebastian Berg 474220e3a5
gh-99947: Ensure unreported errors are chained for SystemError during import (GH-99946) 2022-12-23 15:43:19 -08:00
Dennis Sweeney a98d9ea56e
gh-94155: Reduce hash collisions for code objects (#100183)
* Uses a better hashing algorithm to get better dispersion and remove commutativity.

* Incorporates `co_firstlineno`, `Py_SIZE(co)`, and bytecode instructions.

* This is now the entire set of criteria used in `code_richcompare`, except for `_PyCode_ConstantKey` (which would incorporate the types of `co_consts` rather than just their values).
2022-12-23 13:15:47 -05:00
Pieter Eendebak 7fc7909677
gh-92216: improve performance of `hasattr` for type objects (GH-99979) 2022-12-23 23:23:36 +08:00
Bill Fisher 88d565f32a
gh-99110: Initialize `frame->previous` in init_frame to fix segmentation fault when accessing `frame.f_back` (#100182) 2022-12-23 20:15:53 +05:30
yonillasky 432117cd1f
gh-99540: Constant hash for _PyNone_Type to aid reproducibility (GH-99541)
Needed for ASLR builds of Python.
2022-12-16 12:36:13 -08:00
Mark Shannon 48e352a241
Move stats for the method cache into the `Py_STAT` machinery (GH-100255) 2022-12-15 09:45:03 +00:00
Itamar Ostricher ae83c78215
GH-100000: Cleanup and polish various watchers code (GH-99998)
* Initialize `type_watchers` array to `NULL`s
* Optimize code watchers notification
* Optimize func watchers notification
2022-12-14 19:14:16 +00:00
Eric Snow aa8591e9ca
gh-90111: Minor Cleanup for Runtime-Global Objects (gh-100254)
* move _PyRuntime.global_objects.interned to _PyRuntime.cached_objects.interned_strings (and use _Py_CACHED_OBJECT())
* rename _PyRuntime.global_objects to _PyRuntime.static_objects

(This also relates to gh-96075.)

https://github.com/python/cpython/issues/90111
2022-12-14 11:53:57 -07:00
Mark Shannon 6997e77bdf
GH-100222: Redefine _Py_CODEUNIT as a union to clarify structure of code unit. (GH-100223) 2022-12-14 11:12:53 +00:00
Ned Batchelder 935ef59321
clarify the 4300-digit limit on int-str conversion (#100175) 2022-12-12 13:39:54 +02:00
Mark Shannon fb713b2183
GH-98522: Add version number to code objects. (GH-98525)
* Add version number to code object for better versioning of functions.

* Improves specialization for closures and list comprehensions.
2022-12-09 12:18:45 +00:00
Ken Jin 748c6c0921
GH-100110: Specialize FOR_ITER for tuples (GH-100109)
* Specialize FOR_ITER for tuples
2022-12-09 10:27:01 +00:00
Eric Snow 91a8e002c2
gh-81057: Move More Globals to _PyRuntimeState (gh-100092)
https://github.com/python/cpython/issues/81057
2022-12-07 15:56:31 -07:00
Serhiy Storchaka a87c46eab3
bpo-15999: Accept arbitrary values for boolean parameters. (#15609)
builtins and extension module functions and methods that expect boolean values for parameters now accept any Python object rather than just a bool or int type. This is more consistent with how native Python code itself behaves.
2022-12-03 11:52:21 -08:00
Itamar Ostricher 3c137dc613
GH-91054: Add code object watchers API (GH-99859)
* Add API to allow extensions to set callback function on creation and destruction of PyCodeObject

Co-authored-by: Ye11ow-Flash <janshah@cs.stonybrook.edu>
2022-12-02 17:28:27 +00:00
Serhiy Storchaka f08e52ccb0
gh-99612: Fix PyUnicode_DecodeUTF8Stateful() for ASCII-only data (GH-99613)
Previously *consumed was not set in this case.
2022-12-01 14:54:51 +02:00
Serhiy Storchaka 787764219f
gh-89189: More compact range iterator (GH-27986) 2022-11-30 23:04:30 +02:00
Victor Stinner 85dd6cb6df
gh-99845: Use size_t type in __sizeof__() methods (#99846)
The implementation of __sizeof__() methods using _PyObject_SIZE() now
use an unsigned type (size_t) to compute the size, rather than a signed
type (Py_ssize_t).

Cast explicitly signed (Py_ssize_t) values to unsigned type
(Py_ssize_t).
2022-11-30 17:22:52 +01:00
Victor Stinner 74d5f61ebd
gh-99845: Clean up _PyObject_VAR_SIZE() usage (#99847)
* code_sizeof() now uses an unsigned type (size_t) to compute the result.
* Fix _PyObject_ComputedDictPointer(): cast _PyObject_VAR_SIZE() to
  Py_ssize_t, rather than long: it's a different type on 64-bit Windows.
* Clarify that _PyObject_VAR_SIZE() uses an unsigned type (size_t).
2022-11-29 12:15:21 +01:00
Victor Stinner 4246fe977d
gh-99845: Change _PyDict_KeysSize() return type to size_t (#99848)
* Change _PyDict_KeysSize() and shared_keys_usable_size() return type
  from signed (Py_ssize_t) to unsigned (size_t) type.
* new_values() argument type is now unsigned (size_t).
* init_inline_values() now uses size_t rather than int for the 'i'
  iterator variable.
* type.__sizeof__() implementation now uses unsigned (size_t) type.
2022-11-29 12:12:17 +01:00
Ivan Savov 25bc115df9
gh-89682: [doc] reword docstring of __contains__ to clarify that it returns a bool (GH-29043) 2022-11-26 22:24:04 +00:00
Victor Stinner 5556d3e02c
gh-98724: Fix warnings on Py_SETREF() usage (#99781)
Cast argument to the expected type.
2022-11-26 00:30:37 +01:00
Victor Stinner 81f7359f67
gh-99537: Use Py_SETREF(var, NULL) in C code (#99687)
Replace "Py_DECREF(var); var = NULL;" with "Py_SETREF(var, NULL);".
2022-11-23 14:57:50 +01:00
Victor Stinner 135ec7cefb
gh-99537: Use Py_SETREF() function in C code (#99657)
Fix potential race condition in code patterns:

* Replace "Py_DECREF(var); var = new;" with "Py_SETREF(var, new);"
* Replace "Py_XDECREF(var); var = new;" with "Py_XSETREF(var, new);"
* Replace "Py_CLEAR(var); var = new;" with "Py_XSETREF(var, new);"

Other changes:

* Replace "old = var; var = new; Py_DECREF(var)"
  with "Py_SETREF(var, new);"
* Replace "old = var; var = new; Py_XDECREF(var)"
  with "Py_XSETREF(var, new);"
* And remove the "old" variable.
2022-11-22 13:39:11 +01:00
mpage 3db0a21f73
gh-91053: Add an optional callback that is invoked whenever a function is modified (#98175) 2022-11-22 13:06:44 +01:00
Victor Stinner 20d9749a0f
gh-99537: Use Py_SETREF() function in longobject C code (#99655)
Replace "Py_DECREF(var); var = new;" with "Py_SETREF(var, new);"
in longobject.c and _testcapi/long.c.
2022-11-22 13:04:19 +01:00
Irit Katriel c8c6113398
gh-99553: fix bug where an ExceptionGroup subclass can wrap a BaseException (GH-99572) 2022-11-18 15:44:43 +00:00
Hood Chatham bc390dd935
gh-99443: `descr_set_trampoline_call` return type should be `int` not `PyObject*` (#99444) 2022-11-16 15:20:17 -08:00
Brandt Bucher aa8b58cb33
Remove old comment (GH-99489) 2022-11-16 13:43:31 -08:00
Eric Snow 9db1e17c80
gh-81057: Move the global Dict-Related Versions to _PyRuntimeState (gh-99497)
We also move the global func version.

https://github.com/python/cpython/issues/81057
2022-11-16 10:37:29 -07:00
Victor Stinner 8211cf5d28
gh-99300: Replace Py_INCREF() with Py_NewRef() (#99530)
Replace Py_INCREF() and Py_XINCREF() using a cast with Py_NewRef()
and Py_XNewRef().
2022-11-16 18:34:24 +01:00
Eric Snow 5f55067e23
gh-81057: Move More Globals in Core Code to _PyRuntimeState (gh-99516)
https://github.com/python/cpython/issues/81057
2022-11-16 09:37:14 -07:00
Victor Stinner 3ed8803ef5
gh-99300: Replace Py_INCREF() with Py_NewRef() (#99513)
Replace Py_INCREF() and Py_XINCREF() using a cast with Py_NewRef()
and Py_XNewRef().
2022-11-16 10:39:47 +01:00
Eric Snow 3c57971a2d
gh-81057: Move Globals in Core Code to _PyRuntimeState (gh-99496)
This is the first of several changes to consolidate non-object globals in core code.

https://github.com/python/cpython/issues/81057
2022-11-15 09:45:11 -07:00
Eric Snow a088290f9d
gh-81057: Move Global Variables Holding Objects to _PyRuntimeState. (gh-99487)
This moves nearly all remaining object-holding globals in core code (other than static types).

https://github.com/python/cpython/issues/81057
2022-11-14 13:50:56 -07:00
Victor Stinner 6788303f5c
gh-91248: Optimize PyFrame_GetVar() (#99252)
PyFrame_GetVar() no longer creates a temporary dictionary to get a
variable.
2022-11-13 15:37:03 +01:00
Eric Snow 67807cfc87
gh-81057: Move the Allocators to _PyRuntimeState (gh-99217)
The global allocators were stored in 3 static global variables: _PyMem_Raw, _PyMem, and _PyObject.  State for the "small block" allocator was stored in another 13.  That makes a total of 16 global variables. We are moving all 16 to the _PyRuntimeState struct as part of the work for gh-81057.  (If PEP 684 is accepted then we will follow up by moving them all to PyInterpreterState.)

https://github.com/python/cpython/issues/81057
2022-11-11 16:30:46 -07:00
Eric Snow dd36b71fa6
gh-81057: Move the Extension Modules Cache to _PyRuntimeState (gh-99355)
We also move the closely related max_module_number and add comments documenting the group of struct members.

https://github.com/python/cpython/issues/81057
2022-11-11 14:16:28 -07:00
Victor Stinner 3a1dde8f29
gh-99300: Use Py_NewRef() in Objects/ directory (#99354)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in C files of the Objects/ directory.
2022-11-10 23:58:07 +01:00
Victor Stinner 1960eb005e
gh-99300: Use Py_NewRef() in Objects/ directory (#99351)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in C files of the Objects/ directory.
2022-11-10 23:40:31 +01:00
Victor Stinner 584e55bd34
gh-99300: Use Py_NewRef() in Objects/ directory (#99335)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in C files of the Objects/ directory.
2022-11-10 22:22:02 +01:00
Victor Stinner dbf8613a2e
gh-99300: Use Py_NewRef() in Objects/listobject.c (#99336)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in Objects/listobject.c.
2022-11-10 22:09:18 +01:00
Victor Stinner 6dedf42527
gh-99300: Use Py_NewRef() in Objects/dictobject.c (#99333)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in Objects/dictobject.c.
2022-11-10 16:27:53 +01:00
Victor Stinner c0feb99187
gh-99300: Use Py_NewRef() in Objects/ directory (#99332)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in C files of the Objects/ directory.
2022-11-10 16:27:32 +01:00
Mark Shannon 1e197e63e2
GH-96421: Insert shim frame on entry to interpreter (GH-96319)
* Adds EXIT_INTERPRETER instruction to exit PyEval_EvalDefault()

* Simplifies RETURN_VALUE, YIELD_VALUE and RETURN_GENERATOR instructions as they no longer need to check for entry frames.
2022-11-10 12:34:57 +00:00