Commit Graph

8225 Commits

Author SHA1 Message Date
Eric Wieser 3d2a46845b
gh-83791: Raise TypeError for len(memoryview_0d) (#18463)
Changes the behaviour of `len` on a zero-dimensional `memoryview` to raise `TypeError`. Previously, `len` would return `1`.
2023-04-22 17:32:47 +01:00
AN Long d83faf7f1b
gh-103092: Isolate winreg (#103250) 2023-04-17 12:30:48 -06:00
Mark Shannon 411b169281
GH-103082: Implementation of PEP 669: Low Impact Monitoring for CPython (GH-103083)
* The majority of the monitoring code is in instrumentation.c

* The new instrumentation bytecodes are in bytecodes.c

* legacy_tracing.c adapts the new API to the old sys.setrace and sys.setprofile APIs
2023-04-12 12:04:55 +01:00
Nikita Sobolev 50b4b15984
gh-87864: Use correct function definition syntax in the docs (#103312) 2023-04-11 16:50:25 +03:00
Irit Katriel 55c99d97e1
gh-77757: replace exception wrapping by PEP-678 notes in typeobject's __set_name__ (#103402) 2023-04-11 11:53:06 +01:00
Nikita Sobolev 059bb04245
gh-102213: Revert "gh-102213: Optimize the performance of `__getattr__` (GH-102248)" (GH-103332)
This reverts commit aa0a73d1bc.
2023-04-07 17:22:36 +08:00
Nikita Sobolev 2a721258a1
gh-101865: Deprecate `co_lnotab` from code objects as per PEP 626 (#101866)
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
2023-04-03 17:35:04 +02:00
Brandt Bucher 121057aa36
GH-89987: Shrink the BINARY_SUBSCR caches (GH-103022) 2023-03-29 15:53:30 -07:00
Eric Snow ba65a065cf
gh-100227: Move the Dict of Interned Strings to PyInterpreterState (gh-102339)
We can revisit the options for keeping it global later, if desired.  For now the approach seems quite complex, so we've gone with the simpler isolation solution in the meantime.

https://github.com/python/cpython/issues/100227
2023-03-28 12:52:28 -06:00
Eric Snow 89e67ada69
gh-100227: Revert gh-102925 "gh-100227: Make the Global Interned Dict Safe for Isolated Interpreters" (gh-103063)
This reverts commit 87be8d9.

This approach to keeping the interned strings safe is turning out to be too complex for my taste (due to obmalloc isolation). For now I'm going with the simpler solution, making the dict per-interpreter. We can revisit that later if we want a sharing solution.
2023-03-27 16:53:05 -06:00
Nikita Sobolev 2cdc5189a6
gh-102941: Fix "‘subobj’ may be used uninitialized in this function" warning in `bytes_methods.c` (#102942) 2023-03-26 17:05:06 -07:00
Brandt Bucher 0444ae2487
GH-100982: Break up COMPARE_AND_BRANCH (GH-102801) 2023-03-23 15:25:09 -07:00
Eric Snow 87be8d9522
gh-100227: Make the Global Interned Dict Safe for Isolated Interpreters (gh-102925)
This is effectively two changes.  The first (the bulk of the change) is where we add _Py_AddToGlobalDict() (and _PyRuntime.cached_objects.main_tstate, etc.).  The second (much smaller) change is where we update PyUnicode_InternInPlace() to use _Py_AddToGlobalDict() instead of calling PyDict_SetDefault() directly.

Basically, _Py_AddToGlobalDict() is a wrapper around PyDict_SetDefault() that should be used whenever we need to add a value to a runtime-global dict object (in the few cases where we are leaving the container global rather than moving it to PyInterpreterState, e.g. the interned strings dict).  _Py_AddToGlobalDict() does all the necessary work to make sure the target global dict is shared safely between isolated interpreters.  This is especially important as we move the obmalloc state to each interpreter (gh-101660), as well as, potentially, the GIL (PEP 684).

https://github.com/python/cpython/issues/100227
2023-03-22 18:30:04 -06:00
Irit Katriel 3468c768ce
gh-102859: Remove JUMP_IF_FALSE_OR_POP and JUMP_IF_TRUE_OR_POP (#102870) 2023-03-22 18:10:48 +00:00
Mark Shannon 7559f5fda9
GH-101291: Rearrange the size bits in PyLongObject (GH-102464)
* Eliminate all remaining uses of Py_SIZE and Py_SET_SIZE on PyLongObject, adding asserts.

* Change layout of size/sign bits in longobject to support future addition of immortal ints and tagged medium ints.

* Add functions to hide some internals of long object, and for setting sign and digit count.

* Replace uses of IS_MEDIUM_VALUE macro with _PyLong_IsCompact().
2023-03-22 14:49:51 +00:00
Irit Katriel 76350e85eb
gh-102406: replace exception chaining by PEP-678 notes in codecs (#102407) 2023-03-21 21:36:31 +00:00
Eric Snow 743687434c
gh-102304: Move the Total Refcount to PyInterpreterState (gh-102545)
Moving it valuable with a per-interpreter GIL.  However, it is also useful without one, since it allows us to identify refleaks within a single interpreter or where references are escaping an interpreter.  This becomes more important as we move the obmalloc state to PyInterpreterState.

https://github.com/python/cpython/issues/102304
2023-03-21 11:46:09 -06:00
Eric Snow ad77d16a62
gh-102304: Move _Py_RefTotal to _PyRuntimeState (gh-102543)
The essentially eliminates the global variable, with the associated benefits. This is also a precursor to isolating this bit of state to PyInterpreterState.

Folks that currently read _Py_RefTotal directly would have to start using _Py_GetGlobalRefTotal() instead.

https://github.com/python/cpython/issues/102304
2023-03-20 10:03:04 -06:00
Inada Naoki 65fb7c4055
gh-102701: Fix overflow in dictobject.c (GH-102750) 2023-03-17 22:39:09 +09:00
Irit Katriel 51d693c584
gh-102594: PyErr_SetObject adds note to exception raised on normalization error (#102675) 2023-03-16 10:16:01 +00:00
Kumar Aditya 3d872a74c8
GH-100227: cleanup initialization of global interned dict (#102682) 2023-03-14 14:22:21 +05:30
Thomas Krennwallner ced13c96a4
gh-79940: add introspection API for asynchronous generators to `inspect` module (#11590) 2023-03-11 18:49:40 +05:30
wangxiang-hz aa0a73d1bc
gh-102213: Optimize the performance of `__getattr__` (GH-102248)
When __getattr__ is defined, python with try to find an attribute using _PyObject_GenericGetAttrWithDict
find nothing is reasonable so we don't need an exception, it will hurt performance.
2023-03-11 19:11:37 +08:00
Max Bachmann c6858d1e7f
gh-102255: Improve build support for Windows API partitions (GH-102256)
Add `MS_WINDOWS_DESKTOP`, `MS_WINDOWS_APPS`, `MS_WINDOWS_SYSTEM` and `MS_WINDOWS_GAMES` preprocessor definitions to allow switching off functionality missing from particular API partitions ("partitions" are used in Windows to identify overlapping subsets of APIs).
CPython only officially supports `MS_WINDOWS_DESKTOP` and `MS_WINDOWS_SYSTEM` (APPS is included by normal desktop builds, but APPS without DESKTOP is not covered). Other configurations are a convenience for people building their own runtimes.
`MS_WINDOWS_GAMES` is for the Xbox subset of the Windows API, which is also available on client OS, but is restricted compared to `MS_WINDOWS_DESKTOP`. These restrictions may change over time, as they relate to the build headers rather than the OS support, and so we assume that Xbox builds will use the latest available version of the GDK.
2023-03-09 21:09:12 +00:00
Eric Snow b45d14b886
gh-100227: Move dict_state.global_version to PyInterpreterState (gh-102338)
https://github.com/python/cpython/issues/100227
2023-03-09 08:16:30 -07:00
Eric Snow 5e5acd291f
gh-100227: Move next_keys_version to PyInterpreterState (gh-102335)
https://github.com/python/cpython/issues/100227
2023-03-08 18:04:16 -07:00
Eric Snow 66ff374d4f
gh-100227: Move func_state.next_version to PyInterpreterState (gh-102334)
https://github.com/python/cpython/issues/100227
2023-03-08 15:56:36 -07:00
Eric Snow cbb0aa71d0
gh-102304: Consolidate Direct Usage of _Py_RefTotal (gh-102514)
This simplifies further changes to _Py_RefTotal (e.g. make it atomic or move it to PyInterpreterState).

https://github.com/python/cpython/issues/102304
2023-03-08 12:03:50 -07:00
Irit Katriel 11a2c6ce51
gh-102192: Replace PyErr_Fetch/Restore etc by more efficient alternatives (in Objects/) (#102218) 2023-03-08 17:03:18 +00:00
Kumar Aditya e499680100
fix typo in async generator code field name `ag_code` (#102448) 2023-03-08 09:09:50 +05:30
Carl Meyer 1e703a4733
gh-102381: don't call watcher callback with dead object (#102382)
Co-authored-by: T. Wouters <thomas@python.org>
2023-03-07 17:10:58 -07:00
Eric Snow 8606697f49
gh-90110: Fix the c-analyzer Tool (#102483)
Some incompatible changes had gone in, and the "ignore" lists weren't properly undated. This change fixes that. It's necessary prior to enabling test_check_c_globals, which I hope to do soon.

Note that this does include moving last_resort_memory_error to PyInterpreterState.

https://github.com/python/cpython/issues/90110
2023-03-06 19:40:09 -07:00
Raj 705487c655
gh-101892: Fix `SystemError` when a callable iterator call exhausts the iterator (#101896)
Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net>
2023-03-04 19:51:29 +05:30
Irit Katriel 71db5dbcd7
gh-102371: move _Py_Mangle from compile.c to symtable.c (#102372) 2023-03-02 18:38:22 +00:00
Michael K 73250000ac
Fix typos in documentation and comments (GH-102374)
Found some duplicate `to`s in the documentation and some code comments and fixed them.


[Misc/NEWS.d/3.12.0a1.rst](ed55c69ebd/Misc/NEWS.d/3.12.0a1.rst) also contains two duplicate `to`s, but I wasn't sure if it's ok to touch that file.  Looks auto generated.  I'm happy to amend the PR if requested. :)

Automerge-Triggered-By: GH:AlexWaygood
2023-03-02 07:26:49 -08:00
Petr Viktorin 6b2d7c0ddb
gh-101101: Unstable C API tier (PEP 689) (GH-101102) 2023-02-28 09:31:01 +01:00
Sergey B Kirpichev 4624987b29
gh-101825: Clarify that as_integer_ratio() output is always normalized (#101843)
Make docstrings for `as_integer_ratio` consistent across types, and document that
the returned pair is always normalized (coprime integers, with positive denominator).

---------

Co-authored-by: Owain Davies <116417456+OTheDev@users.noreply.github.com>
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2023-02-27 19:11:28 +00:00
Jelle Zijlstra 8d0f09b1be
gh-101765: unicodeobject: use Py_XDECREF correctly (#102283) 2023-02-26 14:45:37 -08:00
Jelle Zijlstra d71edbd1b7
gh-101765: Fix refcount issues in list and unicode pickling (#102265)
Followup from #101769.
2023-02-25 16:01:58 -08:00
Ionite 54dfa14c5a
gh-101765: Fix SystemError / segmentation fault in iter `__reduce__` when internal access of `builtins.__dict__` exhausts the iterator (#101769) 2023-02-24 15:02:04 -08:00
Mark Shannon 22b8d77b98
GH-100719: Remove redundant `gi_code` field from generator object. (GH-100749) 2023-02-23 10:19:01 +00:00
Irit Katriel 572223f9ce
Revert "bpo-46978: Correct docstrings for in-place builtin operators #31802) (#102146)
Revert "bpo-46978: Correct docstrings for in-place builtin operators (#31802)"

This reverts commit 128379b8cd.
2023-02-23 10:17:44 +00:00
Steve Dower a99eb5cd99
gh-101907: Stop using `_Py_OPCODE` and `_Py_OPARG` macros (GH-101912)
* gh-101907: Removes use of non-standard C++ extension from Include/cpython/code.h

* Make cases_generator correct on Windows
2023-02-20 14:56:48 +00:00
Furkan Onder 61f1e67c6f
GH-84783: Make the slice object hashable (GH-101264) 2023-02-18 18:22:02 -06:00
Nicko van Someren 128379b8cd
bpo-46978: Correct docstrings for in-place builtin operators (#31802) 2023-02-18 18:44:41 +00:00
Eric Snow b2fc549278
gh-101758: Clean Up Uses of Import State (gh-101919)
This change is almost entirely moving code around and hiding import state behind internal API.  We introduce no changes to behavior, nor to non-internal API.  (Since there was already going to be a lot of churn, I took this as an opportunity to re-organize import.c into topically-grouped sections of code.)  The motivation is to simplify a number of upcoming changes.

Specific changes:

* move existing import-related code to import.c, wherever possible
* add internal API for interacting with import state (both global and per-interpreter)
* use only API outside of import.c (to limit churn there when changing the location, etc.)
* consolidate the import-related state of PyInterpreterState into a single struct field (this changes layout slightly)
* add macros for import state in import.c (to simplify changing the location)
* group code in import.c into sections
*remove _PyState_AddModule()

https://github.com/python/cpython/issues/101758
2023-02-15 15:32:31 -07:00
Mark Shannon 160f2fe2b9
GH-87849: Simplify stack effect of SEND and specialize it for generators and coroutines. (GH-101788) 2023-02-13 11:24:55 +00:00
Nikita Sobolev 34c50ceb1e
gh-101747: Fix refleak in new `OrderedDict` repr (GH-101748) 2023-02-09 21:00:58 -06:00
Dong-hee Na 5b946d3719
gh-101430: Update tracemalloc to handle presize properly. (gh-101745) 2023-02-10 08:30:03 +09:00
Kumar Aditya d9de079248
GH-101696: invalidate type version tag in `_PyStaticType_Dealloc` (#101697) 2023-02-08 23:32:15 +05:30
Mark Shannon feec49c407
GH-101578: Normalize the current exception (GH-101607)
* Make sure that the current exception is always normalized.

* Remove redundant type and traceback fields for the current exception.

* Add new API functions: PyErr_GetRaisedException, PyErr_SetRaisedException

* Add new API functions: PyException_GetArgs, PyException_SetArgs
2023-02-08 09:31:12 +00:00
Nikita Sobolev 790ff6bc6a
gh-101446: Change `repr` of `collections.OrderedDict` (#101661) 2023-02-07 20:01:10 -06:00
penguin_wwy 753fc8a5d6
gh-101632: Add the new RETURN_CONST opcode (#101633) 2023-02-07 22:32:21 +00:00
Matthieu Dartiailh ae62bddaf8
gh-101072: support default and kw default in PyEval_EvalCodeEx for 3.11+ (#101127)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2023-02-07 10:34:21 +01:00
Mark Dickinson 39017e04b5
gh-101266: Fix __sizeof__ for subclasses of int (#101394)
Fix the behaviour of the `__sizeof__` method (and hence the results returned by `sys.getsizeof`) for subclasses of `int`. Previously, `int` subclasses gave identical results to the `int` base class, ignoring the presence of the instance dictionary.

<!-- gh-issue-number: gh-101266 -->
* Issue: gh-101266
<!-- /gh-issue-number -->
2023-02-05 10:02:53 +00:00
Erlend E. Aasland 2753cf2ed6
gh-101409: Improve generated clinic code for self type checks (#101411) 2023-01-31 21:42:03 +01:00
Mark Shannon c1b1f51cd1
GH-101291: Refactor the `PyLongObject` struct into object header and PyLongValue struct. (GH-101292) 2023-01-30 10:03:04 +00:00
Mark Shannon f02fa64bf2
GH-100762: Don't call `gen.throw()` in `gen.close()`, unless necessary. (GH-101013)
* Store exception stack depth in YIELD_VALUE's oparg and use it avoid expensive gen.throw() in gen.close() where possible.
2023-01-24 17:25:37 +00:00
Eric Snow 7b20a0f55a
gh-59956: Allow the "Trashcan" Mechanism to Work Without a Thread State (gh-101209)
We've factored out a struct from the two PyThreadState fields. This accomplishes two things:

* make it clear that the trashcan-related code doesn't need any other parts of PyThreadState
* allows us to use the trashcan mechanism even when there isn't a "current" thread state

We still expect the caller to hold the GIL.

https://github.com/python/cpython/issues/59956
2023-01-23 08:30:20 -07:00
Pieter Eendebak f63f525e16
gh-100726: Optimize construction of range object for medium sized integers (#100810)
Use C long arithmetic instead of PyLong arithmetic to compute the range length, where possible.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2023-01-21 19:33:08 +00:00
Mark Dickinson 401fdf9c85
gh-101037: Fix potential memory underallocation for zeros of int subtypes (#101038)
This PR fixes object allocation in long_subtype_new to ensure that there's at least one digit in all cases, and makes sure that the value of that digit is copied over from the source long.

Needs backport to 3.11, but not any further: the change to require at least one digit was only introduced for Python 3.11.

Fixes #101037.
2023-01-21 10:23:59 +00:00
Mark Shannon 7b14c2ef19
GH-100982: Add `COMPARE_AND_BRANCH` instruction (GH-100983) 2023-01-16 12:35:21 +00:00
Nikita Sobolev b1a74a182d
gh-101056: Fix memory leak in `formatfloat()` in `bytesobject.c` (#101057) 2023-01-16 16:16:07 +05:30
Steve Dower b5d4347950
gh-86682: Adds sys._getframemodulename as an alternative to using _getframe (GH-99520)
Also updates calls in collections, doctest, enum, and typing modules to use _getframemodulename first when available.
2023-01-13 11:31:06 +00:00
Raymond Hettinger 94fc7706b7
GH-100942: Fix incorrect cast in property_copy(). (#100965) 2023-01-12 16:13:56 -06:00
Neil Schemenauer c549fcccbb
GH-81381: Add longer comment _PyType_AllocNoTrack() (GH-100954)
The details on the "nitems+1" expression is a bit subtle so add a longer
comment about it.
2023-01-12 10:03:50 -08:00
Brandt Bucher f07daaf4f7
GH-100117: Make `co_lines` more efficient (GH-100447) 2023-01-10 10:56:53 +00:00
Brandt Bucher 61762b9387
GH-100126: Skip incomplete frames in more places (GH-100613) 2023-01-09 12:20:04 -08:00
Irit Katriel 15c44789bb
gh-100758: Refactor initialisation of frame headers into a single function (_PyFrame_Initialize) (GH-100759) 2023-01-06 14:55:56 +00:00
Mark Shannon 78068126a1
GH-99005: More intrinsics (GH-100774)
* Remove UNARY_POSITIVE, LIST_TO_TUPLE and ASYNC_GEN_WRAP, replacing them with intrinsics.
2023-01-06 14:47:57 +00:00
Mark Shannon 15aecf8dd7
GH-100719: Remove the `co_nplaincellvars` field from code objects. (GH-100721) 2023-01-04 15:41:39 +00:00
Irit Katriel c31e356a10
gh-100720: refactor calculation of number of frame slots for a code object into the new function _PyFrame_NumSlotsForCodeObject (#100722) 2023-01-04 13:37:06 +00:00
L. A. F. Pereira e6d4440782
gh-100146: Steal references from stack when building a list (#100147)
When executing the BUILD_LIST opcode, steal the references from the stack,
in a manner similar to the BUILD_TUPLE opcode.  Implement this by offloading
the logic to a new private API, _PyList_FromArraySteal(), that works similarly
to _PyTuple_FromArraySteal().

This way, instead of performing multiple stack pointer adjustments while the
list is being initialized, the stack is adjusted only once and a fast memory
copy operation is performed in one fell swoop.
2023-01-03 10:49:49 -08:00
Ionite d7e7f79ca7
gh-100637: Fix int and bool __sizeof__ calculation to include the 1 element ob_digit array for 0 and False (#100663)
Fixes behaviour where int (and subtypes like bool) __sizeof__ under-reports true size as it did not take into account the size 1 `ob_digit` array for the zero int.

Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2023-01-02 21:11:49 +00:00
Géry Ogam ba1342ce99
gh-87980: Fix the error message for disallowed __weakref__ slots (#25362)
Fix the error message for disallowed `__weakref__` slots.
2023-01-01 21:11:33 +05:30
Pieter Eendebak b3da698952
gh-94603: micro optimize list.pop (gh-94604) 2022-12-27 19:55:54 +09:00
Shantanu 3e46f9fe05
gh-100268: Add is_integer method to int (#100439)
This improves the lives of type annotation users of `float` - which type checkers implicitly treat as `int|float` because that is what most code actually wants. Before this change a `.is_integer()` method could not be assumed to exist on things annotated as `: float` due to the method not existing on both types.
2022-12-23 18:30:27 -08:00
Sebastian Berg 474220e3a5
gh-99947: Ensure unreported errors are chained for SystemError during import (GH-99946) 2022-12-23 15:43:19 -08:00
Dennis Sweeney a98d9ea56e
gh-94155: Reduce hash collisions for code objects (#100183)
* Uses a better hashing algorithm to get better dispersion and remove commutativity.

* Incorporates `co_firstlineno`, `Py_SIZE(co)`, and bytecode instructions.

* This is now the entire set of criteria used in `code_richcompare`, except for `_PyCode_ConstantKey` (which would incorporate the types of `co_consts` rather than just their values).
2022-12-23 13:15:47 -05:00
Pieter Eendebak 7fc7909677
gh-92216: improve performance of `hasattr` for type objects (GH-99979) 2022-12-23 23:23:36 +08:00
Bill Fisher 88d565f32a
gh-99110: Initialize `frame->previous` in init_frame to fix segmentation fault when accessing `frame.f_back` (#100182) 2022-12-23 20:15:53 +05:30
yonillasky 432117cd1f
gh-99540: Constant hash for _PyNone_Type to aid reproducibility (GH-99541)
Needed for ASLR builds of Python.
2022-12-16 12:36:13 -08:00
Mark Shannon 48e352a241
Move stats for the method cache into the `Py_STAT` machinery (GH-100255) 2022-12-15 09:45:03 +00:00
Itamar Ostricher ae83c78215
GH-100000: Cleanup and polish various watchers code (GH-99998)
* Initialize `type_watchers` array to `NULL`s
* Optimize code watchers notification
* Optimize func watchers notification
2022-12-14 19:14:16 +00:00
Eric Snow aa8591e9ca
gh-90111: Minor Cleanup for Runtime-Global Objects (gh-100254)
* move _PyRuntime.global_objects.interned to _PyRuntime.cached_objects.interned_strings (and use _Py_CACHED_OBJECT())
* rename _PyRuntime.global_objects to _PyRuntime.static_objects

(This also relates to gh-96075.)

https://github.com/python/cpython/issues/90111
2022-12-14 11:53:57 -07:00
Mark Shannon 6997e77bdf
GH-100222: Redefine _Py_CODEUNIT as a union to clarify structure of code unit. (GH-100223) 2022-12-14 11:12:53 +00:00
Ned Batchelder 935ef59321
clarify the 4300-digit limit on int-str conversion (#100175) 2022-12-12 13:39:54 +02:00
Mark Shannon fb713b2183
GH-98522: Add version number to code objects. (GH-98525)
* Add version number to code object for better versioning of functions.

* Improves specialization for closures and list comprehensions.
2022-12-09 12:18:45 +00:00
Ken Jin 748c6c0921
GH-100110: Specialize FOR_ITER for tuples (GH-100109)
* Specialize FOR_ITER for tuples
2022-12-09 10:27:01 +00:00
Eric Snow 91a8e002c2
gh-81057: Move More Globals to _PyRuntimeState (gh-100092)
https://github.com/python/cpython/issues/81057
2022-12-07 15:56:31 -07:00
Serhiy Storchaka a87c46eab3
bpo-15999: Accept arbitrary values for boolean parameters. (#15609)
builtins and extension module functions and methods that expect boolean values for parameters now accept any Python object rather than just a bool or int type. This is more consistent with how native Python code itself behaves.
2022-12-03 11:52:21 -08:00
Itamar Ostricher 3c137dc613
GH-91054: Add code object watchers API (GH-99859)
* Add API to allow extensions to set callback function on creation and destruction of PyCodeObject

Co-authored-by: Ye11ow-Flash <janshah@cs.stonybrook.edu>
2022-12-02 17:28:27 +00:00
Serhiy Storchaka f08e52ccb0
gh-99612: Fix PyUnicode_DecodeUTF8Stateful() for ASCII-only data (GH-99613)
Previously *consumed was not set in this case.
2022-12-01 14:54:51 +02:00
Serhiy Storchaka 787764219f
gh-89189: More compact range iterator (GH-27986) 2022-11-30 23:04:30 +02:00
Victor Stinner 85dd6cb6df
gh-99845: Use size_t type in __sizeof__() methods (#99846)
The implementation of __sizeof__() methods using _PyObject_SIZE() now
use an unsigned type (size_t) to compute the size, rather than a signed
type (Py_ssize_t).

Cast explicitly signed (Py_ssize_t) values to unsigned type
(Py_ssize_t).
2022-11-30 17:22:52 +01:00
Victor Stinner 74d5f61ebd
gh-99845: Clean up _PyObject_VAR_SIZE() usage (#99847)
* code_sizeof() now uses an unsigned type (size_t) to compute the result.
* Fix _PyObject_ComputedDictPointer(): cast _PyObject_VAR_SIZE() to
  Py_ssize_t, rather than long: it's a different type on 64-bit Windows.
* Clarify that _PyObject_VAR_SIZE() uses an unsigned type (size_t).
2022-11-29 12:15:21 +01:00
Victor Stinner 4246fe977d
gh-99845: Change _PyDict_KeysSize() return type to size_t (#99848)
* Change _PyDict_KeysSize() and shared_keys_usable_size() return type
  from signed (Py_ssize_t) to unsigned (size_t) type.
* new_values() argument type is now unsigned (size_t).
* init_inline_values() now uses size_t rather than int for the 'i'
  iterator variable.
* type.__sizeof__() implementation now uses unsigned (size_t) type.
2022-11-29 12:12:17 +01:00
Ivan Savov 25bc115df9
gh-89682: [doc] reword docstring of __contains__ to clarify that it returns a bool (GH-29043) 2022-11-26 22:24:04 +00:00
Victor Stinner 5556d3e02c
gh-98724: Fix warnings on Py_SETREF() usage (#99781)
Cast argument to the expected type.
2022-11-26 00:30:37 +01:00
Victor Stinner 81f7359f67
gh-99537: Use Py_SETREF(var, NULL) in C code (#99687)
Replace "Py_DECREF(var); var = NULL;" with "Py_SETREF(var, NULL);".
2022-11-23 14:57:50 +01:00
Victor Stinner 135ec7cefb
gh-99537: Use Py_SETREF() function in C code (#99657)
Fix potential race condition in code patterns:

* Replace "Py_DECREF(var); var = new;" with "Py_SETREF(var, new);"
* Replace "Py_XDECREF(var); var = new;" with "Py_XSETREF(var, new);"
* Replace "Py_CLEAR(var); var = new;" with "Py_XSETREF(var, new);"

Other changes:

* Replace "old = var; var = new; Py_DECREF(var)"
  with "Py_SETREF(var, new);"
* Replace "old = var; var = new; Py_XDECREF(var)"
  with "Py_XSETREF(var, new);"
* And remove the "old" variable.
2022-11-22 13:39:11 +01:00
mpage 3db0a21f73
gh-91053: Add an optional callback that is invoked whenever a function is modified (#98175) 2022-11-22 13:06:44 +01:00
Victor Stinner 20d9749a0f
gh-99537: Use Py_SETREF() function in longobject C code (#99655)
Replace "Py_DECREF(var); var = new;" with "Py_SETREF(var, new);"
in longobject.c and _testcapi/long.c.
2022-11-22 13:04:19 +01:00
Irit Katriel c8c6113398
gh-99553: fix bug where an ExceptionGroup subclass can wrap a BaseException (GH-99572) 2022-11-18 15:44:43 +00:00
Hood Chatham bc390dd935
gh-99443: `descr_set_trampoline_call` return type should be `int` not `PyObject*` (#99444) 2022-11-16 15:20:17 -08:00
Brandt Bucher aa8b58cb33
Remove old comment (GH-99489) 2022-11-16 13:43:31 -08:00
Eric Snow 9db1e17c80
gh-81057: Move the global Dict-Related Versions to _PyRuntimeState (gh-99497)
We also move the global func version.

https://github.com/python/cpython/issues/81057
2022-11-16 10:37:29 -07:00
Victor Stinner 8211cf5d28
gh-99300: Replace Py_INCREF() with Py_NewRef() (#99530)
Replace Py_INCREF() and Py_XINCREF() using a cast with Py_NewRef()
and Py_XNewRef().
2022-11-16 18:34:24 +01:00
Eric Snow 5f55067e23
gh-81057: Move More Globals in Core Code to _PyRuntimeState (gh-99516)
https://github.com/python/cpython/issues/81057
2022-11-16 09:37:14 -07:00
Victor Stinner 3ed8803ef5
gh-99300: Replace Py_INCREF() with Py_NewRef() (#99513)
Replace Py_INCREF() and Py_XINCREF() using a cast with Py_NewRef()
and Py_XNewRef().
2022-11-16 10:39:47 +01:00
Eric Snow 3c57971a2d
gh-81057: Move Globals in Core Code to _PyRuntimeState (gh-99496)
This is the first of several changes to consolidate non-object globals in core code.

https://github.com/python/cpython/issues/81057
2022-11-15 09:45:11 -07:00
Eric Snow a088290f9d
gh-81057: Move Global Variables Holding Objects to _PyRuntimeState. (gh-99487)
This moves nearly all remaining object-holding globals in core code (other than static types).

https://github.com/python/cpython/issues/81057
2022-11-14 13:50:56 -07:00
Victor Stinner 6788303f5c
gh-91248: Optimize PyFrame_GetVar() (#99252)
PyFrame_GetVar() no longer creates a temporary dictionary to get a
variable.
2022-11-13 15:37:03 +01:00
Eric Snow 67807cfc87
gh-81057: Move the Allocators to _PyRuntimeState (gh-99217)
The global allocators were stored in 3 static global variables: _PyMem_Raw, _PyMem, and _PyObject.  State for the "small block" allocator was stored in another 13.  That makes a total of 16 global variables. We are moving all 16 to the _PyRuntimeState struct as part of the work for gh-81057.  (If PEP 684 is accepted then we will follow up by moving them all to PyInterpreterState.)

https://github.com/python/cpython/issues/81057
2022-11-11 16:30:46 -07:00
Eric Snow dd36b71fa6
gh-81057: Move the Extension Modules Cache to _PyRuntimeState (gh-99355)
We also move the closely related max_module_number and add comments documenting the group of struct members.

https://github.com/python/cpython/issues/81057
2022-11-11 14:16:28 -07:00
Victor Stinner 3a1dde8f29
gh-99300: Use Py_NewRef() in Objects/ directory (#99354)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in C files of the Objects/ directory.
2022-11-10 23:58:07 +01:00
Victor Stinner 1960eb005e
gh-99300: Use Py_NewRef() in Objects/ directory (#99351)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in C files of the Objects/ directory.
2022-11-10 23:40:31 +01:00
Victor Stinner 584e55bd34
gh-99300: Use Py_NewRef() in Objects/ directory (#99335)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in C files of the Objects/ directory.
2022-11-10 22:22:02 +01:00
Victor Stinner dbf8613a2e
gh-99300: Use Py_NewRef() in Objects/listobject.c (#99336)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in Objects/listobject.c.
2022-11-10 22:09:18 +01:00
Victor Stinner 6dedf42527
gh-99300: Use Py_NewRef() in Objects/dictobject.c (#99333)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in Objects/dictobject.c.
2022-11-10 16:27:53 +01:00
Victor Stinner c0feb99187
gh-99300: Use Py_NewRef() in Objects/ directory (#99332)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in C files of the Objects/ directory.
2022-11-10 16:27:32 +01:00
Mark Shannon 1e197e63e2
GH-96421: Insert shim frame on entry to interpreter (GH-96319)
* Adds EXIT_INTERPRETER instruction to exit PyEval_EvalDefault()

* Simplifies RETURN_VALUE, YIELD_VALUE and RETURN_GENERATOR instructions as they no longer need to check for entry frames.
2022-11-10 12:34:57 +00:00
Eric Snow 52f91c642b
gh-90868: Adjust the Generated Objects (gh-99223)
We do the following:

* move the generated _PyUnicode_InitStaticStrings() to its own file
* move the generated _PyStaticObjects_CheckRefcnt() to its own file
* include pycore_global_objects.h in extension modules instead of pycore_runtime_init.h

These changes help us avoid including things that aren't needed.

https://github.com/python/cpython/issues/90868
2022-11-08 10:03:03 -07:00
Victor Stinner 4d5fcca273
gh-91248: Add PyFrame_GetVar() function (#95712)
Add PyFrame_GetVar() and PyFrame_GetVarString() functions to get a
frame variable by its name.

Move PyFrameObject C API tests from test_capi to test_frame.
2022-11-08 17:40:27 +01:00
Brandt Bucher c7065ce019
GH-93143: Don't turn LOAD_FAST into LOAD_FAST_CHECK (GH-99075) 2022-11-08 07:50:46 -08:00
Irit Katriel c43714fbcd
gh-99181: fix except* on unhashable exceptions (GH-99192) 2022-11-08 09:32:20 +00:00
Mark Shannon 4a1c58d504
GH-96793: Specialize FOR_ITER for generators. (GH-98772) 2022-11-07 14:49:51 +00:00
jonasdlindner ede6cb2615
Correct some typos in comments (GH-98194)
Automerge-Triggered-By: GH:AlexWaygood
2022-11-06 08:54:44 -08:00
Kaushik Kulkarni 67ade403a2
gh-98284: better error message for undefined abstractmethod (#97971) 2022-11-05 09:31:57 -07:00
Victor Stinner 387f72588d
gh-90716: Fix pylong_int_from_string() refleak (#99094)
Fix validated by:

    $ ./python -m test -R 3:3 test_int
    Tests result: SUCCESS
2022-11-04 14:24:10 +01:00
Gregory P. Smith 4c4b5ce2e5
gh-90716: bugfixes and more tests for _pylong. (#99073)
* Properly decref on _pylong import error.
* Improve the error message on _pylong TypeError.
* Fix the assertion error in pydebug builds to be a TypeError.
* Tie the return value comments together.

These are minor followups to issues not caught among the reviewers on
https://github.com/python/cpython/pull/96673.
2022-11-03 16:18:38 -07:00
Mark Shannon f4adb97506
GH-96793: Implement PEP 479 in bytecode. (GH-99006)
* Handle converting StopIteration to RuntimeError in bytecode.

* Add custom instruction for converting StopIteration into RuntimeError.
2022-11-03 11:38:51 +00:00
Brandt Bucher 276d77724f
GH-98686: Quicken everything (GH-98687) 2022-11-02 10:42:57 -07:00
Serhiy Storchaka 0e15c31c7e
gh-98852: Fix subscription of type aliases (GH-98920)
Fix subscription of type aliases containing bare generic types or types
like TypeVar: for example tuple[A, T][int] and tuple[TypeVar, T][int],
where A is a generic type, and T is a type variable.
2022-11-01 09:33:09 +02:00
Nikita Sobolev 76f989dc3e
gh-98783: Fix crashes when `str` subclasses are used in `_PyUnicode_Equal` (#98806) 2022-10-30 02:23:20 -04:00
Benjamin Peterson bded5edd9a
obmalloc: Remove unused variable. (GH-98770) 2022-10-27 09:06:49 -07:00
Mark Shannon 22863df7ca
GH-96793: Change `FOR_ITER` to not pop the iterator on exhaustion. (GH-96801)
Change FOR_ITER to have the same stack effect regardless of whether it branches or not.
Performance is unchanged as FOR_ITER (and specialized forms jump over the cleanup code).
2022-10-27 11:55:03 +01:00
Eric Snow 731909ebef
gh-98627: Use a Switch in PyModule_FromDefAndSpec2() (gh-98734)
This helps simplify some changes in follow-up PRs.  It also matches what we're doing in PyModule_ExecDef().
2022-10-26 21:20:54 -06:00
Neil Schemenauer de6981680b
gh-90716: add _pylong.py module (#96673)
Add Python implementations of certain longobject.c functions. These use
asymptotically faster algorithms that can be used for operations on
integers with many digits. In those cases, the performance overhead of
the Python implementation is not significant since the asymptotic
behavior is what dominates runtime. Functions provided by this module
should be considered private and not part of any public API.

Co-author: Tim Peters <tim.peters@gmail.com>
Co-author: Mark Dickinson <dickinsm@gmail.com>
Co-author: Bjorn Martinsson
2022-10-25 22:00:50 -07:00
Pablo Galindo Salgado 7cfbb49fcd
gh-91058: Add error suggestions to 'import from' import errors (#98305) 2022-10-25 23:56:59 +01:00
Pablo Galindo Salgado 1f737edb67
gh-96143: Move the perf trampoline files to the Python directory (#98675) 2022-10-25 23:34:22 +01:00
Carl Meyer 82ccbf69a8
gh-91051: allow setting a callback hook on PyType_Modified (GH-97875) 2022-10-21 14:41:51 +01:00
MonadChains c60b3b3cca
gh-98421: Clean Up PyObject_Print (GH-98422)
Work on test coverage for `PyObject_Print` made it clear that some lines can't get executed.
Simplify the function by excluding the checks for non-string types.
Also eliminate creating a temporary bytes object.
2022-10-20 15:59:10 +02:00
Eric Snow 9c8dde0fa5
gh-98417: Store int_max_str_digits on the Interpreter State (GH-98418) 2022-10-19 13:27:46 -07:00
Victor Stinner db03c8066a
gh-98393: os module reject bytes-like, only accept bytes (#98394)
The os module and the PyUnicode_FSDecoder() function no longer accept
bytes-like paths, like bytearray and memoryview types: only the exact
bytes type is accepted for bytes strings.
2022-10-18 17:52:31 +02:00
Nikita Sobolev ccab67ba79
gh-97982: Factorize PyUnicode_Count() and unicode_count() code (#98025)
Add unicode_count_impl() to factorize PyUnicode_Count()
and unicode_count() code.
2022-10-12 18:27:53 +02:00
Victor Stinner df3a6d9beb
gh-97982: Remove asciilib_count() (#98164)
asciilib_count() is the same than ucs1lib_count(): the code is not
specialized for ASCII strings, so it's not worth it to have a
separated function. Remove asciilib_count() function.
2022-10-11 17:59:58 +02:00
Ken Jin 7ec2e279fe
gh-95756: Free and NULL-out code caches when needed (GH-98181) 2022-10-11 23:11:46 +08:00
Ken Jin b399115ef1
gh-95756: Lazily created cached co_* attrs (GH-97791) 2022-10-11 11:26:08 +08:00
Carl Meyer e82d977eb0
gh-91052: Add PyDict_Unwatch for unwatching a dictionary (#98055) 2022-10-07 17:37:46 -07:00
Carl Meyer a4b7794887
GH-91052: Add C API for watching dictionaries (GH-31787) 2022-10-07 01:08:00 +01:00
larryhastings 6bfb0be804
gh-97943: PyFunction_GetAnnotations should return a borrowed reference. (#97949) 2022-10-06 12:23:20 -07:00
Serhiy Storchaka 77f0249308
gh-95196: Disable incorrect pickling of the C implemented classmethod descriptors (GH-96383) 2022-10-05 14:21:16 +03:00
Brandt Bucher 0ff8fd6583
GH-97779: Ensure that *all* frame objects are backed by "complete" frames (GH-97845) 2022-10-04 17:30:03 -07:00
Michael 07b8e85d0e
gh-96526: Clarify format and __format__ docstrings (gh-96648) 2022-10-03 15:28:02 -07:00
Gregory P. Smith b0f89cb431
gh-96512: Move int_max_str_digits setting to PyConfig (#96944)
It had to live as a global outside of PyConfig for stable ABI reasons in
the pre-3.12 backports.

This removes the `_Py_global_config_int_max_str_digits` and gets rid of
the equivalent field in the internal `struct _is PyInterpreterState` as
code can just use the existing nested config struct within that.

Adds tests to verify unique settings and configs in subinterpreters.
2022-10-03 13:55:45 -07:00
Nikita Sobolev e990c6af08
gh-94808: `_PyLineTable_StartsLine` was not used (GH-96609) 2022-10-03 19:35:43 +01:00
Ofey Chan d639438609
gh-97591: In `Exception.__setstate__()` acquire strong references before calling `tp_hash` slot (#97700) 2022-10-01 20:57:17 -07:00
Ofey Chan 83a3de4e06
gh-96348: Deprecate the 3-arg signature of coroutine.throw and generator.throw (GH-96428) 2022-09-30 09:43:02 +01:00
Victor Stinner a5f092f3c4
gh-97616: list_resize() checks for integer overflow (#97617)
Fix multiplying a list by an integer (list *= int): detect the
integer overflow when the new allocated length is close to the
maximum size.  Issue reported by Jordan Limor.

list_resize() now checks for integer overflow before multiplying the
new allocated length by the list item size (sizeof(PyObject*)).
2022-09-29 00:07:07 +02:00
LiarPrincess 0c1d7a06ed
bpo-47243: Duplicate entry in 'Objects/unicodetype_db.h' (GH-32376)
Fix for duplicate 1st entry in 'Objects/unicodetype_db.h':

```c
/* a list of unique character type descriptors */
const _PyUnicode_TypeRecord _PyUnicode_TypeRecords[] = {
    {0, 0, 0, 0, 0, 0},
    {0, 0, 0, 0, 0, 0}, <--- HERE
    {0, 0, 0, 0, 0, 32},
    {0, 0, 0, 0, 0, 48},
    …
```

https://bugs.python.org/issue47243

Automerge-Triggered-By: GH:isidentical
2022-09-28 06:57:14 -07:00
Pablo Galindo Salgado aab01e3524
gh-96670: Raise SyntaxError when parsing NULL bytes (#97594) 2022-09-27 23:23:42 +01:00
Oscar Benjamin 817fa28f81
gh-90716: Refactor PyLong_FromString to separate concerns (GH-96808)
This is a preliminary PR to refactor `PyLong_FromString` which is currently quite messy and has spaghetti like code that mixes up different concerns as well as duplicating logic.

In particular:

- `PyLong_FromString` now only handles sign, base and prefix detection and calls a new function `long_from_string_base` to parse the main body of the string.
- The `long_from_string_base` function handles all string validation and then calls `long_from_binary_base` or a new function `long_from_non_binary_base` to construct the actual `PyLong`.
- The existing `long_from_binary_base` function is simplified by factoring duplicated logic to `long_from_string_base`.
- The new function `long_from_non_binary_base` factors out much of the code from `PyLong_FromString` including in particular the quadratic algorithm reffered to in gh-95778 so that this can be seen separately from unrelated concerns such as string validation.
2022-09-25 10:09:50 +01:00
Samuel 8563966be4
Fix minor comment typo in dictobject.c (GH-96960)
Fix a minor comment typo in the Objects/dictobject.c file.

Automerge-Triggered-By: GH:methane
2022-09-20 05:17:40 -07:00
Victor Stinner e841ffc915
gh-95778: Mention sys.set_int_max_str_digits() in error message (#96874)
When ValueError is raised if an integer is larger than the limit,
mention sys.set_int_max_str_digits() in the error message.
2022-09-16 20:04:37 +02:00
adphrost a41ed975e8
GH-91049: Introduce set vectorcall field API for PyFunctionObject (GH-92257)
Co-authored-by: Andrew Frost <adfrost@fb.com>
Co-authored-by: Itamar Ostricher <itamarost@gmail.com>
2022-09-15 16:42:37 +01:00
Benjamin Peterson fd1e477f53
closes gh-96734: Update to Unicode 15.0.0. (GH-96809) 2022-09-13 15:45:12 -07:00
Dennis Sweeney 69d9a08099
gh-94808: improve comments and coverage of fastsearch.h (GH-96760) 2022-09-13 14:25:10 -04:00
Dong-hee Na 8d75a13fde
gh-90751: memoryview now supports half-float (#96738)
Co-authored-by: Antoine Pitrou <antoine@python.org>
2022-09-10 22:44:10 +02:00
Nikita Sobolev 30cc1901ef
gh-96364: Fix text signatures of `__getitem__` for `list` and `dict` (GH-96365) 2022-09-09 17:37:02 +09:00
philg314 b9634ac776
gh-96352: Set AttributeError context in _PyObject_GenericGetAttrWithDict (#96353) 2022-09-08 12:12:14 +01:00
Pablo Galindo Salgado 3fedfcf19b
gh-96143: Clear instruction cache after mprotect call (#96476) 2022-09-08 12:04:41 +01:00
Kumar Aditya 4e4bfffe2d
GH-90699: use statically allocated interned strings in typeobject's slotdefs (GH-94706) 2022-09-07 15:02:08 -07:00
Mark Dickinson b126196838
gh-95778: Correctly pre-check for int-to-str conversion (#96537)
Converting a large enough `int` to a decimal string raises `ValueError` as expected. However, the raise comes _after_ the quadratic-time base-conversion algorithm has run to completion. For effective DOS prevention, we need some kind of check before entering the quadratic-time loop. Oops! =)

The quick fix: essentially we catch _most_ values that exceed the threshold up front. Those that slip through will still be on the small side (read: sufficiently fast), and will get caught by the existing check so that the limit remains exact.

The justification for the current check. The C code check is:
```c
max_str_digits / (3 * PyLong_SHIFT) <= (size_a - 11) / 10
```

In GitHub markdown math-speak, writing $M$ for `max_str_digits`, $L$ for `PyLong_SHIFT` and $s$ for `size_a`, that check is:
$$\left\lfloor\frac{M}{3L}\right\rfloor \le \left\lfloor\frac{s - 11}{10}\right\rfloor$$

From this it follows that
$$\frac{M}{3L} < \frac{s-1}{10}$$
hence that
$$\frac{L(s-1)}{M} > \frac{10}{3} > \log_2(10).$$
So
$$2^{L(s-1)} > 10^M.$$
But our input integer $a$ satisfies $|a| \ge 2^{L(s-1)}$, so $|a|$ is larger than $10^M$. This shows that we don't accidentally capture anything _below_ the intended limit in the check.

<!-- gh-issue-number: gh-95778 -->
* Issue: gh-95778
<!-- /gh-issue-number -->

Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
2022-09-04 09:21:18 -07:00
Kumar Aditya 6dab8c95bd
GH-96458: Statically initialize utf8 representation of static strings (#96481) 2022-09-02 23:43:08 -07:00
Gregory P. Smith 511ca94520
gh-95778: CVE-2020-10735: Prevent DoS by very large int() (#96499)
Integer to and from text conversions via CPython's bignum `int` type is not safe against denial of service attacks due to malicious input. Very large input strings with hundred thousands of digits can consume several CPU seconds.

This PR comes fresh from a pile of work done in our private PSRT security response team repo.

Signed-off-by: Christian Heimes [Red Hat] <christian@python.org>
Tons-of-polishing-up-by: Gregory P. Smith [Google] <greg@krypto.org>
Reviews via the private PSRT repo via many others (see the NEWS entry in the PR).

<!-- gh-issue-number: gh-95778 -->
* Issue: gh-95778
<!-- /gh-issue-number -->

I wrote up [a one pager for the release managers](https://docs.google.com/document/d/1KjuF_aXlzPUxTK4BMgezGJ2Pn7uevfX7g0_mvgHlL7Y/edit#). Much of that text wound up in the Issue. Backports PRs already exist. See the issue for links.
2022-09-02 09:35:08 -07:00
Irit Katriel 4c72517cad
gh-93554: Conditional jump opcodes only jump forward (GH-96318) 2022-09-01 21:36:47 +01:00
Irit Katriel a91f25577c
gh-96455: update example in exception_handling_notes.txt to the 3.11RC bytecode (GH-96456) 2022-09-01 14:21:39 +01:00
Pablo Galindo Salgado f49dd54b72
gh-96143: Add some comments and minor fixes missed in the original PR (#96433)
* gh-96132: Add some comments and minor fixes missed in the original PR

* Update Doc/using/cmdline.rst

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>

Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2022-08-30 19:37:22 +01:00
Pablo Galindo Salgado 6d791a9736
gh-96143: Allow Linux perf profiler to see Python calls (GH-96123)
⚠️  ⚠️ Note for reviewers, hackers and fellow systems/low-level/compiler engineers ⚠️ ⚠️ 

If you have a lot of experience with this kind of shenanigans and want to improve the **first** version, **please make a PR against my branch** or **reach out by email** or **suggest code changes directly on GitHub**. 

If you have any **refinements or optimizations** please, wait until the first version is merged before starting hacking or proposing those so we can keep this PR productive.
2022-08-30 10:11:18 -07:00
Mark Shannon c09fa7542c
GH-96237: Allow non-functions as reference-holder in frames. (GH-96238) 2022-08-25 10:16:55 +01:00
Mark Shannon 575f8880bf
GH-96068: Document object layout (GH-96069) 2022-08-23 13:55:43 +01:00
Pablo Galindo Salgado 16ebae4cd4
GH-96187: Prevent _PyCode_GetExtra to return garbage for negative indexes (GH-96188) 2022-08-23 11:13:53 +01:00
Kumar Aditya 129998bd7b
GH-96075: move interned dict under runtime state (GH-96077) 2022-08-22 12:05:21 -07:00
Christian Heimes 53e6a9a725
gh-96046: Initialize ht_cached_keys in PyType_Ready() (GH-96047) 2022-08-21 22:24:03 -07:00
Brandt Bucher 5bfb3c372b
GH-90997: Wrap yield from/await in a virtual try/except StopIteration (GH-96010) 2022-08-19 12:33:44 -07:00
Christian Heimes d9c1b746b5
gh-96017: Fix some compiler warnings (GH-96018)
- "comparison of integers of different signs" in typeobject.c
- only define static_builtin_index_is_set in DEBUG builds
- only define recreate_gil with ifdef HAVE_FORK
2022-08-19 08:08:43 +02:00
Matthias Görgens 4a6fa89465
Remove dead code in _PyDict_GetItemHint and rename to _PyDict_LookupIndex (GH-95948) 2022-08-18 10:19:21 +01:00
Mark Shannon b73e3b6d4a
GH-95589: Dont crash when subclassing extension classes with multiple inheritance (GH-96028)
* Treat tp_weakref and tp_dictoffset like other opaque slots for multiple inheritance.

* Document Py_TPFLAGS_MANAGED_DICT and Py_TPFLAGS_MANAGED_WEAKREF in what's new.
2022-08-17 12:50:53 +01:00
Ken Jin 7276ca25f5
GH-93911: Specialize `LOAD_ATTR` for custom `__getattribute__` (GH-93988) 2022-08-17 12:37:07 +01:00
Christian Heimes 48174fa0b9
gh-96005: Handle WASI ENOTCAPABLE in getpath (GH-96006)
- On WASI `ENOTCAPABLE` is now mapped to `PermissionError`.
- The `errno` modules exposes the new error number.
- `getpath.py` now ignores `PermissionError` when it cannot open landmark
  files `pybuilddir.txt` and `pyenv.cfg`.
2022-08-16 20:20:15 +02:00
Mark Shannon 5a8c15819c
GH-95245: Move weakreflist into the pre-header. (GH-95996) 2022-08-16 13:57:18 +01:00
Mark Shannon 3ef3c6306d
GH-95707: Fix uses of `Py_TPFLAGS_MANAGED_DICT` (GH-95854)
* Make sure that tp_dictoffset is correct with Py_TPFLAGS_MANAGED_DICT is set.

* Avoid traversing managed dict twice when subclassing class with Py_TPFLAGS_MANAGED_DICT set.
2022-08-15 12:29:27 +01:00
Kumar Aditya f2afdf3352
GH-95977: Speed up calling pure python descriptor __get__ with vectorcall (gh-95978) 2022-08-15 00:13:42 +09:00
Erlend E. Aasland f07adf82f3
gh-90928: Improve static initialization of keywords tuple in AC (#95907) 2022-08-13 12:09:40 +02:00
Eric Snow 6f6a4e6cc5
gh-90928: Statically Initialize the Keywords Tuple in Clinic-Generated Code (gh-95860)
We only statically initialize for core code and builtin modules.  Extension modules still create
the tuple at runtime.  We'll solve that part of interpreter isolation separately.

This change includes generated code. The non-generated changes are in:

* Tools/clinic/clinic.py
* Python/getargs.c
* Include/cpython/modsupport.h
* Makefile.pre.in (re-generate global strings after running clinic)
* very minor tweaks to Modules/_codecsmodule.c and Python/Python-tokenize.c

All other changes are generated code (clinic, global strings).
2022-08-11 15:25:49 -06:00
Mark Dickinson 97e9cfa75a
gh-95605: Fix `float(s)` error message when `s` contains only whitespace (GH-95665)
This PR fixes the error message from float(s) in the case where s contains only whitespace.
2022-08-10 19:25:39 +01:00
Petr Viktorin 71c3d649b5
gh-95504: Fix negative numbers in PyUnicode_FromFormat (GH-95848)
Co-authored-by: philg314 <110174000+philg314@users.noreply.github.com>
2022-08-10 13:12:40 +02:00
Mark Shannon 8d37c62c2a
GH-92678: Document that you shouldn't be doing your own dictionary offset calculations. (GH-95598)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Stanley <46876382+slateny@users.noreply.github.com>
2022-08-09 14:26:37 +01:00
Serhiy Storchaka 62f06508e7
gh-95781: More strict format string checking in PyUnicode_FromFormatV() (GH-95784)
An unrecognized format character in PyUnicode_FromFormat() and
PyUnicode_FromFormatV() now sets a SystemError.
In previous versions it caused all the rest of the format string to be
copied as-is to the result string, and any extra arguments discarded.
2022-08-08 19:21:07 +03:00
Petr Viktorin 656dad702d
gh-93274: Expose receiving vectorcall in the Limited API (GH-95717) 2022-08-08 14:12:05 +02:00
Tim Gates 32b49f613a
docs: Fix a few typos (#94899)
- overriden => overridden
- calcualation => calculation

Signed-off-by: Tim Gates <tim.gates@iress.com>
2022-08-08 10:02:45 +02:00
Eric Snow 87154d8dd8
gh-94673: Add Per-Interpreter tp_subclasses for Static Builtin Types (gh-95301) 2022-08-04 19:26:59 -06:00
Eric Snow bdbadb905a
gh-94673: Recover Weaklist Lookup Performance (gh-95544)
gh-95302 seems to have introduced a small performance regression. Here we make some minor changes to recover that lost performance.
2022-08-04 11:28:15 -06:00
Petr Viktorin 7b370b7305
gh-93274: Make vectorcall safe on mutable classes & inherit it by default (#95437) 2022-08-04 17:19:29 +02:00
Petr Viktorin a613fedd6e
gh-95388: Deprecate creating immutable types with mutable bases (GH-95533) 2022-08-04 16:13:45 +02:00
Ken Jin 42b102bbf9
gh-94936: C getters: co_varnames, co_cellvars, co_freevars (#95008) 2022-08-04 06:53:31 -07:00
Mark Shannon 906e450932
GH-92678: Fix tp_dictoffset inheritance. (GH-95596)
* Add test for inheriting explicit __dict__ and weakref.

* Restore 3.10 behavior for multiple inheritance of C extension classes that store their dictionary at the end of the struct.
2022-08-03 18:56:24 +01:00
Brandt Bucher c7e5bbaee8
GH-95150: Use position and exception tables for code hashing and equality (GH-95509) 2022-08-01 11:02:56 -07:00
Mark Shannon de388c0a7b
GH-95245: Store object values and dict pointers in single tagged pointer. (GH-95278) 2022-08-01 14:34:54 +01:00
Dong-hee Na fb75d015f4
gh-91146: More reduce allocation size of list from str.split/rsplit (gh-95493)
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2022-08-01 22:15:07 +09:00
Dong-hee Na 50b2261bda
gh-91146: Reduce allocation size of list from str.split()/rsplit() (gh-95473) 2022-07-31 12:14:53 +09:00
Eric Snow 3e7cad3bca
gh-94673: Add Per-Interpreter tp_weaklist for Static Builtin Types (#95302)
* Store tp_weaklist on the interpreter state for static builtin types.

* Factor out _PyStaticType_GET_WEAKREFS_LISTPTR().

* Add _PyStaticType_ClearWeakRefs().

* Add a comment about how _PyStaticType_ClearWeakRefs() loops.

* Document the change.

* Update Doc/whatsnew/3.12.rst

* Fix a typo.
2022-07-28 19:23:47 -06:00
Pamela Fox 70068b9336
Fix Unicode doc and replace use of macro with PyMem_New function (GH-94088) 2022-07-28 23:32:16 +01:00
Irit Katriel bceb197947
gh-95369: add missing decref in error case of exception group's split (GH-95370) 2022-07-28 11:38:46 +01:00
Pablo Galindo Salgado f40bc7fa49
gh-95324: Emit a warning if an object doesn't call PyObject_GC_UnTrack during deallocation in debug mode (#95325) 2022-07-27 16:03:38 +01:00
Aivars Kalvāns 8c88e360e7
gh-95005: Replace PyAccu with PyUnicodeWriter (gh-95006) 2022-07-27 17:43:34 +09:00
Eric Snow 47e75a0025
gh-94673: Add Per-Interpreter Storage for Static Builtin Types (#95255)
This is the last precursor to storing tp_subclasses (and tp_weaklist) on the interpreter state for static builtin types.

Here we add per-type storage on PyInterpreterState, but only for the static builtin types.  This involves the following:

* add PyInterpreterState.types
   * move PyInterpreterState.type_cache to it
   * add a "num_builtins_initialized" field
   * add a "builtins" field (a static array big enough for all the static builtin types)
* add _PyStaticType_GetState() to look up a static builtin type's state
* (temporarily) add PyTypeObject.tp_static_builtin_index (to hold the type's index into PyInterpreterState.types.builtins)

We will be eliminating tp_static_builtin_index in a later change.
2022-07-26 17:26:43 -06:00
Pieter Eendebak 2ef73be891
gh-91247: Use memcpy for list and tuple repeat (#91482)
* Add _Py_memory_repeat function to pycore_list

* Add _Py_RefcntAdd function to pycore_object

* Use the new functions in tuplerepeat, list_repeat, and list_inplace_repeat
2022-07-25 22:10:23 -04:00
Mark Shannon 27055d766a
GH-92678: Expose managed dict clear and visit functions (#95246) 2022-07-25 22:30:53 +01:00
Eric Snow 2d26449b06
gh-94673: Always Finalize Static Builtin Types (#95153)
Static builtin types are finalized by calling _PyStaticType_Dealloc().  Before this change, we were skipping finalizing such a type if it still had subtypes (i.e. its tp_subclasses hadn't been cleared yet).  The problem is that types hold several heap objects, which leak if we skip the type's finalization.  This change addresses that.

For context, there's an old comment (from e9e3eab0b8) that says the following:

   // If a type still has subtypes, it cannot be deallocated.
   // A subtype can inherit attributes and methods of its parent type,
   // and a type must no longer be used once it's deallocated.

However, it isn't clear that is actually still true.  Clearing tp_dict should mean it isn't a problem.

Furthermore, the only subtypes that might still be around come from extension modules that didn't clean them up when unloaded (i.e. extensions that do not implement multi-phase initialization, AKA PEP 489).  Those objects are already leaking, so this change doesn't change anything in that regard.  Instead, this change means more objects gets cleaned up that before.
2022-07-25 14:23:41 -06:00
Eric Snow 4a1dd73431
gh-94673: Add _PyStaticType_InitBuiltin() (#95152)
This is the first of several precursors to storing tp_subclasses (and tp_weaklist) on the interpreter state for static builtin types.

We do the following:

* add `_PyStaticType_InitBuiltin()`
* add `_Py_TPFLAGS_STATIC_BUILTIN`
* set it on all static builtin types in `_PyStaticType_InitBuiltin()`
* shuffle some code around to be able to use _PyStaticType_InitBuiltin()
    * rename `_PyStructSequence_InitType()` to `_PyStructSequence_InitBuiltinWithFlags()`
    * add `_PyStructSequence_InitBuiltin()`.
2022-07-25 12:47:31 -06:00
Pablo Galindo Salgado 9007dec606
gh-95173: Revert commit 51ed2c56a1 (#95176) 2022-07-24 22:33:06 +01:00
Brandt Bucher e4d3a96a11
GH-94438: Handle extended arguments and conditional pops in mark_stacks (GH-95110) 2022-07-22 16:28:03 -07:00
Brandt Bucher e402b26b7f
GH-95113: Don't use EXTENDED_ARG_QUICK in unquickened code (GH-95121) 2022-07-22 11:04:20 -07:00
Ken Jin a6daaf2a13
Fix PyCode_Addr2Location when addrq < 0 (GH-95091) 2022-07-21 14:49:49 +01:00
Kumar Aditya 1834133e66
GH-90699: fix ref counting of static immortal strings (gh-94850) 2022-07-20 15:23:30 +09:00
Brandt Bucher f36589510b
GH-91153: Handle mutating __index__ methods in bytearray item assignment (GH-94891) 2022-07-19 09:42:40 -07:00
Mark Shannon 3f738600f6
GH-94979: Pop the exception stack when jumping out of exception handlers. (#95003) 2022-07-19 16:33:18 +01:00
Eric Snow 7a1a85d640
gh-94673: [c-analyzer] Add a Script to Identify Static Types (#94989)
issue: https://github.com/python/cpython/issues/94673
2022-07-18 19:03:57 -06:00
Brandt Bucher b407312a24
GH-94893: Ignore caches when adding LOAD_FAST_CHECKs (GH-94894) 2022-07-18 09:00:41 -07:00
Mark Shannon 2f8bff6879
GH-94739: Mark stacks of exception handling blocks for setting frame.f_lineno in the debugger. (GH-94958) 2022-07-18 16:06:42 +01:00
neonene 9b3f779209
gh-94841: Ensure arena_map_get() is inlined in PyObject_Free() (#94842) 2022-07-14 11:33:25 -07:00
Ken Jin 6442a9dd21
gh-94607: Fix subclassing generics (GH-94610)
Co-authored-by: Serhiy Storchaka <3659035+serhiy-storchaka@users.noreply.github.com>
2022-07-09 12:18:01 +08:00
Kumar Aditya 9dff9f4814
GH-90699: Intern statically allocated strings (GH-93597)
This is similar to how strings are interned for deepfreeze.
2022-07-08 10:47:37 -07:00
Irit Katriel 50b9a7762f
gh-94438: in frameobject's mark_stacks switch, the PUSH_EXC_INFO and POP_EXCEPT cases are no longer reachable (GH-94582) 2022-07-06 07:15:59 +01:00
Mark Shannon be80db14c4
GH-94438: Account for NULLs on evaluation stack when jumping lines. (GH-94444) 2022-07-01 14:01:14 +01:00
Mark Shannon 544531de23
GH-94262: Don't create frame objects for frames that aren't yet complete. (GH-94371) 2022-07-01 11:08:20 +01:00
Pablo Galindo Salgado c485ec014c
gh-88116: Avoid undefined behavior when decoding varints in code objects (#94375) 2022-06-28 14:24:54 +01:00
Serhiy Storchaka efdc9d68de
gh-87995: Make MappingProxyType hashable (GH-94252) 2022-06-28 11:54:58 +02:00
Mark Shannon c0453a40fa
GH-94163: Add BINARY_SLICE and STORE_SLICE instructions. (GH-94168) 2022-06-27 12:24:23 +01:00
Ken Jin 50a5ab2c0b
gh-93382: Sync up `co_code` changes with 3.11 (GH-94227)
Sync up co_code changes with 3.11 commit 852b4d4bcd.
2022-06-25 01:55:18 +08:00
Kumar Aditya 5b6e5762ca
GH-91742: Fix pdb crash after jump (GH-94171) 2022-06-23 15:19:13 +01:00
Jelle Zijlstra 4e08fbcfdf
gh-93021: Fix __text_signature__ for __get__ (GH-93023)
Because of the way wrap_descr_get is written, the second argument
to __get__ methods implemented through the wrapper is always
optional.
2022-06-21 21:45:38 +02:00
Dennis Sweeney 5fcfdd87c9
GH-91432: Specialize FOR_ITER (GH-91713)
* Adds FOR_ITER_LIST and FOR_ITER_RANGE specializations.

* Adds _PyLong_AssignValue() internal function to avoid temporary boxing of ints.
2022-06-21 11:19:26 +01:00
Irit Katriel 1603a1029f
GH-93249: relax overly strict assertion on bounds->ar_start (GH-93961) 2022-06-20 17:13:39 +01:00
Mark Shannon ab0e601016
GH-93516: Speedup line number checks when tracing. (GH-93763)
* Use a lookup table to reduce overhead of getting line numbers during tracing.
2022-06-20 13:00:42 +01:00
Mark Shannon 45e62a2bc1
GH-93897: Store frame size in code object and de-opt if insufficient space on thread frame stack. (GH-93908) 2022-06-20 12:59:25 +01:00
Christian Heimes 774ef28814
gh-84461: Silence some compiler warnings on WASM (GH-93978) 2022-06-20 13:34:40 +02:00
Victor Stinner 27b9894033
gh-93937, C API: Move PyFrame_GetBack() to Python.h (#93938)
Move the follow functions and type from frameobject.h to pyframe.h,
so the standard <Python.h> provide frame getter functions:

* PyFrame_Check()
* PyFrame_GetBack()
* PyFrame_GetBuiltins()
* PyFrame_GetGenerator()
* PyFrame_GetGlobals()
* PyFrame_GetLasti()
* PyFrame_GetLocals()
* PyFrame_Type

Remove #include "frameobject.h" from many C files. It's no longer
needed.
2022-06-19 12:02:33 +02:00
Kumar Aditya 726448ebe1
GH-93990: fix refcounting bug in `add_subclass` in `typeobject.c` (GH-93989) 2022-06-19 14:06:37 +08:00
Ken Jin fea1e9bc5c
gh-93955: Use unbound methods for slot `__getattr__` and `__getattribute__` (GH-93956) 2022-06-18 22:42:42 +08:00
Serhiy Storchaka f9433fff47
gh-89828: Do not relay the __class__ attribute in GenericAlias (#93754)
list[int].__class__ returned type, and isinstance(list[int], type)
returned True. It caused numerous problems in code that checks
isinstance(x, type).
2022-06-18 11:34:57 +03:00
Ken Jin 11190c4ad0
gh-92888: Fix memoryview bad `__index__` use after free (GH-92946)
Co-authored-by: chilaxan <35645806+chilaxan@users.noreply.github.com>
Co-authored-by: Serhiy Storchaka <3659035+serhiy-storchaka@users.noreply.github.com>
2022-06-17 23:14:53 +08:00
Ken Jin a51742ab82
gh-93911: Specialize `LOAD_ATTR_PROPERTY` (GH-93912) 2022-06-17 23:13:17 +08:00
Pamela Fox ff095e13df
Update includes in call.c (GH-93786) 2022-06-15 13:24:12 +01:00
Mark Shannon 3cd1a5d3ec
GH-93516: Store offset of first traceable instruction in code object (GH-93769) 2022-06-14 11:09:30 +01:00
Petr Viktorin 3597c12941
gh-89546: Clean up PyType_FromMetaclass (GH-93686)
When changing PyType_FromMetaclass recently (GH-93012, GH-93466, GH-28748)
I found a bunch of opportunities to improve the code. Here they are.

Fixes: #89546

Automerge-Triggered-By: GH:encukou
2022-06-14 02:13:29 -07:00
Serhiy Storchaka 8352e322e8
gh-79512: Fixed names and __module__ value of weakref classes (GH-93719)
Classes ReferenceType, ProxyType and CallableProxyType have now correct
atrtributes __module__, __name__ and __qualname__.
It makes them (types, not instances) pickleable.
2022-06-14 07:18:03 +03:00
Serhiy Storchaka 6fd4c8ec77
gh-93741: Add private C API _PyImport_GetModuleAttrString() (GH-93742)
It combines PyImport_ImportModule() and PyObject_GetAttrString()
and saves 4-6 lines of code on every use.

Add also _PyImport_GetModuleAttr() which takes Python strings as arguments.
2022-06-14 07:15:26 +03:00
Kumar Aditya 9331087966
GH-90699: use statically allocated strings in typeobject.c (gh-93751) 2022-06-13 01:38:18 +09:00
Dong-hee Na ae1ca74513
gh-93747: Fix Refleak when handling multiple Py_tp_doc slots (gh-93749) 2022-06-13 00:55:01 +09:00
Kumar Aditya 8f728e5baa
gh-93728: fix memory leak in deepfrozen code objects (GH-93729) 2022-06-12 22:42:18 +08:00
Serhiy Storchaka 3473817106
gh-91162: Support splitting of unpacked arbitrary-length tuple over TypeVar and TypeVarTuple parameters (alt) (GH-93412)
For example:

  A[T, *Ts][*tuple[int, ...]] -> A[int, *tuple[int, ...]]
  A[*Ts, T][*tuple[int, ...]] -> A[*tuple[int, ...], int]
2022-06-12 16:22:01 +03:00
Petr Viktorin 21a9a85ff4
gh-93466: Document PyType_Spec doesn't accept repeated slot IDs; raise where this was problematic (GH-93471) 2022-06-10 15:55:09 +02:00
Sebastian Berg 7fef847662
bpo-45383: Get metaclass from bases in PyType_From* (GH-28748)
This checks the bases of of a type created using the FromSpec
API to inherit the bases metaclasses.  The metaclass's alloc
function will be called as is done in `tp_new` for classes
created in Python.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@protonmail.com>
2022-06-09 17:11:08 +02:00
Mark Dickinson e12f34b6d8
Fix missing word in sys.float_info docstring (GH-93489) 2022-06-04 17:28:14 +01:00
Ken Jin d52ffc1d1f
gh-93382: Cache result of `PyCode_GetCode` in codeobject (GH-93383)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
Co-authored-by: Dennis Sweeney <36520290+sweeneyde@users.noreply.github.com>
2022-06-04 00:41:18 +08:00
Dennis Sweeney f425f3bb27
gh-93143: Avoid NULL check in LOAD_FAST based on analysis in the compiler (GH-93144) 2022-05-31 16:32:30 -04:00
Serhiy Storchaka f545fc955a
gh-93345: Fix a crash in substitution of nested TypeVar after TypeVarTuple (GH-93346)
For example: tuple[*Ts, list[T]][int, str, bool]
2022-05-31 08:46:16 +03:00
Sergei Izmailov 66f5add1f0
bpo-41287: Handle `doc` argument of `property.__init__` in subclasses (#23205)
Fixes #85459

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-05-28 20:25:51 -07:00
Eric Snow caa279d6fd
bpo-40514: Drop EXPERIMENTAL_ISOLATED_SUBINTERPRETERS (gh-93185)
This was added for bpo-40514 (gh-84694) to test out a per-interpreter GIL. However, it has since proven unnecessary to keep the experiment in the repo. (It can be done as a branch in a fork like normal.) So here we are removing:

* the configure option
* the macro
* the code enabled by the macro
2022-05-27 17:38:01 -06:00
Mark Shannon bbcf42449e
GH-90230: Add stats to breakdown the origin of calls to `PyEval_EvalFrame` (GH-93284) 2022-05-27 16:31:41 +01:00
Kumar Aditya cb04a09d2d
GH-93207: Remove HAVE_STDARG_PROTOTYPES configure check for stdarg.h (#93215) 2022-05-27 13:30:45 +02:00
Wenzel Jakob 5e34b494a0
gh-60074: add new stable API function PyType_FromMetaclass (GH-93012)
Added a new stable API function ``PyType_FromMetaclass``, which mirrors
the behavior of ``PyType_FromModuleAndSpec`` except that it takes an
additional metaclass argument. This is, e.g., useful for language
binding tools that need to store additional information in the type
object.
2022-05-27 10:27:39 +02:00
Victor Stinner 5f8c3fb997
gh-91924: Optimize unicode_check_encoding_errors() (#93200)
Avoid _PyCodec_Lookup() and PyCodec_LookupError() for most common
built-in encodings and error handlers to avoid creating a temporary
Unicode string object, whereas these encodings and error handlers are
known to be valid.
2022-05-27 00:39:49 +02:00
oda-gitso 854db1a606
Remove unnecessary for loop initializer in long_lshift1() (GH-93071)
* Remove unnecessary for loop initialization.
2022-05-25 18:57:33 +01:00
Victor Stinner 71d8775fee
gh-93202: Always use %zd printf formatter (#93201)
Python now always use the ``%zu`` and ``%zd`` printf formats to
format a size_t or Py_ssize_t number. Building Python 3.12 requires a
C11 compiler, so these printf formats are now always supported.

* PyObject_Print() and _PyObject_Dump() now use the printf %zd format
  to display an object reference count.
* Update PY_FORMAT_SIZE_T comment.
* Remove outdated notes about the %zd format in PyBytes_FromFormat()
  and PyUnicode_FromFormat() documentations.
* configure no longer checks for the %zd format and no longer defines
  PY_FORMAT_SIZE_T macro in pyconfig.h.
* pymacconfig.h no longer undefines PY_FORMAT_SIZE_T: macOS 10.4 is
  no longer supported. Python 3.12 now requires macOS 10.6 (Snow
  Leopard) or newer.
2022-05-25 14:21:36 +02:00
oda-gitso 9485a0dbdd
gh-93040 Wrap unused parameters in Objects/obmalloc.c with Py_UNUSED (#93175) 2022-05-25 12:39:06 +02:00
goldsteinn 7108bdf27c
gh-93033: Use wmemchr in stringlib (GH-93034)
Generally comparable perf for the "good" case where memchr doesn't
return any collisions (false matches on lower byte) but clearly faster
with collisions.
2022-05-24 10:45:31 +09:00
Kumar Aditya c5f5f978ca
GH-92955: fix memory leak in code object lines and positions iterators (gh-92956) 2022-05-19 22:55:22 +09:00
Serhiy Storchaka 8a6af5a346
gh-92914: Round the allocated size for lists up to the even number (GH-92915) 2022-05-19 08:43:50 +03:00
Mark Shannon a4460f2eb8
Split refcount stats into 'interpreter' and 'non-interpreter' (GH-92919) 2022-05-18 14:38:43 +01:00
Kumar Aditya d923fdf54b
GH-92804: Fix memory leak in memoryview iterator (gh-92805) 2022-05-14 23:24:20 +09:00
Dong-hee Na 2e8f721c0f
gh-90861: Memory optimization for set.issubset (gh-92799) 2022-05-14 17:58:19 +09:00
Victor Stinner 059b5baf98
gh-85858: Remove PyUnicode_InternImmortal() function (#92579)
Remove the PyUnicode_InternImmortal() function and the
SSTATE_INTERNED_IMMORTAL macro.

The PyUnicode_InternImmortal() function is still exported in the
stable ABI. The function is removed from the API.

PyASCIIObject.state.interned size is now a single bit, rather than 2
bits.

Keep SSTATE_NOT_INTERNED and SSTATE_INTERNED_MORTAL macros for
backward compatibility, but no longer use them internally since the
interned member is now a single bit and so can only have two values
(interned or not interned).

Update stats of _PyUnicode_ClearInterned().
2022-05-13 13:40:22 +02:00
Victor Stinner f62ad4f2c4
gh-89653: Use int type for Unicode kind (#92704)
Use the same type that PyUnicode_FromKindAndData() kind parameter
type (public C API): int.
2022-05-13 12:41:05 +02:00
Victor Stinner db388df1d9
gh-89653: PEP 670: Convert PyUnicode_KIND() macro to function (#92705)
In the limited C API version 3.12, PyUnicode_KIND() is now
implemented as a static inline function. Keep the macro for the
regular C API and for the limited C API version 3.11 and older to
prevent introducing new compiler warnings.

Update _decimal.c and stringlib/eq.h for PyUnicode_KIND().
2022-05-13 11:49:56 +02:00
ravi140222 079f0dd719
gh-91578: improved error message when trying to instantiate an abstract class with missing methods (gh-47246) 2022-05-12 11:26:11 -04:00
Inada Naoki f9c9354a7a
gh-92536: PEP 623: Remove wstr and legacy APIs from Unicode (GH-92537) 2022-05-12 14:48:38 +09:00
Victor Stinner 897f14d38d
gh-89653: PEP 670: Convert PyCell macros to functions (#92653)
Convert the following macros to static inline functions:

* PyCell_GET()
* PyCell_SET()

Limited C API version 3.12 no longer casts arguments.

Fix also usage of PyCell_SET(): only delete the old value after
setting the new value.
2022-05-11 23:24:48 +02:00
Victor Stinner ffcc7cd57f
gh-89653: PEP 670: Convert pycore_gc.h macros to functions (#92649)
Convert the following macros to static inline functions:

* _Py_AS_GC()
* _PyGCHead_FINALIZED(), _PyGCHead_SET_FINALIZED()
* _PyGCHead_NEXT(), _PyGCHead_SET_NEXT()
* _PyGCHead_PREV(), _PyGCHead_SET_PREV()
* _PyGC_FINALIZED(), _PyGC_SET_FINALIZED()
* _PyObject_GC_IS_TRACKED()
* _PyObject_GC_MAY_BE_TRACKED()

Add a macro wrapping the _PyObject_GC_IS_TRACKED() function to cast
the argument to PyObject*.
2022-05-11 13:37:18 +02:00
Serhiy Storchaka 9d25db9db1
gh-91162: Fix substitution of unpacked tuples in generic aliases (GH-92335) 2022-05-08 18:32:32 +03:00
Tim Peters 9652900969
Issues/88027: A potential double free in list_sort_impl (#92367)
merge_freemem(): set keys to NULL do it's harmless to call this again.
2022-05-05 23:14:09 -05:00
Alexey Izbyshev 85354ed78c
gh-92112: Fix crash triggered by an evil custom `mro()` (#92113) 2022-05-05 21:01:15 -07:00
Serhiy Storchaka 1ed8d035f1
gh-87390: Fix starred tuple equality and pickling (GH-92337) 2022-05-05 20:16:06 +03:00
Géry Ogam a95138b2c5
bpo-43857: Improve the AttributeError message when deleting a missing attribute (#25424)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-05-05 06:37:26 -07:00
Michael J. Sullivan a918589578
bpo-46764: Fix wrapping bound method with @classmethod (#31367) 2022-05-04 23:00:21 -05:00
Victor Stinner d716a0dfe2
Use static inline function Py_EnterRecursiveCall() (#91988)
Currently, calling Py_EnterRecursiveCall() and
Py_LeaveRecursiveCall() may use a function call or a static inline
function call, depending if the internal pycore_ceval.h header file
is included or not. Use a different name for the static inline
function to ensure that the static inline function is always used in
Python internals for best performance. Similar approach than
PyThreadState_GET() (function call) and _PyThreadState_GET() (static
inline function).

* Rename _Py_EnterRecursiveCall() to _Py_EnterRecursiveCallTstate()
* Rename _Py_LeaveRecursiveCall() to _Py_LeaveRecursiveCallTstate()
* pycore_ceval.h: Rename Py_EnterRecursiveCall() to
  _Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() and
  _Py_LeaveRecursiveCall()
2022-05-04 13:30:23 +02:00
Mark Shannon 836b17c9c3
Add more stats for freelist use and allocations. (GH-92211) 2022-05-03 16:40:24 -06:00
Victor Stinner 804f2529d8
gh-91320: Use _PyCFunction_CAST() (#92251)
Replace "(PyCFunction)(void(*)(void))func" cast with
_PyCFunction_CAST(func).

Change generated by the command:

sed -i -e \
  's!(PyCFunction)(void(\*)(void)) *\([A-Za-z0-9_]\+\)!_PyCFunction_CAST(\1)!g' \
  $(find -name "*.c")
2022-05-03 21:42:14 +02:00
Victor Stinner b270b82f11
gh-91320: Argument Clinic uses _PyCFunction_CAST() (#32210)
Replace "(PyCFunction)(void(*)(void))func" cast with
_PyCFunction_CAST(func).
2022-05-03 20:25:41 +02:00
Dennis Sweeney b156578bd6
gh-92031: Deoptimize Static Code at Finalization (GH-92039) 2022-05-03 08:59:12 -06:00
Ken Jin 6c7249f265
gh-92154: Expose PyCode_GetCode in the C API (GH-92168) 2022-05-03 21:13:13 +08:00
larryhastings c96da83a8e
Fix the closure argument to PyEval_EvalCodeEx. (GH-92175) 2022-05-02 14:08:22 -06:00
Jelle Zijlstra ff88f7e007
gh-87390: Add __unpacked__ attribute to types.GenericAlias (#92059) 2022-05-02 13:21:59 -06:00
Mark Dickinson 0ed91a26fe
gh-90213: Speed up right shifts of negative integers (GH-30277) 2022-05-02 11:19:03 -06:00
Serhiy Storchaka 18b07d773e
bpo-36819: Fix crashes in built-in encoders with weird error handlers (GH-28593)
If the error handler returns position less or equal than the starting
position of non-encodable characters, most of built-in encoders didn't
properly re-size the output buffer. This led to out-of-bounds writes,
and segfaults.
2022-05-02 12:37:48 +03:00
Serhiy Storchaka 4d10f703d7
gh-92114: Improve error message for types with __class_getitem__ = None (GH-92115) 2022-05-02 08:29:49 +03:00
Serhiy Storchaka 3483299a24
gh-81548: Deprecate octal escape sequences with value larger than 0o377 (GH-91668) 2022-04-30 13:16:27 +03:00
Serhiy Storchaka e8c2f72b94
bpo-43224: Implement substitution of unpacked TypeVarTuple in C (GH-31828)
Co-authored-by: Matthew Rahtz <mrahtz@gmail.com>
2022-04-30 08:22:46 +03:00
Yurii Karabas 0ef8d921f5
gh-91603: Speed up isinstance/issubclass on union types (GH-91631)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2022-04-28 23:24:19 +08:00
Serhiy Storchaka cd1fbbc817
gh-91603: Speed up operator "|" for UnionType (GH-91955)
Reduce the complexity from O((M+N)^2) to O(M*N), where M and N are the length
of __args__ for both operands (1 for operand which is not a UnionType).

As a consequence, the complexity of parameter substitution in UnionType has
been reduced from O(N^3) to O(N^2).

Co-authored-by: Yurii Karabas <1998uriyyo@gmail.com>
2022-04-28 13:25:33 +03:00
Victor Stinner 64a54e511d
gh-91719: Add pycore_opcode.h internal header file (#91906)
Move the following API from Include/opcode.h (public C API) to a new
Include/internal/pycore_opcode.h header file (internal C API):

* EXTRA_CASES
* _PyOpcode_Caches
* _PyOpcode_Deopt
* _PyOpcode_Jump
* _PyOpcode_OpName
* _PyOpcode_RelativeJump
2022-04-26 00:14:30 +02:00
Victor Stinner 364ed94092
gh-89373: _Py_Dealloc() checks tp_dealloc exception (#32357)
If Python is built in debug mode, _Py_Dealloc() now ensures that the
tp_dealloc function leaves the current exception unchanged.
2022-04-21 23:04:01 +02:00
Mark Shannon d44815cabc
GH-88116: Document that PyCodeNew is dangerous, and make PyCode_NewEmpty less dangerous. (GH-91790) 2022-04-21 19:08:36 +01:00
Mark Shannon 944fffee89
GH-88116: Use a compact format to represent end line and column offsets. (GH-91666)
* Stores all location info in linetable to conform to PEP 626.

* Remove column table from code objects.

* Remove end-line table from code objects.

* Document new location table format
2022-04-21 16:10:37 +01:00
Dennis Sweeney f2b4e458b3
gh-91636: Don't clear required fields of function objects (GH-91651) 2022-04-21 02:06:35 -04:00
Victor Stinner 7cdaf87ec5
gh-91731: Replace Py_BUILD_ASSERT() with static_assert() (#91730)
Python 3.11 now uses C11 standard which adds static_assert()
to <assert.h>.

* In pytime.c, replace Py_BUILD_ASSERT() with preprocessor checks on
  SIZEOF_TIME_T with #error.
* On macOS, py_mach_timebase_info() now accepts timebase members with
  the same size than _PyTime_t.
* py_get_monotonic_clock() now saturates GetTickCount64() to
  _PyTime_MAX: GetTickCount64() is unsigned, whereas _PyTime_t is
  signed.
2022-04-20 19:26:40 +02:00
Dennis Sweeney d7d7e6c007
Cast to (destructor) to fix compiler warnings (GH-91711) 2022-04-20 16:15:45 +01:00
Dong-hee Na f571c26fc1
gh-91632: Fix generic_alias_iterator to be finalized at exit. (GH-91727) 2022-04-20 23:10:41 +09:00
Inada Naoki 4d2403fd50
gh-91020: Add `PyBytes_Type.tp_alloc` for subclass (GH-91686) 2022-04-20 14:06:29 +09:00
Dennis Sweeney da6c78584b
gh-90667: Add specializations of Py_DECREF when types are known (GH-30872) 2022-04-19 19:02:19 +01:00
Kumar Aditya ab0d35d70d
bpo-46712: share more global strings in deepfreeze (gh-32152)
(for gh-90868)
2022-04-19 11:41:36 -06:00
Victor Stinner aa5c0a9f8d
bpo-40421: Cleanup PyFrame C API (GH-32417) 2022-04-19 09:53:10 +02:00
Oleg Iarygin 7fbc7f6128
gh-91098: Use Argument Clinic for Object/classobject.c to fix docstrings (#31711)
Closes GH-91098.

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-04-18 19:56:53 -07:00
Oleg Iarygin 2f0fc521f4
gh-91102: Use Argument Clinic for EncodingMap (#31725)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-04-18 13:43:56 -07:00
Kumar Aditya 8c54c3dacc
gh-91576: Speed up iteration of strings (#91574) 2022-04-18 07:18:27 -07:00
Oleg Iarygin a573cb2fec
gh-91118: Fix docstrings that do not honor --without-doc-strings (#31769)
Co-authored-by: Éric <merwok@netwok.org>
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-04-17 20:39:32 -07:00
Irit Katriel d4c4a76ed1
gh-89770: Implement PEP-678 - Exception notes (GH-31317) 2022-04-16 19:59:52 +01:00
Pieter Eendebak 355cbaadbb
gh-91266: refactor bytearray strip methods (GH-32096) 2022-04-13 22:20:38 -04:00
Pablo Galindo Salgado 37a53fb6bd
gh-91502: Add a new API to check if a frame is an entry frame (GH-91503) 2022-04-13 14:06:56 +01:00
Tobias Stoeckmann 0859368335
gh-91421: Use constant value check during runtime (GH-91422)
The left-hand side expression of the if-check can be converted to a
constant by the compiler, but the addition on the right-hand side is
performed during runtime.

Move the addition from the right-hand side to the left-hand side by
turning it into a subtraction there. Since the values are known to
be large enough to not turn negative, this is a safe operation.

Prevents a very unlikely integer overflow on 32 bit systems.

Fixes GH-91421.
2022-04-12 20:01:02 -07:00
Dennis Sweeney 8be8949116
gh-91117: Ensure integer mod and pow operations use cached small ints (GH-31843) 2022-04-11 16:07:09 -04:00
Mark Shannon f6e43e834c
GH-89480: Document motivation, design and implementation of 3.11 frame stack. (GH-32304) 2022-04-11 16:05:20 +01:00
John Belmonte b0b836b20c
bpo-45995: add "z" format specifer to coerce negative 0 to zero (GH-30049)
Add "z" format specifier to coerce negative 0 to zero.

See https://github.com/python/cpython/issues/90153 (originally https://bugs.python.org/issue45995) for discussion.
This covers `str.format()` and f-strings.  Old-style string interpolation is not supported.

Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2022-04-11 15:34:18 +01:00
Irit Katriel dd207a6ac5
bpo-47120: make POP_JUMP_IF_TRUE/FALSE/NONE/NOT_NONE relative (GH-32400) 2022-04-11 10:40:24 +01:00
Raymond Hettinger d6fb104690
Fix bad grammar and import docstring for split/rsplit (GH-32381) 2022-04-08 08:36:20 -05:00
Mark Shannon 5b4a4b6f09
Add new PyFrame_GetLasti C-API function (GH-32413) 2022-04-08 12:18:57 +01:00
Dong-hee Na e2d78baed3
bpo-47250: Fix refleak from object.__getstate__() (GH-32403)
Co-authored-by: Brandt Bucher <brandt@python.org>
2022-04-08 08:27:00 +09:00
Brandt Bucher ef6a482b02
bpo-47177: Replace `f_lasti` with `prev_instr` (GH-32208) 2022-04-07 12:31:01 -07:00
Victor Stinner 85addfb9c6
bpo-35134: Remove the Include/code.h header file (GH-32385)
Remove the Include/code.h header file. C extensions should only
include the main <Python.h> header file.

Python.h includes directly Include/cpython/code.h instead.
2022-04-07 02:29:52 +02:00
Serhiy Storchaka 884eba3c76
bpo-26579: Add object.__getstate__(). (GH-2821)
Copying and pickling instances of subclasses of builtin types
bytearray, set, frozenset, collections.OrderedDict, collections.deque,
weakref.WeakSet, and datetime.tzinfo now copies and pickles instance attributes
implemented as slots.
2022-04-06 20:00:14 +03:00
Serhiy Storchaka a69a4a917c
bpo-46721: Optimize set.issuperset() for non-set arguments (GH-31280) 2022-04-06 19:57:13 +03:00
Serhiy Storchaka 31cd25f4e1
bpo-43464: Optimize set.intersection() for non-set arguments (GH-31316) 2022-04-06 19:56:28 +03:00
Irit Katriel 0aa8d5cbd8
bpo-47120: make JUMP_NO_INTERRUPT relative (GH-32221) 2022-04-05 12:49:08 +01:00
Irit Katriel 32091df41c
bpo-47186: Replace JUMP_IF_NOT_EG_MATCH by CHECK_EG_MATCH + jump (GH-32309) 2022-04-05 12:06:22 +01:00
Dong-hee Na b183f48649
no-issue: Add assertion to PyModule_GetName for understanding (GH-32236) 2022-04-02 09:56:30 +09:00
Irit Katriel 04e07c258f
bpo-47186: Replace JUMP_IF_NOT_EXC_MATCH by CHECK_EXC_MATCH + jump (GH-32231) 2022-04-01 13:59:38 +01:00
Brandt Bucher ae9de82e32
bpo-46841: Use a `bytes` object for `_co_code_adaptive` (GH-32205) 2022-04-01 12:28:50 +01:00
Brandt Bucher bd2e47c883
bpo-46841: Avoid unnecessary allocations in code object comparisons (GH-32222) 2022-04-01 11:42:46 +01:00
Dennis Sweeney a0ea7a116c
bpo-47009: Streamline list.append for the common case (GH-31864) 2022-04-01 11:23:42 +01:00
Mark Shannon 74b95d86e0
bpo-40421: Add missing getters for frame object attributes to C-API. (GH-32114) 2022-03-31 17:13:25 +01:00
Christian Heimes 44e915028d
bpo-47182: Fix crash by named unicode characters after interpreter reinitialization (GH-32212)
Automerge-Triggered-By: GH:tiran
2022-03-31 08:14:50 -07:00
Dave Goncalves 8be7c2bc5a
bpo-14911: Corrected generator.throw() documentation (GH-32207)
Co-authored-by: Andrew Svetlov <andrew.svetlov@gmail.com>
2022-03-31 16:56:48 +03:00
Irit Katriel a00518d9ad
bpo-47120: Replace the JUMP_ABSOLUTE opcode by the relative JUMP_BACKWARD (GH-32115) 2022-03-31 14:14:15 +01:00
Victor Stinner c14d7e4b81
bpo-47164: Add _PyASCIIObject_CAST() macro (GH-32191)
Add macros to cast objects to PyASCIIObject*, PyCompactUnicodeObject*
and PyUnicodeObject*: _PyASCIIObject_CAST(),
_PyCompactUnicodeObject_CAST() and _PyUnicodeObject_CAST(). Using
these new macros make the code more readable and check their argument
with: assert(PyUnicode_Check(op)).

Remove redundant assert(PyUnicode_Check(op)) in macros using directly
or indirectly these new CAST macros.

Replacing existing casts with these macros.
2022-03-31 09:59:27 +02:00
Christian Heimes 581c4434de
bpo-47162: Add call trampoline to mitigate bad fpcasts on Emscripten (GH-32189) 2022-03-30 12:28:33 -07:00
Dong-hee Na d0c67ea064
bpo-46775: OSError should call winerror_to_errno unconditionally on Windows (GH-32179) 2022-03-30 18:31:33 +01:00
Pieter Eendebak 850687df47
bpo-47070: Add _PyBytes_Repeat() (GH-31999)
Use it where appropriate: the repeat functions of `array.array`, `bytes`, `bytearray`, and `str`.
2022-03-28 04:43:45 -04:00
Pieter Eendebak c23ddf5ec2
bpo-47116: use _PyLong_FromUnsignedChar instead of PyLong_FromLong (GH-32110) 2022-03-26 15:47:38 -04:00
Mark Shannon d7163bb35d
bpo-42197: Don't create `f_locals` dictionary unless we actually need it. (GH-32055)
* `PyFrame_FastToLocalsWithError` and `PyFrame_LocalsToFast` are no longer called during profile and tracing.
 (Contributed by Fabio Zadrozny)

* Make accesses to a frame's `f_locals` safe from C code, not relying on calls to `PyFrame_FastToLocals` or `PyFrame_LocalsToFast`.

* Document new `PyFrame_GetLocals` C-API function.
2022-03-25 12:57:50 +00:00
Kumar Aditya bd1cf6ecee
bpo-47012: speed up iteration of bytes and bytearray (GH-31867) 2022-03-23 04:30:05 -04:00
Mark Shannon 49daf6dba8
bpo-47045: Remove `f_state` field (GH-31963)
* Remove the f_state field from _PyInterpreterFrame

*  Make ownership of the frame explicit, replacing the is_generator field with an owner field.
2022-03-22 12:57:19 +00:00
Jeremy Kloth 88872a29f1
bpo-47084: Clear Unicode cached representations on finalization (GH-32032) 2022-03-22 13:53:51 +01:00
penguin_wwy 1ea055bd53
bpo-47067: Optimize calling GenericAlias objects (GH-31996)
Use vectorcall, and replace `PyObject_SetAttrString` with `PyObject_SetAttr` and a global string.
2022-03-21 16:33:02 -04:00
Brandt Bucher 2bde6827ea
bpo-46841: Quicken code in-place (GH-31888)
* Moves the bytecode to the end of the corresponding PyCodeObject, and quickens it in-place.

* Removes the almost-always-unused co_varnames, co_freevars, and co_cellvars member caches

* _PyOpcode_Deopt is a new mapping from all opcodes to their un-quickened forms.

* _PyOpcode_InlineCacheEntries is renamed to _PyOpcode_Caches

* _Py_IncrementCountAndMaybeQuicken is renamed to _PyCode_Warmup

* _Py_Quicken is renamed to _PyCode_Quicken

* _co_quickened is renamed to _co_code_adaptive (and is now a read-only memoryview).

* Do not emit unused nonzero opargs anymore in the compiler.
2022-03-21 11:11:17 +00:00
Victor Stinner 2217462bda
bpo-45786: Remove _PyFrame_Fini() and _PyFrame_DebugMallocStats() (GH-31874)
Remove private empty _PyFrame_Fini() and _PyFrame_DebugMallocStats()
functions.
2022-03-18 11:16:55 +01:00
Pieter Eendebak ac8308d3ea
bpo-47005: Improve performance of bytearray_repeat and bytearray_irepeat (GH-31856) 2022-03-17 19:10:36 -04:00
Mark Shannon 49e1e1e1bd
bpo-46841: Don't scan backwards in bytecode (GH-31901) 2022-03-15 17:08:37 -07:00
Kumar Aditya 6dfe09fc5f
bpo-46993: Speed up bytearray creation from list and tuple (GH-31834) 2022-03-15 15:27:30 +02:00
Irit Katriel e3d348a525
bpo-43721: Fix docstrings for property.getter/setter/deleter (GH-31046) 2022-03-14 23:28:54 +00:00
Oleg Iarygin a52f82baf2
bpo-46920: Remove disabled debug code added decades ago and likely unnecessary (GH-31812) 2022-03-14 17:03:21 +01:00
Oleg Iarygin e885ac3d5f
bpo-46920: Remove code that has no explainer why it was disabled (GH-31814) 2022-03-14 17:02:32 +01:00
Crowthebird 2153daf0a0
bpo-39829: Fix `__len__()` is called twice in list() constructor (GH-31816) 2022-03-14 10:23:59 +09:00
Matthew Rahtz af2277e461
bpo-43224: Implement PEP 646 changes to genericaliasobject.c (GH-31019)
Specifically, prepare for starring of tuples via a new genericalias iter type. GenericAlias also partially supports the iterator protocol after this change.

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
2022-03-12 20:20:12 +08:00
Victor Stinner 882d8096c2
bpo-46906: Add PyFloat_Pack8() to the C API (GH-31657)
Add new functions to pack and unpack C double (serialize and
deserialize):

* PyFloat_Pack2(), PyFloat_Pack4(), PyFloat_Pack8()
* PyFloat_Unpack2(), PyFloat_Unpack4(), PyFloat_Unpack8()

Document these functions and add unit tests.

Rename private functions and move them from the internal C API
to the public C API:

* _PyFloat_Pack2() => PyFloat_Pack2()
* _PyFloat_Pack4() => PyFloat_Pack4()
* _PyFloat_Pack8() => PyFloat_Pack8()
* _PyFloat_Unpack2() => PyFloat_Unpack2()
* _PyFloat_Unpack4() => PyFloat_Unpack4()
* _PyFloat_Unpack8() => PyFloat_Unpack8()

Replace the "unsigned char*" type with "char*" which is more common
and easy to use.
2022-03-12 00:10:02 +01:00
Kumar Aditya 304197b382
bpo-46944: use FASTCALL calling convention in generator.throw (GH-31723) 2022-03-11 14:37:14 +00:00
Jelle Zijlstra 54ab9ad312
bpo-46881: Fix refleak from GH-31616 (GH-31805) 2022-03-11 17:05:08 +08:00
Serhiy Storchaka b6a5d8590c
bpo-44796: Unify TypeVar and ParamSpec substitution (GH-31143)
Add methods __typing_subst__() in TypeVar and ParamSpec.
Simplify code by using more object-oriented approach, especially
the C code for types.GenericAlias and the Python code for
collections.abc.Callable.
2022-03-11 10:47:26 +02:00
Kumar Aditya 8714b6fa27
bpo-46881: Statically allocate and initialize the latin1 characters. (GH-31616) 2022-03-09 15:02:00 -08:00
Brandt Bucher f193631387
bpo-46841: Use inline caching for calls (GH-31709) 2022-03-07 11:45:00 -08:00
Pablo Galindo Salgado 3b3be05a16
bpo-46940: Don't override existing AttributeError suggestion information (GH-31710)
When an exception is created in a nested call to PyObject_GetAttr, any
external calls will override the context information of the
AttributeError that we have already placed in the most internal call.
This will cause the suggestions we create to nor work properly as the
attribute name and object that we will be using are the incorrect ones.

To avoid this, we need to check first if these attributes are already
set and bail out if that's the case.
2022-03-07 12:23:11 +00:00
Ken Jin 602024e6e1
bpo-46921: Vectorcall support for `super()` (GH-31687)
Co-Authored-By: Dong-hee Na <donghee.na@python.org>
2022-03-06 14:21:28 +08:00
Inada Naoki 2d8b764210
bpo-46864: Deprecate PyBytesObject.ob_shash. (GH-31598) 2022-03-06 11:39:10 +09:00
Serhiy Storchaka ab9301a28f
bpo-46927: Include the type's name in the error message for subscripting non-generic types (GH-31694) 2022-03-05 15:59:24 +02:00
Mark Shannon 03c2a36b2b
bpo-46903: Handle str-subclasses in virtual instance dictionaries. (GH-31658) 2022-03-04 11:31:29 +00:00
Mark Shannon 751c9ed801
bpo-46891: Fix creating a new instance of a module subclass with slots (GH-31643) 2022-03-03 10:38:27 +00:00
Inada Naoki 3241cba4ec
dict: Fix refleak (GH-31650) 2022-03-03 14:30:58 +09:00
Victor Stinner b6b711a1aa
bpo-46848: Move _PyBytes_Find() to internal C API (GH-31642)
Move _PyBytes_Find() and _PyBytes_ReverseFind() functions to the
internal C API.

bytesobject.c now includes pycore_bytesobject.h.
2022-03-02 14:15:26 +01:00
Inada Naoki 03642df1a1
dict: Internal cleanup (GH-31641)
* Make empty_key from split table to combined table.
* Use unicode_get_hash() when possible.
2022-03-02 19:05:12 +09:00
Dennis Sweeney 6ddb09f35b
bpo-46848: Use stringlib/fastsearch in mmap (GH-31625)
Speed up mmap.find(). Add _PyBytes_Find() and _PyBytes_ReverseFind().
2022-03-01 23:46:30 -05:00
Inada Naoki 9833bb91e4
bpo-46845: Reduce dict size when all keys are Unicode (GH-31564) 2022-03-02 08:09:28 +09:00
Mark Dickinson c60e6b6ad7
bpo-46311: Clean up PyLong_FromLong and PyLong_FromLongLong (GH-30496) 2022-03-01 14:20:52 +00:00
Eric Snow 08deed1af5
bpo-46753: Add the empty tuple to the _PyRuntimeState.global_objects. (gh-31345)
https://bugs.python.org/issue46753
2022-02-28 15:15:48 -07:00
Victor Stinner ad56919c5e
bpo-46857: Fix refleak in OSError INIT_ALIAS() (GH-31594)
_Py_GetRefTotal() no longer decrements _PySet_Dummy refcount.
2022-02-27 00:28:24 +01:00
Kumar Aditya 0d9b565e62
Propagate errors (however unlikely) from _Py_Deepfreeze_Init() (GH-31596) 2022-02-26 08:35:03 -08:00
Victor Stinner 5ab745fc51
bpo-46852: Remove the float.__set_format__() method (GH-31585)
Remove the undocumented private float.__set_format__() method,
previously known as float.__set_format__() in Python 3.7. Its
docstring said: "You probably don't want to use this function. It
exists mainly to be used in Python's test suite."
2022-02-26 00:53:27 +01:00
Victor Stinner 87af12bff3
bpo-46836: Rename InterpreterFrame to _PyInterpreterFrame (GH-31583)
Rename also struct _interpreter_frame to struct _PyInterpreterFrame.

Reduce risk of name conflicts if a project includes pycore_frame.h.
2022-02-25 16:22:00 +01:00
Victor Stinner 4a0c7a1aac
bpo-45316: Move private PyCode C API to internal C API (GH-31576)
Rename private functions (no exported), add an underscore prefix:

* PyLineTable_InitAddressRange() => _PyLineTable_InitAddressRange()
* PyLineTable_NextAddressRange() => _PyLineTable_NextAddressRange()
* PyLineTable_PreviousAddressRange() => _PyLineTable_PreviousAddressRange()

Move private functions to the internal C API:

* _PyCode_Addr2EndLine()
* _PyCode_Addr2EndOffset()
* _PyCode_Addr2Offset()
* _PyCode_InitAddressRange()
* _PyCode_InitEndAddressRange(
* _PyLineTable_InitAddressRange()
* _PyLineTable_NextAddressRange()
* _PyLineTable_PreviousAddressRange()

No longer export the following internal functions:

* _PyCode_GetVarnames()
* _PyCode_GetCellvars()
* _PyCode_GetFreevars()
* _Py_GetSpecializationStats()

Add "extern" to pycore_code.h functions to identify them more easiliy
(they are still not exported).
2022-02-25 15:41:32 +01:00
Victor Stinner 4657bf7016
bpo-1635741: Fix winreg reference leaks (GH-31560)
Clear also the PyHKEY_Type static type at exit.
2022-02-25 12:34:00 +01:00
Victor Stinner 7d03c8be5a
bpo-46852: Rename float.__set_format__() to float.__setformat__() (GH-31558)
Rename the private undocumented float.__set_format__() method to
float.__setformat__() to fix a typo introduced in Python 3.7. The
method is only used by test_float.

The change enables again test_float tests on the float format which
were previously skipped because of the typo.

The typo was introduced in Python 3.7 by bpo-20185
in commit b5c51d3dd9.
2022-02-25 03:05:25 +01:00
Victor Stinner 1b2611eb02
bpo-46656: Remove Py_NO_NAN macro (GH-31160)
Building Python now requires support for floating point Not-a-Number
(NaN): remove the Py_NO_NAN macro.
2022-02-25 01:32:57 +01:00
Victor Stinner 042f31da55
bpo-45459: C API uses type names rather than structure names (GH-31528)
Thanks to the new pytypedefs.h, it becomes to use type names like
PyObject rather like structure names like "struct _object".
2022-02-24 17:51:59 +01:00
Mark Shannon ae3adbeaed
Add (undocumented) _co_quickened attribute for code object. (GH-31552) 2022-02-24 12:10:53 +00:00
Victor Stinner 9bbdde2180
bpo-45412: Add _PY_SHORT_FLOAT_REPR macro (GH-31171)
Remove the HAVE_PY_SET_53BIT_PRECISION macro (moved to the internal
C API).

* Move HAVE_PY_SET_53BIT_PRECISION macro to pycore_pymath.h.
* Replace PY_NO_SHORT_FLOAT_REPR macro with _PY_SHORT_FLOAT_REPR
  macro which is always defined. gcc -Wundef emits a warning when
  using _PY_SHORT_FLOAT_REPR but the macro is not defined, if
  pycore_pymath.h include was forgotten.
2022-02-23 18:16:23 +01:00
Eric Snow 1f455361ec
bpo-46765: Replace Locally Cached Strings with Statically Initialized Objects (gh-31366)
https://bugs.python.org/issue46765
2022-02-22 17:23:51 -07:00
Irit Katriel 38b5acf867
bpo-46729: add number of sub-exceptions in str() of BaseExceptionGroup (GH-31294) 2022-02-22 18:28:58 +00:00
Inada Naoki 1e344684d8
dict: Add dk_log2_index_bytes (GH-31439) 2022-02-22 11:03:15 +00:00
Jelle Zijlstra 0a222db2bc
fix __bool__ docstring (GH-31301) 2022-02-21 07:42:27 -08:00
Inada Naoki 5543d9c559
dict: Use DK_LOG_SIZE in hot loop. (GH-31405)
DK_LOG_SIZE(key) < 8 is faster than DK_SIZE(key) <= 0xff, at least on GCC.
2022-02-19 13:15:20 +09:00
Christian Heimes 9e06d03672
bpo-46730: Fix refleak and tighten NULL checks (GH-31389)
``PyType_GetQualName`` returns a new reference.

Signed-off-by: Christian Heimes <christian@python.org>
2022-02-17 14:27:42 -05:00
Robert-André Mauchin 482a4b6c3f
bpo-46745: Fix typo in PositionsIterator (#31322)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-02-16 21:20:06 -08:00
Alex-Blade 0cb765b2ce
bpo-46730: Add more info to @property AttributeError messages (GH-31311)
On `obj.read_only_property = x`, raise `AttributeError: property 'read_only_property' of 'A' object has no setter`.
2022-02-16 02:07:34 -05:00
Eric Snow 12360aa159
bpo-46541: Discover the global strings. (gh-31346)
Instead of manually enumerating the global strings in generate_global_objects.py, we extrapolate the list from usage of _Py_ID() and _Py_STR() in the source files.

This is partly inspired by gh-31261.

https://bugs.python.org/issue46541
2022-02-14 17:36:51 -07:00
Raymond Hettinger 0ae4019179
Improve docstring for set.discard() (GH-31315) 2022-02-13 15:42:59 -06:00
Dennis Sweeney 4a66615ba7
bpo-46615: Don't crash when set operations mutate the sets (GH-31120)
Ensure strong references are acquired whenever using `set_next()`. Added randomized test cases for `__eq__` methods that sometimes mutate sets when called.
2022-02-11 11:25:08 -05:00
Petr Viktorin 204946986f
bpo-46613: Add PyType_GetModuleByDef to the public API (GH-31081)
* Make PyType_GetModuleByDef public (remove underscore)

Co-authored-by: Victor Stinner <vstinner@python.org>
2022-02-11 17:22:11 +01:00
Kumar Aditya c0a5ebeb12
bpo-46430: Intern strings in deep-frozen modules (GH-30683) 2022-02-09 08:52:42 -08:00
Eric Snow 81c72044a1
bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized global objects. (gh-30928)
We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code.  It is still used in a number of non-builtin stdlib modules.

The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime.  A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings).

https://bugs.python.org/issue46541#msg411799 explains the rationale for this change.

The core of the change is in:

* (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros
* Include/internal/pycore_runtime_init.h - added the static initializers for the global strings
* Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState
* Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers

I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings.  That check is added to the PR CI config.

The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()).  This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *.

The following are not changed (yet):

* stop using _Py_IDENTIFIER() in the stdlib modules
* (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API
* (maybe) intern the strings during runtime init

https://bugs.python.org/issue46541
2022-02-08 13:39:07 -07:00
Mark Shannon 25db2b361b
bpo-46675: Allow object value arrays and split key dictionaries larger than 16 (GH-31191) 2022-02-08 11:50:38 +00:00
Victor Stinner 760349198d
bpo-46670: Remove unused macros in the Objects directory (GH-31193) 2022-02-07 16:21:41 +01:00
Victor Stinner b556f53785
bpo-46670: Test if a macro is defined, not its value (GH-31178)
* audioop.c: #ifdef WORDS_BIGENDIAN
* ctypes.h: #ifdef USING_MALLOC_CLOSURE_DOT_C
* _ctypes/malloc_closure.c: #ifdef HAVE_FFI_CLOSURE_ALLOC
  and #ifdef USING_APPLE_OS_LIBFFI
* pytime.c: #ifdef __APPLE__
* unicodeobject.c: #ifdef HAVE_NON_UNICODE_WCHAR_T_REPRESENTATION
2022-02-07 01:46:51 +01:00
Victor Stinner 097f74a5a3
bpo-46670: Define all macros for stringlib (GH-31176)
bytesobject.c, bytearrayobject.c and unicodeobject.c now define all
macros used by stringlib, to avoid using undefined macros.
Fix "gcc -Wundef" warnings.
2022-02-07 01:26:58 +01:00
Victor Stinner 16f96a4cf9
bpo-46670: Remove unused get_frame_state() function (GH-31177)
The PyFrame_MAXFREELIST macro is no longer defined.
2022-02-07 01:25:44 +01:00
Mark Shannon da4d4ec185
Pass reference to func, as well as args, when pushing frame. (GH-31100) 2022-02-03 18:36:28 +00:00
Zackery Spytz b4bd1e1422
bpo-44977: Deprecate delegation of int to __trunc__ (GH-31031)
Calling int(a) when type(a) implements __trunc__ but not __int__
or __index__ now raises a DeprecationWarning.
2022-02-03 11:43:25 +02:00
Petr Viktorin 0ef0853012
bpo-46433: _PyType_GetModuleByDef: handle static types in MRO (GH-30696)
Automerge-Triggered-By: GH:encukou
2022-02-02 07:57:51 -08:00
Mark Shannon 187930f74c
bpo-46072: Add some frame stats. (GH-31060) 2022-02-02 11:01:33 +00:00
Mark Shannon 48be46ec1f
bpo-46072: Add some object layout and allocation stats (GH-31051) 2022-02-01 15:05:18 +00:00
Kumar Aditya b9ebde8db7
bpo-46564: do not create frame object for super object (GH-31002) 2022-02-01 21:40:46 +08:00
Ken Jin 768569325a
bpo-46407: Fix long_mod refleak (GH-31025) 2022-01-31 11:41:14 +01:00
Victor Stinner 9a24127113
bpo-46417: _PyStructSequence_FiniType() updates _Py_RefTotal (GH-30988) 2022-01-28 14:08:32 +01:00
Crowthebird f10dafc430
bpo-46407: Optimizing some modulo operations (GH-30653)
Added new internal functions to compute mod without also computing the quotient.

The loops can be leaner then, which leads to modestly but reliably faster execution in contexts that know they don't need the quotient.

Code by Jeremiah Vivian (Pascual).
2022-01-27 18:46:45 -06:00
Victor Stinner 18ea973c21
bpo-40170: Remove PyHeapType_GET_MEMBERS() macro (GH-30942)
Remove the PyHeapType_GET_MEMBERS() macro. It was exposed in the
public C API by mistake, it must only be used by Python internally.
Use the PyTypeObject.tp_members member instead.

Rename PyHeapType_GET_MEMBERS() to _PyHeapType_GET_MEMBERS() and move
it to the internal C API.
2022-01-28 00:39:52 +01:00
Victor Stinner 0575551f69
bpo-40170: Move _Py_GetAllocatedBlocks() to pycore_pymem.h (GH-30943)
Move _Py_GetAllocatedBlocks() and _PyObject_DebugMallocStats()
declarations to pycore_pymem.h. These functions are related to memory
allocators, not to the PyObject structure.
2022-01-27 21:23:22 +01:00
Christian Heimes 26b0482393
bpo-46476: Simplify and fix _PyStaticCode_Dealloc (GH-30965) 2022-01-27 19:32:12 +00:00
Kumar Aditya c7f810b34d
bpo-46476: Fix memory leak in code objects generated by deepfreeze (GH-30853)
Add _Py_Deepfreeze_Fini() and _PyStaticCode_Dealloc() functions.
2022-01-27 14:03:47 +01:00
Victor Stinner 6b491b9dc0
bpo-40170: Remove _Py_GetAllocatedBlocks() function (GH-30940)
Move _Py_GetAllocatedBlocks() and _PyObject_DebugMallocStats()
private functions to the internal C API.
2022-01-27 03:35:51 +01:00
Victor Stinner af32b3ef1f
bpo-40170: PyType_SUPPORTS_WEAKREFS() becomes a regular function (GH-30938)
Convert the PyType_SUPPORTS_WEAKREFS() macro to a regular function.
It no longer access the PyTypeObject.tp_weaklistoffset member
directly.

Add _PyType_SUPPORTS_WEAKREFS() static inline functions, used
internally by Python for best performance.
2022-01-27 03:00:55 +01:00
Jelle Zijlstra ac0c6e128c
bpo-46527: allow calling enumerate(iterable=...) again (GH-30904) 2022-01-27 00:46:48 +09:00
Tim Peters 7c26472d09
bpo-46504: faster code for trial quotient in x_divrem() (GH-30856)
* bpo-46504: faster code for trial quotient in x_divrem()

This brings x_divrem() back into synch with x_divrem1(), which was changed
in bpo-46406 to generate faster code to find machine-word division
quotients and remainders. Modern processors compute both with a single
machine instruction, but convincing C to exploit that requires writing
_less_ "clever" C code.
2022-01-24 19:06:00 -06:00
Irit Katriel 573b545157
bpo-46431: improve error message on invalid calls to BaseExceptionGroup.__new__ (GH-30854) 2022-01-24 21:47:40 +00:00
Mark Shannon 0367a36fdc
bpo-43683: Streamline YIELD_VALUE and SEND (GH-30723)
* Split YIELD_VALUE into ASYNC_GEN_WRAP; YIELD_VALUE for async generators.

* Split SEND into SEND; YIELD_VALUE.

* Document new opcodes.
2022-01-24 11:08:53 +00:00
Dong-hee Na 76dc047a0e
bpo-46481: Implement vectorcall for weakref.ref.__call__ method. (GH-30820) 2022-01-24 00:39:45 +09:00
Gregory P. Smith c7f20f1cc8
bpo-46406: Faster single digit int division. (#30626)
* bpo-46406: Faster single digit int division.

This expresses the algorithm in a more basic manner resulting in better
instruction generation by todays compilers.

See https://mail.python.org/archives/list/python-dev@python.org/thread/ZICIMX5VFCX4IOFH5NUPVHCUJCQ4Q7QM/#NEUNFZU3TQU4CPTYZNF3WCN7DOJBBTK5
2022-01-23 10:00:41 +00:00
Victor Stinner 976dec9b3b
bpo-46417: _PyList_Fini() clears indexerr (GH-30815)
_PyList_Fini() now clears the 'indexerr' error message.
2022-01-23 02:20:44 +01:00
Victor Stinner a1444f4358
bpo-46417: Fix _PyStaticType_Dealloc() (GH-30810)
_PyStaticType_Dealloc() now only calls PyObject_ClearWeakRefs()
if the call is not going to fail.
2022-01-23 00:32:05 +01:00
Victor Stinner 1626bf4ac7
bpo-46417: Clear Unicode static types at exit (GH-30806)
Add _PyUnicode_FiniTypes() function, called by
finalize_interp_types(). It clears these static types:

* EncodingMapType
* PyFieldNameIter_Type
* PyFormatterIter_Type

_PyStaticType_Dealloc() now does nothing if tp_subclasses
is not NULL.
2022-01-22 22:55:39 +01:00
Victor Stinner 621a45ccac
bpo-46417: Py_Finalize() clears static exceptioins (GH-30805)
The Py_Finalize() function now clears exceptions implemented as
static types.

Add _PyExc_FiniTypes() function, called by _PyExc_Fini().
2022-01-22 22:31:44 +01:00
Victor Stinner f1bcdeaca6
bpo-46417: Factorize _PyExc_InitTypes() code (GH-30804)
Add 'static_exceptions' list to factorize code between
_PyExc_InitTypes() and _PyBuiltins_AddExceptions().

_PyExc_InitTypes() does nothing if it's not the main interpreter.

Sort exceptions in Lib/test/exception_hierarchy.txt.
2022-01-22 21:48:56 +01:00
Victor Stinner 6cacdb4245
bpo-46417: _PyTypes_FiniTypes() clears object and type (GH-30798) 2022-01-22 19:31:24 +01:00
Victor Stinner 3a4c15bb98
bpo-46417: Cleanup typeobject.c code (GH-30795)
* Add comment to recurse_down_subclasses() explaining why it's safe
  to use a borrowed reference to tp_subclasses.
* remove_all_subclasses() no longer accept NULL cases
* type_set_bases() now relies on the fact that new_bases is not NULL.
* type_dealloc_common() avoids PyErr_Fetch/PyErr_Restore if tp_bases
  is NULL.
* remove_all_subclasses() makes sure that no exception is raised.
* Don't test at runtime if tp_mro only contains types: rely on
  _PyType_CAST() assertion for that.
* _PyStaticType_Dealloc() no longer clears tp_subclasses which is
  already NULL.
* mro_hierarchy() avoids calling _PyType_GetSubclasses() if
  tp_subclasses is NULL.

Coding style:

* Use Py_NewRef().
* Add braces and move variable declarations to the first variable
  assignement.
* Rename a few variables and parameters to use better names.
2022-01-22 18:56:11 +01:00
Victor Stinner 500c146387
bpo-46417: Clear more static types (GH-30796)
* Move PyContext static types into object.c static_types list.
* Rename PyContextTokenMissing_Type to _PyContextTokenMissing_Type
  and declare it in pycore_context.h.
* _PyHamtItems types are no long exported: replace PyAPI_DATA() with
  extern.
2022-01-22 18:55:48 +01:00
Victor Stinner 2d03b73cc9
bpo-46417: remove_subclass() clears tp_subclasses (GH-30793)
The remove_subclass() function now deletes the dictionary when
removing the last subclass (if the dictionary becomes empty) to save
memory: set PyTypeObject.tp_subclasses to NULL. remove_subclass() is
called when a type is deallocated.

_PyType_GetSubclasses() no longer holds a reference to tp_subclasses:
its loop cannot modify tp_subclasses.
2022-01-22 16:53:30 +01:00
Victor Stinner f1c6ae3270
bpo-46417: Fix race condition on setting type __bases__ (GH-30788)
Fix a race condition on setting a type __bases__ attribute: the
internal function add_subclass() now gets the
PyTypeObject.tp_subclasses member after calling PyWeakref_NewRef()
which can trigger a garbage collection which can indirectly modify
PyTypeObject.tp_subclasses.
2022-01-22 15:08:42 +01:00
Victor Stinner ac1f152421
bpo-46417: Use _PyType_CAST() in Objects directory (GH-30764) 2022-01-21 23:33:43 +01:00
Victor Stinner 8ee07dda13
bpo-46417: Add _PyType_GetSubclasses() function (GH-30761)
Add a new _PyType_GetSubclasses() function to get type's subclasses.

_PyType_GetSubclasses(type) returns a list which holds strong
refererences to subclasses. It is safer than iterating on
type->tp_subclasses which yields weak references and can be modified
in the loop.

_PyType_GetSubclasses(type) now holds a reference to the tp_subclasses
dict while creating the list of subclasses.

set_collection_flag_recursive() of _abc.c now uses
_PyType_GetSubclasses().
2022-01-21 23:29:10 +01:00
Victor Stinner bc67f189fd
bpo-46417: Add _PyType_CAST() macro (GH-30760)
In debug mode, the macro makes sure that its argument is a type using
an assertion.
2022-01-21 21:39:01 +01:00
Victor Stinner a1bf329bca
bpo-46417: Add missing types of _PyTypes_InitTypes() (GH-30749)
Add types removed by mistake by the commit adding
_PyTypes_FiniTypes().

Move also PyBool_Type at the end, since it depends on PyLong_Type.

PyBytes_Type and PyUnicode_Type no longer depend explicitly on
PyBaseObject_Type: it's the default of PyType_Ready().
2022-01-21 17:53:13 +01:00
Victor Stinner fda8886498
bpo-46417: Revert remove_subclass() change (GH-30750)
remove_subclass() doesn't clear the tp_subclasses dict if the dict
becomes empty.
2022-01-21 16:45:14 +01:00
Victor Stinner 595225e86d
bpo-46417: Py_Finalize() clears static types (GH-30743)
Add _PyTypes_FiniTypes() best-effort function to clear static types:
don't deallocate a type if it still has subclasses.

remove_subclass() now sets tp_subclasses to NULL when removing the
last subclass.
2022-01-21 13:06:34 +01:00
Victor Stinner 1781d55eb3
bpo-46417: _curses uses PyStructSequence_NewType() (GH-30736)
The _curses module now creates its ncurses_version type as a heap
type using PyStructSequence_NewType(), rather than using a static
type.

* Move _PyStructSequence_FiniType() definition to pycore_structseq.h.
* test.pythoninfo: log curses.ncurses_version.
2022-01-21 03:30:20 +01:00
Victor Stinner e9e3eab0b8
bpo-46417: Finalize structseq types at exit (GH-30645)
Add _PyStructSequence_FiniType() and _PyStaticType_Dealloc()
functions to finalize a structseq static type in Py_Finalize().
Currrently, these functions do nothing if Python is built in release
mode.

Clear static types:

* AsyncGenHooksType: sys.set_asyncgen_hooks()
* FlagsType: sys.flags
* FloatInfoType: sys.float_info
* Hash_InfoType: sys.hash_info
* Int_InfoType: sys.int_info
* ThreadInfoType: sys.thread_info
* UnraisableHookArgsType: sys.unraisablehook
* VersionInfoType: sys.version
* WindowsVersionType: sys.getwindowsversion()
2022-01-21 01:42:25 +01:00
Mark Shannon b04dfbbe4b
bpo-46409: Make generators in bytecode (GH-30633)
* Add RETURN_GENERATOR and JUMP_NO_INTERRUPT opcodes.

* Trim frame and generator by word each.

* Minor refactor of frame.c

* Update test.test_sys to account for smaller frames.

* Treat generator functions as normal functions when evaluating and specializing.
2022-01-20 11:46:39 +00:00
Joshua Bronson 2d10fa9bc4
docs: correct outdated MappingProxyType docstrings (#30281)
The docstrings for MappingProxyType's keys(), values(), and items()
methods were never updated to reflect the changes that Python 3 brought
to these APIs, namely returning views rather than lists.
2022-01-19 13:20:43 -08:00
Raymond Hettinger 243c31667c
bpo-42161: Hoist the _PyLong_GetOne() call out of the inner loop. (GH-30656) 2022-01-18 00:02:35 -08:00
Brandt Bucher 5cd9a162cd
bpo-46361: Fix "small" `int` caching (GH-30583) 2022-01-16 16:06:37 +00:00
Tim Peters fc05e6bfce
bpo-46020: Optimize long_pow for the common case (GH-30555)
This cuts a bit of overhead by not initializing the table of small
odd powers unless it's needed for a large exponent.
2022-01-12 12:55:02 -06:00
Eric Snow cf496d657a
bpo-45953: Statically allocate and initialize global bytes objects. (gh-30096)
The empty bytes object (b'') and the 256 one-character bytes objects were allocated at runtime init.  Now we statically allocate and initialize them.

https://bugs.python.org/issue45953
2022-01-11 09:37:24 -07:00
Dennis Sweeney ad1d5908ad
bpo-46235: Do all ref-counting at once during list/tuple multiplication (GH-30346)
When multiplying lists and tuples by `n`, increment each element's refcount, by `n`, just once.

Saves `n-1` increments per element, and allows for a leaner & faster copying loop.

Code by  sweeneyde (Dennis Sweeney).
2022-01-07 21:47:58 -06:00
Victor Stinner 35d6540c90
bpo-46006: Revert "bpo-40521: Per-interpreter interned strings (GH-20085)" (GH-30422)
This reverts commit ea251806b8.

Keep "assert(interned == NULL);" in _PyUnicode_Fini(), but only for
the main interpreter.

Keep _PyUnicode_ClearInterned() changes avoiding the creation of a
temporary Python list object.
2022-01-06 08:53:44 +01:00
Inada Naoki 46e4c257e7
bpo-46236: Fix PyFunction_GetAnnotations() returned tuple. (GH-30409)
Automerge-Triggered-By: GH:pablogsal
2022-01-05 04:25:54 -08:00
Brandt Bucher 31e43cbe5f
bpo-46009: Remove GEN_START (GH-30367) 2022-01-04 11:38:32 -08:00
Irit Katriel a94461d718
bpo-46202: Remove opcode POP_EXCEPT_AND_RERAISE (GH-30302)
* bpo-46202: remove opcode POP_EXCEPT_AND_RERAISE

* do not assume that an exception group is truthy
2022-01-04 10:37:12 +00:00
Tim Peters 3aa5242b54
bpo-46233: Minor speedup for bigint squaring (GH-30345)
x_mul()'s squaring code can do some redundant and/or useless
work at the end of each digit pass. A more careful analysis
of worst-case carries at various digit positions allows
making that code leaner.
2022-01-03 20:41:16 -06:00
Irit Katriel 65e7c1f90e
bpo-46219, 46221: simplify except* implementation following exc_info changes. Move helpers to exceptions.c. Do not assume that exception groups are truthy. (GH-30289) 2022-01-02 23:22:42 +00:00
Tim Peters 863729e9c6
bpo-46218: Change long_pow() to sliding window algorithm (GH-30319)
* bpo-46218: Change long_pow() to sliding window algorithm

The primary motivation is to eliminate long_pow's reliance on that the number of bits in a long "digit" is a multiple of 5. Now it no longer cares how many bits are in a digit.

But the sliding window approach also allows cutting the precomputed table of small powers in half, which reduces initialization overhead enough that the approach pays off for smaller exponents too. Depending on exponent bit patterns, a sliding window may also be able to save some bigint multiplies (sometimes when at least 5 consecutive exponent bits are 0, regardless of their starting bit position modulo 5).

Note: boosting the window width to 6 didn't work well overall. It give marginal speed improvements for huge exponents, but the increased overhead (the small-power table needs twice as many entries) made it a loss for smaller exponents.

Co-authored-by: Oleg Iarygin <dralife@yandex.ru>
2022-01-02 13:18:20 -06:00
Dong-hee Na fb44d05896
bpo-46085: Fix iterator cache mechanism of OrderedDict. (GH-30290) 2021-12-30 12:35:45 +09:00
Xinhang Xu 3581c7abbe
bpo-46055: Speed up binary shifting operators (GH-30044)
Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
2021-12-27 18:36:55 +00:00
Mark Dickinson 360fedc2d2
bpo-46055: Streamline inner loop for right shifts (#30243) 2021-12-27 18:04:36 +00:00
Serhiy Storchaka ad4857884b
bpo-43413: Revert changes in set.__init__ (GH-28403)
Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
2021-12-26 13:27:01 +02:00
David Hewitt 31ff96712e
bpo-46140: take more Py_buffer arguments as const * (GH-30217) 2021-12-22 15:07:46 +02:00
Irit Katriel c66fc0fb53
bpo-46107: ExceptionGroup.subgroup()/split() should copy __note__ to the parts (GH-30159) 2021-12-21 10:12:26 +00:00
Raymond Hettinger 29ea68bd1d
Revert "bpo-46131: add fastpath for PyFloat_Check() (GH-30200)" (GH-30208)
This reverts commit 2ef06d4125.
2021-12-19 15:05:46 -06:00
Matti Picus 2ef06d4125
bpo-46131: add fastpath for PyFloat_Check() (#30200) 2021-12-19 14:24:30 -06:00
Irit Katriel 396b58345f
bpo-45711: Remove type and traceback from exc_info (GH-30122)
* Do not PUSH/POP traceback or type to the stack as part of exc_info

* Remove exc_traceback and exc_type from _PyErr_StackItem

* Add to what's new, because this change breaks things like Cython
2021-12-17 14:46:22 +00:00
Mark Shannon 0b50a4f0cd
bpo-46039: Split yield from in two (GH-30035)
* Split YIELD_FROM opcode into SEND and JUMP_ABSOLUTE.

* Remove YIELD_FROM opcode.
2021-12-15 10:30:09 +00:00
Mark Shannon 9f8f45144b
bpo-44525: Split calls into PRECALL and CALL (GH-30011)
* Add 3 new opcodes for calls: PRECALL_METHOD, CALL_NO_KW, CALL_KW.

* Update specialization to handle new CALL opcodes.

* Specialize call to method descriptors.

* Remove old CALL opcodes: CALL_FUNCTION, CALL_METHOD, CALL_METHOD_KW, CALL_FUNCTION_KW.
2021-12-14 18:22:44 +00:00
Irit Katriel d60457a667
bpo-45292: [PEP-654] add except* (GH-29581) 2021-12-14 16:48:15 +00:00
Eric Snow 121f1f893a
bpo-45953: Statically initialize the small ints. (gh-30092)
The array of small PyLong objects has been statically declared. Here I also statically initialize them. Consequently they are no longer initialized dynamically during runtime init.

I've also moved them under a new sub-struct in _PyRuntimeState, in preparation for static allocation and initialization of other global objects.

https://bugs.python.org/issue45953
2021-12-13 18:04:05 -07:00
Georg Brandl f4095e53ab
bpo-45855: document that `no_block` has no use anymore in PyCapsule_Import (#29665) 2021-12-12 10:49:50 +01:00
Kumar Aditya 41026c3155
bpo-45855: Replaced deprecated `PyImport_ImportModuleNoBlock` with PyImport_ImportModule (GH-30046) 2021-12-12 10:45:20 +02:00
Eric Snow c8749b5783
bpo-46008: Make runtime-global object/type lifecycle functions and state consistent. (gh-29998)
This change is strictly renames and moving code around.  It helps in the following ways:

* ensures type-related init functions focus strictly on one of the three aspects (state, objects, types)
* passes in PyInterpreterState * to all those functions, simplifying work on moving types/objects/state to the interpreter
* consistent naming conventions help make what's going on more clear
* keeping API related to a type in the corresponding header file makes it more obvious where to look for it

https://bugs.python.org/issue46008
2021-12-09 12:59:26 -07:00
Andrew Svetlov d4363d2140
bpo-45813: Drop redundant assertion from frame.clear() (GH-29990)
* bpo-45813: Drop redundant assertion from frame.clear()

* Move assertion to frame_dealloc()
2021-12-08 16:05:00 +00:00
Mark Shannon 69806b9516
bpo-46009: Do not exhaust generator when send() method raises (GH-29986) 2021-12-08 12:09:26 +00:00
Mark Shannon 8319114fee
bpo-45947: Place dict and values pointer at fixed (negative) offset just before GC header. (GH-29879)
* Place __dict__ immediately before GC header for plain Python objects.

* Fix up lazy dict creation logic to use managed dict pointers.

* Manage values pointer, placing them directly before managed dict pointers.

* Convert hint-based load/store attr specialization target managed dict classes.

* Specialize LOAD_METHOD for managed dict objects.

* Remove unsafe _PyObject_GC_Calloc function.

* Remove unsafe _PyObject_GC_Malloc() function.

* Add comment explaning use of Py_TPFLAGS_MANAGED_DICT.
2021-12-07 16:02:53 +00:00
Mark Shannon 299483c95d
bpo-45963: Make space for the InterpreterFrame of a generator in that generator. (GH-29891)
* Make generator, coroutine and async gen structs all the same size.

* Store interpreter frame in generator (and coroutine). Reduces the number of allocations neeeded for a generator from two to one.
2021-12-06 10:13:49 +00:00
Irit Katriel 5bb7ef2768
bpo-45607: Make it possible to enrich exception displays via setting their __note__ field (GH-29880) 2021-12-03 22:01:15 +00:00
Dennis Sweeney 03768c4d13
bpo-45885: Specialize COMPARE_OP (GH-29734)
* Add COMPARE_OP_ADAPTIVE adaptive instruction.

* Add COMPARE_OP_FLOAT_JUMP, COMPARE_OP_INT_JUMP and COMPARE_OP_STR_JUMP specialized instructions.

* Introduce and use _PyUnicode_Equal
2021-12-03 11:29:12 +00:00
Victor Stinner a6c3b0faa1
bpo-45954: Rename PyConfig.no_debug_ranges to code_debug_ranges (GH-29886)
Rename PyConfig.no_debug_ranges to PyConfig.code_debug_ranges and
invert the value.

Document -X no_debug_ranges and PYTHONNODEBUGRANGES env var in
PyConfig.code_debug_ranges documentation.
2021-12-02 11:43:37 +01:00
Mark Shannon 60929576e4
bpo-45786: Allocate space for frame in frame object. (GH-29729) 2021-11-29 12:34:59 +00:00
Steve Dower db55f3faba
bpo-44530: Reverts a change to the 'code.__new__' audit event (GH-29809) 2021-11-27 00:26:45 +00:00
Dong-hee Na db671b010c
bpo-45760: Remove PyNumber_InMatrixMultiply (GH-29751) 2021-11-26 23:10:10 +09:00
Mark Shannon 135cabd328
bpo-44525: Copy free variables in bytecode to allow calls to inner functions to be specialized (GH-29595)
* Make internal APIs that take PyFrameConstructor take a PyFunctionObject instead.

* Add reference to function to frame, borrow references to builtins and globals.

* Add COPY_FREE_VARS instruction to allow specialization of calls to inner functions.
2021-11-23 09:53:24 +00:00
Mark Shannon 7fd92a8b7e
bpo-45813: Make sure that frame->generator is NULLed when generator is deallocated. (GH-29700) 2021-11-22 14:01:23 +00:00
Dennis Sweeney 036fead695
bpo-45609: Specialize STORE_SUBSCR (GH-29242)
* Specialize STORE_SUBSCR for list[int], and dict[object]

* Adds _PyDict_SetItem_Take2 which consumes references to the key and values.
2021-11-19 10:30:37 +00:00
Dong-hee Na 345ba3f080
bpo-45510: Specialize BINARY_SUBTRACT (GH-29523) 2021-11-18 09:19:58 +00:00
Brandt Bucher 6a84d61c55
bpo-45636: Simplify BINARY_OP (GH-29565) 2021-11-16 05:53:57 -08:00
Christian Heimes cbab997efb
bpo-45723: Prepare support for autoconf 2.71 (GH-29441) 2021-11-08 19:31:14 +01:00
Mark Dickinson 9127520729
bpo-45392: Update the docstring of the 'type' built-in (GH-29439) 2021-11-06 20:08:44 +01:00
Itamar Ostricher 2c045bd567
bpo-45697: Use PyObject_TypeCheck in type_call (GH-29392) 2021-11-04 12:39:50 +02:00
Mark Shannon acc89db923
bpo-45691: Make array of small ints static to fix use-after-free error. (GH-29366) 2021-11-03 16:22:32 +00:00
Mark Shannon 4fc68560ea
Store actual ints, not pointers to them in the interpreter state. (GH-29274) 2021-10-28 17:35:43 +01:00
Mark Shannon 7f61d9d848
bpo-45256: Rationalize code around Python-to-Python calls a bit. (GH-29235) 2021-10-28 16:14:59 +01:00
Mark Shannon f291404a80
bpo-45637: Store the frame pointer in the cframe (GH-29267)
* Rename 'frame' to 'current_frame'
2021-10-28 13:59:11 +01:00
Tim Peters 51ed2c56a1
bpo-45530: speed listobject.c's unsafe_tuple_compare() (GH-29076)
Keep track of whether unsafe_tuple_compare() calls are resolved by the very
first tuple elements, and adjust strategy accordingly. This can significantly
cut the number of calls made to the full-blown PyObject_RichCompareBool(),
and especially when duplicates are rare.

Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-10-24 22:27:24 -05:00
Irit Katriel f30ad65dbf
bpo-45292: [PEP 654] add the ExceptionGroup and BaseExceptionGroup classes (GH-28569) 2021-10-23 00:13:46 +01:00
Dennis Sweeney 423fa1c181
bpo-30570: Use Py_EnterRecursiveCall() in issubclass() (GH-29048)
* Use Py_EnterRecursiveCall() in issubclass()

Reviewed-by: Gregory P. Smith <greg@krypto.org> [Google]
2021-10-22 14:24:08 -07:00
Dong-hee Na 83f202a802
bpo-43706: Use PEP 590 vectorcall to speed up enumerate() (GH-25154) 2021-10-22 08:20:43 +09:00
Neil Schemenauer 0224b7180b
bpo-45526: obmalloc radix use 64 addr bits (GH-29062)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2021-10-21 23:05:46 +02:00
Neil Schemenauer 311910b31a
bpo-45521: Fix a bug in the obmalloc radix tree code. (GH-29051)
MAP_BOT_LENGTH was incorrectly used to compute MAP_TOP_MASK instead of
MAP_TOP_LENGTH. On 64-bit machines, the error causes the tree to hold
46-bits of virtual addresses, rather than the intended 48-bits.
2021-10-21 06:38:36 -07:00
Christian Heimes 9942f42a93
bpo-45522: Allow to disable freelists on build time (GH-29056)
Freelists for object structs can now be disabled. A new ``configure``
option ``--without-freelists`` can be used to disable all freelists
except empty tuple singleton. Internal Py*_MAXFREELIST macros can now
be defined as 0 without causing compiler warnings and segfaults.

Signed-off-by: Christian Heimes <christian@python.org>
2021-10-21 06:12:20 -07:00
Petr Viktorin 8a310dd5f4
bpo-45315: PyType_FromSpec: Copy spec->name and have the type own the memory for its name (GH-29103) 2021-10-21 11:46:20 +02:00
Victor Stinner 5f09bb021a
bpo-35134: Add Include/cpython/longobject.h (GH-29044)
Move Include/longobject.h non-limited API to a new
Include/cpython/longobject.h header file.

Move the following definitions to the internal C API:

* _PyLong_DigitValue
* _PyLong_FormatAdvancedWriter()
* _PyLong_FormatWriter()
2021-10-19 02:04:52 +02:00
Mark Shannon 70945d57e7
bpo-45256: Avoid C calls for most Python to Python calls. (GH-28937)
* Avoid making C calls for most calls to Python functions.

* Change initialize_locals(steal=true) and _PyTuple_FromArraySteal to consume the argument references regardless of whether they succeed or fail.
2021-10-18 09:57:24 +01:00
Victor Stinner 354c35220d
bpo-45482: Rename namespaceobject.h to pycore_namespace.h (GH-28975)
Rename Include/namespaceobject.h to
Include/internal/pycore_namespace.h.

The _testmultiphase extension is now built with the
Py_BUILD_CORE_MODULE macro defined to access _PyNamespace_Type.

object.c: remove unused "pycore_context.h" include.
2021-10-15 15:21:21 +02:00
Victor Stinner 063abd931f
bpo-35081: Move interpreteridobject.h to Include/internal/ (GH-28969)
Move the interpreteridobject.h header file from Include/ to
Include/internal/. It only provides private functions.
2021-10-15 11:56:34 +02:00
Victor Stinner 8e5de40f90
bpo-35134: Move classobject.h to Include/cpython/ (GH-28968)
Move classobject.h, context.h, genobject.h and longintrepr.h header
files from Include/ to Include/cpython/.

Remove redundant "#ifndef Py_LIMITED_API" in context.h.

Remove explicit #include "longintrepr.h" in C files. It's not needed,
Python.h already includes it.
2021-10-15 09:46:29 +02:00
Benjamin Peterson 9ce9cfe595
bpo-45479: Futher simplify Py_UniversalNewlineFgets. (GH-28967)
Thank you to Eryk Sun for the suggestions in https://github.com/python/cpython/pull/28965#discussion_r729527143.
2021-10-14 23:10:52 -07:00
Benjamin Peterson 160c38df7f
closes bpo-45479: Degunkify Py_UniversalNewlineFgets. (GH-28965)
Remove dead variables and control flow.
2021-10-14 21:28:52 -07:00
Victor Stinner 0a883a76cd
bpo-35134: Add Include/cpython/floatobject.h (GH-28957)
Split Include/floatobject.h into sub-files: add
Include/cpython/floatobject.h and
Include/internal/pycore_floatobject.h.
2021-10-14 23:41:06 +02:00
Victor Stinner 3cc56c828d
bpo-45439: Move _PyObject_VectorcallTstate() to pycore_call.h (GH-28893)
* Move _PyObject_VectorcallTstate() and _PyObject_FastCallTstate() to
  pycore_call.h (internal C API).
* Convert PyObject_CallOneArg(), PyObject_Vectorcall(),
  _PyObject_FastCall() and PyVectorcall_Function() static inline
  functions to regular functions.
* Add _PyVectorcall_FunctionInline() static inline function.
* PyObject_Vectorcall(), _PyObject_FastCall(), and
  PyObject_CallOneArg() now call _PyThreadState_GET() rather
  than PyThreadState_Get().
2021-10-14 21:53:04 +02:00
Serhiy Storchaka 39aa98346d
bpo-45467: Fix IncrementalDecoder and StreamReader in the "raw-unicode-escape" codec (GH-28944)
They support now splitting escape sequences between input chunks.

Add the third parameter "final" in codecs.raw_unicode_escape_decode().
It is True by default to match the former behavior.
2021-10-14 20:04:19 +03:00
Dennis Sweeney 3b3d30e8f7
bpo-45367: Specialize BINARY_MULTIPLY (GH-28727) 2021-10-14 15:56:33 +01:00
Serhiy Storchaka c96d1546b1
bpo-45461: Fix IncrementalDecoder and StreamReader in the "unicode-escape" codec (GH-28939)
They support now splitting escape sequences between input chunks.

Add the third parameter "final" in codecs.unicode_escape_decode().
It is True by default to match the former behavior.
2021-10-14 13:17:00 +03:00
Victor Stinner aac29af678
bpo-45434: pyport.h no longer includes <stdlib.h> (GH-28914)
Include <stdlib.h> explicitly in C files.

Python.h includes <wchar.h>.
2021-10-13 19:25:53 +02:00
Victor Stinner bbe7497c5a
bpo-45434: Remove pystrhex.h header file (GH-28923)
Move Include/pystrhex.h to Include/internal/pycore_strhex.h.
The header file only contains private functions.

The following C extensions are now built with Py_BUILD_CORE_MODULE
macro defined to get access to the internal C API:

* _blake2
* _hashopenssl
* _md5
* _sha1
* _sha3
* _ssl
* binascii
2021-10-13 15:22:35 +02:00
Mark Shannon a8b9350964
bpo-45340: Don't create object dictionaries unless actually needed (GH-28802)
* Never change types' cached keys. It could invalidate inline attribute objects.

* Lazily create object dictionaries.

* Update specialization of LOAD/STORE_ATTR.

* Don't update shared keys version for deletion of value.

* Update gdb support to handle instance values.

* Rename SPLIT_KEYS opcodes to INSTANCE_VALUE.
2021-10-13 14:19:34 +01:00
Victor Stinner 7cdc2a0f4b
pycore_pystate.h no longer redefines PyThreadState_GET() (GH-28921)
Redefining the PyThreadState_GET() macro in pycore_pystate.h is
useless since it doesn't affect files not including it. Either use
_PyThreadState_GET() directly, or don't use pycore_pystate.h internal
C API. For example, the _testcapi extension don't use the internal C
API, but use the public PyThreadState_Get() function instead.

Replace PyThreadState_Get() with _PyThreadState_GET(). The
_PyThreadState_GET() macro is more efficient than PyThreadState_Get()
and PyThreadState_GET() function calls which call fail with a fatal
Python error.

posixmodule.c and _ctypes extension now include <windows.h> before
pycore header files (like pycore_call.h).

_PyTraceback_Add() now uses _PyErr_Fetch()/_PyErr_Restore() instead
of PyErr_Fetch()/PyErr_Restore().

The _decimal and _xxsubinterpreters extensions are now built with the
Py_BUILD_CORE_MODULE macro defined to get access to the internal C
API.
2021-10-13 14:09:13 +02:00
Victor Stinner d943d19172
bpo-45439: Move _PyObject_CallNoArgs() to pycore_call.h (GH-28895)
* Move _PyObject_CallNoArgs() to pycore_call.h (internal C API).
* _ssl, _sqlite and _testcapi extensions now call the public
  PyObject_CallNoArgs() function, rather than _PyObject_CallNoArgs().
* _lsprof extension is now built with Py_BUILD_CORE_MODULE macro
  defined to get access to internal _PyObject_CallNoArgs().
2021-10-12 08:38:19 +02:00
Victor Stinner ce3489cfdb
bpo-45439: Rename _PyObject_CallNoArg() to _PyObject_CallNoArgs() (GH-28891)
Fix typo in the private _PyObject_CallNoArg() function name: rename
it to _PyObject_CallNoArgs() to be consistent with the public
function PyObject_CallNoArgs().
2021-10-12 00:42:23 +02:00
Victor Stinner fb8f208a4d
bpo-45439: _PyObject_Call() only checks tp_vectorcall_offset once (GH-28890)
Add _PyVectorcall_Call() helper function.

Add "assert(PyCallable_Check(callable));" to PyVectorcall_Call(),
similar check than PyVectorcall_Function().
2021-10-12 00:18:26 +02:00
Victor Stinner 2f92e2a590
bpo-45412: Remove Py_SET_ERRNO_ON_MATH_ERROR() macro (GH-28820)
Remove the following math macros using the errno variable:

* Py_ADJUST_ERANGE1()
* Py_ADJUST_ERANGE2()
* Py_OVERFLOWED()
* Py_SET_ERANGE_IF_OVERFLOW()
* Py_SET_ERRNO_ON_MATH_ERROR()

Create pycore_pymath.h internal header file.

Rename Py_ADJUST_ERANGE1() and Py_ADJUST_ERANGE2() to
_Py_ADJUST_ERANGE1() and _Py_ADJUST_ERANGE2(), and convert these
macros to static inline functions.

Move the following macros to pycore_pymath.h:

* _Py_IntegralTypeSigned()
* _Py_IntegralTypeMax()
* _Py_IntegralTypeMin()
* _Py_InIntegralTypeRange()
2021-10-11 21:00:25 +02:00
Pablo Galindo Salgado b4903afd4d
bpo-45256: Remove the usage of the C stack in Python to Python calls (GH-28488)
Ths commit inlines calls to Python functions in the eval loop and steals all the arguments in the call from the caller for
performance.
2021-10-09 16:51:30 +01:00
Dong-hee Na e6ff4eba6d
bpo-45385: Fix reference leak from descr_check (#28719) 2021-10-07 09:51:56 +09:00
Christian Clauss 5f401f1040
Fix typos in the Objects directory (GH-28766) 2021-10-06 16:57:10 -07:00
Mark Shannon a7252f88d3
bpo-40116: Add insertion order bit-vector to dict values to allow dicts to share keys more freely. (GH-28520) 2021-10-06 13:19:53 +01:00
Serhiy Storchaka 252b7bcb23
bpo-45355: More use of sizeof(_Py_CODEUNIT) (GH-28720) 2021-10-04 14:11:26 +03:00
Serhiy Storchaka 60b9e040c9
bpo-45355: Use sizeof(_Py_CODEUNIT) instead of literal 2 for the size of the code unit (GH-28711) 2021-10-03 21:22:42 +03:00