Commit Graph

1401 Commits

Author SHA1 Message Date
adphrost a41ed975e8
GH-91049: Introduce set vectorcall field API for PyFunctionObject (GH-92257)
Co-authored-by: Andrew Frost <adfrost@fb.com>
Co-authored-by: Itamar Ostricher <itamarost@gmail.com>
2022-09-15 16:42:37 +01:00
Nikita Sobolev e37ac5fbb6
gh-96751: Remove dead code from `CALL_FUNCTION_EX` opcode (GH-96752) 2022-09-15 10:33:13 +01:00
Kevin Modzelewski 4781535a57
ceval: Use _PyTuple_FromArraySteal in BUILD_TUPLE (GH-96516) 2022-09-14 12:44:32 +09:00
Mark Shannon 12c5f328d2
GH-96754: Check whether the interpreter frame is complete before creating frame object. (GH-96776) 2022-09-13 09:25:16 +01:00
Mark Shannon 50a70a083d
GH-96678: Fix undefined behavior in ceval.c (#96708) 2022-09-10 01:12:06 +01:00
Brandt Bucher 72b29b2611
Replace ad-hoc labels with JUMP_TO_INSTRUCTION (GH-96634) 2022-09-09 14:43:10 -07:00
Mark Shannon aa3b4cf779
GH-96636: Remove all uses of NOTRACE_DISPATCH (GH-96643)
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2022-09-08 17:16:48 +01:00
Itamar Ostricher 4f523a7054
gh-96653: Remove duplicate CALL_STAT_INC(inlined_py_calls) in BINARY_SUBSCR_GETITEM (GH-96654) 2022-09-08 16:22:32 +08:00
Brandt Bucher 0d04b8d9e1
GH-91432: Remove the iterator_exhausted_no_error label (GH-96517) 2022-09-06 21:07:20 -07:00
Kumar Aditya 67444902a0
GH-96572: fix use after free in trace refs build mode (#96618) 2022-09-06 11:02:41 -07:00
Dennis Sweeney ac18665472
ceval.c's GETITEM should have asserts, not set exceptions (GH-96518) 2022-09-04 19:00:24 -04:00
Irit Katriel 4c72517cad
gh-93554: Conditional jump opcodes only jump forward (GH-96318) 2022-09-01 21:36:47 +01:00
Brandt Bucher 88671a9d69
Remove the binary_subscr_dict_error label (GH-96443) 2022-08-30 15:45:24 -07:00
Mark Shannon c09fa7542c
GH-96237: Allow non-functions as reference-holder in frames. (GH-96238) 2022-08-25 10:16:55 +01:00
Pablo Galindo Salgado e34c82abeb
GH-93503: Add thread-specific APIs to set profiling and tracing functions in the C-API (#93504)
* gh-93503: Add APIs to set profiling and tracing functions in all threads in the C-API

* Use a separate API

* Fix NEWS entry

* Add locks around the loop

* Document ignoring exceptions

* Use the new APIs in the sys module

* Update docs
2022-08-24 23:21:39 +01:00
Mark Shannon a4a9f2e879
GH-96177: Move GIL and eval breaker code out of ceval.c into ceval_gil.c. (GH-96204) 2022-08-24 14:21:01 +01:00
Brandt Bucher 5bfb3c372b
GH-90997: Wrap yield from/await in a virtual try/except StopIteration (GH-96010) 2022-08-19 12:33:44 -07:00
Ken Jin 7276ca25f5
GH-93911: Specialize `LOAD_ATTR` for custom `__getattribute__` (GH-93988) 2022-08-17 12:37:07 +01:00
zhanpon 23a757f44f
Update _PyEval_AddPendingCall comment (#95817) 2022-08-11 13:32:52 +02:00
Mark Shannon de388c0a7b
GH-95245: Store object values and dict pointers in single tagged pointer. (GH-95278) 2022-08-01 14:34:54 +01:00
Mark Shannon b8b2990fb3
GH-90081: Run python tracers at full speed (GH-95328) 2022-07-28 10:17:22 +01:00
Brandt Bucher e402b26b7f
GH-95113: Don't use EXTENDED_ARG_QUICK in unquickened code (GH-95121) 2022-07-22 11:04:20 -07:00
Mark Shannon 2f8bff6879
GH-94739: Mark stacks of exception handling blocks for setting frame.f_lineno in the debugger. (GH-94958) 2022-07-18 16:06:42 +01:00
Brandt Bucher 8a285df806
GH-93252: Fix error handling for failed Python calls (GH-94693) 2022-07-08 18:52:26 -07:00
Brandt Bucher 23ee4a8067
gh-94215: Fix error handling for line-tracing events (GH-94681)
* Re-enable crasher
* Fix error handling for line-tracing events
* blurb add
2022-07-08 14:40:35 +02:00
Pablo Galindo Salgado 40d81fd63b
gh-94510: Raise on re-entrant calls to sys.setprofile and sys.settrace (GH-94511)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2022-07-05 20:18:47 +02:00
neonene ea39b77de9
gh-91719: Reload opcode on unknown error so that C can optimize the dispatching in ceval.c (#94364) 2022-06-30 08:38:29 -07:00
Mark Shannon c0453a40fa
GH-94163: Add BINARY_SLICE and STORE_SLICE instructions. (GH-94168) 2022-06-27 12:24:23 +01:00
Mark Shannon 6f8875eba3
GH-93841: Allow stats to be turned on and off, cleared and dumped at runtime. (GH-93843) 2022-06-21 15:40:54 +01:00
Dennis Sweeney 5fcfdd87c9
GH-91432: Specialize FOR_ITER (GH-91713)
* Adds FOR_ITER_LIST and FOR_ITER_RANGE specializations.

* Adds _PyLong_AssignValue() internal function to avoid temporary boxing of ints.
2022-06-21 11:19:26 +01:00
Mark Shannon ab0e601016
GH-93516: Speedup line number checks when tracing. (GH-93763)
* Use a lookup table to reduce overhead of getting line numbers during tracing.
2022-06-20 13:00:42 +01:00
Mark Shannon 45e62a2bc1
GH-93897: Store frame size in code object and de-opt if insufficient space on thread frame stack. (GH-93908) 2022-06-20 12:59:25 +01:00
Victor Stinner 27b9894033
gh-93937, C API: Move PyFrame_GetBack() to Python.h (#93938)
Move the follow functions and type from frameobject.h to pyframe.h,
so the standard <Python.h> provide frame getter functions:

* PyFrame_Check()
* PyFrame_GetBack()
* PyFrame_GetBuiltins()
* PyFrame_GetGenerator()
* PyFrame_GetGlobals()
* PyFrame_GetLasti()
* PyFrame_GetLocals()
* PyFrame_Type

Remove #include "frameobject.h" from many C files. It's no longer
needed.
2022-06-19 12:02:33 +02:00
Ken Jin a51742ab82
gh-93911: Specialize `LOAD_ATTR_PROPERTY` (GH-93912) 2022-06-17 23:13:17 +08:00
Victor Stinner 1735710873
gh-77782: Py_FdIsInteractive() now uses PyConfig.interactive (#93916) 2022-06-17 15:19:28 +02:00
Ken Jin ab45c1dde0
Fix BINARY_SUBSCR_GETITEM stats (GH-93903) 2022-06-16 15:02:07 +01:00
Ken Jin b083450f88
GH-93429: Merge `LOAD_METHOD` back into `LOAD_ATTR` (GH-93430) 2022-06-14 11:36:22 +01:00
Mark Shannon 3cd1a5d3ec
GH-93516: Store offset of first traceable instruction in code object (GH-93769) 2022-06-14 11:09:30 +01:00
Ken Jin 75260925ec
Fix MSVC compiler warnings in ceval.c (#93569) 2022-06-07 11:32:42 +01:00
Mark Shannon f012df706c
Shrink the LOAD_METHOD cache by one codeunit. (#93537) 2022-06-07 10:28:53 +01:00
Dennis Sweeney f425f3bb27
gh-93143: Avoid NULL check in LOAD_FAST based on analysis in the compiler (GH-93144) 2022-05-31 16:32:30 -04:00
Mark Shannon eb618d5ff0
GH-93354: Use exponential backoff to avoid excessive specialization attempts. (GH-93355) 2022-05-31 11:58:26 +01:00
Eric Snow caa279d6fd
bpo-40514: Drop EXPERIMENTAL_ISOLATED_SUBINTERPRETERS (gh-93185)
This was added for bpo-40514 (gh-84694) to test out a per-interpreter GIL. However, it has since proven unnecessary to keep the experiment in the repo. (It can be done as a branch in a fork like normal.) So here we are removing:

* the configure option
* the macro
* the code enabled by the macro
2022-05-27 17:38:01 -06:00
Mark Shannon bbcf42449e
GH-90230: Add stats to breakdown the origin of calls to `PyEval_EvalFrame` (GH-93284) 2022-05-27 16:31:41 +01:00
Ken Jin 5e6e5b98a8
gh-92777: Add LOAD_METHOD_LAZY_DICT (GH-92778) 2022-05-25 14:06:15 +01:00
Victor Stinner 5695c0e0a2
gh-91924: Fix __lltrace__ for non-UTF-8 stdout encoding (#93199)
Fix __lltrace__ debug feature if the stdout encoding is not UTF-8.

If the stdout encoding is not UTF-8, the first call to
lltrace_resume_frame() indirectly sets lltrace to 0 when calling
unicode_check_encoding_errors() which calls
encodings.search_function().
2022-05-25 11:45:33 +02:00
Mark Shannon 3fd8610002
GH-89914: Make the oparg of the YIELD_VALUE instruction equal the stack depth. (GH-92960) 2022-05-19 17:49:29 +01:00
Mark Shannon e48ac9c100
GH-90690: Remove `PRECALL` instruction (GH-92925) 2022-05-19 11:05:26 +01:00
Mark Shannon a4460f2eb8
Split refcount stats into 'interpreter' and 'non-interpreter' (GH-92919) 2022-05-18 14:38:43 +01:00
Mark Shannon fa2b8b75eb
Improve object stats (#92845)
* Add incref/decref stats

* Show ratios for allocation in summary
2022-05-16 14:35:11 +01:00
Mark Shannon 22a1db378c
GH-92236: Remove spurious "line" event when starting coroutine or generator. (GH-92722) 2022-05-13 11:24:45 +01:00
Crowthebird bdf9969197
Update outdated `LOAD_METHOD` comments in `Python/ceval.c` (GH-92641) 2022-05-12 20:37:55 +08:00
Mark Shannon f8a2fab212
GH-92239: Make sure that PEP 523 is supported, even when specializing first. (GH-92245) 2022-05-04 09:31:21 -06:00
Victor Stinner d716a0dfe2
Use static inline function Py_EnterRecursiveCall() (#91988)
Currently, calling Py_EnterRecursiveCall() and
Py_LeaveRecursiveCall() may use a function call or a static inline
function call, depending if the internal pycore_ceval.h header file
is included or not. Use a different name for the static inline
function to ensure that the static inline function is always used in
Python internals for best performance. Similar approach than
PyThreadState_GET() (function call) and _PyThreadState_GET() (static
inline function).

* Rename _Py_EnterRecursiveCall() to _Py_EnterRecursiveCallTstate()
* Rename _Py_LeaveRecursiveCall() to _Py_LeaveRecursiveCallTstate()
* pycore_ceval.h: Rename Py_EnterRecursiveCall() to
  _Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() and
  _Py_LeaveRecursiveCall()
2022-05-04 13:30:23 +02:00
Dennis Sweeney 868b1afa05
gh-92063: Enforce types in specialized PRECALL opcodes (GH-92068)
* Check the types of PRECALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS

* fix PRECALL_NO_KW_METHOD_DESCRIPTOR_NOARGS as well

* fix PRECALL_NO_KW_METHOD_DESCRIPTOR_O

* fix PRECALL_NO_KW_METHOD_DESCRIPTOR_FAST
2022-04-30 12:35:33 -06:00
Dennis Sweeney 37c6db60f9
gh-91869: Fix tracing of specialized instructions with extended args (GH-91945) 2022-04-27 22:36:34 -06:00
Victor Stinner 64a54e511d
gh-91719: Add pycore_opcode.h internal header file (#91906)
Move the following API from Include/opcode.h (public C API) to a new
Include/internal/pycore_opcode.h header file (internal C API):

* EXTRA_CASES
* _PyOpcode_Caches
* _PyOpcode_Deopt
* _PyOpcode_Jump
* _PyOpcode_OpName
* _PyOpcode_RelativeJump
2022-04-26 00:14:30 +02:00
Guido van Rossum 2f233fceae
gh-89279: In ceval.c, redefine some macros for speed (#32387)
Macros Py_DECREF, Py_XDECREF, Py_IS_TYPE, _Py_atomic_load_32bit_impl
and _Py_DECREF_SPECIALIZED are redefined as macros
that completely replace the inline functions of the same name.
These three came out in the top four of functions that (in MSVC)
somehow weren't inlined.

Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Dennis Sweeney <36520290+sweeneyde@users.noreply.github.com>
2022-04-22 07:46:26 -07:00
Guido van Rossum f8dc6186d1
GH-91719: Make MSVC generate somewhat faster switch code (#91718)
Apparently a switch on an 8-bit quantity where all cases are
present generates a more efficient jump (doing only one indexed
memory load instead of two).

So we make opcode and use_tracing uint8_t, and generate a macro
full of extra `case NNN:` lines for all unused opcodes.

See https://github.com/faster-cpython/ideas/issues/321#issuecomment-1103263673
2022-04-21 11:53:57 -07:00
Dennis Sweeney d7d7e6c007
Cast to (destructor) to fix compiler warnings (GH-91711) 2022-04-20 16:15:45 +01:00
Dennis Sweeney da6c78584b
gh-90667: Add specializations of Py_DECREF when types are known (GH-30872) 2022-04-19 19:02:19 +01:00
Dennis Sweeney cec5d858f5
gh-91625: Don't ignore extended args of adaptive opcodes (GH-91626) 2022-04-17 14:04:29 -04:00
Dennis Sweeney 37965d2fb4
gh-78607: Replace __ltrace__ with __lltrace__ (GH-91619) 2022-04-16 18:57:00 -04:00
Dennis Sweeney 8560f4a0f2
gh-91462: Make lltrace output human-readable. (GH-91463)
* Transform opcodes into opnames
* Print the whole stack at each opcode, and eliminate prtrace output at each (push/pop/stackadj)
* Display info about the function at each resume_frame
2022-04-16 17:36:29 -04:00
Irit Katriel ea2ae02607
gh-91276: Make JUMP_IF_TRUE_OR_POP/JUMP_IF_FALSE_OR_POP relative (GH-32215) 2022-04-15 20:19:24 +01:00
Dennis Sweeney 72965981d1
gh-90699: Use _Py_STR(empty) instead of PyUnicode_New(0, 0) for BUILD_STRING (GH-91476) 2022-04-15 23:21:35 +09:00
Irit Katriel dd207a6ac5
bpo-47120: make POP_JUMP_IF_TRUE/FALSE/NONE/NOT_NONE relative (GH-32400) 2022-04-11 10:40:24 +01:00
jonasdlindner f52d987abf
Fix some typos in comments (GH-32422) 2022-04-09 18:12:15 +09:00
Brandt Bucher ef6a482b02
bpo-47177: Replace `f_lasti` with `prev_instr` (GH-32208) 2022-04-07 12:31:01 -07:00
Victor Stinner 85addfb9c6
bpo-35134: Remove the Include/code.h header file (GH-32385)
Remove the Include/code.h header file. C extensions should only
include the main <Python.h> header file.

Python.h includes directly Include/cpython/code.h instead.
2022-04-07 02:29:52 +02:00
Christian Heimes 9e88b572fb
bpo-47009: Fix assert on big endian (GH-32332) 2022-04-05 18:18:02 +02:00
Irit Katriel 0aa8d5cbd8
bpo-47120: make JUMP_NO_INTERRUPT relative (GH-32221) 2022-04-05 12:49:08 +01:00
Irit Katriel 32091df41c
bpo-47186: Replace JUMP_IF_NOT_EG_MATCH by CHECK_EG_MATCH + jump (GH-32309) 2022-04-05 12:06:22 +01:00
Dennis Sweeney 6c6e0408a6
bpo-47009: Let PRECALL_NO_KW_LIST_APPEND do its own POP_TOP (GH-32239) 2022-04-05 11:18:30 +01:00
Hood Chatham 087d0fa5b9
bpo-47176: Interrupt handling for wasm32-emscripten builds without pthreads (GH-32209)
Co-authored-by: Christian Heimes <christian@python.org>
Co-authored-by: Brett Cannon <brett@python.org>
2022-04-03 22:58:52 +02:00
Irit Katriel 04e07c258f
bpo-47186: Replace JUMP_IF_NOT_EXC_MATCH by CHECK_EXC_MATCH + jump (GH-32231) 2022-04-01 13:59:38 +01:00
Dennis Sweeney a0ea7a116c
bpo-47009: Streamline list.append for the common case (GH-31864) 2022-04-01 11:23:42 +01:00
Irit Katriel a00518d9ad
bpo-47120: Replace the JUMP_ABSOLUTE opcode by the relative JUMP_BACKWARD (GH-32115) 2022-03-31 14:14:15 +01:00
Mark Shannon 04acfa94bb
Merge deoptimization blocks in interpreter (GH-32155) 2022-03-30 13:11:33 +01:00
Dennis Sweeney 788154919c
bpo-47053: Refactor BINARY_OP_INPLACE_ADD_UNICODE (GH-32122) 2022-03-28 22:07:05 -04:00
Kumar Aditya 58448cbd96
bpo-47127: Specialize calls for fastcall c methods with keywords (GH-32125)
* add PRECALL_METHOD_DESCRIPTOR_FAST_WITH_KEYWORDS
2022-03-28 03:53:25 +08:00
Dennis Sweeney cca43b7d64
bpo-47053: Reduce deoptimization in BINARY_OP_INPLACE_ADD_UNICODE (GH-31318)
* Don't deopt if refcounts are too big

* Detect more at specialization time
2022-03-25 16:13:19 +00:00
Irit Katriel 2f49b97cc5
bpo-46841: remove no-longer-used macro UPDATE_PREV_INSTR_OPARG (GH-32100) 2022-03-24 18:54:35 +00:00
neonene cd05d0a423
bpo-43166: Disable ceval.c optimizations for Windows debug builds (GH-32023)
Also increases the stack allocation when run with `python_d.exe` to account for the extra stack checks that are added.
2022-03-23 00:35:25 +00:00
Mark Shannon 49daf6dba8
bpo-47045: Remove `f_state` field (GH-31963)
* Remove the f_state field from _PyInterpreterFrame

*  Make ownership of the frame explicit, replacing the is_generator field with an owner field.
2022-03-22 12:57:19 +00:00
Brandt Bucher 2bde6827ea
bpo-46841: Quicken code in-place (GH-31888)
* Moves the bytecode to the end of the corresponding PyCodeObject, and quickens it in-place.

* Removes the almost-always-unused co_varnames, co_freevars, and co_cellvars member caches

* _PyOpcode_Deopt is a new mapping from all opcodes to their un-quickened forms.

* _PyOpcode_InlineCacheEntries is renamed to _PyOpcode_Caches

* _Py_IncrementCountAndMaybeQuicken is renamed to _PyCode_Warmup

* _Py_Quicken is renamed to _PyCode_Quicken

* _co_quickened is renamed to _co_code_adaptive (and is now a read-only memoryview).

* Do not emit unused nonzero opargs anymore in the compiler.
2022-03-21 11:11:17 +00:00
Victor Stinner e63894b3ee
bpo-46850: Remove _PyEval_CallTracing() function (GH-32019)
Remove the private undocumented function _PyEval_CallTracing() from
the C API. Call the public sys.call_tracing() function instead.
2022-03-21 03:03:22 +01:00
Victor Stinner 9087243e2c
bpo-46850: Remove _PyEval_GetCoroutineOriginTrackingDepth() (GH-32018)
Remove the private undocumented function
_PyEval_GetCoroutineOriginTrackingDepth() from the C API. Call the
public sys.get_coroutine_origin_tracking_depth() function instead.

Change the internal function
_PyEval_SetCoroutineOriginTrackingDepth():

* Remove the 'tstate' parameter;
* Add return value and raises an exception if depth is negative;
* No longer export the function: call the public
  sys.set_coroutine_origin_tracking_depth() function instead.

Uniformize also function declarations in pycore_ceval.h.
2022-03-21 02:24:00 +01:00
Mark Shannon 3011a097bd
Use low bit of LOAD_GLOBAL's oparg to indicate whether it should push an additional NULL. (GH-31933) 2022-03-17 16:14:57 +00:00
Mark Shannon 49e1e1e1bd
bpo-46841: Don't scan backwards in bytecode (GH-31901) 2022-03-15 17:08:37 -07:00
Mark Shannon 099f756141
bpo-45923: Decouple suspension of tracing from tracing flag. (GH-31908) 2022-03-15 17:06:21 +00:00
Oleg Iarygin 13b0412223
bpo-46920: Remove code that has explainers why it was disabled (GH-31813) 2022-03-14 17:04:22 +01:00
Brandt Bucher a89c29fbcc
bpo-46841: Add a _Py_SET_OPCODE macro (GH-31780) 2022-03-11 16:31:50 -08:00
Brandt Bucher 5498a61c7c
bpo-46841: Don't use an oparg counter for `STORE_SUBSCR` (GH-31742) 2022-03-08 15:53:22 +00:00
Brandt Bucher f193631387
bpo-46841: Use inline caching for calls (GH-31709) 2022-03-07 11:45:00 -08:00
Pablo Galindo Salgado 3b3be05a16
bpo-46940: Don't override existing AttributeError suggestion information (GH-31710)
When an exception is created in a nested call to PyObject_GetAttr, any
external calls will override the context information of the
AttributeError that we have already placed in the most internal call.
This will cause the suggestions we create to nor work properly as the
attribute name and object that we will be using are the incorrect ones.

To avoid this, we need to check first if these attributes are already
set and bail out if that's the case.
2022-03-07 12:23:11 +00:00
Brandt Bucher c4d2d57eef
bpo-46841: Fix BINARY_OP's handling of inline caches (GH-31671) 2022-03-04 10:51:27 -08:00
Brandt Bucher 586b24d3be
bpo-46841: Fix error message hacks in `GET_AWAITABLE` (GH-31664) 2022-03-04 12:41:17 +00:00
Brandt Bucher 05a8bc1c94
bpo-46841: Use inline caching for attribute accesses (GH-31640) 2022-03-03 15:31:00 -08:00
Mark Shannon b35603532b
Move check for str-only keys in LOAD_GLOBAL specializations to specialization time. (GH-31659) 2022-03-03 15:17:18 +00:00