Commit Graph

59 Commits

Author SHA1 Message Date
Victor Stinner 9c03215a3e
gh-107149: Make PyUnstable_ExecutableKinds public (#108440)
Move PyUnstable_ExecutableKinds and associated macros from the
internal C API to the public C API.

Rename constants: replace "PY_" prefix with "PyUnstable_" prefix.
2023-08-31 09:56:06 +02:00
Victor Stinner 21c0844742
gh-108220: Internal header files require Py_BUILD_CORE to be defined (#108221)
* pycore_intrinsics.h does nothing if included twice
  (add #ifndef and #define).
* Update Tools/cases_generator/generate_cases.py to generate the
  Py_BUILD_CORE test.
* _bz2, _lzma, _opcode and zlib extensions now define the
  Py_BUILD_CORE_MODULE macro to use internal headers
  (pycore_code.h, pycore_intrinsics.h and pycore_blocks_output_buffer.h).
2023-08-21 19:15:52 +02:00
Mark Shannon 006e44f950
GH-108035: Remove the `_PyCFrame` struct as it is no longer needed for performance. (GH-108036) 2023-08-17 11:16:03 +01:00
Victor Stinner 1a3faba9f1
gh-106869: Use new PyMemberDef constant names (#106871)
* Remove '#include "structmember.h"'.
* If needed, add <stddef.h> to get offsetof() function.
* Update Parser/asdl_c.py to regenerate Python/Python-ast.c.
* Replace:

  * T_SHORT => Py_T_SHORT
  * T_INT => Py_T_INT
  * T_LONG => Py_T_LONG
  * T_FLOAT => Py_T_FLOAT
  * T_DOUBLE => Py_T_DOUBLE
  * T_STRING => Py_T_STRING
  * T_OBJECT => _Py_T_OBJECT
  * T_CHAR => Py_T_CHAR
  * T_BYTE => Py_T_BYTE
  * T_UBYTE => Py_T_UBYTE
  * T_USHORT => Py_T_USHORT
  * T_UINT => Py_T_UINT
  * T_ULONG => Py_T_ULONG
  * T_STRING_INPLACE => Py_T_STRING_INPLACE
  * T_BOOL => Py_T_BOOL
  * T_OBJECT_EX => Py_T_OBJECT_EX
  * T_LONGLONG => Py_T_LONGLONG
  * T_ULONGLONG => Py_T_ULONGLONG
  * T_PYSSIZET => Py_T_PYSSIZET
  * T_NONE => _Py_T_NONE
  * READONLY => Py_READONLY
  * PY_AUDIT_READ => Py_AUDIT_READ
  * READ_RESTRICTED => Py_AUDIT_READ
  * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED
  * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED)
2023-07-25 15:28:30 +02:00
Carl Meyer 104d7b760f
gh-105340: include hidden fast-locals in locals() (#105715)
* gh-105340: include hidden fast-locals in locals()
2023-07-05 17:05:02 -06:00
Mark Shannon 04492cbc9a
GH-91095: Specialize calls to normal Python classes. (GH-99331) 2023-06-22 09:48:19 +01:00
Mark Shannon 7199584ac8
GH-100987: Allow objects other than code objects as the "executable" of an internal frame. (GH-105727)
* Add table describing possible executable classes for out-of-process debuggers.

* Remove shim code object creation code as it is no longer needed.

* Make lltrace a bit more robust w.r.t. non-standard frames.
2023-06-14 13:46:37 +01:00
Pablo Galindo Salgado 6a8b862357
Revert "Move observability-relevant structure fields to the top" (#105512) 2023-06-08 15:59:38 +00:00
Gabriele N. Tornetta 410c2f13e5
Move observability-relevant structure fields to the top (#105271) 2023-06-08 11:41:24 +01:00
Mark Shannon 45f5aa8fc7
GH-103082: Filter LINE events in VM, to simplify tool implementation. (GH-104387)
When monitoring LINE events, instrument all instructions that can have a predecessor on a different line.
Then check that the a new line has been hit in the instrumentation code.
This brings the behavior closer to that of 3.11, simplifying implementation and porting of tools.
2023-05-12 12:21:20 +01:00
Mark Shannon a0df9ee8fc
GH-96803: Add three C-API functions to make _PyInterpreterFrame less opaque for users of PEP 523. (GH-96849) 2023-05-05 17:53:07 +01:00
Mark Shannon 738c226786
GH-103082: Code cleanup in instrumentation code (#103474) 2023-04-29 04:51:55 +00:00
Mark Shannon efb8a2553c
GH-103488: Use return-offset, not yield-offset. (GH-103502)
* Use return-offset, not yield-offset, so that instruction pointer is correct when sending to a generator or coroutine.
2023-04-13 16:19:07 +01:00
Mark Shannon 411b169281
GH-103082: Implementation of PEP 669: Low Impact Monitoring for CPython (GH-103083)
* The majority of the monitoring code is in instrumentation.c

* The new instrumentation bytecodes are in bytecodes.c

* legacy_tracing.c adapts the new API to the old sys.setrace and sys.setprofile APIs
2023-04-12 12:04:55 +01:00
Mark Shannon 22b8d77b98
GH-100719: Remove redundant `gi_code` field from generator object. (GH-100749) 2023-02-23 10:19:01 +00:00
Mark Shannon d9199175c7
GH-100987: Refactor `_PyInterpreterFrame` a bit, to assist generator improvement. (GH-100988)
Refactor _PyInterpreterFrame a bit, to assist generator improvement.
2023-02-13 11:31:15 +00:00
Brandt Bucher 61762b9387
GH-100126: Skip incomplete frames in more places (GH-100613) 2023-01-09 12:20:04 -08:00
Irit Katriel 15c44789bb
gh-100758: Refactor initialisation of frame headers into a single function (_PyFrame_Initialize) (GH-100759) 2023-01-06 14:55:56 +00:00
Irit Katriel c31e356a10
gh-100720: refactor calculation of number of frame slots for a code object into the new function _PyFrame_NumSlotsForCodeObject (#100722) 2023-01-04 13:37:06 +00:00
Bill Fisher 88d565f32a
gh-99110: Initialize `frame->previous` in init_frame to fix segmentation fault when accessing `frame.f_back` (#100182) 2022-12-23 20:15:53 +05:30
Mark Shannon 1e197e63e2
GH-96421: Insert shim frame on entry to interpreter (GH-96319)
* Adds EXIT_INTERPRETER instruction to exit PyEval_EvalDefault()

* Simplifies RETURN_VALUE, YIELD_VALUE and RETURN_GENERATOR instructions as they no longer need to check for entry frames.
2022-11-10 12:34:57 +00:00
Mark Shannon 4a1c58d504
GH-96793: Specialize FOR_ITER for generators. (GH-98772) 2022-11-07 14:49:51 +00:00
Mark Shannon 222f10ca2d
GH-96569: Add two NULL checks to avoid undefined behavior. (GH-96585) 2022-09-06 16:45:43 +01:00
Mark Shannon c09fa7542c
GH-96237: Allow non-functions as reference-holder in frames. (GH-96238) 2022-08-25 10:16:55 +01:00
Mark Shannon 544531de23
GH-94262: Don't create frame objects for frames that aren't yet complete. (GH-94371) 2022-07-01 11:08:20 +01:00
Mark Shannon 45e62a2bc1
GH-93897: Store frame size in code object and de-opt if insufficient space on thread frame stack. (GH-93908) 2022-06-20 12:59:25 +01:00
Mark Shannon f6e43e834c
GH-89480: Document motivation, design and implementation of 3.11 frame stack. (GH-32304) 2022-04-11 16:05:20 +01:00
Brandt Bucher ef6a482b02
bpo-47177: Replace `f_lasti` with `prev_instr` (GH-32208) 2022-04-07 12:31:01 -07:00
Mark Shannon 8a349eb30b
Revert "bpo-44800: Document internal frame naming conventions (GH-32281)" (#32301)
This reverts commit 124227c95f.
2022-04-04 15:09:40 +01:00
Nick Coghlan 124227c95f
bpo-44800: Document internal frame naming conventions (GH-32281)
The fact interpreter frames were split out from full frame objects
rather than always being part of the eval loop implementation means
that it's tricky to infer the expected naming conventions simply
from looking at the code.

Documenting the de facto conventions in pycore_frame.h means future
readers of the code will have a clear explanation of the rationale
for those conventions (i.e. minimising non-functional code churn).
2022-04-03 16:55:55 +10:00
Mark Shannon d7163bb35d
bpo-42197: Don't create `f_locals` dictionary unless we actually need it. (GH-32055)
* `PyFrame_FastToLocalsWithError` and `PyFrame_LocalsToFast` are no longer called during profile and tracing.
 (Contributed by Fabio Zadrozny)

* Make accesses to a frame's `f_locals` safe from C code, not relying on calls to `PyFrame_FastToLocals` or `PyFrame_LocalsToFast`.

* Document new `PyFrame_GetLocals` C-API function.
2022-03-25 12:57:50 +00:00
Mark Shannon 49daf6dba8
bpo-47045: Remove `f_state` field (GH-31963)
* Remove the f_state field from _PyInterpreterFrame

*  Make ownership of the frame explicit, replacing the is_generator field with an owner field.
2022-03-22 12:57:19 +00:00
Victor Stinner 2217462bda
bpo-45786: Remove _PyFrame_Fini() and _PyFrame_DebugMallocStats() (GH-31874)
Remove private empty _PyFrame_Fini() and _PyFrame_DebugMallocStats()
functions.
2022-03-18 11:16:55 +01:00
Victor Stinner 7496f95873
bpo-45431: Rename CFrame to _PyCFrame in the C API (GH-31584)
Rename also struct _cframe to struct _PyCFrame.

Add a comment suggesting using public functions rather than using
directly the private _PyCFrame structure.
2022-02-28 16:03:57 +01:00
Victor Stinner 87af12bff3
bpo-46836: Rename InterpreterFrame to _PyInterpreterFrame (GH-31583)
Rename also struct _interpreter_frame to struct _PyInterpreterFrame.

Reduce risk of name conflicts if a project includes pycore_frame.h.
2022-02-25 16:22:00 +01:00
Victor Stinner 8f2a337a80
bpo-45316: Move private functions to internal C API (GH-31579)
Move the unexported private functions to the internal C API:

* pycore_frame.h: _PyFrame_New_NoTrack()
* pycore_function.h: _PyFunction_GetVersionForCurrentState()
* pycore_genobject.h: _PyAsyncGenValueWrapperNew()
* pycore_genobject.h: _PyCoro_GetAwaitableIter()
* pycore_genobject.h: _PyGen_yf()
2022-02-25 16:07:14 +01:00
Victor Stinner 18b5dd68c6
bpo-46836: Move PyFrameObject to pycore_frame.h (GH-31530)
Move the PyFrameObject type definition (struct _frame) to the
internal C API pycore_frame.h header file.
2022-02-25 12:53:19 +01:00
Mark Shannon da4d4ec185
Pass reference to func, as well as args, when pushing frame. (GH-31100) 2022-02-03 18:36:28 +00:00
Mark Shannon 89fd7c3452
bpo-46329: Split calls into precall and call instructions. (GH-30855)
* Add PRECALL_FUNCTION opcode.

* Move 'call shape' varaibles into struct.

* Replace CALL_NO_KW and CALL_KW with KW_NAMES and CALL instructions.

* Specialize for builtin methods taking using the METH_FASTCALL | METH_KEYWORDS protocol.

* Allow kwnames for specialized calls to builtin types.

* Specialize calls to tuple(arg) and str(arg).
2022-01-28 12:42:30 +00:00
Mark Shannon b04dfbbe4b
bpo-46409: Make generators in bytecode (GH-30633)
* Add RETURN_GENERATOR and JUMP_NO_INTERRUPT opcodes.

* Trim frame and generator by word each.

* Minor refactor of frame.c

* Update test.test_sys to account for smaller frames.

* Treat generator functions as normal functions when evaluating and specializing.
2022-01-20 11:46:39 +00:00
Brandt Bucher 332e6b9725
bpo-45256: Don't track the exact depth of each `InterpreterFrame` (GH-30372) 2022-01-05 11:30:26 +00:00
Brandt Bucher 77195cd44b
bpo-46090: Allow PyThreadState.datastack_* members to be NULL (GH-30234) 2021-12-28 09:49:48 -08:00
Mark Shannon 0b50a4f0cd
bpo-46039: Split yield from in two (GH-30035)
* Split YIELD_FROM opcode into SEND and JUMP_ABSOLUTE.

* Remove YIELD_FROM opcode.
2021-12-15 10:30:09 +00:00
Eric Snow c8749b5783
bpo-46008: Make runtime-global object/type lifecycle functions and state consistent. (gh-29998)
This change is strictly renames and moving code around.  It helps in the following ways:

* ensures type-related init functions focus strictly on one of the three aspects (state, objects, types)
* passes in PyInterpreterState * to all those functions, simplifying work on moving types/objects/state to the interpreter
* consistent naming conventions help make what's going on more clear
* keeping API related to a type in the corresponding header file makes it more obvious where to look for it

https://bugs.python.org/issue46008
2021-12-09 12:59:26 -07:00
Mark Shannon 299483c95d
bpo-45963: Make space for the InterpreterFrame of a generator in that generator. (GH-29891)
* Make generator, coroutine and async gen structs all the same size.

* Store interpreter frame in generator (and coroutine). Reduces the number of allocations neeeded for a generator from two to one.
2021-12-06 10:13:49 +00:00
Mark Shannon 49444fb807
bpo-45753: Interpreter internal tweaks (GH-29575)
* Split exit paths into exceptional and non-exceptional.

* Move exit tracing code to individual bytecodes.

* Wrap all trace entry and exit events in macros to make them clearer and easier to enhance.

* Move return sequence into RETURN_VALUE, YIELD_VALUE and YIELD_FROM. Distinguish between normal trace events and dtrace events.
2021-12-01 12:09:36 +00:00
Mark Shannon 60929576e4
bpo-45786: Allocate space for frame in frame object. (GH-29729) 2021-11-29 12:34:59 +00:00
Mark Shannon 135cabd328
bpo-44525: Copy free variables in bytecode to allow calls to inner functions to be specialized (GH-29595)
* Make internal APIs that take PyFrameConstructor take a PyFunctionObject instead.

* Add reference to function to frame, borrow references to builtins and globals.

* Add COPY_FREE_VARS instruction to allow specialization of calls to inner functions.
2021-11-23 09:53:24 +00:00
Mark Shannon 7f61d9d848
bpo-45256: Rationalize code around Python-to-Python calls a bit. (GH-29235) 2021-10-28 16:14:59 +01:00
Pablo Galindo Salgado b4903afd4d
bpo-45256: Remove the usage of the C stack in Python to Python calls (GH-28488)
Ths commit inlines calls to Python functions in the eval loop and steals all the arguments in the call from the caller for
performance.
2021-10-09 16:51:30 +01:00