Commit Graph

1090 Commits

Author SHA1 Message Date
Mark Shannon 069e81ab3d
bpo-43977: Use tp_flags for collection matching (GH-25723)
* Add Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING, add to all relevant standard builtin classes.

* Set relevant flags on collections.abc.Sequence and Mapping.

* Use flags in MATCH_SEQUENCE and MATCH_MAPPING opcodes.

* Inherit Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING.

* Add NEWS

* Remove interpreter-state map_abc and seq_abc fields.
2021-04-30 09:50:28 +01:00
Pablo Galindo 5bf8bf2267
bpo-38530: Offer suggestions on NameError (GH-25397)
When printing NameError raised by the interpreter, PyErr_Display
will offer suggestions of simmilar variable names in the function that the exception
was raised from:

    >>> schwarzschild_black_hole = None
    >>> schwarschild_black_hole
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    NameError: name 'schwarschild_black_hole' is not defined. Did you mean: schwarzschild_black_hole?
2021-04-14 15:10:33 +01:00
Mark Shannon 9e7b2076fb
bpo-43760: Speed up check for tracing in interpreter dispatch (#25276)
* Remove redundant tracing_possible field from interpreter state.

* Move 'use_tracing' from tstate onto C stack, for fastest possible checking in dispatch logic.

* Add comments stressing the importance stack discipline when dealing with CFrames.

* Add NEWS
2021-04-13 11:08:14 +01:00
Victor Stinner 09bbebea16
bpo-43753: Add Py_Is() and Py_IsNone() functions (GH-25227)
Add the Py_Is(x, y) function to test if the 'x' object is the 'y'
object, the same as "x is y" in Python. Add also the Py_IsNone(),
Py_IsTrue(), Py_IsFalse() functions to test if an object is,
respectively, the None singleton, the True singleton or the False
singleton.
2021-04-11 00:17:39 +02:00
Mark Shannon 28d28e053d
bpo-43760: Streamline dispatch sequence for machines without computed gotos. (GH-25244)
* Do fetch and decode at end of opcode then jump directly to switch.
  Should allow compilers that don't support computed-gotos, specifically MSVC,
  to generate better code.
2021-04-08 11:22:55 +01:00
Mark Shannon b37181e692
bpo-43683: Handle generator entry in bytecode (GH-25138)
* Handle check for sending None to starting generator and coroutine into bytecode.

* Document new bytecode and make it fail gracefully if mis-compiled.
2021-04-06 11:48:59 +01:00
Brandt Bucher f84d5a1136
bpo-42128: __match_args__ can't be a list anymore (GH-25203) 2021-04-05 19:17:08 -07:00
Mark Shannon fcb55c0037
bpo-27129: Use instruction offsets, not byte offsets, in bytecode and internally. (GH-25069)
* Use instruction offset, rather than bytecode offset. Streamlines interpreter dispatch a bit, and removes most EXTENDED_ARGs for jumps.

* Change some uses of PyCode_Addr2Line to PyFrame_GetLineNumber
2021-04-01 16:00:31 +01:00
Mark Shannon d41bdddbed
Move big block of macros out of function to improve readability. (GH-25020) 2021-03-25 12:00:30 +00:00
Mark Shannon 4958f5d69d
Only check evalbreaker after calls and on backwards egdes. Makes sure that __exit__ or __aexit__ is called in with statments in case of interrupt. (GH-18334) 2021-03-24 17:56:12 +00:00
Victor Stinner fc980e0be1
bpo-43541: Fix PyEval_EvalCodeEx() regression (GH-24918)
* Remove an assertion which required CO_NEWLOCALS and CO_OPTIMIZED
  code flags. It is ok to call this function on a code with these
  flags set.
* Fix reference counting on builtins: remove Py_DECREF().
  Fix regression introduced in the
  commit 46496f9d12.

Add also a comment to document that _PyEval_BuiltinsFromGlobals()
returns a borrowed reference.
2021-03-18 14:51:24 +01:00
Mark Shannon 8e1b406275
Make tracing info (bounds, and previous instruction offset) a bit more robust. (GH-24726) 2021-03-05 14:45:50 +00:00
db3l 131d551640
bpo-43271: Re-enable ceval.c optimizations for Windows debug builds (GH-24739)
Partially reverts commit b74396c316

The optimizations are necessary to prevent the interpreter from
crashing in a number of tests involving recursion.
2021-03-03 19:09:48 -08:00
Pablo Galindo af5fa13ef6
bpo-37146: Deactivate opcode cache only when using huntrleaks in the test suite (GH-24643) 2021-02-28 22:41:09 +00:00
Brandt Bucher 145bf269df
bpo-42128: Structural Pattern Matching (PEP 634) (GH-22917)
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Talin <viridia@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-02-26 14:51:55 -08:00
Victor Stinner d5fc998737
bpo-42093: Cleanup _PyDict_GetItemHint() (GH-24582)
* No longer save/restore the current exception. It is no longer used
  with an exception raised.
* No longer clear the current exception on error: it's now up to the
  caller.
2021-02-21 12:02:04 +01:00
Victor Stinner 46496f9d12
bpo-42990: Functions inherit current builtins (GH-24564)
The types.FunctionType constructor now inherits the current builtins
if the globals dictionary has no "__builtins__" key, rather than
using {"None": None} as builtins: same behavior as eval() and exec()
functions.

Defining a function with "def function(...): ..." in Python is not
affected, globals cannot be overriden with this syntax: it also
inherits the current builtins.

PyFrame_New(), PyEval_EvalCode(), PyEval_EvalCodeEx(),
PyFunction_New() and PyFunction_NewWithQualName() now inherits the
current builtins namespace if the globals dictionary has no
"__builtins__" key.

* Add _PyEval_GetBuiltins() function.
* _PyEval_BuiltinsFromGlobals() now uses _PyEval_GetBuiltins() if
  builtins cannot be found in globals.
* Add tstate parameter to _PyEval_BuiltinsFromGlobals().
2021-02-20 15:17:18 +01:00
Victor Stinner bcb094b41f
bpo-43268: Pass interp rather than tstate to internal functions (GH-24580)
Pass the current interpreter (interp) rather than the current Python
thread state (tstate) to internal functions which only use the
interpreter.

Modified functions:

* _PyXXX_Fini() and _PyXXX_ClearFreeList() functions
* _PyEval_SignalAsyncExc(), make_pending_calls()
* _PySys_GetObject(), sys_set_object(), sys_set_object_id(), sys_set_object_str()
* should_audit(), set_flags_from_config(), make_flags()
* _PyAtExit_Call()
* init_stdio_encoding()
* etc.
2021-02-19 15:10:45 +01:00
Victor Stinner a486054b24
bpo-43270: Remove private _PyErr_OCCURRED() macro (GH-24579)
Remove the private _PyErr_OCCURRED() macro: use the public
PyErr_Occurred() function instead.

CPython internals must use the internal _PyErr_Occurred(tstate)
function instead: it is the most efficient way to check if an
exception was raised.
2021-02-19 15:08:54 +01:00
Victor Stinner 101bf69ff1
bpo-43268: _Py_IsMainInterpreter() now expects interp (GH-24577)
The _Py_IsMainInterpreter() function now expects interp rather than
tstate.
2021-02-19 13:33:31 +01:00
Victor Stinner 44085a3fc9
bpo-42990: Refactor _PyFrame_New_NoTrack() (GH-24566)
* Refactor _PyFrame_New_NoTrack() and PyFunction_NewWithQualName()
  code.
* PyFrame_New() checks for _PyEval_BuiltinsFromGlobals() failure.
* Fix a ref leak in _PyEval_BuiltinsFromGlobals() error path.
* Complete PyFunction_GetModule() documentation: it returns a
  borrowed reference and it can return NULL.
* Move _PyEval_BuiltinsFromGlobals() definition to the internal C
  API.
* PyFunction_NewWithQualName() uses _Py_IDENTIFIER() API for the
  "__name__" string to make it compatible with subinterpreters.
2021-02-18 19:20:16 +01:00
Dong-hee Na 3cf0833f42
bpo-43152: Update assert statement to remove unused warning (GH-24473) 2021-02-14 15:54:39 +09:00
Mark Shannon 0332e569c1
bpo-42990: Further refactoring of PyEval_ functions. (GH-24368)
* Further refactoring of PyEval_EvalCode and friends. Break into make-frame, and eval-frame parts.

* Simplify function vector call using new _PyEval_Vector.

* Remove unused internal functions: _PyEval_EvalCodeWithName and _PyEval_EvalCode.

* Don't use legacy function PyEval_EvalCodeEx.
2021-02-01 10:42:03 +00:00
Guido van Rossum 5c5a938573
bpo-42927: Inline cache for attributes defined with '__slots__' (#24216) 2021-01-29 18:02:29 -08:00
Mark Shannon d6c33fbd34
bpo-42990: Introduce 'frame constructor' struct to simplify API for PyEval_CodeEval and friends (GH-24298)
* Introduce 'frame constructor' to simplify API for frame creation

* Embed struct using a macro to conform to PEP 7
2021-01-29 13:24:55 +00:00
Mark Shannon 8643345bdb
Update frame.f_lineno before any call to the (C) tracing function. (GH-24150)
* Minimizes breakage of C extensions and ensures PyFrame_GetLineNumber is efficient.
2021-01-07 16:49:02 +00:00
Mark Shannon ee9f98d9f4
bpo-42823: Fix frame lineno when frame.f_trace is set (GH-24099)
* Add test for frame.f_lineno with/without tracing.

* Make sure that frame.f_lineno is correct regardless of whether frame.f_trace is set.

* Update importlib

* Add NEWS
2021-01-05 12:04:10 +00:00
Mark Shannon bf353f3c2d
bpo-42246: Make sure that `f_lasti`, and thus `f_lineno`, is set correctly after raising or reraising an exception (GH-23803)
* Ensure that f_lasti is set correctly after an exception is raised to conform to PEP 626.

* Update importlib

* Add NEWS.
2020-12-17 13:55:28 +00:00
Mark Shannon 4e7a69bdb6
bpo-42500: Fix recursion in or after except (GH-23568)
* Use counter, rather boolean state when handling soft overflows.
2020-12-02 13:30:55 +00:00
Yurii Karabas 7301979b23
bpo-42202: Store func annotations as a tuple (GH-23316)
Reduce memory footprint and improve performance of loading modules having many func annotations.

  >>> sys.getsizeof({"a":"int","b":"int","return":"int"})
  232
  >>> sys.getsizeof(("a","int","b","int","return","int"))
  88

The tuple is converted into dict on the fly when `func.__annotations__` is accessed first.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2020-11-25 19:43:18 +09:00
Victor Stinner d96a7a8313
bpo-42296: On Windows, fix CTRL+C regression (GH-23257)
On Windows, fix a regression in signal handling which prevented to
interrupt a program using CTRL+C. The signal handler can be run in a
thread different than the Python thread, in which case the test
deciding if the thread can handle signals is wrong.

On Windows, _PyEval_SignalReceived() now always sets eval_breaker to
1 since it cannot test _Py_ThreadCanHandleSignals(), and
  eval_frame_handle_pending() always calls
  _Py_ThreadCanHandleSignals() to recompute eval_breaker.
2020-11-13 14:44:42 +01:00
Mark Shannon 877df851c3
bpo-42246: Partial implementation of PEP 626. (GH-23113)
* Implement new line number table format, as defined in PEP 626.
2020-11-12 09:43:29 +00:00
Pablo Galindo 80449f243b
bpo-42266: Handle monkey-patching descriptors in LOAD_ATTR cache (GH-23157) 2020-11-05 09:23:15 +00:00
Neil Schemenauer 0564aafb71
bpo-42099: Fix reference to ob_type in unionobject.c and ceval (GH-22829)
* Use Py_TYPE() rather than o->ob_type.
2020-10-27 18:55:52 +00:00
Serhiy Storchaka fb5db7ec58
bpo-42006: Stop using PyDict_GetItem, PyDict_GetItemString and _PyDict_GetItemId. (GH-22648)
These functions are considered not safe because they suppress all internal errors
and can return wrong result.  PyDict_GetItemString and _PyDict_GetItemId can
also silence current exception in rare cases.

Remove no longer used _PyDict_GetItemId.
Add _PyDict_ContainsId and rename _PyDict_Contains into
_PyDict_Contains_KnownHash.
2020-10-26 08:43:39 +02:00
Pablo Galindo 109826c850
bpo-42093: Add opcode cache for LOAD_ATTR (GH-22803) 2020-10-20 06:22:44 +01:00
Vladimir Matveev 037245c5ac
bpo-41756: Add PyIter_Send function (#22443) 2020-10-09 17:15:15 -07:00
Serhiy Storchaka dcc54215ac
bpo-41936. Remove macros Py_ALLOW_RECURSION/Py_END_ALLOW_RECURSION (GH-22552) 2020-10-05 12:32:00 +03:00
Victor Stinner bd0a08ea90
bpo-21955: Change my nickname in BINARY_ADD comment (GH-22481) 2020-10-01 18:57:37 +02:00
Mark Shannon 17b5be0c0a
bpo-41670: Remove outdated predict macro invocation. (GH-22026)
Remove PREDICTion of POP_BLOCK from FOR_ITER.
2020-09-29 10:09:13 +01:00
Victor Stinner b7d8d8dbfe
bpo-40941: Fix stackdepth compiler warnings (GH-22377)
Explicitly cast a difference of two pointers to int:
PyFrameObject.f_stackdepth is an int.
2020-09-23 14:07:16 +02:00
Victor Stinner 19c3ac92bf
bpo-41834: Remove _Py_CheckRecursionLimit variable (GH-22359)
Remove the global _Py_CheckRecursionLimit variable: it has been
replaced by ceval.recursion_limit of the PyInterpreterState
structure.

There is no need to keep the variable for the stable ABI, since
Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() were not usable
in Python 3.8 and older: these macros accessed PyThreadState members,
whereas the PyThreadState structure is opaque in the limited C API.
2020-09-23 14:04:57 +02:00
Vladimir Matveev 2b05361bf7
bpo-41756: Introduce PyGen_Send C API (GH-22196)
The new API allows to efficiently send values into native generators
and coroutines avoiding use of StopIteration exceptions to signal 
returns.

ceval loop now uses this method instead of the old "private"
_PyGen_Send C API. This translates to 1.6x increased performance
of 'await' calls in micro-benchmarks.

Aside from CPython core improvements, this new API will also allow 
Cython to generate more efficient code, benefiting high-performance
IO libraries like uvloop.
2020-09-18 18:38:38 -07:00
Mark Shannon cb9879b948
bpo-40941: Unify implicit and explicit state in the frame and generator objects into a single value. (GH-20803)
* Merge gen and frame state variables into one.

* Replace stack pointer with depth in PyFrameObject. Makes code easier to read and saves a word of memory.
2020-07-17 11:44:23 +01:00
Victor Stinner 384621c42f
bpo-41078: Rename pycore_tupleobject.h to pycore_tuple.h (GH-21056) 2020-06-22 17:27:35 +02:00
Dong-hee Na 33faf5c4f4
bpo-40925: Remove unused stack macro SET_VALUE (GH-20783) 2020-06-11 10:39:18 +01:00
Victor Stinner 1bcc32f062
bpo-39465: Use _PyInterpreterState_GET() (GH-20788)
Replace _PyThreadState_GET() with _PyInterpreterState_GET() in:

* get_small_int()
* gcmodule.c: add also get_gc_state() function
* _PyTrash_deposit_object()
* _PyTrash_destroy_chain()
* warnings_get_state()
* Py_GetRecursionLimit()

Cleanup listnode.c: add 'parser' variable.
2020-06-10 20:08:26 +02:00
Victor Stinner 232dda6cbc
bpo-40679: Fix _PyEval_EvalCode() crash if qualname is NULL (GH-20615)
If name is NULL, name is now set to co->co_name.
If qualname is NULL, qualname is now set to name.

qualname must not be NULL: it is used to build error messages.

Cleanup also the code: declare variables where they are initialized.
Rename "name" local variables to "varname" to avoid overriding "name"
parameter.
2020-06-04 15:19:02 +02:00
Victor Stinner 317bab0bf6
PyOS_AfterFork_Child() pass tstate to _PyEval_ReInitThreads() (GH-20598) 2020-06-02 18:44:54 +02:00
Victor Stinner 26881c8fae
PyOS_AfterFork_Child() uses PyStatus (GH-20596)
PyOS_AfterFork_Child() helper functions now return a PyStatus:
PyOS_AfterFork_Child() is now responsible to handle errors.

* Move _PySignal_AfterFork() to the internal C API
* Add #ifdef HAVE_FORK on _PyGILState_Reinit(), _PySignal_AfterFork()
  and _PyInterpreterState_DeleteExceptMain().
2020-06-02 15:51:37 +02:00