Commit Graph

1521 Commits

Author SHA1 Message Date
Brandt Bucher 9178f533ff
bpo-45636: Merge all numeric operators (GH-29482) 2021-11-10 22:56:22 -08:00
Irit Katriel 4cdeee5978
bpo-45711: remove unnecessary DUP_TOP and POP in exception handling (GH-29495) 2021-11-10 18:08:28 +00:00
Mark Shannon 7f61d9d848
bpo-45256: Rationalize code around Python-to-Python calls a bit. (GH-29235) 2021-10-28 16:14:59 +01:00
Ken Jin 0a1a36b74b
bpo-44525: Add recursive checks for `CALL_FUNCTION_BUILTIN_O` (GH-29271) 2021-10-28 16:02:34 +01:00
Mark Shannon f291404a80
bpo-45637: Store the frame pointer in the cframe (GH-29267)
* Rename 'frame' to 'current_frame'
2021-10-28 13:59:11 +01:00
Mark Shannon 4776b07d17
Don't make a call at the C level when calling bound-methods from Python code. (GH-29238) 2021-10-27 15:26:22 +01:00
Brandt Bucher 82a662e521
bpo-44511: Improve the bytecode for class and mapping patterns (GH-26922)
* Refactor mapping patterns and speed up class patterns.

* Simplify MATCH_KEYS and MATCH_CLASS.

* Add COPY opcode.
2021-10-27 10:45:35 +01:00
Mark Shannon bc85eb7a4f
bpo-45527: Don't count cache hits, just misses. (GH-29092) 2021-10-20 20:53:48 +02:00
Mark Shannon 8863a0fcc5
bpo-44525: Specialize simple Python calls. (GH-29033) 2021-10-20 16:08:28 +01:00
Ken Jin 3163e68c34
bpo-44525: Specialize ``CALL_FUNCTION`` for C function calls (GH-26934) 2021-10-20 00:16:36 +01:00
Ken Jin 6e35b096ac
Record cache hits for BINARY_SUBSCR specializations (GH-29060) 2021-10-19 17:25:31 +01:00
Victor Stinner 034f607906
bpo-43760: Rename _PyThreadState_DisableTracing() (GH-29032)
* Rename _PyThreadState_DisableTracing()
  to _PyThreadState_PauseTracing()
* Rename _PyThreadState_ResetTracing()
  to _PyThreadState_ResumeTracing()
2021-10-18 18:40:43 +02:00
Mark Shannon 70945d57e7
bpo-45256: Avoid C calls for most Python to Python calls. (GH-28937)
* Avoid making C calls for most calls to Python functions.

* Change initialize_locals(steal=true) and _PyTuple_FromArraySteal to consume the argument references regardless of whether they succeed or fail.
2021-10-18 09:57:24 +01:00
Victor Stinner 547d26aa08
bpo-43760: Add PyThreadState_EnterTracing() (GH-28542)
Add PyThreadState_EnterTracing() and PyThreadState_LeaveTracing()
functions to the limited C API to suspend and resume tracing and
profiling.

Add an unit test on the PyThreadState C API to _testcapi.

Add also internal _PyThreadState_DisableTracing() and
_PyThreadState_ResetTracing().
2021-10-15 16:06:30 +02:00
Dennis Sweeney 3b3d30e8f7
bpo-45367: Specialize BINARY_MULTIPLY (GH-28727) 2021-10-14 15:56:33 +01:00
Brandt Bucher e71662c1ae
Ensure that instruction cases are self-contained (GH-28938) 2021-10-13 15:34:11 -07:00
Pablo Galindo Salgado 3901c08114
bpo-45256: Fix cleanup of stolen locals for Python-to-Python calls (GH-28905) 2021-10-13 14:38:41 +01:00
Mark Shannon a8b9350964
bpo-45340: Don't create object dictionaries unless actually needed (GH-28802)
* Never change types' cached keys. It could invalidate inline attribute objects.

* Lazily create object dictionaries.

* Update specialization of LOAD/STORE_ATTR.

* Don't update shared keys version for deletion of value.

* Update gdb support to handle instance values.

* Rename SPLIT_KEYS opcodes to INSTANCE_VALUE.
2021-10-13 14:19:34 +01:00
Victor Stinner 7cdc2a0f4b
pycore_pystate.h no longer redefines PyThreadState_GET() (GH-28921)
Redefining the PyThreadState_GET() macro in pycore_pystate.h is
useless since it doesn't affect files not including it. Either use
_PyThreadState_GET() directly, or don't use pycore_pystate.h internal
C API. For example, the _testcapi extension don't use the internal C
API, but use the public PyThreadState_Get() function instead.

Replace PyThreadState_Get() with _PyThreadState_GET(). The
_PyThreadState_GET() macro is more efficient than PyThreadState_Get()
and PyThreadState_GET() function calls which call fail with a fatal
Python error.

posixmodule.c and _ctypes extension now include <windows.h> before
pycore header files (like pycore_call.h).

_PyTraceback_Add() now uses _PyErr_Fetch()/_PyErr_Restore() instead
of PyErr_Fetch()/PyErr_Restore().

The _decimal and _xxsubinterpreters extensions are now built with the
Py_BUILD_CORE_MODULE macro defined to get access to the internal C
API.
2021-10-13 14:09:13 +02:00
Victor Stinner d943d19172
bpo-45439: Move _PyObject_CallNoArgs() to pycore_call.h (GH-28895)
* Move _PyObject_CallNoArgs() to pycore_call.h (internal C API).
* _ssl, _sqlite and _testcapi extensions now call the public
  PyObject_CallNoArgs() function, rather than _PyObject_CallNoArgs().
* _lsprof extension is now built with Py_BUILD_CORE_MODULE macro
  defined to get access to internal _PyObject_CallNoArgs().
2021-10-12 08:38:19 +02:00
Victor Stinner ce3489cfdb
bpo-45439: Rename _PyObject_CallNoArg() to _PyObject_CallNoArgs() (GH-28891)
Fix typo in the private _PyObject_CallNoArg() function name: rename
it to _PyObject_CallNoArgs() to be consistent with the public
function PyObject_CallNoArgs().
2021-10-12 00:42:23 +02:00
Mark Shannon fcb3d2ff63
Restore PEP 523 functionality. (GH-28871) 2021-10-11 11:34:02 +01:00
Pablo Galindo Salgado 543acbce5a
bpo-45256: Small cleanups for the code that inlines Python-to-Python calls in ceval.c (GH-28836) 2021-10-09 17:52:05 +01:00
Pablo Galindo Salgado b4903afd4d
bpo-45256: Remove the usage of the C stack in Python to Python calls (GH-28488)
Ths commit inlines calls to Python functions in the eval loop and steals all the arguments in the call from the caller for
performance.
2021-10-09 16:51:30 +01:00
Mark Shannon a7252f88d3
bpo-40116: Add insertion order bit-vector to dict values to allow dicts to share keys more freely. (GH-28520) 2021-10-06 13:19:53 +01:00
Mark Shannon f6eafe18c0
Normalize jumps in compiler. All forward jumps to use JUMP_FORWARD. (GH-28755) 2021-10-06 13:05:45 +01:00
Mark Shannon bd627eb7ed
bpo-43760: Check for tracing using 'bitwise or' instead of branch in dispatch. (GH-28723) 2021-10-05 11:01:11 +01:00
Pablo Galindo Salgado 07cf10bafc
Fix compiler warning in ceval.c regarding signed comparison (GH-28716) 2021-10-04 12:13:46 +01:00
Serhiy Storchaka 60b9e040c9
bpo-45355: Use sizeof(_Py_CODEUNIT) instead of literal 2 for the size of the code unit (GH-28711) 2021-10-03 21:22:42 +03:00
Mark Shannon cd760ceb67
Fix a couple of compiler warnings. (GH-28677) 2021-10-01 15:44:19 +01:00
Ken Jin 70bed6f993
bpo-45107: Make LOAD_METHOD_CLASS safer and faster, clean up comments (GH-28177)
* Improve comments

* Check cls is a type, remove dict calculation
2021-09-17 18:47:36 +08:00
Dong-hee Na e6497fe698
bpo-45045: Optimize mapping patterns of structural pattern matching (GH-28043) 2021-08-30 19:02:32 +09:00
Mark Shannon d3eaf0cc5b
bpo-44945: Specialize BINARY_ADD (GH-27967) 2021-08-27 09:21:01 +01:00
Mark Shannon f9242d50b1
bpo-44990: Change layout of evaluation frames. "Layout B" (GH-27933)
Places the locals between the specials and stack. This is the more "natural" layout for a C struct, makes the code simpler and gives a slight speedup (~1%)
2021-08-25 13:44:20 +01:00
Ken Jin 96346cb6d0
bpo-44889: Specialize LOAD_METHOD with PEP 659 adaptive interpreter (GH-27722)
Adds four new instructions:

* LOAD_METHOD_ADAPTIVE
* LOAD_METHOD_CACHED
* LOAD_METHOD_MODULE
* LOAD_METHOD_CLASS
2021-08-17 15:55:55 +01:00
Mark Shannon 4f51fa9e2d
bpo-44900: Add five superinstructions. (GH-27741)
* LOAD_FAST LOAD_FAST
* STORE_FAST LOAD_FAST
* LOAD_FAST LOAD_CONST
* LOAD_CONST LOAD_FAST
* STORE_FAST STORE_FAST
2021-08-16 12:23:13 +01:00
Irit Katriel 8ac0886091
bpo-44890: collect specialization stats if Py_DEBUG (GH-27731) 2021-08-12 12:15:06 +01:00
Mark Shannon a530a9538f
bpo-44878: Remove loop from interpreter. All dispatching is done by gotos. (GH-27727) 2021-08-12 11:47:38 +01:00
Mark Shannon f66d00fdd7
bpo-44878: Remove the switch from the main interpreter loop when using computed gotos. (GH-27726)
* Refactor dispatch logic to make flow of control clearer. Moves lltrace and dxprofile instrumentation into DISPATCH macro.

* Remove switch in interpreter loop when using computed gotos. There is no need for two nearly-duplicate dispatch tables.
2021-08-11 14:02:11 +01:00
Mark Shannon 3f3d5dcac3
bpo-44878: _PyEval_EvalFrameDefault readability improvements (GH-27725)
* Move a few variable declarations to point of definition.

* Factor out tracing of function entry into helper function.
2021-08-11 11:47:52 +01:00
Mark Shannon c174eafc33
Add missing DISPATCH() (GH-27715) 2021-08-11 09:25:26 +01:00
Mark Shannon c7ea1e3dce
Fix stats for STORE_ATTR specialization. (GH-27708) 2021-08-10 11:40:37 +01:00
Mark Shannon ac75f6bdd4
bpo-44826: Specialize STORE_ATTR (GH-27590)
* Generalize cache names for LOAD_ATTR to allow store and delete specializations.

* Factor out specialization of attribute dictionary access.

* Specialize STORE_ATTR.
2021-08-09 10:40:21 +01:00
Mark Shannon c83919bd63
Add option to write specialization stats to files and script to summarize. (GH-27575)
* Add option to write stats to random file in a directory.

* Add script to summarize stats.
2021-08-04 11:39:52 +01:00
Mark Shannon 2116909b3e
Minor fixes to specialization stats. (GH-27457)
* Use class, not value for fail stats for BINARY_SUBSCR.

* Fix counts for unquickened instructions.
2021-07-29 20:50:03 +01:00
Mark Shannon ae0a2b7562
bpo-44590: Lazily allocate frame objects (GH-27077)
* Convert "specials" array to InterpreterFrame struct, adding f_lasti, f_state and other non-debug FrameObject fields to it.

* Refactor, calls pushing the call to the interpreter upward toward _PyEval_Vector.

* Compute f_back when on thread stack, only filling in value when frame object outlives stack invocation.

* Move ownership of InterpreterFrame in generator from frame object to generator object.

* Do not create frame objects for Python calls.

* Do not create frame objects for generators.
2021-07-26 11:22:16 +01:00
Mark Shannon d09c134178
bpo-44645: Check for interrupts on any potentially backwards edge (GH-27216) 2021-07-19 11:10:21 +01:00
Pablo Galindo Salgado c90c591e51
Revert "bpo-44645: Check for interrupts on any potentially backwards edge. (GH-27167)" (#27194)
This reverts commit 000e70ad52.
2021-07-16 19:05:47 +02:00
Mark Shannon 000e70ad52
bpo-44645: Check for interrupts on any potentially backwards edge. (GH-27167) 2021-07-16 10:59:31 +01:00
Pablo Galindo Salgado 4cb7263f0c
Remove sys._deactivate_opcache() now that is not needed (GH-27154) 2021-07-15 14:43:59 +01:00
Irit Katriel 641345d636
bpo-26280: Port BINARY_SUBSCR to PEP 659 adaptive interpreter (GH-27043) 2021-07-15 13:13:12 +01:00
Mark Shannon da6414f0ac
bpo-44570: Fix line tracing for forwards jumps to duplicated tails (GH-27068) 2021-07-08 19:21:09 +01:00
Mark Shannon 514f76bbac
bpo-44581: Don't execute quickened instructions if tracing is on (GH-27064) 2021-07-08 13:33:13 +01:00
Gabriele N. Tornetta 2f180ce2cb
bpo-44530: Add co_qualname field to PyCodeObject (GH-26941) 2021-07-07 12:21:51 +01:00
Serhiy Storchaka 20a88004ba
bpo-12022: Change error type for bad objects in "with" and "async with" (GH-26809)
A TypeError is now raised instead of an AttributeError in
"with" and "async with" statements for objects which do not
support the context manager or asynchronous context manager
protocols correspondingly.
2021-06-29 11:27:04 +03:00
Mark Shannon c3f52b4d70
bpo-44486: Make sure that modules always have a dictionary. (GH-26847)
* Make sure that modules always have a dictionary.
2021-06-23 10:00:43 +01:00
Pablo Galindo 06cda808f1
bpo-44472: Fix ltrace functionality when exceptions are raised (GH-26822) 2021-06-21 16:23:53 +01:00
Mark Shannon fb68791a26
bpo-44337: Improve LOAD_ATTR specialization (GH-26759)
* Specialize obj.__class__ with LOAD_ATTR_SLOT

* Specialize instance attribute lookup with attribute on class, provided attribute on class is not an overriding descriptor.

* Add stat for how many times the unquickened instruction has executed.
2021-06-21 11:49:21 +01:00
Mark Shannon 0982ded179
bpo-44032: Move pointer to code object from frame-object to frame specials array. (GH-26771) 2021-06-18 11:00:29 +01:00
Eric Snow ac38a9f2df
bpo-43693: Eliminate unused "fast locals". (gh-26587)
Currently, if an arg value escapes (into the closure for an inner function) we end up allocating two indices in the fast locals even though only one gets used.  Additionally, using the lower index would be better in some cases, such as with no-arg `super()`.  To address this, we update the compiler to fix the offsets so each variable only gets one "fast local".  As a consequence, now some cell offsets are interspersed with the locals (only when an arg escapes to an inner function).

https://bugs.python.org/issue43693
2021-06-15 16:35:25 -06:00
Mark Shannon 358aa6197c
Remove accidentally duplicated STAT_INC (GH-26718) 2021-06-14 13:38:16 +01:00
Mark Shannon eecbc7c390
bpo-44338: Port LOAD_GLOBAL to PEP 659 adaptive interpreter (GH-26638)
* Add specializations of LOAD_GLOBAL.

* Add more stats.

* Remove old opcache; it is no longer used.

* Add NEWS
2021-06-14 11:04:09 +01:00
Mark Shannon 54cb63863f
bpo-44348: Move trace-info to thread-state (GH-26623)
* Move trace-info to thread state.

* Correct output for pdb when turning on tracing in middle of line
2021-06-10 08:46:59 +01:00
Mark Shannon e117c02837
bpo-44337: Port LOAD_ATTR to PEP 659 adaptive interpreter (GH-26595)
* Specialize LOAD_ATTR with  LOAD_ATTR_SLOT and LOAD_ATTR_SPLIT_KEYS

* Move dict-common.h to internal/pycore_dict.h

* Add LOAD_ATTR_WITH_HINT specialized opcode.

* Quicken in function if loopy

* Specialize LOAD_ATTR for module attributes.

* Add specialization stats
2021-06-10 08:46:01 +01:00
Eric Snow 3e1c7167d8
bpo-43693: Un-revert commit f3fa63e. (#26609)
This was reverted in GH-26596 (commit 6d518bb) due to some bad memory accesses.

* Add the MAKE_CELL opcode. (gh-26396)

The memory accesses have been fixed.

https://bugs.python.org/issue43693
2021-06-08 16:01:34 -06:00
Pablo Galindo 3fe921cd49
Revert "bpo-43693: Add the MAKE_CELL opcode and interleave fast locals offsets. (gh-26396)" (GH-26597)
This reverts commit 631f9938b1.
2021-06-08 13:17:55 +01:00
Eric Snow 631f9938b1
bpo-43693: Add the MAKE_CELL opcode and interleave fast locals offsets. (gh-26396)
This moves logic out of the frame initialization code and into the compiler and eval loop.  Doing so simplifies the runtime code and allows us to optimize it better.

https://bugs.python.org/issue43693
2021-06-07 16:52:00 -06:00
Eric Snow 2ab27c4af4
bpo-43693: Un-revert commits 2c1e258 and b2bf2bc. (gh-26577)
These were reverted in gh-26530 (commit 17c4edc) due to refleaks.

* 2c1e258 - Compute deref offsets in compiler (gh-25152)
* b2bf2bc - Add new internal code objects fields: co_fastlocalnames and co_fastlocalkinds. (gh-26388)

This change fixes the refleaks.

https://bugs.python.org/issue43693
2021-06-07 12:22:26 -06:00
Mark Shannon 001eb520b5
bpo-44187: Quickening infrastructure (GH-26264)
* Add co_firstinstr field to code object.

* Implement barebones quickening.

* Use non-quickened bytecode when tracing.

* Add NEWS item

* Add new file to Windows build.

* Don't specialize instructions with EXTENDED_ARG.
2021-06-07 18:38:06 +01:00
Pablo Galindo 17c4edc4e0
bpo-43693: Revert commits 2c1e2583fd and b2bf2bc1ec (GH-26530)
* Revert "bpo-43693: Compute deref offsets in compiler (gh-25152)"

This reverts commit b2bf2bc1ec.

* Revert "bpo-43693: Add new internal code objects fields: co_fastlocalnames and co_fastlocalkinds. (gh-26388)"

This reverts commit 2c1e2583fd.

These two commits are breaking the refleak buildbots.
2021-06-04 17:51:05 +01:00
Mark Shannon b2bf2bc1ec
bpo-43693: Compute deref offsets in compiler (gh-25152)
Merges locals and cells into a single array.
Saves a pointer in the interpreter and means that we don't need the LOAD_CLOSURE opcode any more

https://bugs.python.org/issue43693
2021-06-03 18:03:54 -06:00
Eric Snow 2c1e2583fd
bpo-43693: Add new internal code objects fields: co_fastlocalnames and co_fastlocalkinds. (gh-26388)
A number of places in the code base (notably ceval.c and frameobject.c) rely on mapping variable names to indices in the frame "locals plus" array (AKA fast locals), and thus opargs.  Currently the compiler indirectly encodes that information on the code object as the tuples co_varnames, co_cellvars, and co_freevars.  At runtime the dependent code must calculate the proper mapping from those, which isn't ideal and impacts performance-sensitive sections.  This is something we can easily address in the compiler instead.

This change addresses the situation by replacing internal use of co_varnames, etc. with a single combined tuple of names in locals-plus order, along with a minimal array mapping each to its kind (local vs. cell vs. free).  These two new PyCodeObject fields, co_fastlocalnames and co_fastllocalkinds, are not exposed to Python code for now, but co_varnames, etc. are still available with the same values as before (though computed lazily).

Aside from the (mild) performance impact, there are a number of other benefits:

* there's now a clear, direct relationship between locals-plus and variables
* code that relies on the locals-plus-to-name mapping is simpler
* marshaled code objects are smaller and serialize/de-serialize faster

Also note that we can take this approach further by expanding the possible values in co_fastlocalkinds to include specific argument types (e.g. positional-only, kwargs).  Doing so would allow further speed-ups in _PyEval_MakeFrameVector(), which is where args get unpacked into the locals-plus array.  It would also allow us to shrink marshaled code objects even further.

https://bugs.python.org/issue43693
2021-06-03 10:28:27 -06:00
Irit Katriel 7b1f527d5b
bpo-17792: more accurate error message for unbound variable access exceptions (GH-24976) 2021-06-02 14:09:04 +01:00
Eric Snow 6cc800d363
bpo-43693: Clean up the PyCodeObject fields. (GH-26364)
* Move up the comment about fields using in hashing/comparision.

* Group the fields more clearly.

* Add co_ncellvars and co_nfreevars.

* Raise ValueError if nlocals != len(varnames), rather than aborting.
2021-05-26 20:15:40 +01:00
Mark Shannon b11a951f16
bpo-44032: Move data stack to thread from FrameObject. (GH-26076)
* Remove 'zombie' frames. We won't need them once we are allocating fixed-size frames.

* Add co_nlocalplus field to code object to avoid recomputing size of locals + frees + cells.

* Move locals, cells and freevars out of frame object into separate memory buffer.

* Use per-threadstate allocated memory chunks for local variables.

* Move globals and builtins from frame object to per-thread stack.

* Move (slow) locals frame object to per-thread stack.

* Move internal frame functions to internal header.
2021-05-21 10:57:35 +01:00
Pablo Galindo f13a6a2d67
Remove unused function in ceval.c (GH-26246) 2021-05-19 19:12:19 +01:00
Ken Jin f24afda591
bpo-26110: Add ``CALL_METHOD_KW`` opcode to speedup method calls with keywords (GH-26014)
* Add CALL_METHOD_KW

* Make CALL_METHOD branchless too since it shares the same code

* Place parentheses in STACK_SHRINK
2021-05-15 16:15:23 +01:00
Mark Shannon 117bfd2b71
Remove PyTryblock struct (GH-26059) 2021-05-12 14:04:38 +01:00
Dennis Sweeney 45862f9f5e
Prevent access outside buffer (GH-26012) 2021-05-10 10:10:22 +01:00
Mark Shannon adcd220556
bpo-40222: "Zero cost" exception handling (GH-25729)
"Zero cost" exception handling.

* Uses a lookup table to determine how to handle exceptions.
* Removes SETUP_FINALLY and POP_TOP block instructions, eliminating (most of) the runtime overhead of try statements.
* Reduces the size of the frame object by about 60%.
2021-05-07 15:19:19 +01:00
Brandt Bucher 0ad1e0384c
bpo-43754: Eliminate bindings for partial pattern matches (GH-25229) 2021-05-02 13:02:10 -07:00
Mark Shannon 069e81ab3d
bpo-43977: Use tp_flags for collection matching (GH-25723)
* Add Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING, add to all relevant standard builtin classes.

* Set relevant flags on collections.abc.Sequence and Mapping.

* Use flags in MATCH_SEQUENCE and MATCH_MAPPING opcodes.

* Inherit Py_TPFLAGS_SEQUENCE and Py_TPFLAGS_MAPPING.

* Add NEWS

* Remove interpreter-state map_abc and seq_abc fields.
2021-04-30 09:50:28 +01:00
Pablo Galindo 5bf8bf2267
bpo-38530: Offer suggestions on NameError (GH-25397)
When printing NameError raised by the interpreter, PyErr_Display
will offer suggestions of simmilar variable names in the function that the exception
was raised from:

    >>> schwarzschild_black_hole = None
    >>> schwarschild_black_hole
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    NameError: name 'schwarschild_black_hole' is not defined. Did you mean: schwarzschild_black_hole?
2021-04-14 15:10:33 +01:00
Mark Shannon 9e7b2076fb
bpo-43760: Speed up check for tracing in interpreter dispatch (#25276)
* Remove redundant tracing_possible field from interpreter state.

* Move 'use_tracing' from tstate onto C stack, for fastest possible checking in dispatch logic.

* Add comments stressing the importance stack discipline when dealing with CFrames.

* Add NEWS
2021-04-13 11:08:14 +01:00
Victor Stinner 09bbebea16
bpo-43753: Add Py_Is() and Py_IsNone() functions (GH-25227)
Add the Py_Is(x, y) function to test if the 'x' object is the 'y'
object, the same as "x is y" in Python. Add also the Py_IsNone(),
Py_IsTrue(), Py_IsFalse() functions to test if an object is,
respectively, the None singleton, the True singleton or the False
singleton.
2021-04-11 00:17:39 +02:00
Mark Shannon 28d28e053d
bpo-43760: Streamline dispatch sequence for machines without computed gotos. (GH-25244)
* Do fetch and decode at end of opcode then jump directly to switch.
  Should allow compilers that don't support computed-gotos, specifically MSVC,
  to generate better code.
2021-04-08 11:22:55 +01:00
Mark Shannon b37181e692
bpo-43683: Handle generator entry in bytecode (GH-25138)
* Handle check for sending None to starting generator and coroutine into bytecode.

* Document new bytecode and make it fail gracefully if mis-compiled.
2021-04-06 11:48:59 +01:00
Brandt Bucher f84d5a1136
bpo-42128: __match_args__ can't be a list anymore (GH-25203) 2021-04-05 19:17:08 -07:00
Mark Shannon fcb55c0037
bpo-27129: Use instruction offsets, not byte offsets, in bytecode and internally. (GH-25069)
* Use instruction offset, rather than bytecode offset. Streamlines interpreter dispatch a bit, and removes most EXTENDED_ARGs for jumps.

* Change some uses of PyCode_Addr2Line to PyFrame_GetLineNumber
2021-04-01 16:00:31 +01:00
Mark Shannon d41bdddbed
Move big block of macros out of function to improve readability. (GH-25020) 2021-03-25 12:00:30 +00:00
Mark Shannon 4958f5d69d
Only check evalbreaker after calls and on backwards egdes. Makes sure that __exit__ or __aexit__ is called in with statments in case of interrupt. (GH-18334) 2021-03-24 17:56:12 +00:00
Victor Stinner fc980e0be1
bpo-43541: Fix PyEval_EvalCodeEx() regression (GH-24918)
* Remove an assertion which required CO_NEWLOCALS and CO_OPTIMIZED
  code flags. It is ok to call this function on a code with these
  flags set.
* Fix reference counting on builtins: remove Py_DECREF().
  Fix regression introduced in the
  commit 46496f9d12.

Add also a comment to document that _PyEval_BuiltinsFromGlobals()
returns a borrowed reference.
2021-03-18 14:51:24 +01:00
Mark Shannon 8e1b406275
Make tracing info (bounds, and previous instruction offset) a bit more robust. (GH-24726) 2021-03-05 14:45:50 +00:00
db3l 131d551640
bpo-43271: Re-enable ceval.c optimizations for Windows debug builds (GH-24739)
Partially reverts commit b74396c316

The optimizations are necessary to prevent the interpreter from
crashing in a number of tests involving recursion.
2021-03-03 19:09:48 -08:00
Pablo Galindo af5fa13ef6
bpo-37146: Deactivate opcode cache only when using huntrleaks in the test suite (GH-24643) 2021-02-28 22:41:09 +00:00
Brandt Bucher 145bf269df
bpo-42128: Structural Pattern Matching (PEP 634) (GH-22917)
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Talin <viridia@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-02-26 14:51:55 -08:00
Victor Stinner d5fc998737
bpo-42093: Cleanup _PyDict_GetItemHint() (GH-24582)
* No longer save/restore the current exception. It is no longer used
  with an exception raised.
* No longer clear the current exception on error: it's now up to the
  caller.
2021-02-21 12:02:04 +01:00
Victor Stinner 46496f9d12
bpo-42990: Functions inherit current builtins (GH-24564)
The types.FunctionType constructor now inherits the current builtins
if the globals dictionary has no "__builtins__" key, rather than
using {"None": None} as builtins: same behavior as eval() and exec()
functions.

Defining a function with "def function(...): ..." in Python is not
affected, globals cannot be overriden with this syntax: it also
inherits the current builtins.

PyFrame_New(), PyEval_EvalCode(), PyEval_EvalCodeEx(),
PyFunction_New() and PyFunction_NewWithQualName() now inherits the
current builtins namespace if the globals dictionary has no
"__builtins__" key.

* Add _PyEval_GetBuiltins() function.
* _PyEval_BuiltinsFromGlobals() now uses _PyEval_GetBuiltins() if
  builtins cannot be found in globals.
* Add tstate parameter to _PyEval_BuiltinsFromGlobals().
2021-02-20 15:17:18 +01:00
Victor Stinner bcb094b41f
bpo-43268: Pass interp rather than tstate to internal functions (GH-24580)
Pass the current interpreter (interp) rather than the current Python
thread state (tstate) to internal functions which only use the
interpreter.

Modified functions:

* _PyXXX_Fini() and _PyXXX_ClearFreeList() functions
* _PyEval_SignalAsyncExc(), make_pending_calls()
* _PySys_GetObject(), sys_set_object(), sys_set_object_id(), sys_set_object_str()
* should_audit(), set_flags_from_config(), make_flags()
* _PyAtExit_Call()
* init_stdio_encoding()
* etc.
2021-02-19 15:10:45 +01:00
Victor Stinner a486054b24
bpo-43270: Remove private _PyErr_OCCURRED() macro (GH-24579)
Remove the private _PyErr_OCCURRED() macro: use the public
PyErr_Occurred() function instead.

CPython internals must use the internal _PyErr_Occurred(tstate)
function instead: it is the most efficient way to check if an
exception was raised.
2021-02-19 15:08:54 +01:00
Victor Stinner 101bf69ff1
bpo-43268: _Py_IsMainInterpreter() now expects interp (GH-24577)
The _Py_IsMainInterpreter() function now expects interp rather than
tstate.
2021-02-19 13:33:31 +01:00
Victor Stinner 44085a3fc9
bpo-42990: Refactor _PyFrame_New_NoTrack() (GH-24566)
* Refactor _PyFrame_New_NoTrack() and PyFunction_NewWithQualName()
  code.
* PyFrame_New() checks for _PyEval_BuiltinsFromGlobals() failure.
* Fix a ref leak in _PyEval_BuiltinsFromGlobals() error path.
* Complete PyFunction_GetModule() documentation: it returns a
  borrowed reference and it can return NULL.
* Move _PyEval_BuiltinsFromGlobals() definition to the internal C
  API.
* PyFunction_NewWithQualName() uses _Py_IDENTIFIER() API for the
  "__name__" string to make it compatible with subinterpreters.
2021-02-18 19:20:16 +01:00
Dong-hee Na 3cf0833f42
bpo-43152: Update assert statement to remove unused warning (GH-24473) 2021-02-14 15:54:39 +09:00
Mark Shannon 0332e569c1
bpo-42990: Further refactoring of PyEval_ functions. (GH-24368)
* Further refactoring of PyEval_EvalCode and friends. Break into make-frame, and eval-frame parts.

* Simplify function vector call using new _PyEval_Vector.

* Remove unused internal functions: _PyEval_EvalCodeWithName and _PyEval_EvalCode.

* Don't use legacy function PyEval_EvalCodeEx.
2021-02-01 10:42:03 +00:00
Guido van Rossum 5c5a938573
bpo-42927: Inline cache for attributes defined with '__slots__' (#24216) 2021-01-29 18:02:29 -08:00
Mark Shannon d6c33fbd34
bpo-42990: Introduce 'frame constructor' struct to simplify API for PyEval_CodeEval and friends (GH-24298)
* Introduce 'frame constructor' to simplify API for frame creation

* Embed struct using a macro to conform to PEP 7
2021-01-29 13:24:55 +00:00
Mark Shannon 8643345bdb
Update frame.f_lineno before any call to the (C) tracing function. (GH-24150)
* Minimizes breakage of C extensions and ensures PyFrame_GetLineNumber is efficient.
2021-01-07 16:49:02 +00:00
Mark Shannon ee9f98d9f4
bpo-42823: Fix frame lineno when frame.f_trace is set (GH-24099)
* Add test for frame.f_lineno with/without tracing.

* Make sure that frame.f_lineno is correct regardless of whether frame.f_trace is set.

* Update importlib

* Add NEWS
2021-01-05 12:04:10 +00:00
Mark Shannon bf353f3c2d
bpo-42246: Make sure that `f_lasti`, and thus `f_lineno`, is set correctly after raising or reraising an exception (GH-23803)
* Ensure that f_lasti is set correctly after an exception is raised to conform to PEP 626.

* Update importlib

* Add NEWS.
2020-12-17 13:55:28 +00:00
Mark Shannon 4e7a69bdb6
bpo-42500: Fix recursion in or after except (GH-23568)
* Use counter, rather boolean state when handling soft overflows.
2020-12-02 13:30:55 +00:00
Yurii Karabas 7301979b23
bpo-42202: Store func annotations as a tuple (GH-23316)
Reduce memory footprint and improve performance of loading modules having many func annotations.

  >>> sys.getsizeof({"a":"int","b":"int","return":"int"})
  232
  >>> sys.getsizeof(("a","int","b","int","return","int"))
  88

The tuple is converted into dict on the fly when `func.__annotations__` is accessed first.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2020-11-25 19:43:18 +09:00
Victor Stinner d96a7a8313
bpo-42296: On Windows, fix CTRL+C regression (GH-23257)
On Windows, fix a regression in signal handling which prevented to
interrupt a program using CTRL+C. The signal handler can be run in a
thread different than the Python thread, in which case the test
deciding if the thread can handle signals is wrong.

On Windows, _PyEval_SignalReceived() now always sets eval_breaker to
1 since it cannot test _Py_ThreadCanHandleSignals(), and
  eval_frame_handle_pending() always calls
  _Py_ThreadCanHandleSignals() to recompute eval_breaker.
2020-11-13 14:44:42 +01:00
Mark Shannon 877df851c3
bpo-42246: Partial implementation of PEP 626. (GH-23113)
* Implement new line number table format, as defined in PEP 626.
2020-11-12 09:43:29 +00:00
Pablo Galindo 80449f243b
bpo-42266: Handle monkey-patching descriptors in LOAD_ATTR cache (GH-23157) 2020-11-05 09:23:15 +00:00
Neil Schemenauer 0564aafb71
bpo-42099: Fix reference to ob_type in unionobject.c and ceval (GH-22829)
* Use Py_TYPE() rather than o->ob_type.
2020-10-27 18:55:52 +00:00
Serhiy Storchaka fb5db7ec58
bpo-42006: Stop using PyDict_GetItem, PyDict_GetItemString and _PyDict_GetItemId. (GH-22648)
These functions are considered not safe because they suppress all internal errors
and can return wrong result.  PyDict_GetItemString and _PyDict_GetItemId can
also silence current exception in rare cases.

Remove no longer used _PyDict_GetItemId.
Add _PyDict_ContainsId and rename _PyDict_Contains into
_PyDict_Contains_KnownHash.
2020-10-26 08:43:39 +02:00
Pablo Galindo 109826c850
bpo-42093: Add opcode cache for LOAD_ATTR (GH-22803) 2020-10-20 06:22:44 +01:00
Vladimir Matveev 037245c5ac
bpo-41756: Add PyIter_Send function (#22443) 2020-10-09 17:15:15 -07:00
Serhiy Storchaka dcc54215ac
bpo-41936. Remove macros Py_ALLOW_RECURSION/Py_END_ALLOW_RECURSION (GH-22552) 2020-10-05 12:32:00 +03:00
Victor Stinner bd0a08ea90
bpo-21955: Change my nickname in BINARY_ADD comment (GH-22481) 2020-10-01 18:57:37 +02:00
Mark Shannon 17b5be0c0a
bpo-41670: Remove outdated predict macro invocation. (GH-22026)
Remove PREDICTion of POP_BLOCK from FOR_ITER.
2020-09-29 10:09:13 +01:00
Victor Stinner b7d8d8dbfe
bpo-40941: Fix stackdepth compiler warnings (GH-22377)
Explicitly cast a difference of two pointers to int:
PyFrameObject.f_stackdepth is an int.
2020-09-23 14:07:16 +02:00
Victor Stinner 19c3ac92bf
bpo-41834: Remove _Py_CheckRecursionLimit variable (GH-22359)
Remove the global _Py_CheckRecursionLimit variable: it has been
replaced by ceval.recursion_limit of the PyInterpreterState
structure.

There is no need to keep the variable for the stable ABI, since
Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() were not usable
in Python 3.8 and older: these macros accessed PyThreadState members,
whereas the PyThreadState structure is opaque in the limited C API.
2020-09-23 14:04:57 +02:00
Vladimir Matveev 2b05361bf7
bpo-41756: Introduce PyGen_Send C API (GH-22196)
The new API allows to efficiently send values into native generators
and coroutines avoiding use of StopIteration exceptions to signal 
returns.

ceval loop now uses this method instead of the old "private"
_PyGen_Send C API. This translates to 1.6x increased performance
of 'await' calls in micro-benchmarks.

Aside from CPython core improvements, this new API will also allow 
Cython to generate more efficient code, benefiting high-performance
IO libraries like uvloop.
2020-09-18 18:38:38 -07:00
Mark Shannon cb9879b948
bpo-40941: Unify implicit and explicit state in the frame and generator objects into a single value. (GH-20803)
* Merge gen and frame state variables into one.

* Replace stack pointer with depth in PyFrameObject. Makes code easier to read and saves a word of memory.
2020-07-17 11:44:23 +01:00
Victor Stinner 384621c42f
bpo-41078: Rename pycore_tupleobject.h to pycore_tuple.h (GH-21056) 2020-06-22 17:27:35 +02:00
Dong-hee Na 33faf5c4f4
bpo-40925: Remove unused stack macro SET_VALUE (GH-20783) 2020-06-11 10:39:18 +01:00
Victor Stinner 1bcc32f062
bpo-39465: Use _PyInterpreterState_GET() (GH-20788)
Replace _PyThreadState_GET() with _PyInterpreterState_GET() in:

* get_small_int()
* gcmodule.c: add also get_gc_state() function
* _PyTrash_deposit_object()
* _PyTrash_destroy_chain()
* warnings_get_state()
* Py_GetRecursionLimit()

Cleanup listnode.c: add 'parser' variable.
2020-06-10 20:08:26 +02:00
Victor Stinner 232dda6cbc
bpo-40679: Fix _PyEval_EvalCode() crash if qualname is NULL (GH-20615)
If name is NULL, name is now set to co->co_name.
If qualname is NULL, qualname is now set to name.

qualname must not be NULL: it is used to build error messages.

Cleanup also the code: declare variables where they are initialized.
Rename "name" local variables to "varname" to avoid overriding "name"
parameter.
2020-06-04 15:19:02 +02:00
Victor Stinner 317bab0bf6
PyOS_AfterFork_Child() pass tstate to _PyEval_ReInitThreads() (GH-20598) 2020-06-02 18:44:54 +02:00
Victor Stinner 26881c8fae
PyOS_AfterFork_Child() uses PyStatus (GH-20596)
PyOS_AfterFork_Child() helper functions now return a PyStatus:
PyOS_AfterFork_Child() is now responsible to handle errors.

* Move _PySignal_AfterFork() to the internal C API
* Add #ifdef HAVE_FORK on _PyGILState_Reinit(), _PySignal_AfterFork()
  and _PyInterpreterState_DeleteExceptMain().
2020-06-02 15:51:37 +02:00
Victor Stinner 3026cad59b
bpo-40826: Add _Py_EnsureTstateNotNULL() macro (GH-20571)
Add _Py_EnsureTstateNotNULL(tstate) macro: call Py_FatalError() if
tstate is NULL, the error message contains the current function name.
2020-06-01 16:02:40 +02:00
Dennis Sweeney b5cc2089cc
bpo-40679: Use the function's qualname in certain TypeErrors (GH-20236)
Patch by Dennis Sweeney.
2020-05-22 13:40:17 -07:00
Chris Jerdonek 4a12d12186
Fix typo in code comment in main_loop label. (GH-20068) 2020-05-14 19:25:45 -07:00
scoder 4c9ea093cd
bpo-38787: Add PyCFunction_CheckExact() macro for exact type checks (GH-20024)
… now that we allow subtypes of PyCFunction.

Also add PyCMethod_CheckExact() and PyCMethod_Check() for checks against the PyCMethod subtype.
2020-05-12 07:12:41 -07:00
Victor Stinner 4804b5b3df
bpo-39465: Don't access directly _Py_Identifier members (GH-20043)
* Replace id->object with _PyUnicode_FromId(&id)
* Use _Py_static_string_init(str) macro to initialize statically
  name_op in typeobject.c.
2020-05-12 01:43:38 +02:00
Victor Stinner 7be4e350aa
bpo-40513: Per-interpreter GIL (GH-19943)
In the experimental isolated subinterpreters build mode, the GIL is
now per-interpreter.

Move gil from _PyRuntimeState.ceval to PyInterpreterState.ceval.

new_interpreter() always get the config from the main interpreter.
2020-05-05 20:27:47 +02:00
Victor Stinner e838a9324c
bpo-40522: _PyThreadState_Swap() sets autoTSSkey (GH-19939)
In the experimental isolated subinterpreters build mode,
_PyThreadState_GET() gets the autoTSSkey variable and
_PyThreadState_Swap() sets the autoTSSkey variable.

* Add _PyThreadState_GetTSS()
* _PyRuntimeState_GetThreadState() and _PyThreadState_GET()
  return _PyThreadState_GetTSS()
* PyEval_SaveThread() sets the autoTSSkey variable to current Python
  thread state rather than NULL.
* eval_frame_handle_pending() doesn't check that
  _PyThreadState_Swap() result is NULL.
* _PyThreadState_Swap() gets the current Python thread state with
  _PyThreadState_GetTSS() rather than
  _PyRuntimeGILState_GetThreadState().
* PyGILState_Ensure() no longer checks _PyEval_ThreadsInitialized()
  since it cannot access the current interpreter.
2020-05-05 19:56:48 +02:00
Victor Stinner 299b8c61e9
Revert "bpo-40513: Per-interpreter signals pending (GH-19924)" (GH-19932)
This reverts commit 4e01946caf.
2020-05-05 17:40:18 +02:00
Victor Stinner 4e30ed3af0
bpo-40513: Per-interpreter recursion_limit (GH-19929)
Move recursion_limit member from _PyRuntimeState.ceval to
PyInterpreterState.ceval.

* Py_SetRecursionLimit() now only sets _Py_CheckRecursionLimit
  of ceval.c if the current Python thread is part of the main
  interpreter.
* Inline _Py_MakeEndRecCheck() into _Py_LeaveRecursiveCall().
* Convert _Py_RecursionLimitLowerWaterMark() macro into a static
  inline function.
2020-05-05 16:52:52 +02:00
Victor Stinner 0b1e3307e2
bpo-40513: Per-interpreter gil_drop_request (GH-19927)
Move gil_drop_request member from _PyRuntimeState.ceval to
PyInterpreterState.ceval.
2020-05-05 16:14:31 +02:00
Victor Stinner 4e01946caf
bpo-40513: Per-interpreter signals pending (GH-19924)
Move signals_pending from _PyRuntime.ceval to
PyInterpreterState.ceval.
2020-05-05 15:43:37 +02:00
Victor Stinner 6d86a2331e
bpo-40429: PyFrame_GetCode() result cannot be NULL (GH-19772)
Add frame_nslots() to factorize duplicate code.
2020-04-29 00:56:58 +02:00
Victor Stinner 25104949a5
bpo-40048: Fix _PyCode_InitOpcache() error path (GH-19691)
If _PyCode_InitOpcache() fails in _PyEval_EvalFrameDefault(), use
"goto exit_eval_frame;" rather than "return NULL;" to exit the
function in a consistent state. For example, tstate->frame is now
reset properly.
2020-04-24 02:43:18 +02:00
Victor Stinner 4a21e57fe5
bpo-40268: Remove unused structmember.h includes (GH-19530)
If only offsetof() is needed: include stddef.h instead.

When structmember.h is used, add a comment explaining that
PyMemberDef is used.
2020-04-15 02:35:41 +02:00
Victor Stinner 62183b8d6d
bpo-40268: Remove explicit pythread.h includes (#19529)
Remove explicit pythread.h includes: it is always included
by Python.h.
2020-04-15 02:04:42 +02:00
Victor Stinner e560f90602
bpo-40268: Move struct _gc_runtime_state to pycore_gc.h (GH-19515) 2020-04-14 18:30:41 +02:00
Dong-hee Na 62f75fe3dd
bpo-40232: Update PyOS_AfterFork_Child() to use _PyThread_at_fork_reinit() (GH-19450) 2020-04-15 01:16:24 +09:00
Victor Stinner e5014be049
bpo-40268: Remove a few pycore_pystate.h includes (GH-19510) 2020-04-14 17:52:15 +02:00
Victor Stinner 81a7be3fa2
bpo-40268: Rename _PyInterpreterState_GET_UNSAFE() (GH-19509)
Rename _PyInterpreterState_GET_UNSAFE() to _PyInterpreterState_GET()
for consistency with _PyThreadState_GET() and to have a shorter name
(help to fit into 80 columns).

Add also "assert(tstate != NULL);" to the function.
2020-04-14 15:14:01 +02:00
Victor Stinner b54a99d643
bpo-40082: trip_signal() uses the main interpreter (GH-19441)
Fix the signal handler: it now always uses the main interpreter,
rather than trying to get the current Python thread state.

The following function now accepts an interpreter, instead of a
Python thread state:

* _PyEval_SignalReceived()
* _Py_ThreadCanHandleSignals()
* _PyEval_AddPendingCall()
* COMPUTE_EVAL_BREAKER()
* SET_GIL_DROP_REQUEST(), RESET_GIL_DROP_REQUEST()
* SIGNAL_PENDING_CALLS(), UNSIGNAL_PENDING_CALLS()
* SIGNAL_PENDING_SIGNALS(), UNSIGNAL_PENDING_SIGNALS()
* SIGNAL_ASYNC_EXC(), UNSIGNAL_ASYNC_EXC()

Py_AddPendingCall() now uses the main interpreter if it fails to the
current Python thread state.

Convert _PyThreadState_GET() and PyInterpreterState_GET_UNSAFE()
macros to static inline functions.
2020-04-08 23:35:05 +02:00
Victor Stinner cfc3c2f8b3
bpo-37127: Remove _pending_calls.finishing (GH-19439) 2020-04-08 22:10:53 +02:00
Victor Stinner dda5d6e071
bpo-40226: PyInterpreterState_Delete() deletes pending calls (GH-19436)
PyInterpreterState_New() is now responsible to create pending calls,
PyInterpreterState_Delete() now deletes pending calls.

* Rename _PyEval_InitThreads() to _PyEval_InitGIL() and rename
  _PyEval_InitGIL() to _PyEval_FiniGIL().
* _PyEval_InitState() and PyEval_FiniState() now create and delete
  pending calls. _PyEval_InitState() now returns -1 on memory
  allocation failure.
* Add init_interp_create_gil() helper function: code shared by
  Py_NewInterpreter() and Py_InitializeFromConfig().
* init_interp_create_gil() now also calls _PyEval_FiniGIL(),
  _PyEval_InitGIL() and _PyGILState_Init() in subinterpreters, but
  these functions now do nothing when called from a subinterpreter.
2020-04-08 17:54:59 +02:00
Victor Stinner a15e260b70
bpo-40170: Add _PyIndex_Check() internal function (GH-19426)
Add _PyIndex_Check() function to the internal C API: fast inlined
verson of PyIndex_Check().

Add Include/internal/pycore_abstract.h header file.

Replace PyIndex_Check() with _PyIndex_Check() in C files of Objects
and Python subdirectories.
2020-04-08 02:01:56 +02:00
Victor Stinner 1c1e68cf3e
bpo-38644: Use _PySys_Audit(): pass tstate explicitly (GH-19183)
Add the dependency to tstate more explicit.
2020-03-27 15:11:45 +01:00
Victor Stinner 728189884e
bpo-38644: Pass tstate explicitly in signalmodule.c (GH-19184)
PyOS_InterruptOccurred() now checks _Py_ThreadCanHandleSignals()
before checking if SIGINT is tripped.
2020-03-26 22:28:11 +01:00
Zackery Spytz 79ceccd1ec
bpo-38410: Properly handle PySys_Audit() failures (GH-16657) 2020-03-26 12:11:13 +00:00
Victor Stinner 6723e933c4
bpo-39946: Remove _PyThreadState_GetFrame (GH-19094)
Remove _PyRuntime.getframe hook and remove _PyThreadState_GetFrame
macro which was an alias to _PyRuntime.getframe. They were only
exposed by the internal C API. Remove also PyThreadFrameGetter type.
2020-03-20 17:46:56 +01:00
Victor Stinner d83168854e
bpo-40010: Optimize pending calls in multithreaded applications (GH-19091)
If a thread different than the main thread schedules a pending call
(Py_AddPendingCall()), the bytecode evaluation loop is no longer
interrupted at each bytecode instruction to check for pending calls
which cannot be executed. Only the main thread can execute pending
calls.

Previously, the bytecode evaluation loop was interrupted at each
instruction until the main thread executes pending calls.

* Add _Py_ThreadCanHandlePendingCalls() function.
* SIGNAL_PENDING_CALLS() now only sets eval_breaker to 1 if the
  current thread can execute pending calls. Only the main thread can
  execute pending calls.
2020-03-20 14:50:35 +01:00
Victor Stinner d2a8e5b42c
bpo-40010: COMPUTE_EVAL_BREAKER() checks for subinterpreter (GH-19087)
COMPUTE_EVAL_BREAKER() now also checks if the Python thread state
belongs to the main interpreter. Don't break the evaluation loop if
there are pending signals but the Python thread state it belongs to a
subinterpeter.

* Add _Py_IsMainThread() function.
* Add _Py_ThreadCanHandleSignals() function.
2020-03-20 13:38:58 +01:00
Victor Stinner da2914db4b
bpo-40010: Pass tstate to ceval GIL functions (GH-19077)
* Add eval_frame_handle_pending() helper function: cold code path.
* Fix PyEval_ReleaseLock(): don't dereference tstate if it's NULL.
2020-03-20 09:29:08 +01:00
Victor Stinner 5a3a71dddb
bpo-40010: Optimize signal handling in multithreaded applications (GH-19067)
If a thread different than the main thread gets a signal, the
bytecode evaluation loop is no longer interrupted at each bytecode
instruction to check for pending signals which cannot be handled.
Only the main thread of the main interpreter can handle signals.

Previously, the bytecode evaluation loop was interrupted at each
instruction until the main thread handles signals.

Changes:

* COMPUTE_EVAL_BREAKER() and SIGNAL_PENDING_SIGNALS() no longer set
  eval_breaker to 1 if the current thread cannot handle signals.
* take_gil() now always recomputes eval_breaker.
2020-03-19 17:40:12 +01:00
Victor Stinner 50e6e99178
bpo-39984: Move pending calls to PyInterpreterState (GH-19066)
If Py_AddPendingCall() is called in a subinterpreter, the function is
now scheduled to be called from the subinterpreter, rather than being
called from the main interpreter.

Each subinterpreter now has its own list of scheduled calls.

* Move pending and eval_breaker fields from _PyRuntimeState.ceval
  to PyInterpreterState.ceval.
* new_interpreter() now calls _PyEval_InitThreads() to create
  pending calls lock.
* Fix Py_AddPendingCall() for subinterpreters. It now calls
  _PyThreadState_GET() which works in a subinterpreter if the
  caller holds the GIL, and only falls back on
  PyGILState_GetThisThreadState() if _PyThreadState_GET()
  returns NULL.
2020-03-19 02:41:21 +01:00
Victor Stinner 8849e5962b
bpo-39984: trip_signal() uses PyGILState_GetThisThreadState() (GH-19061)
bpo-37127, bpo-39984:

* trip_signal() and Py_AddPendingCall() now get the current Python
  thread state using PyGILState_GetThisThreadState() rather than
  _PyRuntimeState_GetThreadState() to be able to get it even if the
  GIL is released.
* _PyEval_SignalReceived() now expects tstate rather than ceval.
* Remove ceval parameter of _PyEval_AddPendingCall(): ceval is now
  get from tstate parameter.
2020-03-18 19:28:53 +01:00
Victor Stinner 56bfdebfb1
bpo-39984: Pass tstate to _PyEval_SignalAsyncExc() (GH-19049)
_PyEval_SignalAsyncExc() and _PyEval_FiniThreads() now expect tstate,
instead of ceval.
2020-03-18 09:26:25 +01:00
Victor Stinner 23ef89db7a
bpo-39984: _PyThreadState_DeleteCurrent() takes tstate (GH-19051)
* _PyThreadState_DeleteCurrent() now takes tstate rather than
  runtime.
* Add ensure_tstate_not_null() helper to pystate.c.
* Add _PyEval_ReleaseLock() function.
* _PyThreadState_DeleteCurrent() now calls
  _PyEval_ReleaseLock(tstate) and frees PyThreadState memory after
  this call, not before.
* PyGILState_Release(): rename "tcur" variable to "tstate".
2020-03-18 02:26:04 +01:00
Victor Stinner d7fabc1162
bpo-39984: Pass tstate to handle_signals() (GH-19050)
handle_signals() and make_pending_calls() now expect tstate rather
than runtime.
2020-03-18 01:56:21 +01:00
Victor Stinner dab8423d22
bpo-39984: Add PyInterpreterState.ceval (GH-19047)
subinterpreters: Move _PyRuntimeState.ceval.tracing_possible to
PyInterpreterState.ceval.tracing_possible: each interpreter now has
its own variable.

Changes:

* Add _ceval_state structure.
* Add PyInterpreterState.ceval field.
* _PyEval_EvalFrameDefault(): add ceval2 variable (struct _ceval_state*).
* Rename _PyEval_Initialize() to _PyEval_InitRuntimeState().
* Add _PyEval_InitState().
* Don't export internal _Py_FinishPendingCalls() and
  _PyEval_FiniThreads() functions anymore.
2020-03-17 18:56:44 +01:00
Victor Stinner f6a5850782
bpo-35370: PyEval_SetTrace() logs unraisable error (GH-18977)
If PySys_Audit() fails in PyEval_SetProfile() or PyEval_SetTrace(),
log the error as an unraisable exception.
2020-03-16 17:41:44 +01:00
Victor Stinner 309d7cc5df
bpo-35370: Add _PyEval_SetTrace() function (GH-18975)
* sys.settrace(), sys.setprofile() and _lsprof.Profiler.enable() now
  properly report PySys_Audit() error if "sys.setprofile" or
  "sys.settrace" audit event is denied.
* Add _PyEval_SetProfile() and _PyEval_SetTrace() function: similar
  to PyEval_SetProfile() and PyEval_SetTrace() but take a tstate
  parameter and return -1 on error.
* Add _PyObject_FastCallTstate() function.
2020-03-13 16:39:12 +01:00
Victor Stinner 0b72b23fb0
bpo-38500: Add _PyInterpreterState_SetEvalFrameFunc() (GH-17340)
PyInterpreterState.eval_frame function now requires a tstate (Python
thread state) parameter.

Add private functions to the C API to get and set the frame
evaluation function:

* Add tstate parameter to _PyFrameEvalFunction function type.
* Add _PyInterpreterState_GetEvalFrameFunc() and
  _PyInterpreterState_SetEvalFrameFunc() functions.
* Add tstate parameter to _PyEval_EvalFrameDefault().
2020-03-12 23:18:39 +01:00
Victor Stinner b4698ecfdb
bpo-39877: Deprecate PyEval_InitThreads() (GH-18892)
Deprecated PyEval_InitThreads() and PyEval_ThreadsInitialized().
Calling PyEval_InitThreads() now does nothing.
2020-03-10 01:28:54 +01:00
Victor Stinner 175a704abf
bpo-39877: PyGILState_Ensure() don't call PyEval_InitThreads() (GH-18891)
PyGILState_Ensure() doesn't call PyEval_InitThreads() anymore when a
new Python thread state is created. The GIL is created by
Py_Initialize() since Python 3.7, it's not needed to call
PyEval_InitThreads() explicitly.

Add an assertion to ensure that the GIL is already created.
2020-03-10 00:37:48 +01:00
Victor Stinner 85f5a69ae1
bpo-39877: Refactor take_gil() function (GH-18885)
* Remove ceval parameter of take_gil(): get it from tstate.
* Move exit_thread_if_finalizing() call inside take_gil(). Replace
  exit_thread_if_finalizing() with tstate_must_exit(): the caller is
  now responsible to call PyThread_exit_thread().
* Move is_tstate_valid() assertion inside take_gil(). Remove
  is_tstate_valid(): inline code into take_gil().
* Move gil_created() assertion inside take_gil().
2020-03-09 22:12:04 +01:00
Victor Stinner 111e4ee52a
bpo-39877: Py_Initialize() pass tstate to PyEval_InitThreads() (GH-18884) 2020-03-09 21:24:14 +01:00
Victor Stinner 3225b9f973
bpo-39877: Remove useless PyEval_InitThreads() calls (GH-18883)
Py_Initialize() calls PyEval_InitThreads() since Python 3.7. It's no
longer needed to call it explicitly.
2020-03-09 20:56:57 +01:00
Victor Stinner eb4e2ae2b8
bpo-39877: Fix PyEval_RestoreThread() for daemon threads (GH-18811)
* exit_thread_if_finalizing() does now access directly _PyRuntime
  variable, rather than using tstate->interp->runtime since tstate
  can be a dangling pointer after Py_Finalize() has been called.
* exit_thread_if_finalizing() is now called *before* calling
  take_gil(). _PyRuntime.finalizing is an atomic variable,
  we don't need to hold the GIL to access it.
* Add ensure_tstate_not_null() function to check that tstate is not
  NULL at runtime. Check tstate earlier. take_gil() does not longer
  check if tstate is NULL.

Cleanup:

* PyEval_RestoreThread() no longer saves/restores errno: it's already
  done inside take_gil().
* PyEval_AcquireLock(), PyEval_AcquireThread(),
  PyEval_RestoreThread() and _PyEval_EvalFrameDefault() now check if
  tstate is valid with the new is_tstate_valid() function which uses
  _PyMem_IsPtrFreed().
2020-03-08 11:57:45 +01:00
Victor Stinner 9e5d30cc99
bpo-39882: Py_FatalError() logs the function name (GH-18819)
The Py_FatalError() function is replaced with a macro which logs
automatically the name of the current function, unless the
Py_LIMITED_API macro is defined.

Changes:

* Add _Py_FatalErrorFunc() function.
* Remove the function name from the message of Py_FatalError() calls
  which included the function name.
* Update tests.
2020-03-07 00:54:20 +01:00
Victor Stinner 7b3c252dc7
bpo-39877: _PyRuntimeState.finalizing becomes atomic (GH-18816)
Convert _PyRuntimeState.finalizing field to an atomic variable:

* Rename it to _finalizing
* Change its type to _Py_atomic_address
* Add _PyRuntimeState_GetFinalizing() and _PyRuntimeState_SetFinalizing()
  functions
* Remove _Py_CURRENTLY_FINALIZING() function: replace it with testing
  directly _PyRuntimeState_GetFinalizing() value

Convert _PyRuntimeState_GetThreadState() to static inline function.
2020-03-07 00:24:23 +01:00
Andy Lester dffe4c0709
bpo-39573: Finish converting to new Py_IS_TYPE() macro (GH-18601) 2020-03-04 14:15:20 +01:00
Denis Chernikov baf29b2216
Reuse identifier of PREDICT macros as PREDICT_ID (GH-17155)
In function `_PyEval_EvalFrameDefault`, macros PREDICT and PREDICTED use the same identifier creation scheme, which may be shared between them, reducing code repetition, and do ensure that the same identifier is generated.
2020-02-21 10:17:50 +01:00
Andy Lester e6be9b59a9
closes bpo-39605: Fix some casts to not cast away const. (GH-18453)
gcc -Wcast-qual turns up a number of instances of casting away constness of pointers. Some of these can be safely modified, by either:

Adding the const to the type cast, as in:

-    return _PyUnicode_FromUCS1((unsigned char*)s, size);
+    return _PyUnicode_FromUCS1((const unsigned char*)s, size);

or, Removing the cast entirely, because it's not necessary (but probably was at one time), as in:

-    PyDTrace_FUNCTION_ENTRY((char *)filename, (char *)funcname, lineno);
+    PyDTrace_FUNCTION_ENTRY(filename, funcname, lineno);

These changes will not change code, but they will make it much easier to check for errors in consts
2020-02-11 18:28:35 -08:00
Petr Viktorin ffd9753a94
bpo-39245: Switch to public API for Vectorcall (GH-18460)
The bulk of this patch was generated automatically with:

    for name in \
        PyObject_Vectorcall \
        Py_TPFLAGS_HAVE_VECTORCALL \
        PyObject_VectorcallMethod \
        PyVectorcall_Function \
        PyObject_CallOneArg \
        PyObject_CallMethodNoArgs \
        PyObject_CallMethodOneArg \
    ;
    do
        echo $name
        git grep -lwz _$name | xargs -0 sed -i "s/\b_$name\b/$name/g"
    done

    old=_PyObject_FastCallDict
    new=PyObject_VectorcallDict
    git grep -lwz $old | xargs -0 sed -i "s/\b$old\b/$new/g"

and then cleaned up:

- Revert changes to in docs & news
- Revert changes to backcompat defines in headers
- Nudge misaligned comments
2020-02-11 17:46:57 +01:00
Victor Stinner 60ac6ed557
bpo-39573: Use Py_SET_SIZE() function (GH-18402)
Replace direct acccess to PyVarObject.ob_size with usage of
the Py_SET_SIZE() function.
2020-02-07 23:18:08 +01:00
Victor Stinner a102ed7d2f
bpo-39573: Use Py_TYPE() macro in Python and Include directories (GH-18391)
Replace direct access to PyObject.ob_type with Py_TYPE().
2020-02-07 02:24:48 +01:00
Hai Shi 46874c26ee
bpo-39487: Merge duplicated _Py_IDENTIFIER identifiers in C code (GH-18254)
Moving repetitive `_Py_IDENTIFIER` instances to a global location helps identify them more easily in regards to sub-interpreter support.
2020-01-30 15:20:25 -08:00
Victor Stinner 17c68b8107
bpo-38631: Replace Py_FatalError() with assert() in ceval.c (GH-18279)
Replace a few Py_FatalError() calls if tstate is NULL with
assert(tstate != NULL) in ceval.c.

PyEval_AcquireThread(), PyEval_ReleaseThread() and
PyEval_RestoreThread() must never be called with a NULL tstate.
2020-01-30 12:20:48 +01:00
David Carlier aabdeb766b bpo-38960: DTrace build fix for FreeBSD. (GH-17451)
DTrace build fix for FreeBSD.

- allowing passing an extra flag as it need to define the arch size.
- casting some probe's arguments.
2020-01-28 13:53:32 +01:00
Victor Stinner 61f4db8c56
bpo-38644: Pass tstate in ceval.c (GH-18222)
Pass explicitly the Python thread state (tstate) in ceval.c.
2020-01-28 03:37:45 +01:00
Mark Shannon 8a4cd700a7
bpo-39320: Handle unpacking of **values in compiler (GH-18141)
* Add DICT_UPDATE and DICT_MERGE bytecodes. Use them for ** unpacking.

* Remove BUILD_MAP_UNPACK and BUILD_MAP_UNPACK_WITH_CALL, as they are now unused.

* Update magic number for ** unpacking opcodes.

* Update dis.rst to incorporate new bytecodes.

* Add blurb entry.
2020-01-27 09:57:45 +00:00
Mark Shannon 13bc13960c
bpo-39320: Handle unpacking of *values in compiler (GH-17984)
* Add three new bytecodes: LIST_TO_TUPLE, LIST_EXTEND, SET_UPDATE. Use them to implement star unpacking expressions.

* Remove four bytecodes BUILD_LIST_UNPACK, BUILD_TUPLE_UNPACK, BUILD_SET_UNPACK and  BUILD_TUPLE_UNPACK_WITH_CALL opcodes as they are now unused.

* Update magic number and dis.rst for new bytecodes.
2020-01-23 09:25:17 +00:00
Géry Ogam 1d1b97ae64 bpo-39048: Look up __aenter__ before __aexit__ in async with (GH-17609)
* Reorder the __aenter__ and __aexit__ checks for async with
* Add assertions for async with body being skipped
* Swap __aexit__ and __aenter__ loading in the documentation
2020-01-14 21:58:29 +10:00
Mark Shannon 9af0e47b17
bpo-39156: Break up COMPARE_OP into four logically distinct opcodes. (GH-17754)
Break up COMPARE_OP into four logically distinct opcodes:
* COMPARE_OP for rich comparisons
* IS_OP for 'is' and 'is not' tests
* CONTAINS_OP for 'in' and 'is not' tests
* JUMP_IF_NOT_EXC_MATCH for checking exceptions in 'try-except' statements.
2020-01-14 10:12:45 +00:00
Victor Stinner 2b1df4592e
bpo-38644: Pass tstate to _Py_FinishPendingCalls() (GH-17990)
_Py_FinishPendingCalls() now expects a tstate argument, instead of a
runtime argument.
2020-01-13 18:46:59 +01:00
Mark Shannon e7c9f4aae1
Cleanup exit code for interpreter. (GH-17756) 2020-01-13 12:51:26 +00:00
Pablo Galindo 4c53e63cc9 bpo-39166: Fix trace of last iteration of async for loops (#17800) 2020-01-10 09:24:22 +00:00
Pablo Galindo 04ec7a1f7a
bpo-39114: Fix tracing of except handlers with name binding (GH-17769)
When producing the bytecode of exception handlers with name binding (like `except Exception as e`) we need to produce a try-finally block to make sure that the name is deleted after the handler is executed to prevent cycles in the stack frame objects. The bytecode associated with this try-finally block does not have source lines associated and it was causing problems when the tracing functionality was running over it.
2020-01-02 11:38:44 +00:00
Mark Shannon fee552669f
Produce cleaner bytecode for 'with' and 'async with' by generating separate code for normal and exceptional paths. (#6641)
Remove BEGIN_FINALLY, END_FINALLY, CALL_FINALLY and POP_FINALLY bytecodes. Implement finally blocks by code duplication.
Reimplement frame.lineno setter using line numbers rather than bytecode offsets.
2019-11-21 09:11:43 +00:00
Victor Stinner 01b1cc12e7
bpo-36710: Add PyInterpreterState.runtime field (GH-17270)
Add PyInterpreterState.runtime field: reference to the _PyRuntime
global variable. This field exists to not have to pass runtime in
addition to tstate to a function.  Get runtime from tstate:
tstate->interp->runtime.

Remove "_PyRuntimeState *runtime" parameter from functions already
taking a "PyThreadState *tstate" parameter.

_PyGC_Init() first parameter becomes "PyThreadState *tstate".
2019-11-20 02:27:56 +01:00
Victor Stinner b5e170f127
bpo-38644: Add _PyEval_EvalCode() (GH-17183)
_PyFunction_Vectorcall() now pass tstate to function calls.
2019-11-16 01:03:22 +01:00
Victor Stinner 4d231bcc77
bpo-38644: Add _PyObject_Call() (GH-17089)
* Add pycore_call.h internal header file.
* Add _PyObject_Call(): PyObject_Call() with tstate
* Add _PyObject_CallNoArgTstate(): _PyObject_CallNoArg() with tstate
* Add _PyObject_FastCallDictTstate(): _PyObject_FastCallDict()
  with tstate
* _PyObject_Call_Prepend() now takes tstate
* Replace _PyObject_FastCall() calls
  with _PyObject_VectorcallTstate() calls
2019-11-14 13:36:21 +01:00
Victor Stinner b9e681261c
bpo-38644: Add _PyEval_EvalFrame() with tstate (GH-17131)
Add _PyEval_EvalFrame() static inline function to get eval_frame from
tstate->interp.
2019-11-14 12:20:46 +01:00
Jeroen Demeyer bf17d41826 bpo-37645: add new function _PyObject_FunctionStr() (GH-14890)
Additional note: the `method_check_args` function in `Objects/descrobject.c` is written in such a way that it applies to all kinds of descriptors. In particular, a future re-implementation of `wrapper_descriptor` could use that code.

CC @vstinner @encukou 


https://bugs.python.org/issue37645



Automerge-Triggered-By: @encukou
2019-11-05 07:48:04 -08:00
Victor Stinner 1726909094
bpo-38644: Pass tstate to _Py_CheckFunctionResult() (GH-17050)
* Add tstate parameter to _Py_CheckFunctionResult()
* Add _PyErr_FormatFromCauseTstate()
* Replace PyErr_XXX(...) with _PyErr_XXX(state, ...)
2019-11-05 01:22:12 +01:00
Victor Stinner be434dc038
bpo-38644: Pass tstate to Py_EnterRecursiveCall() (GH-16997)
* Add _Py_EnterRecursiveCall() and _Py_LeaveRecursiveCall() which
  require a tstate argument.
* Pass tstate to _Py_MakeRecCheck() and  _Py_CheckRecursiveCall().
* Convert Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() macros
  to static inline functions.

_PyThreadState_GET() is the most efficient way to get the tstate, and
so using it with _Py_EnterRecursiveCall() and
_Py_LeaveRecursiveCall() should be a little bit more efficient than
using Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() which use
the "slower" PyThreadState_GET().
2019-11-05 00:51:22 +01:00
Victor Stinner f4b1e3d7c6
bpo-38644: Add Py_EnterRecursiveCall() to the limited API (GH-17046)
Provide Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() as
regular functions for the limited API. Previously, there were defined
as macros, but these macros didn't work with the limited API which
cannot access PyThreadState.recursion_depth field.

Remove _Py_CheckRecursionLimit from the stable ABI.

Add Include/cpython/ceval.h header file.
2019-11-04 19:48:34 +01:00
Brandt Bucher f185a73249 bpo-38310: Predict BUILD_MAP_UNPACK_WITH_CALL -> CALL_FUNCTION_EX. (GH-16467) 2019-09-28 17:12:49 -07:00
Jeroen Demeyer 7a6873cdb1 bpo-37151: remove special case for PyCFunction from PyObject_Call (GH-14684)
bpo-37151: remove special case for PyCFunction from PyObject_Call

Alse, make the undocumented function PyCFunction_Call an alias
of PyObject_Call and deprecate it.
2019-09-11 12:01:01 +01:00
Pablo Galindo a511c7a496 bpo-38090: Fix reference leak in ceval.c (GH-15848) 2019-09-10 14:32:54 +01:00
Anthony Sottile 65366bc8bd bpo-20490: Improve circular import error message (GH-15308) 2019-09-09 08:17:50 -07:00
Joannah Nanjekye 74b662cf20 bpo-15088 : Remove PyGen_NeedsFinalizing() (GH-15702)
Remove PyGen_NeedsFinalizing(): it was not
documented, tested or used anywhere within CPython after
the implementation of PEP 442.
2019-09-06 17:41:38 +02:00
Zackery Spytz ce6a070414 bpo-34880: Add the LOAD_ASSERTION_ERROR opcode. (GH-15073)
Fix assert statement misbehavior if AssertionError is shadowed.
2019-08-25 12:44:09 +03:00
Jeroen Demeyer 0567786d26 bpo-37540: vectorcall: keyword names must be strings (GH-14682)
The fact that keyword names are strings is now part of the vectorcall and `METH_FASTCALL` protocols. The biggest concrete change is that `_PyStack_UnpackDict` now checks that and raises `TypeError` if not.

CC @markshannon @vstinner 


https://bugs.python.org/issue37540
2019-08-16 03:41:27 -07:00
Jeroen Demeyer 59ad110d7a bpo-37547: add _PyObject_CallMethodOneArg (GH-14685) 2019-07-11 17:59:05 +09:00
Jeroen Demeyer 0d722f3cd6 bpo-36974: separate vectorcall functions for each calling convention (GH-13781) 2019-07-05 14:48:24 +02:00
Jeroen Demeyer 196a530e00 bpo-37483: add _PyObject_CallOneArg() function (#14558) 2019-07-04 19:31:34 +09:00
Jeroen Demeyer 469d1a70ce bpo-37484: use _PyObject_Vectorcall for __exit__ (GH-14557) 2019-07-03 19:52:21 +09:00
Jörn Heissler c8a35417db bpo-35224: Reverse evaluation order of key: value in dict comprehensions (GH-14139)
… as proposed in PEP 572; key is now evaluated before value.





https://bugs.python.org/issue35224
2019-06-22 07:40:55 -07:00
Jeroen Demeyer 7e1a9aacff bpo-37151: remove _PyCFunction_FastCallDict (GH-14269) 2019-06-21 00:38:45 +09:00
Jeroen Demeyer b2f94730d9 bpo-37249: add declaration of _PyObject_GetMethod (GH-14015) 2019-06-14 19:37:15 +09:00
Victor Stinner 838f26402d
bpo-36710: Pass explicitly tstate in sysmodule.c (GH-14060)
* Replace global var Py_VerboseFlag with interp->config.verbose.
* Add _PyErr_NoMemory(tstate) function.
* Add tstate parameter to _PyEval_SetCoroutineOriginTrackingDepth()
  and move the function to the internal API.
* Replace _PySys_InitMain(runtime, interp)
  with _PySys_InitMain(runtime, tstate).
2019-06-13 22:41:23 +02:00
Victor Stinner 0fd2c300c2
Revert "bpo-36818: Add PyInterpreterState.runtime field. (gh-13129)" (GH-13795)
This reverts commit 396e0a8d9d.
2019-06-04 03:15:09 +02:00
Inada Naoki eddef861b4 bpo-37146: disable opcache when Py_DEBUG is defined (GH-13787)
--with-pydebug is commonly used to find memory leaks.
But opcache makes it harder.
So disable opcache when Py_DEBUG is defined.
2019-06-04 00:38:09 +02:00
Victor Stinner e225bebc14
Revert "bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (gh-13714)" (GH-13780)
This reverts commit 6a150bcaeb.
2019-06-03 18:14:24 +02:00
Inada Naoki 395420e2a3
bpo-26219: remove unused code (GH-13775)
This code was for deoptimization, which is removed from
PR-12884.
2019-06-03 22:34:15 +09:00
Inada Naoki 91234a1636
bpo-26219: per opcode cache for LOAD_GLOBAL (GH-12884)
This patch implements per opcode cache mechanism, and use it in
only LOAD_GLOBAL opcode.

Based on Yury's opcache3.patch in bpo-26219.
2019-06-03 21:30:58 +09:00
Eric Snow 6a150bcaeb
bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (gh-13714) 2019-06-01 15:39:46 -06:00
Pablo Galindo cd74e66a8c
bpo-37122: Make co->co_argcount represent the total number of positonal arguments in the code object (GH-13726) 2019-06-01 18:08:04 +01:00
Eric Snow 396e0a8d9d
bpo-36818: Add PyInterpreterState.runtime field. (gh-13129)
https://bugs.python.org/issue36818
2019-05-31 21:16:47 -06:00
Jeroen Demeyer 37788bc23f bpo-36974: rename _FastCallKeywords -> _Vectorcall (GH-13653) 2019-05-30 15:11:22 +02:00
Jeroen Demeyer aacc77fbd7 bpo-36974: implement PEP 590 (GH-13185)
Co-authored-by: Jeroen Demeyer <J.Demeyer@UGent.be>
Co-authored-by: Mark Shannon <mark@hotpy.org>
2019-05-29 20:31:52 +02:00
Matthias Bussonnier 3880f263d2 bpo-36933: Remove sys.set_coroutine_wrapper (marked for removal in 3.8) (GH-13577)
It has been documented as deprecated and to be removed in 3.8; 

From a comment on another thread – which I can't find ; leave get_coro_wrapper() for now, but always return `None`.


https://bugs.python.org/issue36933
2019-05-28 00:10:59 -07:00
Victor Stinner 438a12dd9d
bpo-36710: Add tstate parameter in ceval.c (GH-13547)
* Fix a possible reference leak in _PyErr_Print() if exception
  is NULL.
* PyErr_BadInternalCall(): replace PyErr_Format() with _PyErr_SetString().
* Add pycore_pyerrors.h header file.
* New functions:

  * _PyErr_Clear()
  * _PyErr_Fetch()
  * _PyErr_Print()
  * _PyErr_Restore()
  * _PyErr_SetObject()
  * _PyErr_SetString()

* Add 'tstate' parameter to _PyEval_AddPendingCall().
2019-05-24 17:01:38 +02:00
Steve Dower b82e17e626
bpo-36842: Implement PEP 578 (GH-12613)
Adds sys.audit, sys.addaudithook, io.open_code, and associated C APIs.
2019-05-23 08:45:22 -07:00
Victor Stinner d5d9e81ce9
bpo-36728: Remove PyEval_ReInitThreads() from C API (GH-13241)
Remove the PyEval_ReInitThreads() function from the Python C API.
It should not be called explicitly: use PyOS_AfterFork_Child()
instead.

Rename PyEval_ReInitThreads() to _PyEval_ReInitThreads() and add a
'runtime' parameter.
2019-05-13 12:35:37 +02:00
Victor Stinner 09532feeec
bpo-36710: Add 'ceval' local variable to ceval.c (GH-12934)
Add "struct _ceval_runtime_state *ceval = &_PyRuntime.ceval;" local
variables to function to better highlight the dependency on the
global variable _PyRuntime and to point directly to _PyRuntime.ceval
field rather than on the larger _PyRuntime.

Changes:

* Add _PyRuntimeState_GetThreadState(runtime) macro.
* Add _PyEval_AddPendingCall(ceval, ...) and
  _PyThreadState_Swap(gilstate, ...) functions.
* _PyThreadState_GET() macro now calls
  _PyRuntimeState_GetThreadState() using &_PyRuntime.
* Add 'ceval' parameter to COMPUTE_EVAL_BREAKER(),
  SIGNAL_PENDING_SIGNALS(), _PyEval_SignalAsyncExc(),
  _PyEval_SignalReceived() and _PyEval_FiniThreads() macros and
  functions.
* Add 'tstate' parameter to call_function(), do_call_core() and
  do_raise().
* Add 'runtime' parameter to _Py_CURRENTLY_FINALIZING(),
  _Py_FinishPendingCalls() and _PyThreadState_DeleteExcept()
  macros and functions.
* Declare 'runtime', 'tstate', 'ceval' and 'eval_breaker' variables
  as constant.
2019-05-10 23:39:09 +02:00
Pablo Galindo f00828a742
bpo-36851: Clean the frame stack if the execution ends with a return and the stack is not empty (GH-13191) 2019-05-09 16:52:02 +01:00
Eric V. Smith 9a4135e939
bpo-36817: Add f-string debugging using '='. (GH-13123)
If a "=" is specified a the end of an f-string expression, the f-string will evaluate to the text of the expression, followed by '=', followed by the repr of the value of the expression.
2019-05-08 16:28:48 -04:00
Victor Stinner c664b342a4
bpo-36475: Make PyThread_exit_thread with _Py_NO_RETURN (GH-13068) 2019-05-04 11:48:05 -04:00
Pablo Galindo 8c77b8cb91
bpo-36540: PEP 570 -- Implementation (GH-12701)
This commit contains the implementation of PEP570: Python positional-only parameters.

* Update Grammar/Grammar with new typedarglist and varargslist

* Regenerate grammar files

* Update and regenerate AST related files

* Update code object

* Update marshal.c

* Update compiler and symtable

* Regenerate importlib files

* Update callable objects

* Implement positional-only args logic in ceval.c

* Regenerate frozen data

* Update standard library to account for positional-only args

* Add test file for positional-only args

* Update other test files to account for positional-only args

* Add News entry

* Update inspect module and related tests
2019-04-29 13:36:57 +01:00
Victor Stinner 99fcc616d4
Revert "bpo-36356: Destroy the GIL at exit (GH-12453)" (GH613006)
This reverts commit b36e5d627d.
2019-04-29 13:04:07 +02:00
Victor Stinner b36e5d627d
bpo-36356: Destroy the GIL at exit (GH-12453)
* Add _PyEval_FiniThreads2(). _PyEval_FiniThreads() now only clears
  the pending lock, whereas _PyEval_FiniThreads2() destroys the GIL.
* pymain_free() now calls _PyEval_FiniThreads2().
* Py_FinalizeEx() now calls _PyEval_FiniThreads().
2019-04-29 11:15:56 +02:00
Joannah Nanjekye f781d202a2 bpo-36475: Finalize PyEval_AcquireLock() and PyEval_AcquireThread() properly (GH-12667)
PyEval_AcquireLock() and PyEval_AcquireThread() now
terminate the current thread if called while the interpreter is
finalizing, making them consistent with PyEval_RestoreThread(),
Py_END_ALLOW_THREADS, and PyGILState_Ensure().
2019-04-29 10:38:45 +02:00
Andrey b021ba5028 Fix typo in 'tandem' word (GH-12998) (GH-12998) 2019-04-28 21:33:26 -07:00
Victor Stinner 5c75f37d47
bpo-36635: Change pyport.h for Py_BUILD_CORE_MODULE define (GH-12853)
Change PyAPI_FUNC(type), PyAPI_DATA(type) and PyMODINIT_FUNC macros
of pyport.h when Py_BUILD_CORE_MODULE is defined.

The Py_BUILD_CORE_MODULE define must be now be used to build a C
extension as a dynamic library accessing Python internals: export the
PyInit_xxx() function in DLL exports on Windows.

Changes:

* Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE now imply
  Py_BUILD_CORE directy in pyport.h.
* ceval.c compilation now fails with an error if Py_BUILD_CORE is not
  defined, just to ensure that Python is build with the correct
  defines.
* setup.py now compiles _pickle.c with Py_BUILD_CORE_MODULE define.
* setup.py compiles _json.c with Py_BUILD_CORE_MODULE define, rather
  than Py_BUILD_CORE_BUILTIN define
* PCbuild/pythoncore.vcxproj: Add Py_BUILD_CORE_BUILTIN define.
2019-04-17 23:02:26 +02:00
Eric Snow b75b1a3504
bpo-33608: Revert "Factor out a private, per-interpreter _Py_AddPendingCall()." (gh-12806)
This reverts commit f13c5c8b94 (gh-12360).
2019-04-12 10:20:10 -06:00
Eric Snow f13c5c8b94
bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (gh-12360)
This is effectively an un-revert of #11617 and #12024 (reverted in #12159). Portions of those were merged in other PRs (with lower risk) and this represents the remainder. Note that I found 3 different bugs in the original PRs and have fixed them here.
2019-04-12 09:18:16 -06:00
Stefan Krah 027b09c5a1
bpo-36370: Check for PyErr_Occurred() after PyImport_GetModule() (GH-12504) 2019-03-25 21:50:58 +01:00
Victor Stinner a712679a2b
bpo-36333, bpo-36356: Fix _PyEval_FiniThreads() (GH-12432)
_PyEval_FiniThreads() now free the pending lock.
2019-03-19 14:19:38 +01:00
Eric Snow 842a2f07f2
bpo-33608: Deal with pending calls relative to runtime shutdown. (gh-12246) 2019-03-15 15:47:51 -06:00
Victor Stinner e3f4070aee
bpo-33608: Fix PyEval_InitThreads() warning (GH-12346)
The function has no return value.

Fix the following warning on Windows:

    python\ceval.c(180): warning C4098: 'PyEval_InitThreads':
    'void' function returning a value
2019-03-15 16:04:20 +01:00
Eric Snow 8479a3426e
bpo-33608: Make sure locks in the runtime are properly re-created. (gh-12245) 2019-03-08 23:44:33 -07:00
Eric Snow 5be45a6105
bpo-33608: Minor cleanup related to pending calls. (gh-12247) 2019-03-08 22:47:07 -07:00
Eric Snow 7bda9de550
Simplify DISPATCH by hoisting eval_breaker ahead of time. (gh-12243) 2019-03-08 17:25:54 -07:00
Victor Stinner 4d61e6e3b8
Revert: bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617) (GH-12159)
* Revert "bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (#12003)"

This reverts commit bcfa450f21.

* Revert "bpo-33608: Simplify ceval's DISPATCH by hoisting eval_breaker ahead of time. (gh-12062)"

This reverts commit bda918bf65.

* Revert "bpo-33608: Use _Py_AddPendingCall() in _PyCrossInterpreterData_Release(). (gh-12024)"

This reverts commit b05b711a2c.

* Revert "bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617)"

This reverts commit ef4ac967e2.
2019-03-04 14:21:28 +01:00
Eric Snow bda918bf65
bpo-33608: Simplify ceval's DISPATCH by hoisting eval_breaker ahead of time. (gh-12062)
This includes fixes to various _Py_atomic_* macros.
2019-03-01 13:15:45 -07:00
Sergey Fedoseev 234531b446 bpo-36030: Add _PyTuple_FromArray() function (GH-11954) 2019-02-25 17:59:12 +01:00
Serhiy Storchaka a24107b04c
bpo-35459: Use PyDict_GetItemWithError() instead of PyDict_GetItem(). (GH-11112) 2019-02-25 17:59:46 +02:00
Eric Snow ef4ac967e2
bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617)
This involves moving the global "pending calls" state to PyInterpreterState.

https://bugs.python.org/issue33608
2019-02-24 15:40:47 -08:00
Eric Snow 64d6cc826d
bpo-35724: Explicitly require the main interpreter for signal-handling. (GH-11530)
Ensure that the main interpreter is active (in the main thread) for signal-handling operations. This is increasingly relevant as people use subinterpreters more.

https://bugs.python.org/issue35724
2019-02-23 15:40:43 -07:00
Inada Naoki 001fee14e0
bpo-12822: use monotonic clock for condvar if possible (GH-11723) 2019-02-20 10:00:09 +09:00
Serhiy Storchaka f1ec3cefad
bpo-35634: Raise an error when first passed kwargs contains duplicated keys. (GH-11438) 2019-01-12 10:12:24 +02:00
Eric Snow fdf282d609
bpo-35423: Stop using the "pending calls" machinery for signals. (gh-10972)
This change separates the signal handling trigger in the eval loop from the "pending calls" machinery. There is no semantic change and the difference in performance is insignificant.

The change makes both components less confusing. It also eliminates the risk of changes to the pending calls affecting signal handling. This is particularly relevant for some upcoming pending calls changes I have in the works.
2019-01-11 14:26:55 -07:00
Serhiy Storchaka 8905fcc85a
bpo-35454: Fix miscellaneous minor issues in error handling. (#11077)
* bpo-35454: Fix miscellaneous minor issues in error handling.

* Fix a null pointer dereference.
2018-12-11 08:38:03 +02:00
Serhiy Storchaka bb86bf4c4e
bpo-35444: Unify and optimize the helper for getting a builtin object. (GH-11047)
This speeds up pickling of some iterators.

This fixes also error handling in pickling methods when fail to
look up builtin "getattr".
2018-12-11 08:28:18 +02:00
Zackery Spytz 99d56b5356 bpo-35441: Remove dead and buggy code related to PyList_SetItem(). (GH-11033)
In _localemodule.c and selectmodule.c, remove dead code that would
cause double decrefs if run.

In addition, replace PyList_SetItem() with PyList_SET_ITEM() in cases
where a new list is populated and there is no possibility of an error.

In addition, check if the list changed size in the loop in array_array_fromlist().
2018-12-08 16:16:55 +02:00
Victor Stinner ec13b9322d
bpo-35081: Add Include/internal/pycore_tupleobject.h (GH-10705)
Move _PyTuple_ITEMS() to a new header file:
Include/internal/pycore_tupleobject.h
2018-11-25 23:56:17 +01:00
Victor Stinner bcda8f1d42
bpo-35081: Add Include/internal/pycore_object.h (GH-10640)
Move _PyObject_GC_TRACK() and _PyObject_GC_UNTRACK() from
Include/objimpl.h to Include/internal/pycore_object.h.
2018-11-21 22:27:47 +01:00
Victor Stinner 621cebe81b
bpo-35081: Rename internal headers (GH-10275)
Rename Include/internal/ headers:

* pycore_hash.h -> pycore_pyhash.h
* pycore_lifecycle.h -> pycore_pylifecycle.h
* pycore_mem.h -> pycore_pymem.h
* pycore_state.h -> pycore_pystate.h

Add missing headers to Makefile.pre.in and PCbuild:

* pycore_condvar.h.
* pycore_hamt.h
* pycore_pyhash.h
2018-11-12 16:53:38 +01:00
Victor Stinner d17a693fa0
bpo-35199: Add an internal _PyTuple_ITEMS() macro (GH-10434)
* _PyTuple_ITEMS() gives access to the tuple->ob_item field and cast the
  first argument to PyTupleObject*. This internal macro is only usable if
  Py_BUILD_CORE is defined.
* Replace &PyTuple_GET_ITEM(ob, 0) with _PyTuple_ITEMS(ob).
* Replace PyTuple_GET_ITEM(op, 1) with &_PyTuple_ITEMS(ob)[1].
2018-11-09 16:56:48 +01:00
Victor Stinner 50b48572d9
bpo-35081: Add _PyThreadState_GET() internal macro (GH-10266)
If Py_BUILD_CORE is defined, the PyThreadState_GET() macro access
_PyRuntime which comes from the internal pycore_state.h header.
Public headers must not require internal headers.

Move PyThreadState_GET() and _PyInterpreterState_GET_UNSAFE() from
Include/pystate.h to Include/internal/pycore_state.h, and rename
PyThreadState_GET() to _PyThreadState_GET() there.

The PyThreadState_GET() macro of pystate.h is now redefined when
pycore_state.h is included, to use the fast _PyThreadState_GET().

Changes:

* Add _PyThreadState_GET() macro
* Replace "PyThreadState_GET()->interp" with
  _PyInterpreterState_GET_UNSAFE()
* Replace PyThreadState_GET() with _PyThreadState_GET() in internal C
  files (compiled with Py_BUILD_CORE defined), but keep
  PyThreadState_GET() in the public header files.
* _testcapimodule.c: replace PyThreadState_GET() with
  PyThreadState_Get(); the module is not compiled with Py_BUILD_CORE
  defined.
* pycore_state.h now requires Py_BUILD_CORE to be defined.
2018-11-01 01:51:40 +01:00
Victor Stinner 27e2d1f219
bpo-35081: Add pycore_ prefix to internal header files (GH-10263)
* Rename Include/internal/ header files:

  * pyatomic.h -> pycore_atomic.h
  * ceval.h -> pycore_ceval.h
  * condvar.h -> pycore_condvar.h
  * context.h -> pycore_context.h
  * pygetopt.h -> pycore_getopt.h
  * gil.h -> pycore_gil.h
  * hamt.h -> pycore_hamt.h
  * hash.h -> pycore_hash.h
  * mem.h -> pycore_mem.h
  * pystate.h -> pycore_state.h
  * warnings.h -> pycore_warnings.h

* PCbuild project, Makefile.pre.in, Modules/Setup: add the
  Include/internal/ directory to the search paths of header files.
* Update includes. For example, replace #include "internal/mem.h"
  with #include "pycore_mem.h".
2018-11-01 00:52:28 +01:00
Victor Stinner 9204fb8623
bpo-35081: Cleanup pystate.c and pystate.h (GH-10240)
* Remove _PyThreadState_Current
* Replace GET_TSTATE() with PyThreadState_GET()
* Replace GET_INTERP_STATE() with _PyInterpreterState_GET_UNSAFE()
* Replace direct access to _PyThreadState_Current with
  PyThreadState_GET()
* Replace _PyThreadState_Current with
  _PyRuntime.gilstate.tstate_current
* Rename SET_TSTATE() to _PyThreadState_SET(), name more
  consistent with _PyThreadState_GET()
* Update outdated comments
2018-10-30 15:13:17 +01:00
Quan Tian 3bd0d620d6 Fix typos in comments (GH-9905) 2018-10-19 23:30:03 +02:00
Zackery Spytz eef059657d bpo-31370: Remove references to threadless builds (#8805)
Support for threadless builds was removed in a6a4dc81.
2018-09-29 10:07:11 -06:00
jdemeyer e89de73987 bpo-34125: Enable profiling of method_descriptor in all cases (GH-8416)
`list.append([], None)` was profiled but `list.append([], None, **{})` was not profiled.
Enable profiling for later case.

https://bugs.python.org/issue34125
2018-09-19 03:06:20 -07:00
Benjamin Peterson ddd1949fea
closes bpo-34673: Tweaks to make ceval more editable. (GH-9289)
Two major changes:
- Move case statements out of the TARGET macro.
- Move PREDICT macro invocations after the case label.
2018-09-16 22:38:02 -07:00
Victor Stinner caba55b3b7
bpo-34301: Add _PyInterpreterState_Get() helper function (GH-8592)
sys_setcheckinterval() now uses a local variable to parse arguments,
before writing into interp->check_interval.
2018-08-03 15:33:52 +02:00
costypetrisor 8ed317f1ca bpo-34113: Fix a crash when using LLTRACE is on (GH-8517)
Fix a crash on negative STACKADJ() when Low-Level trace (LLTRACE) is enabled.
2018-07-31 22:55:14 +02:00
jdemeyer 147d95511f bpo-34190: Fix reference leak in call_function() (GH-8413) 2018-07-23 18:41:20 +02:00
jdemeyer 56868f940e bpo-34126: Fix crashes while profiling invalid calls. (GH-8300) 2018-07-21 11:30:59 +03:00
Serhiy Storchaka 3f4d90d4d7 bpo-34066: Disabled interruption before SETUP_WITH and BEFORE_ASYNC_WITH. (GH-8159)
This will prevent emitting a resource warning when the execution was
interrupted by Ctrl-C between calling open() and entering a 'with' block
in "with open()".
2018-07-09 22:40:14 +10:00
Marcel Plch 3a9ccee0e5 bpo-29673: fix gdb scripts pystack and pystackv (GH-6126) 2018-04-06 17:22:04 -04:00
Serhiy Storchaka a68f2f0578
bpo-29922: Improve error messages in 'async with' (GH-6352)
when __aenter__() or __aexit__() return non-awaitable object.
2018-04-03 01:41:38 +03:00
Xiang Zhang d8b291a742
bpo-32932: More revealing error message when non-str objects in __all__ (GH-5848) 2018-03-24 18:39:36 +08:00
Serhiy Storchaka 702f8f3611
bpo-33041: Rework compiling an "async for" loop. (#6142)
* Added new opcode END_ASYNC_FOR.
* Setting global StopAsyncIteration no longer breaks "async for" loops.
* Jumping into an "async for" loop is now disabled.
* Jumping out of an "async for" loop no longer corrupts the stack.
* Simplify the compiler.
2018-03-23 14:34:35 +02:00
Serhiy Storchaka 520b7ae27e
bpo-17611. Move unwinding of stack for "pseudo exceptions" from interpreter to compiler. (GH-5006)
Co-authored-by: Mark Shannon <mark@hotpy.org>
Co-authored-by: Antoine Pitrou <antoine@python.org>
2018-02-22 23:33:30 +02:00
Mark Shannon 332cd5ee4f bpo-32550. Remove the STORE_ANNOTATION bytecode. (GH-5181) 2018-01-29 16:41:04 -08:00
Victor Stinner 2914bb32e2
bpo-20891: Py_Initialize() now creates the GIL (#4700)
The GIL is no longer created "on demand" to fix a race condition when
PyGILState_Ensure() is called in a non-Python thread.
2018-01-29 11:57:45 +01:00
Xiang Zhang 997478eb79
bpo-32688: Make why the line is commented out clear (GH-5405) 2018-01-29 11:32:12 +08:00
Serhiy Storchaka f320be77ff bpo-32571: Avoid raising unneeded AttributeError and silencing it in C code (GH-5222)
Add two new private APIs: _PyObject_LookupAttr() and _PyObject_LookupAttrId()
2018-01-25 17:49:40 +09:00
Nathaniel J. Smith fc2f407829 bpo-32591: Add native coroutine origin tracking (#5250)
* Add coro.cr_origin and sys.set_coroutine_origin_tracking_depth
* Use coroutine origin information in the unawaited coroutine warning
* Stop using set_coroutine_wrapper in asyncio debug mode
* In BaseEventLoop.set_debug, enable debugging in the correct thread
2018-01-21 09:44:07 -05:00
Serhiy Storchaka 13a6c098c2
bpo-32259: Make a TypeError message when unpack non-iterable more specific. (#4903) 2017-12-26 12:30:41 +02:00
Serhiy Storchaka a5552f023e
bpo-32240: Add the const qualifier to declarations of PyObject* array arguments. (#4746) 2017-12-15 13:11:11 +02:00
pdox 1896793520 bpo-31857: Make the behavior of USE_STACKCHECK deterministic (#4098) 2017-10-25 23:03:01 -07:00
Benjamin Peterson 95f1a7bb09 remove support for splitting the ceval switch into multiple switches (#4099)
This kludge is from 1992. Any C99 compiler is going to be able to handle the
ceval dispatch switch.

Anyway, we have much bigger switches than the ceval dispatch one around. (See,
e.g., Objects/unicodetype_db.h.)
2017-10-24 22:35:15 -07:00
Mark Shannon ae3087c638 Move exc state to generator. Fixes bpo-25612 (#1773)
Move exception state information from frame objects to coroutine (generator/thread) object where it belongs.
2017-10-22 23:41:51 +02:00
George King 20faa68bbb Move opcode tracing to occur after the possible update to f_lineno. (GH-3798) 2017-10-19 10:44:22 +10:00
Yury Selivanov faa135acbf bpo-31709: Drop support for asynchronous __aiter__. (#3903) 2017-10-06 02:08:57 -04:00
Oren Milman 6db7033192 bpo-31492: Fix assertion failures in case of a module with a bad __name__ attribute. (#3620) 2017-09-19 14:23:01 +03:00
Eric Snow 3f9eee6eb4 bpo-28411: Support other mappings in PyInterpreterState.modules. (#3593)
The concrete PyDict_* API is used to interact with PyInterpreterState.modules in a number of places. This isn't compatible with all dict subclasses, nor with other Mapping implementations. This patch switches the concrete API usage to the corresponding abstract API calls.

We also add a PyImport_GetModule() function (and some other helpers) to reduce a bunch of code duplication.
2017-09-15 16:35:20 -06:00
Barry Warsaw b2e5794870 bpo-31338 (#3374)
* Add Py_UNREACHABLE() as an alias to abort().
* Use Py_UNREACHABLE() instead of assert(0)
* Convert more unreachable code to use Py_UNREACHABLE()
* Document Py_UNREACHABLE() and a few other macros.
2017-09-14 18:13:16 -07:00
Eric Snow 93c92f7d1d bpo-31404: Revert "remove modules from Py_InterpreterState (#1638)" (#3565)
PR #1638, for bpo-28411, causes problems in some (very) edge cases. Until that gets sorted out, we're reverting the merge. PR #3506, a fix on top of #1638, is also getting reverted.
2017-09-13 23:46:04 -07:00
Serhiy Storchaka e3b2b4b8d9 bpo-31393: Fix the use of PyUnicode_READY(). (#3451) 2017-09-08 09:58:51 +03:00
Eric Snow 2ebc5ce42a bpo-30860: Consolidate stateful runtime globals. (#3397)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-07 23:51:28 -06:00
Nick Coghlan 5a8516701f bpo-31344: Per-frame control of trace events (GH-3417)
f_trace_lines: enable/disable line trace events
f_trace_opcodes: enable/disable opcode trace events

These are intended primarily for testing of the interpreter
itself, as they make it much easier to emulate signals
arriving at unfortunate times.
2017-09-08 10:14:16 +10:00
Antoine Pitrou a6a4dc816d bpo-31370: Remove support for threads-less builds (#3385)
* Remove Setup.config
* Always define WITH_THREAD for compatibility.
2017-09-07 18:56:24 +02:00
Eric Snow 05351c1bd8 Revert "bpo-30860: Consolidate stateful runtime globals." (#3379)
Windows buildbots started failing due to include-related errors.
2017-09-05 21:43:08 -07:00
Eric Snow 76d5abc868 bpo-30860: Consolidate stateful runtime globals. (#2594)
* group the (stateful) runtime globals into various topical structs
* consolidate the topical structs under a single top-level _PyRuntimeState struct
* add a check-c-globals.py script that helps identify runtime globals

Other globals are excluded (see globals.txt and check-c-globals.py).
2017-09-05 18:26:16 -07:00
Eric Snow 86b7afdfee bpo-28411: Remove "modules" field from Py_InterpreterState. (#1638)
sys.modules is the one true source.
2017-09-04 17:54:09 -06:00
Stefan Krah f432a3234f bpo-30923: Silence fall-through warnings included in -Wextra since gcc-7.0. (#3157) 2017-08-21 13:09:59 +02:00
Serhiy Storchaka 25e4f779d7 bpo-31071: Avoid masking original TypeError in call with * unpacking (#2957)
when other arguments are passed.
2017-08-03 11:37:15 +03:00
Ville Skyttä 49b2734bf1 Spelling fixes (#2902) 2017-08-03 09:00:59 +03:00
Zackery Spytz c6ea8974e2 bpo-30640: Fix undefined behavior in _PyFunction_FastCallDict() and PyEval_EvalCodeEx() (#2919)
k + 1 was calculated with k = NULL.
2017-07-31 17:24:37 +03:00
Masayuki Yamamoto 0c31163093 bpo-30854: Fix compile error when --without-threads (#2581)
* bpo-30854: Fix compile error when --without-threads

* bpo-30854: fix news
2017-07-05 10:39:17 +02:00
Antoine Pitrou c08177a1cc bpo-30703: Improve signal delivery (#2415)
* Improve signal delivery

Avoid using Py_AddPendingCall from signal handler, to avoid calling signal-unsafe functions.

* Remove unused function

* Improve comments

* Add stress test

* Adapt for --without-threads

* Add second stress test

* Add NEWS blurb

* Address comments @haypo
2017-06-28 23:29:29 +02:00
Adrian Wielgosik 50c2850fc8 Trivial cleanup: remove redundant variable stores in ceval.c (#2012)
Redundant code leftover from cleanup in #16191: the variable `err` is being written to, even though it wasn't used after that point.
2017-06-23 22:35:41 +02:00
Dino Viehland f3cffd2b78 bpo-30604: clean up co_extra support (#2144)
bpo-30604: port fix from 3.6 dropping binary compatibility tweaks
2017-06-21 17:44:36 -04:00
Antoine Pitrou 4a8bcdf79c bpo-16500: Use register_at_fork() in the threading module (#1843)
* bpo-16500: Use register_at_fork() in the threading module

* Update comment at top of _after_fork()
2017-05-28 14:02:26 +02:00
Antoine Pitrou f7ecfac0c1 Doc nits for bpo-16500 (#1841)
* Doc nits for bpo-16500

* Fix more references
2017-05-28 11:35:14 +02:00
Nathaniel J. Smith ab4413a7e9 bpo-30039: Don't run signal handlers while resuming a yield from stack (#1081)
If we have a chain of generators/coroutines that are 'yield from'ing
each other, then resuming the stack works like:

- call send() on the outermost generator
- this enters _PyEval_EvalFrameDefault, which re-executes the
  YIELD_FROM opcode
- which calls send() on the next generator
- which enters _PyEval_EvalFrameDefault, which re-executes the
  YIELD_FROM opcode
- ...etc.

However, every time we enter _PyEval_EvalFrameDefault, the first thing
we do is to check for pending signals, and if there are any then we
run the signal handler. And if it raises an exception, then we
immediately propagate that exception *instead* of starting to execute
bytecode. This means that e.g. a SIGINT at the wrong moment can "break
the chain" – it can be raised in the middle of our yield from chain,
with the bottom part of the stack abandoned for the garbage collector.

The fix is pretty simple: there's already a special case in
_PyEval_EvalFrameEx where it skips running signal handlers if the next
opcode is SETUP_FINALLY. (I don't see how this accomplishes anything
useful, but that's another story.) If we extend this check to also
skip running signal handlers when the next opcode is YIELD_FROM, then
that closes the hole – now the exception can only be raised at the
innermost stack frame.

This shouldn't have any performance implications, because the opcode
check happens inside the "slow path" after we've already determined
that there's a pending signal or something similar for us to process;
the vast majority of the time this isn't true and the new check
doesn't run at all.
2017-05-17 13:33:23 -07:00
Xiang Zhang 2ddf5a19c3 bpo-30281: Fix the default value for stop in PySlice_Unpack() (#1480) 2017-05-10 18:19:41 +08:00
Serhiy Storchaka d4edfc9abf bpo-29935: Fixed error messages in the index() method of tuple, list and deque (#887)
when pass indices of wrong type.
2017-03-30 18:29:23 +03:00
Serhiy Storchaka aefa7ebf0f bpo-6532: Make the thread id an unsigned integer. (#781)
* bpo-6532: Make the thread id an unsigned integer.

From C API side the type of results of PyThread_start_new_thread() and
PyThread_get_thread_ident(), the id parameter of
PyThreadState_SetAsyncExc(), and the thread_id field of PyThreadState
changed from "long" to "unsigned long".

* Restore a check in thread_get_ident().
2017-03-23 14:48:39 +01:00
Xiang Zhang 4830f581af bpo-29849: fix a memory leak in import_from (GH-712) 2017-03-21 11:13:42 +08:00
Serhiy Storchaka 80ec8364f1 bpo-29748: Added the slice index converter in Argument Clinic. (#549) 2017-03-19 19:37:40 +02:00
INADA Naoki 93fac8dd35 bpo-29676: fix lsprof can't profile C method call. (GH523)
When LOAD_METHOD is used for calling C mehtod, PyMethodDescrObject
was passed to profilefunc from 5566bbb.
But lsprof traces only PyCFunctionObject. Additionally, there can be
some third party extension which assumes passed arg is
PyCFunctionObject without calling PyCFunction_Check().

So make PyCFunctionObject from PyMethodDescrObject when
tstate->c_profilefunc is set.
2017-03-07 14:24:37 +09:00
Yury Selivanov 398ff91ac0 bpo-28893: Set __cause__ for errors in async iteration protocol (#407) 2017-03-02 22:20:00 -05:00
Matthias Bussonnier 160edb4357 bpo-29655: Fixed possible reference leaks in `import *`. (#301)
Patch by Matthias Bussonnier.
2017-02-26 07:58:05 +02:00
Martijn Pieters d7e64337ef bpo-28598: Support __rmod__ for RHS subclasses of str in % string formatting operations (#51)
When you use `'%s' % SubClassOfStr()`, where `SubClassOfStr.__rmod__` exists, the reverse operation is ignored as normally such string formatting operations use the `PyUnicode_Format()` fast path. This patch tests for subclasses of `str` first and picks the slow path in that case.

Patch by Martijn Pieters.
2017-02-23 15:38:04 +02:00
Matthias Bussonnier 1bc156430b bpo-29546: Improve from-import error message with location (#103)
bpo-29546: Improve from-import error message with location
2017-02-22 10:06:50 -05:00
Matthias Bussonnier bc4bed4405 bpo-29546: Set 'path' on ImportError for ``from ... import ...`` (GH-91) 2017-02-14 16:05:25 -08:00
Victor Stinner c22bfaae83 bpo-29524: Add Objects/call.c file (#12)
* Move all functions to call objects in a new Objects/call.c file.
* Rename fast_function() to _PyFunction_FastCallKeywords().
* Copy null_error() from Objects/abstract.c
* Inline type_error() in call.c to not have to copy it, it was only
  called once.
* Export _PyEval_EvalCodeWithName() since it is now called
  from call.c.
2017-02-12 19:27:05 +01:00
Victor Stinner d2306cec4d Backed out changeset f23fa1f7b68f
Sorry, I didn't want to push this change before the review :-( I was pushing a
change into the 2.7 branch.
2017-02-10 14:19:36 +01:00
Victor Stinner 766af559ad Issue #29465: Add Objects/call.c file
* Move all functions to call objects in a new Objects/call.c file.
* Rename fast_function() to _PyFunction_FastCallKeywords().
* Copy null_error() from Objects/abstract.c
* Inline type_error() in call.c to not have to copy it, it was only
  called once.
* Export _PyEval_EvalCodeWithName() since it is now called
  from call.c.
2017-02-10 13:32:29 +01:00
INADA Naoki 5566bbb8d5 Issue #29263: LOAD_METHOD support for C methods
Calling builtin method is at most 10% faster.
2017-02-03 07:43:03 +09:00
Victor Stinner 122fb562d4 Document that _PyFunction_FastCallDict() must copy kwargs
Issue #29318: Caller and callee functions must not share the dictionary:
kwargs must be copied.
2017-02-01 17:04:52 +01:00
Victor Stinner a8cb515a29 Rephrase !PyErr_Occurred() comment: may=>can
Issue #29259.
2017-01-18 14:12:51 +01:00
INADA Naoki 015bce64b3 Issue #26110: Add document for LOAD_METHOD and CALL_METHOD opcode.
Changed stack layout bit for "easy to explain."
2017-01-16 17:23:30 +09:00
Victor Stinner 231d1f3439 _PyEval_EvalCodeWithName(): remove redundant check
Replace the runtime check with an assertion (just in case).
2017-01-11 02:12:06 +01:00
Victor Stinner 415c5107be Inline call_function()
Issue #29227: Inline call_function() into _PyEval_EvalFrameDefault() using
Py_LOCAL_INLINE to reduce the stack consumption.

It reduces the stack consumption, bytes per call, before => after:

test_python_call: 1152 => 1040 (-112 B)
test_python_getitem: 1008 => 976 (-32 B)
test_python_iterator: 1232 => 1120 (-112 B)

=> total: 3392 => 3136 (- 256 B)
2017-01-11 00:54:57 +01:00
Victor Stinner 865a0f621f Optimize _PyFunction_FastCallDict() when kwargs is {}
Issue #28839: Optimize _PyFunction_FastCallDict() when kwargs is an empty
dictionary, avoid the creation of an useless empty tuple.
2017-01-03 02:01:42 +01:00
INADA Naoki 9c15776fac Issue #29049: Remove unnecessary Py_DECREF 2016-12-26 18:52:46 +09:00
INADA Naoki 6a3cedf8dd Issue #29049: Fix refleak introduced by f5eb0c4f5d37. 2016-12-26 18:01:46 +09:00
INADA Naoki 5a625d0aa6 Issue #29049: Call _PyObject_GC_TRACK() lazily when calling Python function.
Calling function is up to 5% faster.
2016-12-24 20:19:08 +09:00
Serhiy Storchaka 5bb8b9134b Issue #18896: Python function can now have more than 255 parameters.
collections.namedtuple() now supports tuples with more than 255 elements.
2016-12-16 19:19:02 +02:00
Serhiy Storchaka 5ab81d787f Issue #28959: Added private macro PyDict_GET_SIZE for retrieving the size of dict. 2016-12-16 16:18:57 +02:00
Yury Selivanov f2392133eb Issue #26110: Add LOAD_METHOD/CALL_METHOD opcodes.
Special thanks to INADA Naoki for pushing the patch through
the last mile, Serhiy Storchaka for reviewing the code, and to
Victor Stinner for suggesting the idea (originally implemented
in the PyPy project).
2016-12-13 19:03:51 -05:00
Victor Stinner 59a73276e9 Backed out changeset 99c34e47348b
The change broke test_gdb.
2016-12-09 18:51:13 +01:00
Victor Stinner 0ca246c5e7 Inline PyEval_EvalFrameEx() in callers
The PEP 523 modified PyEval_EvalFrameEx(): it's now an indirection to
interp->eval_frame().

Inline the call in performance critical code. Leave PyEval_EvalFrame()
unchanged, this function is only kept for backward compatibility.
2016-12-09 17:12:17 +01:00
Victor Stinner f17c3de263 Use _PyObject_CallNoArg()
Replace:
    PyObject_CallFunctionObjArgs(callable, NULL)
with:
    _PyObject_CallNoArg(callable)
2016-12-06 18:46:19 +01:00
Victor Stinner a5ed5f000a Use _PyObject_CallNoArg()
Replace:
    PyObject_CallObject(callable, NULL)
with:
    _PyObject_CallNoArg(callable)
2016-12-06 18:45:50 +01:00
Victor Stinner 2d0eb65f45 Uniformize argument names of "call" functions
Issue #28838: Rename parameters of the "calls" functions of the Python C API.

* Rename 'callable_object' and 'func' to 'callable': any Python callable object
  is accepted, not only Python functions
* Rename 'method' and 'nameid' to 'name' (method name)
* Rename 'o' to 'obj'
* Move, fix and update documentation of PyObject_CallXXX() functions
  in abstract.h
* Update also the documentaton of the C API (update parameter names)
2016-12-06 16:27:24 +01:00
Victor Stinner de4ae3d486 Backed out changeset b9c9691c72c5
Issue #28858: The change b9c9691c72c5 introduced a regression. It seems like
_PyObject_CallArg1() uses more stack memory than
PyObject_CallFunctionObjArgs().
2016-12-04 22:59:09 +01:00
Victor Stinner 842cfff321 WITH_CLEANUP_START uses fastcall
Modify WITH_CLEANUP_START bytecode: replace PyObject_CallFunctionObjArgs() with
_PyObject_FastCall().
2016-12-01 14:45:31 +01:00
Victor Stinner 27580c1fb5 Replace PyObject_CallFunctionObjArgs() with fastcall
* PyObject_CallFunctionObjArgs(func, NULL) => _PyObject_CallNoArg(func)
* PyObject_CallFunctionObjArgs(func, arg, NULL) => _PyObject_CallArg1(func, arg)

PyObject_CallFunctionObjArgs() allocates 40 bytes on the C stack and requires
extra work to "parse" C arguments to build a C array of PyObject*.

_PyObject_CallNoArg() and _PyObject_CallArg1() are simpler and don't allocate
memory on the C stack.

This change is part of the fastcall project. The change on listsort() is
related to the issue #23507.
2016-12-01 14:43:22 +01:00
Victor Stinner b69ee8c386 call_function(): document PyMethod optimization 2016-11-28 18:32:31 +01:00
Victor Stinner 048afd98b3 Remove CALL_PROFILE special build
Issue #28799:

* Remove the PyEval_GetCallStats() function.
* Deprecate the untested and undocumented sys.callstats() function.
* Remove the CALL_PROFILE special build

Use the sys.setprofile() function, cProfile or profile module to profile
function calls.
2016-11-28 11:59:04 +01:00
Victor Stinner ed6de7345e Merge 3.6 2016-11-24 22:33:49 +01:00
Victor Stinner f7d199ff32 Fix _PyGen_yf()
Issue #28782: Fix a bug in the implementation ``yield from`` when checking
if the next instruction is YIELD_FROM. Regression introduced by WORDCODE
(issue #26647).

Reviewed by Serhiy Storchaka and Yury Selivanov.
2016-11-24 22:33:01 +01:00
Raymond Hettinger 3fa28fd075 merge 2016-11-22 11:50:56 -08:00
Raymond Hettinger 64e2f9ac86 Issue #27100: Fix ref leak 2016-11-22 11:50:40 -08:00
Raymond Hettinger 9af740b99a merge 2016-11-21 17:24:58 -08:00
Raymond Hettinger a3fec1543d Issue #27100: With statement reports missing __enter__ before __exit__. (Contributed by Jonathan Ellington.) 2016-11-21 17:24:23 -08:00
Serhiy Storchaka 85b0f5beb1 Added the const qualifier to char* variables that refer to readonly internal
UTF-8 represenatation of Unicode objects.
2016-11-20 10:16:47 +02:00
Serhiy Storchaka a98c4a984b Replaced outdated macros _PyUnicode_AsString and _PyUnicode_AsStringAndSize
with PyUnicode_AsUTF8 and PyUnicode_AsUTF8AndSize.
2016-11-20 09:13:40 +02:00
Serhiy Storchaka 06515833fe Replaced outdated macros _PyUnicode_AsString and _PyUnicode_AsStringAndSize
with PyUnicode_AsUTF8 and PyUnicode_AsUTF8AndSize.
2016-11-20 09:13:07 +02:00
Raymond Hettinger c32f9db846 Issue #28665: Use macro form of PyCell_GET/SET 2016-11-12 04:10:35 -05:00
Raymond Hettinger b2b154374d merge 2016-11-11 04:32:11 -08:00
Raymond Hettinger 13527123a1 Issue #28665: Harmonize STORE_DEREF with STORE_FAST and LOAD_DEREF giving a 40% speedup. 2016-11-11 04:31:18 -08:00
Victor Stinner c6944e7edc Issue #28618: Make hot functions using __attribute__((hot))
When Python is not compiled with PGO, the performance of Python on call_simple
and call_method microbenchmarks depend highly on the code placement. In the
worst case, the performance slowdown can be up to 70%.

The GCC __attribute__((hot)) attribute helps to keep hot code close to reduce
the risk of such major slowdown. This attribute is ignored when Python is
compiled with PGO.

The following functions are considered as hot according to statistics collected
by perf record/perf report:

* _PyEval_EvalFrameDefault()
* call_function()
* _PyFunction_FastCall()
* PyFrame_New()
* frame_dealloc()
* PyErr_Occurred()
2016-11-11 02:13:35 +01:00
Serhiy Storchaka 818b5cc6db Fixed possible abort in ceval loop if _PyUnicode_FromId() fails.
Every opcode should end with DISPATCH() or goto error.
2016-11-08 23:14:00 +02:00
Serhiy Storchaka 4678b2f448 Fixed possible abort in ceval loop if _PyUnicode_FromId() fails.
Every opcode should end with DISPATCH() or goto error.
2016-11-08 23:13:36 +02:00
Serhiy Storchaka 70b72f0f96 Fixed possible abort in ceval loop if _PyUnicode_FromId() fails.
Every opcode should end with DISPATCH() or goto error.
2016-11-08 23:12:46 +02:00
Yury Selivanov 0ee446c894 Merge 3.6 (issue #27243) 2016-11-08 15:15:42 -05:00
Yury Selivanov 2edd8a1e2c Issue #27243: Change PendingDeprecationWarning -> DeprecationWarning.
As it was agreed in the issue, __aiter__ returning an awaitable
should result in PendingDeprecationWarning in 3.5 and in
DeprecationWarning in 3.6.
2016-11-08 15:13:07 -05:00
Victor Stinner d65f42a132 Issue #21955: Please don't try to optimize int+int 2016-10-20 12:18:10 +02:00
Raymond Hettinger 4186222e63 Minor fix-up to apply the stack adjustment macros consistent with the other opcodes 2016-10-15 19:03:06 -07:00
Serhiy Storchaka 5665301bae Issue #28257: Improved error message when pass a non-mapping as a var-keyword
argument.
2016-10-07 23:32:41 +03:00
Berker Peksag 419968c235 Issue #27358: Merge from 3.6 2016-10-02 13:08:47 +03:00
Berker Peksag 8e9045d0d8 Issue #27358: Fix typo in error message 2016-10-02 13:08:25 +03:00
Serhiy Storchaka 2e84de3638 Issue #27358: Optimized merging var-keyword arguments and improved error
message when pass a non-mapping as a var-keyword argument.
2016-10-02 11:07:29 +03:00
Serhiy Storchaka e036ef8fa2 Issue #27358: Optimized merging var-keyword arguments and improved error
message when pass a non-mapping as a var-keyword argument.
2016-10-02 11:06:43 +03:00
Serhiy Storchaka 775a0ea0da Issue #28257: Improved error message when pass a non-iterable as
a var-positional argument.  Added opcode BUILD_TUPLE_UNPACK_WITH_CALL.
2016-10-02 10:34:46 +03:00
Serhiy Storchaka 7344285c19 Issue #28257: Improved error message when pass a non-iterable as
a var-positional argument.  Added opcode BUILD_TUPLE_UNPACK_WITH_CALL.
2016-10-02 10:33:46 +03:00
Serhiy Storchaka c019158a4c Issue #27703: Got rid of unnecessary NULL checks in do_raise() in release mode.
Patch by Xiang Zhang.
2016-09-27 11:37:10 +03:00
Serhiy Storchaka 208bbd29d3 Silence GCC warning.
The code was correct, but GCC is not enough clever.
2016-09-22 19:59:46 +03:00
Serhiy Storchaka 63dc548109 Issue #28086: Single var-positional argument of tuple subtype was passed
unscathed to the C-defined function.  Now it is converted to exact tuple.
2016-09-22 19:41:20 +03:00
Victor Stinner 57f91ac95a Document kwnames in _PyObject_FastCallKeywords() and _PyStack_AsDict()
Issue #27213.
2016-09-12 13:37:07 +02:00
Victor Stinner eece2229e2 Issue #27213: Fix reference leaks 2016-09-12 11:16:37 +02:00
Serhiy Storchaka b72810583e Issue #27213: Fixed different issues with reworked CALL_FUNCTION* opcodes.
* BUILD_TUPLE_UNPACK and BUILD_MAP_UNPACK_WITH_CALL no longer generated with
  single tuple or dict.
* Restored more informative error messages for incorrect var-positional and
  var-keyword arguments.
* Removed code duplications in _PyEval_EvalCodeWithName().
* Removed redundant runtime checks and parameters in _PyStack_AsDict().
* Added a workaround and enabled previously disabled test in test_traceback.
* Removed dead code from the dis module.
2016-09-12 00:52:40 +03:00
Serhiy Storchaka ab8740058a Issue #27129: Replaced wordcode related magic constants with macros. 2016-09-11 13:48:15 +03:00
Łukasz Langa a785c87d6e DTrace support: function calls, GC activity, line execution
Tested on macOS 10.11 dtrace, Ubuntu 16.04 SystemTap, and libbcc.

Largely based by an initial patch by Jesús Cea Avión, with some
influence from Dave Malcolm's SystemTap patch and Nikhil Benesch's
unification patch.

Things deliberately left out for simplicity:
- ustack helpers, I have no way of testing them at this point since
they are Solaris-specific
- PyFrameObject * in function__entry/function__return, this is
SystemTap-specific
- SPARC support
- dynamic tracing
- sys module dtrace facility introspection

All of those might be added later.
2016-09-09 17:37:37 -07:00
Benjamin Peterson 75f94a7dff remove more READ_TIMESTAMP 2016-09-09 15:03:18 -07:00
Benjamin Peterson 1cfe1d1f53 remove READ_TIMESTAMP macro 2016-09-09 15:02:11 -07:00
Benjamin Peterson 4fd64b9a6a remove ceval timestamp support 2016-09-09 14:57:58 -07:00
Victor Stinner ae8b69c410 Issue #27810: Add _PyCFunction_FastCallKeywords()
Use _PyCFunction_FastCallKeywords() in ceval.c: it allows to remove a lot of
code from ceval.c which was only used to call C functions.
2016-09-09 14:07:44 -07:00
Victor Stinner d873572095 Add _PyObject_FastCallKeywords()
Issue #27830: Add _PyObject_FastCallKeywords(): avoid the creation of a
temporary dictionary for keyword arguments.

Other changes:

* Cleanup call_function() and fast_function() (ex: rename nk to nkwargs)
* Remove now useless do_call(), replaced with _PyObject_FastCallKeywords()
2016-09-09 12:36:44 -07:00
Victor Stinner f9b760f48a Rework CALL_FUNCTION* opcodes
Issue #27213: Rework CALL_FUNCTION* opcodes to produce shorter and more
efficient bytecode:

* CALL_FUNCTION now only accepts position arguments
* CALL_FUNCTION_KW accepts position arguments and keyword arguments, but keys
  of keyword arguments are packed into a constant tuple.
* CALL_FUNCTION_EX is the most generic, it expects a tuple and a dict for
  positional and keyword arguments.

CALL_FUNCTION_VAR and CALL_FUNCTION_VAR_KW opcodes have been removed.

2 tests of test_traceback are currently broken: skip test, the issue #28050 was
created to track the issue.

Patch by Demur Rumed, design by Serhiy Storchaka, reviewed by Serhiy Storchaka
and Victor Stinner.
2016-09-09 10:17:08 -07:00
Yury Selivanov 50c584f50b ceval: tighten the code of STORE_ANNOTATION 2016-09-08 23:38:21 -07:00