Commit Graph

8192 Commits

Author SHA1 Message Date
Brandt Bucher dfb55d9d7f
Use tstate->interp to get the interpreter state in bytecodes.c (GH-107506) 2023-07-31 14:18:38 -07:00
Brandt Bucher 5e584eb704
GH-104584: Fix incorrect uoperands (GH-107513) 2023-07-31 21:16:57 +00:00
Guido van Rossum 5eb80a61f5
GH-104909: Move unused cache entries from uops to macros (#107444)
There's no need to use a dummy uop to skip unused cache entries. The macro syntax lets you write `unused/1` instead.

Similarly, move `unused/5` from op `_LOAD_ATTR_INSTANCE_VALUE` to macro `LOAD_ATTR_INSTANCE_VALUE`.
2023-07-31 08:55:33 -07:00
Georg Brandl f57b9fd2b6
no-issue: Fix typo in import.c (gh-107498) 2023-07-31 12:07:17 +00:00
Eric Snow 8ba4df91ae
gh-105699: Use a _Py_hashtable_t for the PyModuleDef Cache (gh-106974)
This fixes a crasher due to a race condition, triggered infrequently when two isolated (own GIL) subinterpreters simultaneously initialize their sys or builtins modules.  The crash happened due the combination of the "detached" thread state we were using and the "last holder" logic we use for the GIL.  It turns out it's tricky to use the same thread state for different threads.  Who could have guessed?

We solve the problem by eliminating the one object we were still sharing between interpreters.  We replace it with a low-level hashtable, using the "raw" allocator to avoid tying it to the main interpreter.

We also remove the accommodations for "detached" thread states, which were a dubious idea to start with.
2023-07-28 14:39:08 -06:00
Eric Snow 8bdae1424b
gh-101524: Only Use Public C-API in the _xxsubinterpreters Module (gh-107359)
The _xxsubinterpreters module should not rely on internal API.  Some of the functions it uses were recently moved there however.  Here we move them back (and expose them properly).
2023-07-27 15:30:16 -06:00
Eric Snow 75c974f535
gh-104621: Check for Incompatible Extensions in import_find_extension() (gh-107184)
This fixes a bug where incompatible modules could still be imported if attempted multiple times.
2023-07-27 15:08:38 -06:00
Eric Snow b72947a8d2
gh-106931: Intern Statically Allocated Strings Globally (gh-107272)
We tried this before with a dict and for all interned strings.  That ran into problems due to interpreter isolation.  However, exclusively using a per-interpreter cache caused some inconsistency that can eliminate the benefit of interning.  Here we circle back to using a global cache, but only for statically allocated strings.  We also use a more-basic _Py_hashtable_t for that global cache instead of a dict.

Ideally we would only have the global cache, but the optional isolation of each interpreter's allocator means that a non-static string object must not outlive its interpreter.  Thus we would have to store a copy of each such interned string in the global cache, tied to the main interpreter.
2023-07-27 13:56:59 -06:00
Mark Shannon ac7a0f858a
GH-106898: Add the exception as an argument to the `PY_UNWIND` event callback function. (GH-107347) 2023-07-27 15:47:33 +01:00
Victor Stinner 1dbb427dd6
gh-107196: Remove _PyArg_VaParseTupleAndKeywordsFast() function (#107197)
Remove the private _PyArg_VaParseTupleAndKeywordsFast() function: it
is no longer used.
2023-07-27 16:36:54 +02:00
Mark Shannon c6539b36c1
GH-106895: Raise a `ValueError` when attempting to disable events that cannot be disabled. (GH-107337) 2023-07-27 15:27:11 +01:00
Mark Shannon 766d2518ae
GH-106897: Add `RERAISE` event to `sys.monitoring`. (GH-107291)
* Ensures that exception handling events are balanced. Each [re]raise event has a matching unwind/handled event.
2023-07-27 13:32:30 +01:00
Irit Katriel 507d8bc39a
gh-106149: fix comment on stackdepth of generators (#107321) 2023-07-26 22:31:47 +00:00
Pablo Galindo Salgado da8f87b7ea
gh-107015: Remove async_hacks from the tokenizer (#107018) 2023-07-26 16:34:15 +01:00
Irit Katriel b0202a4e5d
gh-106149: Simplify stack depth calculation. Replace asserts by exceptions. (#107255) 2023-07-26 13:32:47 +01:00
Guido van Rossum 233b878288
gh-107082: Fix instruction size computation for ENTER_EXECUTOR (#107256)
Co-authored-by: Victor Stinner <vstinner@python.org>
2023-07-25 20:01:02 +00:00
Victor Stinner 188000ae4b
Remove unused internal _PyImport_GetModuleId() function (#107235) 2023-07-25 17:02:12 +00:00
Victor Stinner 1a3faba9f1
gh-106869: Use new PyMemberDef constant names (#106871)
* Remove '#include "structmember.h"'.
* If needed, add <stddef.h> to get offsetof() function.
* Update Parser/asdl_c.py to regenerate Python/Python-ast.c.
* Replace:

  * T_SHORT => Py_T_SHORT
  * T_INT => Py_T_INT
  * T_LONG => Py_T_LONG
  * T_FLOAT => Py_T_FLOAT
  * T_DOUBLE => Py_T_DOUBLE
  * T_STRING => Py_T_STRING
  * T_OBJECT => _Py_T_OBJECT
  * T_CHAR => Py_T_CHAR
  * T_BYTE => Py_T_BYTE
  * T_UBYTE => Py_T_UBYTE
  * T_USHORT => Py_T_USHORT
  * T_UINT => Py_T_UINT
  * T_ULONG => Py_T_ULONG
  * T_STRING_INPLACE => Py_T_STRING_INPLACE
  * T_BOOL => Py_T_BOOL
  * T_OBJECT_EX => Py_T_OBJECT_EX
  * T_LONGLONG => Py_T_LONGLONG
  * T_ULONGLONG => Py_T_ULONGLONG
  * T_PYSSIZET => Py_T_PYSSIZET
  * T_NONE => _Py_T_NONE
  * READONLY => Py_READONLY
  * PY_AUDIT_READ => Py_AUDIT_READ
  * READ_RESTRICTED => Py_AUDIT_READ
  * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED
  * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED)
2023-07-25 15:28:30 +02:00
Victor Stinner 2e0744955f
gh-107211: Rename PySymtable_Lookup() to _PySymtable_Lookup() (#107212)
Rename the internal PySymtable_Lookup() function to
_PySymtable_Lookup() and no longer export it.
2023-07-25 00:54:09 +00:00
Irit Katriel 7608fa8fd5
gh-106149: move _PyCfg_BasicblockLastInstr and make it local to flowgraph.c (#107180) 2023-07-24 22:08:59 +01:00
Carl Meyer e5d5522612
gh-106917: fix super classmethod calls to non-classmethods (#106977) 2023-07-24 13:14:56 -07:00
Victor Stinner fd66baf34a
gh-106320: Remove private _PyDict C API (#107145)
Move private _PyDict functions to the internal C API (pycore_dict.h):

* _PyDict_Contains_KnownHash()
* _PyDict_DebugMallocStats()
* _PyDict_DelItemIf()
* _PyDict_GetItemWithError()
* _PyDict_HasOnlyStringKeys()
* _PyDict_MaybeUntrack()
* _PyDict_MergeEx()

No longer export these functions.
2023-07-24 14:02:03 +00:00
Victor Stinner 0d6dfd68d2
gh-106320: Remove private _PyObject C API (#107147)
Move private debug _PyObject functions to the internal C API
(pycore_object.h):

* _PyDebugAllocatorStats()
* _PyObject_CheckConsistency()
* _PyObject_DebugTypeStats()
* _PyObject_IsFreed()

No longer export most of these functions, except of
_PyObject_IsFreed().

Move test functions using _PyObject_IsFreed() from _testcapi to
_testinternalcapi. check_pyobject_is_freed() test no longer catch
_testcapi.error: the tested function cannot raise _testcapi.error.
2023-07-23 20:09:08 +00:00
Victor Stinner 0927a2b25c
GH-103082: Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS (#107069)
Rename private C API constants:

* Rename PY_MONITORING_UNGROUPED_EVENTS to _PY_MONITORING_UNGROUPED_EVENTS
* Rename PY_MONITORING_EVENTS to _PY_MONITORING_EVENTS
2023-07-22 21:35:27 +00:00
Victor Stinner ee15844db8
gh-106320: Move _PyMethodWrapper_Type to internal C API (#107064) 2023-07-22 20:57:59 +00:00
Victor Stinner 5e4af2a3e9
gh-106320: Move private _PySet API to the internal API (#107041)
* Add pycore_setobject.h header file.
* Move the following API to the internal C API:

  * _PySet_Dummy
  * _PySet_NextEntry()
  * _PySet_Update()
2023-07-22 17:04:34 +02:00
Victor Stinner 89f9875448
gh-106320: Move private _PyHash API to the internal C API (#107026)
* No longer export most private _PyHash symbols, only export the ones
  which are needed by shared extensions.
* Modules/_xxtestfuzz/fuzzer.c now uses the internal C API.
2023-07-22 13:49:37 +00:00
Kirill Podoprigora 85ed1d2442
gh-106916: Add missing error check _PyCompile_CleanDoc (#106921)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-07-21 13:56:58 +09:00
Brandt Bucher 8f4de57699
GH-106701: Move _PyUopExecute to Python/executor.c (GH-106924) 2023-07-20 20:37:19 +00:00
Irit Katriel 9c81fc2dbe
gh-105481: do not auto-generate pycore_intrinsics.h (#106913) 2023-07-20 17:46:04 +01:00
Brandt Bucher 214a25dd81
GH-104584: Miscellaneous fixes for -Xuops (GH-106908) 2023-07-20 16:35:39 +00:00
Serhiy Storchaka a293fa5915
gh-86493: Use PyModule_Add() instead of PyModule_AddObjectRef() (GH-106860) 2023-07-18 23:59:53 +03:00
Irit Katriel 40f3f11a77
gh-105481: Generate the opcode lists in dis from data extracted from bytecodes.c (#106758) 2023-07-18 19:42:44 +01:00
Serhiy Storchaka 83ac128490
bpo-42327: C API: Add PyModule_Add() function (GH-23443)
It is a fixed implementation of PyModule_AddObject() which consistently
steals reference both on success and on failure.
2023-07-18 09:42:05 +03:00
Inada Naoki ece3b9d12a
gh-106843: fix memleak in _PyCompile_CleanDoc (#106846) 2023-07-18 03:44:16 +00:00
Guido van Rossum 1e36ca63f9
Small fixes to code generator (#106845)
These repair nits I found in PR gh-106798 (issue gh-106797) and in PR gh-106716 (issue gh-106706).
2023-07-18 01:30:41 +00:00
Guido van Rossum 8e9a1a0322
gh-106603: Make uop struct a triple (opcode, oparg, operand) (#106794) 2023-07-17 12:12:33 -07:00
Guido van Rossum 2b94a05a0e
gh-106581: Add 10 new opcodes by allowing `assert(kwnames == NULL)` (#106707)
By turning `assert(kwnames == NULL)` into a macro that is not in the "forbidden" list, many instructions that formerly were skipped because they contained such an assert (but no other mention of `kwnames`) are now supported in Tier 2. This covers 10 instructions in total (all specializations of `CALL` that invoke some C code):
- `CALL_NO_KW_TYPE_1`
- `CALL_NO_KW_STR_1`
- `CALL_NO_KW_TUPLE_1`
- `CALL_NO_KW_BUILTIN_O`
- `CALL_NO_KW_BUILTIN_FAST`
- `CALL_NO_KW_LEN`
- `CALL_NO_KW_ISINSTANCE`
- `CALL_NO_KW_METHOD_DESCRIPTOR_O`
- `CALL_NO_KW_METHOD_DESCRIPTOR_NOARGS`
- `CALL_NO_KW_METHOD_DESCRIPTOR_FAST`
2023-07-17 11:02:58 -07:00
Guido van Rossum b2b261ab2a
gh-106529: Generate uops for POP_JUMP_IF_[NOT_]NONE (#106796)
These aren't automatically translated because (ironically)
they are macros deferring to POP_JUMP_IF_{TRUE,FALSE},
which are not viable uops (being manually translated).

The hack is that we emit IS_NONE and then set opcode and
jump to the POP_JUMP_IF_{TRUE,FALSE} translation code.
2023-07-17 10:06:05 -07:00
Dong-hee Na 48956cc60e
gh-106797: Remove warning logs from Python/generated_cases.c.h (gh-106798) 2023-07-17 09:09:11 +09:00
Kevin Diem cc25ca16ee
gh-106706: Streamline family syntax in cases generator DSL (#106716)
From `family(opname, STRUCTSIZE) = OPNAME + SPEC1 + ... +  SPECn;`
to `family(OPNAME, STRUCTSIZE) = SPEC1 + ... + SPECn;`
2023-07-16 08:16:34 -07:00
Inada Naoki 2566b74b26
gh-81283: compiler: remove indent from docstring (#106411)
Co-authored-by: Éric <merwok@netwok.org>
2023-07-15 19:33:32 +09:00
Guido van Rossum 0db85eeba7
gh-106529: Fix subtle Tier 2 edge case with list iterator (#106756)
The Tier 2 opcode _IS_ITER_EXHAUSTED_LIST (and _TUPLE)
didn't set it->it_seq to NULL, causing a subtle bug
that resulted in test_exhausted_iterator in list_tests.py
to fail when running all tests with -Xuops.

The bug was introduced in gh-106696.

Added this as an explicit test.

Also fixed the dependencies for ceval.o -- it depends on executor_cases.c.h.
2023-07-14 17:22:06 -07:00
Irit Katriel 6a70edf24c
gh-105481: expose opcode metadata via the _opcode module (#106688) 2023-07-14 18:41:52 +01:00
Guido van Rossum 025995fead
gh-106529: Split FOR_ITER_{LIST,TUPLE} into uops (#106696)
Also rename `_ITER_EXHAUSTED_XXX` to `_IS_ITER_EXHAUSTED_XXX` to make it clear this is a test.
2023-07-13 17:27:35 -07:00
Guido van Rossum e6e0ea0113
gh-106701: Move the hand-written Tier 2 uops to bytecodes.c (#106702)
This moves EXIT_TRACE, SAVE_IP, JUMP_TO_TOP, and
_POP_JUMP_IF_{FALSE,TRUE} from ceval.c to bytecodes.c.

They are no less special than before, but this way
they are discoverable o the copy-and-patch tooling.
2023-07-13 12:14:51 -07:00
Mark Shannon 487861c6ae
GH-104909: Split `LOAD_ATTR_INSTANCE_VALUE` into micro-ops (GH-106678) 2023-07-13 16:36:19 +01:00
Guido van Rossum dd1884dc5d
gh-106529: Split FOR_ITER_RANGE into uops (#106638)
For an example of what this does for Tier 1 and Tier 2, see
https://github.com/python/cpython/issues/106529#issuecomment-1631649920
2023-07-12 10:23:59 -07:00
Guido van Rossum 7f55f58b6c
gh-106656: Remove --emit-line-directives from regen-cases (#106657)
If you prefer to see `#line` directives in generated_cases.c.h, run
```
make regen-cases CASESFLAG=-l
```
But please don't commit the result.
2023-07-12 16:12:39 +00:00
Mark Shannon b03755a234
GH-104909: Break LOAD_GLOBAL specializations in micro-ops. (GH-106677) 2023-07-12 14:34:14 +01:00