Commit Graph

8081 Commits

Author SHA1 Message Date
Mark Shannon 22b8d77b98
GH-100719: Remove redundant `gi_code` field from generator object. (GH-100749) 2023-02-23 10:19:01 +00:00
Mark Shannon 7c106a443f
GH-100982: Restrict `FOR_ITER_RANGE` to a single instruction to allow instrumentation. (GH-101985) 2023-02-22 11:11:57 +00:00
Owain Davies 7346a381be
gh-101903: Remove obsolete undefs for previously removed macros Py_EnterRecursiveCall and Py_LeaveRecursiveCall (#101923) 2023-02-21 11:58:47 +00:00
Irit Katriel 022b44f254
gh-102056: Fix a few bugs in error handling of exception printing code (#102078) 2023-02-20 22:16:09 +00:00
Steve Dower a99eb5cd99
gh-101907: Stop using `_Py_OPCODE` and `_Py_OPARG` macros (GH-101912)
* gh-101907: Removes use of non-standard C++ extension from Include/cpython/code.h

* Make cases_generator correct on Windows
2023-02-20 14:56:48 +00:00
Mark Dickinson b1b375e267
gh-97786: Fix compiler warnings in pytime.c (#101826)
Fixes compiler warnings in pytime.c.
2023-02-19 17:16:11 -08:00
Eclips4 89413bbccb
gh-101967: add a missing error check (#101968) 2023-02-18 00:52:23 +00:00
Eric Snow 4d8959b73a
gh-101758: Add _PyState_AddModule() Back for the Stable ABI (gh-101956)
We're adding the function back, only for the stable ABI symbol and not as any form of API. I had removed it yesterday.

This undocumented "private" function was added with the implementation for PEP 3121 (3.0, 2007) for internal use and later moved out of the limited API (3.6, 2016) and then into the internal API (3.9, 2019). I removed it completely yesterday, including from the stable ABI manifest (where it was added because the symbol happened to be exported). It's unlikely that anyone is using _PyState_AddModule(), especially any stable ABI extensions built against 3.2-3.5, but we're playing it safe.

https://github.com/python/cpython/issues/101758
2023-02-16 14:05:31 -07:00
Eclips4 68bd8c5e2e
gh-101952: Fix possible segfault in `BUILD_SET` opcode (#101958) 2023-02-16 09:46:43 -08:00
Rayyan Ansari 739c026f44
gh-101881: Support (non-)blocking read/write functions on Windows pipes (GH-101882)
* fileutils: handle non-blocking pipe IO on Windows

Handle erroring operations on non-blocking pipes by reading the _doserrno code.
Limit writes on non-blocking pipes that are too large.

* Support blocking functions on Windows

Use the GetNamedPipeHandleState and SetNamedPipeHandleState Win32 API functions to add support for os.get_blocking and os.set_blocking.
2023-02-16 14:52:24 +00:00
penguin_wwy df7ccf6138
gh-101928: fix crash in compiler on multi-line lambda in function call (#101933) 2023-02-16 11:31:41 +00:00
Gregory P. Smith 0b13575e74
gh-99108: Refactor _sha256 & _sha512 into _sha2. (#101924)
This merges their code. They're backed by the same single HACL* static library, having them be a single module simplifies maintenance.

This should unbreak the wasm enscripten builds that currently fail due to linking in --whole-archive mode and the HACL* library appearing twice.

Long unnoticed error fixed: _sha512.SHA384Type was doubly assigned and was actually SHA512Type. Nobody depends on those internal names.

Also rename LIBHACL_ make vars to LIBHACL_SHA2_ in preperation for other future HACL things.
2023-02-15 22:08:20 -08:00
Eric Snow 89ac665891
gh-98627: Add an Optional Check for Extension Module Subinterpreter Compatibility (gh-99040)
Enforcing (optionally) the restriction set by PEP 489 makes sense. Furthermore, this sets the stage for a potential restriction related to a per-interpreter GIL.

This change includes the following:

* add tests for extension module subinterpreter compatibility
* add _PyInterpreterConfig.check_multi_interp_extensions
* add Py_RTFLAGS_MULTI_INTERP_EXTENSIONS
* add _PyImport_CheckSubinterpIncompatibleExtensionAllowed()
* fail iff the module does not implement multi-phase init and the current interpreter is configured to check

https://github.com/python/cpython/issues/98627
2023-02-15 18:16:00 -07:00
Eric Snow 3dea4ba6c1
gh-101758: Fix the wasm Buildbots (gh-101943)
They were broken by gh-101920.

https://github.com/python/cpython/issues/101758
2023-02-15 17:54:05 -07:00
Eric Snow b365d88465
gh-101758: Add a Test For Single-Phase Init Modules in Multiple Interpreters (gh-101920)
The test verifies the behavior of single-phase init modules when loaded in multiple interpreters.

https://github.com/python/cpython/issues/101758
2023-02-15 16:05:07 -07:00
Eric Snow b2fc549278
gh-101758: Clean Up Uses of Import State (gh-101919)
This change is almost entirely moving code around and hiding import state behind internal API.  We introduce no changes to behavior, nor to non-internal API.  (Since there was already going to be a lot of churn, I took this as an opportunity to re-organize import.c into topically-grouped sections of code.)  The motivation is to simplify a number of upcoming changes.

Specific changes:

* move existing import-related code to import.c, wherever possible
* add internal API for interacting with import state (both global and per-interpreter)
* use only API outside of import.c (to limit churn there when changing the location, etc.)
* consolidate the import-related state of PyInterpreterState into a single struct field (this changes layout slightly)
* add macros for import state in import.c (to simplify changing the location)
* group code in import.c into sections
*remove _PyState_AddModule()

https://github.com/python/cpython/issues/101758
2023-02-15 15:32:31 -07:00
Erlend E. Aasland eb0c485b6c
gh-101819: Remove _PyWindowsConsoleIO_Type from the Windows DLL (GH-101904)
Automerge-Triggered-By: GH:erlend-aasland
2023-02-15 05:07:59 -08:00
Mark Shannon c7766245c1
GH-87849: Fix refleak in SEND instruction. (GH-101908)
Fix refleak in SEND instruction.
2023-02-15 12:21:40 +00:00
Eric Snow 096d0097a0
gh-101758: Add a Test For Single-Phase Init Module Variants (gh-101891)
The new test exercises the most important variants for single-phase init extension modules. We also add some explanation about those variants to import.c.

https://github.com/python/cpython/issues/101758
2023-02-14 14:26:03 -07:00
Irit Katriel 81e3aa835c
gh-101799: implement PREP_RERAISE_STAR as an intrinsic function (#101800) 2023-02-14 11:54:13 +00:00
James Lee 95cbb3d908
gh-101810: Remove duplicated st_ino calculation (GH-101811) 2023-02-13 13:49:44 +00:00
Mark Shannon 160f2fe2b9
GH-87849: Simplify stack effect of SEND and specialize it for generators and coroutines. (GH-101788) 2023-02-13 11:24:55 +00:00
Sergey B Kirpichev cb2411886a
gh-101670: typo fix in PyImport_ExtendInittab() (#101723)
Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2023-02-09 18:49:02 +02:00
Guido van Rossum 65b7b6bd23
gh-98831: Use opcode metadata for stack_effect() (#101704)
* Write output and metadata in a single run
  This halves the time to run the cases generator
  (most of the time goes into parsing the input).
* Declare or define opcode metadata based on NEED_OPCODE_TABLES
* Use generated metadata for stack_effect()
* compile.o depends on opcode_metadata.h
* Return -1 from _PyOpcode_num_popped/pushed for unknown opcode
2023-02-08 16:23:19 -08:00
Guido van Rossum 616aec1ff1
gh-98831: Modernize CALL and family (#101508)
Includes a slight improvement to `DECREF_INPUTS()`.
2023-02-08 11:40:10 -08:00
Sergey B Kirpichev 35dd55005e
gh-101670: typo fix in PyImport_AppendInittab() (GH-101672) 2023-02-08 07:49:04 -08:00
David Hewitt 3a88de7a0a
gh-101614: Don't treat python3_d.dll as a Python DLL when checking extension modules for incompatibility (GH-101615) 2023-02-08 14:23:57 +00:00
Mark Shannon feec49c407
GH-101578: Normalize the current exception (GH-101607)
* Make sure that the current exception is always normalized.

* Remove redundant type and traceback fields for the current exception.

* Add new API functions: PyErr_GetRaisedException, PyErr_SetRaisedException

* Add new API functions: PyException_GetArgs, PyException_SetArgs
2023-02-08 09:31:12 +00:00
Guido van Rossum a9f01448a9
gh-98831: Modernize CALL_FUNCTION_EX (#101627)
New generator feature: Move CHECK_EVAL_BREAKER() call to just before DISPATCH().
2023-02-07 20:03:22 -08:00
Guido van Rossum b2b85b5db9
gh-98831: Modernize FORMAT_VALUE (#101628)
Generator update: support balanced parentheses and brackets in conditions and size expressions.
2023-02-07 17:35:55 -08:00
Guido van Rossum aacbdb0c65
gh-98831: Finish the UNPACK_SEQUENCE family (#101666)
New generator feature: Generate useful glue for output arrays, so you can just write values to the output array (no bounds checking). Rewrote UNPACK_SEQUENCE_TWO_TUPLE to use this, and also UNPACK_SEQUENCE_{TUPLE,LIST}.
2023-02-07 15:44:37 -08:00
penguin_wwy 753fc8a5d6
gh-101632: Add the new RETURN_CONST opcode (#101633) 2023-02-07 22:32:21 +00:00
Irit Katriel dec1ab0387
gh-98831: rewrite UNPACK_EX, UNPACK_SEQUENCE, UNPACK_SEQUENCE_TWO_TUPLE in the instruction definition DSL (#101641) 2023-02-07 20:37:43 +00:00
Guido van Rossum d54b8d8fbd
gh-98831: Modernize the FOR_ITER family of instructions (#101626)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2023-02-07 08:28:28 -08:00
Matthieu Dartiailh ae62bddaf8
gh-101072: support default and kw default in PyEval_EvalCodeEx for 3.11+ (#101127)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2023-02-07 10:34:21 +01:00
Irit Katriel 38752760c9
gh-98831: rewrite COPY and SWAP in the instruction definition DSL (#101620) 2023-02-06 22:45:18 +00:00
Eric Snow 132b3f8302
gh-59956: Partial Fix for GILState API Compatibility with Subinterpreters (gh-101431)
The GILState API (PEP 311) implementation from 2003 made the assumption that only one thread state would ever be used for any given OS thread, explicitly disregarding the case of subinterpreters.  However, PyThreadState_Swap() still facilitated switching between subinterpreters, meaning the "current" thread state (holding the GIL), and the GILState thread state could end up out of sync, causing problems (including crashes).

This change addresses the issue by keeping the two in sync in PyThreadState_Swap().  I verified the fix against gh-99040.

Note that the other GILState-subinterpreter incompatibility (with autoInterpreterState) is not resolved here.

https://github.com/python/cpython/issues/59956
2023-02-06 14:39:25 -07:00
Irit Katriel 433fb3ef08
gh-98831: rewrite MAKE_FUNCTION and BUILD_SLICE in the instruction definition DSL (#101529) 2023-02-03 14:40:45 +00:00
Irit Katriel 04e06e20ee
gh-98831: rewrite SEND, GET_YIELD_FROM_ITER, RETURN_GENERATOR in the instruction definition DSL (#101516) 2023-02-03 11:30:21 +00:00
Irit Katriel 0675b8f032
gh-98831: rewrite RERAISE and CLEANUP_THROW in the instruction definition DSL (#101511) 2023-02-02 10:02:57 +00:00
Guido van Rossum ae9b38f424
gh-98831: Modernize the LOAD_GLOBAL family (#101502) 2023-02-01 13:12:49 -08:00
Irit Katriel b91b42d236
gh-98831: rewrite PUSH_EXC_INFO and conditional jumps in the instruction definition DSL (#101481) 2023-02-01 19:38:06 +00:00
Guido van Rossum 7840ff3cdb
gh-98831: Modernize the LOAD_ATTR family (#101488) 2023-02-01 10:56:52 -08:00
Brandt Bucher 76efcb4093
GH-100288: Skip extra work when failing to specialize LOAD_ATTR (GH-101354) 2023-01-31 13:28:32 -08:00
Irit Katriel 0062f538d9
gh-98831: rewrite BEFORE_ASYNC_WITH and END_ASYNC_FOR in the instruction definition DSL (#101458) 2023-01-31 18:47:50 +00:00
Irit Katriel 29a858b85f
gh-98831: rewrite GET_LEN, GET_ITER, BEFORE_WITH and a few simple opcodes in the instruction definition DSL (#101443) 2023-01-31 10:23:15 +00:00
Irit Katriel af7b2db384
gh-99955: use SUCCESS/ERROR return values in optimizer and assembler. Use RETURN_IF_ERROR where appropriate. Fix a couple of bugs. (#101412) 2023-01-31 00:06:19 +00:00
Dong-hee Na e867c1b753
gh-101400: Fix incorrect lineno in exception message on continue/break which are not in a loop (#101413) 2023-01-30 23:33:54 +00:00
Eric Snow e11fc032a7
gh-59956: Clarify Runtime State Status Expectations (gh-101308)
A PyThreadState can be in one of many states in its lifecycle, represented by some status value.  Those statuses haven't been particularly clear, so we're addressing that here.  Specifically:

* made the distinct lifecycle statuses clear on PyThreadState
* identified expectations of how various lifecycle-related functions relate to status
* noted the various places where those expectations don't match the actual behavior

At some point we'll need to address the mismatches.

(This change also includes some cleanup.)

https://github.com/python/cpython/issues/59956
2023-01-30 12:07:48 -07:00
Mark Shannon c1b1f51cd1
GH-101291: Refactor the `PyLongObject` struct into object header and PyLongValue struct. (GH-101292) 2023-01-30 10:03:04 +00:00
Guido van Rossum f5a3d91b6c
gh-98831: Support conditional effects; use for LOAD_ATTR (#101333) 2023-01-29 17:28:39 -08:00
Irit Katriel c4170c36b0
gh-39615: fix warning on return type mismatch (#101407) 2023-01-29 16:41:27 -08:00
Gregory P. Smith 052f53d65d
gh-39615: Add warnings.warn() skip_file_prefixes support (#100840)
`warnings.warn()` gains the ability to skip stack frames based on code
filename prefix rather than only a numeric `stacklevel=` via a new
`skip_file_prefixes=` keyword argument.
2023-01-27 18:35:14 -08:00
Виталий Дмитриев 37f15a5efa
Fix typos in pystate.c file (#101348) 2023-01-26 15:04:11 -08:00
Irit Katriel b400219df5
gh-98831: rewrite RAISE_VARARGS in the instruction definition DSL (#101306) 2023-01-25 22:29:56 +00:00
Irit Katriel 19f90d6b97
gh-98831: add variable stack effect support to cases generator (#101309) 2023-01-25 20:41:03 +00:00
Guido van Rossum 498598e8c2
Fix some comments in ceval.c and fix lltrace output (#101297)
The comment at the top was rather outdated. :-)

Also added a note about the dangers of dump_stack().
2023-01-24 14:58:18 -08:00
Irit Katriel 1a9d8c750b
gh-98831: rewrite pattern matching opcodes in the instruction definition DSL (#101287) 2023-01-24 22:39:13 +00:00
Mark Shannon f02fa64bf2
GH-100762: Don't call `gen.throw()` in `gen.close()`, unless necessary. (GH-101013)
* Store exception stack depth in YIELD_VALUE's oparg and use it avoid expensive gen.throw() in gen.close() where possible.
2023-01-24 17:25:37 +00:00
Irit Katriel 8c183cddd3
gh-98831: rewrite CHECK_EG_MATCH opcode in the instruction definition DSL (#101269) 2023-01-24 09:43:16 +00:00
Eric Snow 7b20a0f55a
gh-59956: Allow the "Trashcan" Mechanism to Work Without a Thread State (gh-101209)
We've factored out a struct from the two PyThreadState fields. This accomplishes two things:

* make it clear that the trashcan-related code doesn't need any other parts of PyThreadState
* allows us to use the trashcan mechanism even when there isn't a "current" thread state

We still expect the caller to hold the GIL.

https://github.com/python/cpython/issues/59956
2023-01-23 08:30:20 -07:00
Ikko Eltociear Ashimine 4510fbb216
Fix typo in comment in compile.c (#101194) 2023-01-20 18:34:36 +00:00
Nikita Sobolev 8be6992620
gh-101181: Fix `unused-variable` warning in `pystate.c` (#101188)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
2023-01-20 23:31:30 +05:30
Eric Snow f30c94024f
gh-59956: Fix Function Groupings in pystate.c (gh-101172)
This is a follow-up to gh-101161.  The objective is to make it easier to read Python/pystate.c by grouping the functions there in a consistent way.  This exclusively involves moving code around and adding various kinds of comments.

https://github.com/python/cpython/issues/59956
2023-01-19 17:23:53 -07:00
Irit Katriel 9ec9b203ea
gh-101169: reduce the implementation of except* by one bytecode instruction (#101170) 2023-01-19 23:30:49 +00:00
Eric Snow 6036c3e856
gh-59956: Clarify GILState-related Code (gh-101161)
The objective of this change is to help make the GILState-related code easier to understand.  This mostly involves moving code around and some semantically equivalent refactors.  However, there are a also a small number of slight changes in structure and behavior:

* tstate_current is moved out of _PyRuntimeState.gilstate
* autoTSSkey is moved out of _PyRuntimeState.gilstate
* autoTSSkey is initialized earlier
* autoTSSkey is re-initialized (after fork) earlier

https://github.com/python/cpython/issues/59956
2023-01-19 16:04:14 -07:00
Irit Katriel e9ccfe4a63
gh-100712: make it possible to disable specialization (for debugging) (#100713) 2023-01-19 18:14:55 +00:00
Guido van Rossum 1f0d0a432c
GH-98831: Move assorted macros from ceval.h to a new header (#101116) 2023-01-18 10:41:07 -08:00
Guido van Rossum 80e3e3423c
GH-98831: Implement array support in cases generator (#100912)
You can now write things like this:
```
inst(BUILD_STRING, (pieces[oparg] -- str)) { ... }
inst(LIST_APPEND, (list, unused[oparg-1], v -- list, unused[oparg-1])) { ... }
```
Note that array output effects are only partially supported (they must be named `unused` or correspond to an input effect).
2023-01-17 15:59:19 -08:00
Mark Shannon 7b14c2ef19
GH-100982: Add `COMPARE_AND_BRANCH` instruction (GH-100983) 2023-01-16 12:35:21 +00:00
Kumar Aditya f6307d4416
GH-100892: consolidate `HEAD_LOCK/HEAD_UNLOCK` macros (#100953) 2023-01-15 20:39:26 +05:30
Guido van Rossum 5134ef4878
GH-98831: Identify instructions that don't use oparg (#100957)
For these the instr_format field uses IX instead of IB.
Register instructions use IX, IB, IBBX, IBBB, etc.

Also: Include the closing '}' in Block.tokens, for completeness
2023-01-13 17:06:45 -08:00
Steve Dower b5d4347950
gh-86682: Adds sys._getframemodulename as an alternative to using _getframe (GH-99520)
Also updates calls in collections, doctest, enum, and typing modules to use _getframemodulename first when available.
2023-01-13 11:31:06 +00:00
Mark Shannon 6e4e14d98f
GH-100923: Embed jump mask in `COMPARE_OP` oparg (GH-100924) 2023-01-11 20:40:43 +00:00
Guido van Rossum 3f3c78e32f
GH-98831: Refactor instr format code and change to enum (#100895) 2023-01-09 15:53:01 -08:00
Brandt Bucher 61762b9387
GH-100126: Skip incomplete frames in more places (GH-100613) 2023-01-09 12:20:04 -08:00
Nikita Sobolev bc0a686f82
gh-87447: Fix walrus comprehension rebind checking (#100581)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2023-01-08 15:51:29 -07:00
Benjamin Peterson 11f99323c2
Update copyright years to 2023. (gh-100848) 2023-01-08 09:13:25 -06:00
Nnarol 0741da8d28
GH-90829: Fix empty iterable error message in min/max (#31181) 2023-01-08 19:21:20 +05:30
Shantanu a2141882f2
gh-100776: Fix misleading default value in help(input) (#100788) 2023-01-08 13:27:41 +05:30
Carl Meyer 7116030a25
gh-88696: clean up dead argument to compiler_make_closure (GH-100806) 2023-01-07 12:20:48 +00:00
Irit Katriel 15c44789bb
gh-100758: Refactor initialisation of frame headers into a single function (_PyFrame_Initialize) (GH-100759) 2023-01-06 14:55:56 +00:00
Mark Shannon 78068126a1
GH-99005: More intrinsics (GH-100774)
* Remove UNARY_POSITIVE, LIST_TO_TUPLE and ASYNC_GEN_WRAP, replacing them with intrinsics.
2023-01-06 14:47:57 +00:00
Zachary Ware af5149f30b
gh-98831: Regenerate Python/opcode_metadata.h (GH-100778) 2023-01-05 15:47:18 -06:00
Guido van Rossum 14b7f00fdf
GH-98831: Update generate_cases.py: register inst, opcode_metadata.h (#100735)
(These aren't used yet, but may be coming soon,
and it's easier to keep this tool the same between branches.)

Added a sanity check for all this to compile.c.

Co-authored-by: Irit Katriel <iritkatriel@yahoo.com>
2023-01-05 13:01:07 -08:00
Mark Shannon 28187141cc
GH-99005: Add `CALL_INTRINSIC_1` instruction (GH-100771)
* Remove PRINT_EXPR instruction

* Remove STOPITERATION_ERROR instruction

* Remove IMPORT_STAR instruction
2023-01-05 16:05:51 +00:00
Mark Shannon f20c553a45
GH-100288: Remove LOAD_ATTR_METHOD_WITH_DICT instruction. (GH-100753) 2023-01-05 12:20:09 +00:00
Irit Katriel 52017dbe16
gh-100747: some compiler macros use c instead of C to access the compiler (#100748) 2023-01-04 20:19:20 +00:00
Mark Shannon 15aecf8dd7
GH-100719: Remove the `co_nplaincellvars` field from code objects. (GH-100721) 2023-01-04 15:41:39 +00:00
L. A. F. Pereira e6d4440782
gh-100146: Steal references from stack when building a list (#100147)
When executing the BUILD_LIST opcode, steal the references from the stack,
in a manner similar to the BUILD_TUPLE opcode.  Implement this by offloading
the logic to a new private API, _PyList_FromArraySteal(), that works similarly
to _PyTuple_FromArraySteal().

This way, instead of performing multiple stack pointer adjustments while the
list is being initialized, the stack is adjusted only once and a fast memory
copy operation is performed in one fell swoop.
2023-01-03 10:49:49 -08:00
Éric 46521826cb
gh-95778: add doc missing in some places (GH-100627) 2022-12-30 13:21:15 -08:00
Nikita Sobolev 5e1adb4f88
gh-100577: Replace `assert(0)` with `Py_UNREACHABLE` in `symtable.c` (#100579) 2022-12-28 21:01:53 +05:30
Guido van Rossum 08e5594cf3
GH-98831: Modernize a ton of simpler instructions (#100545)
* load_const and load_fast aren't families for now
* Don't decref unmoved names
* Modernize GET_ANEXT
* Modernize GET_AWAITABLE
* Modernize ASYNC_GEN_WRAP
* Modernize YIELD_VALUE
* Modernize POP_EXCEPT (in more than one way)
* Modernize PREP_RERAISE_STAR
* Modernize LOAD_ASSERTION_ERROR
* Modernize LOAD_BUILD_CLASS
* Modernize STORE_NAME
* Modernize LOAD_NAME
* Modernize LOAD_CLASSDEREF
* Modernize LOAD_DEREF
* Modernize STORE_DEREF
* Modernize COPY_FREE_VARS (mark it as done)
* Modernize LIST_TO_TUPLE
* Modernize LIST_EXTEND
* Modernize SET_UPDATE
* Modernize SETUP_ANNOTATIONS
* Modernize DICT_UPDATE
* Modernize DICT_MERGE
* Modernize MAP_ADD
* Modernize IS_OP
* Modernize CONTAINS_OP
* Modernize CHECK_EXC_MATCH
* Modernize IMPORT_NAME
* Modernize IMPORT_STAR
* Modernize IMPORT_FROM
* Modernize JUMP_FORWARD (mark it as done)
* Modernize JUMP_BACKWARD (mark it as done)
2022-12-27 17:11:03 -08:00
Nikita Sobolev bdfb694386
gh-100357: Convert several functions in `bltinsmodule` to AC (#100358) 2022-12-24 06:45:47 -08:00
Sebastian Berg 474220e3a5
gh-99947: Ensure unreported errors are chained for SystemError during import (GH-99946) 2022-12-23 15:43:19 -08:00
Raymond Hettinger 5d84966cce
GH-100425: Improve accuracy of builtin sum() for float inputs (GH-100426) 2022-12-23 14:35:58 -08:00
Ken Jin 36d358348d
Revert "gh-100288: Specialise LOAD_ATTR_METHOD for managed dictionaries (GH-100289)" (#100468)
This reverts commit c3c7848a48.
2022-12-24 01:48:43 +08:00
Ken Jin c3c7848a48
gh-100288: Specialise LOAD_ATTR_METHOD for managed dictionaries (GH-100289) 2022-12-24 00:26:42 +08:00
Irit Katriel 2659036c75
GH-100459: fix copy-paste errors in specialization stats (GH-100460) 2022-12-23 14:42:24 +00:00
Brandt Bucher 09edde95f4
GH-99554: Trim trailing whitespace (GH-100435)
Automerge-Triggered-By: GH:brandtbucher
2022-12-22 12:13:51 -08:00
penguin_wwy a02161286a
GH-99770: Make the correct call specialization fail kind show up in the stats (GH-99771) 2022-12-22 12:44:59 +00:00
Pieter Eendebak 2b82c36f17
gh-99761: Add `_PyLong_IsPositiveSingleDigit` function to check for single digit integers (#100064) 2022-12-22 17:00:18 +05:30
Brandt Bucher 3c033a2e6f
GH-99554: Pack location tables more effectively (GH-99556) 2022-12-21 16:41:18 -08:00
Dennis Sweeney c18d831188
gh-100188: Reduce misses in BINARY_SUBSCR_(LIST/TUPLE)_INT (#100189)
Don't specialize if the index is negative.
2022-12-20 15:46:16 -05:00
Jeremy Paige 1cf3d78c92
Correct CVE-2020-10735 documentation (#100306) 2022-12-18 12:41:21 +05:30
colorfulappl efbb1eb9f5
gh-99240: Reset pointer to NULL when the pointed memory is freed in argument parsing (#99890)
Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2022-12-17 12:07:44 +05:30
Guido van Rossum 9cdd2fa63b
GH-98831: Add DECREF_INPUTS(), expanding to DECREF() each stack input (#100205)
The presence of this macro indicates that a particular instruction
may be considered for conversion to a register-based format
(see https://github.com/faster-cpython/ideas/issues/485).

An invariant (currently unchecked) is that `DEOPT_IF()` may only
occur *before* `DECREF_INPUTS()`, and `ERROR_IF()` may only occur
*after* it. One reason not to check this is that there are a few
places where we insert *two* `DECREF_INPUTS()` calls, in different
branches of the code. The invariant checking would have to be able
to do some flow control analysis to understand this.

Note that many instructions, especially specialized ones,
can't be converted to use this macro straightforwardly.
This is because the generator currently only generates plain
`Py_DECREF(variable)` statements, and cannot generate
things like `_Py_DECREF_SPECIALIZED()` let alone deal with
`_PyList_AppendTakeRef()`.
2022-12-16 20:45:55 -08:00
Brandt Bucher 9076455d1b
GH-90043: Handle NaNs in COMPARE_OP_FLOAT_JUMP (GH-100278) 2022-12-16 10:18:31 -08:00
Mark Shannon d4052d835b
Improve stats presentation for calls. (GH-100274) 2022-12-16 15:43:04 +00:00
Mark Shannon 289c1126dd
Better stats for `LOAD_ATTR` and `STORE_ATTR` (GH-100295)
* Don't attempt to specialize for LOAD_ATTR on instance if class has attribute

* Improvement to LOAD_ATTR and STORE_ATTR specialization stats.
2022-12-16 15:41:23 +00:00
Mark Shannon 48e352a241
Move stats for the method cache into the `Py_STAT` machinery (GH-100255) 2022-12-15 09:45:03 +00:00
Carl Meyer bdd86741be
GH-100222: fix typo _py_set_opocde -> _py_set_opcode (GH-100259)
Typo introduced in #100223.

Automerge-Triggered-By: GH:brandtbucher
2022-12-14 16:39:00 -08:00
Itamar Ostricher ae83c78215
GH-100000: Cleanup and polish various watchers code (GH-99998)
* Initialize `type_watchers` array to `NULL`s
* Optimize code watchers notification
* Optimize func watchers notification
2022-12-14 19:14:16 +00:00
Mark Shannon 5693f45b19
Assorted minor fixes for specialization stats. (GH-100219) 2022-12-14 15:50:02 +00:00
Mark Shannon 6997e77bdf
GH-100222: Redefine _Py_CODEUNIT as a union to clarify structure of code unit. (GH-100223) 2022-12-14 11:12:53 +00:00
Irit Katriel 985a71032b
gh-99955: undef ERROR and SUCCESS before redefining (fixes sanitizer warning) (#100215) 2022-12-13 13:55:10 +00:00
Eric Snow 5eb28bca9f
gh-81057: Move Signal-Related Globals to _PyRuntimeState (gh-100085)
https://github.com/python/cpython/issues/81057
2022-12-12 16:50:19 -07:00
Michael Droettboom 1583c6e326
GH-100143: Improve collecting pystats for parts of runs (GH-100144)
* pystats off by default

* Add -Xpystats flag

* Always dump pystats, even if turned off
2022-12-12 14:50:43 +00:00
Irit Katriel e4ea33b178
gh-99955: standardize return values of functions in compiler's code-gen (#100010) 2022-12-12 14:22:15 +00:00
chgnrdv 3221b0de67
gh-96715 Remove redundant NULL check in `profile_trampoline` function (#96716)
Closes #96715
2022-12-12 17:29:27 +05:30
Eric Snow 8d0bd93ae2
gh-81057: Fix the wasm32-wasi Buildbot (gh-100139)
The build was broken by gh-100084.

https://github.com/python/cpython/issues/81057
2022-12-09 10:17:54 -07:00
Mark Shannon fb713b2183
GH-98522: Add version number to code objects. (GH-98525)
* Add version number to code object for better versioning of functions.

* Improves specialization for closures and list comprehensions.
2022-12-09 12:18:45 +00:00
Ken Jin 748c6c0921
GH-100110: Specialize FOR_ITER for tuples (GH-100109)
* Specialize FOR_ITER for tuples
2022-12-09 10:27:01 +00:00
Eric Snow 1160001b34
gh-81057: Move Threading-Related Globals to _PyRuntimeState (#100084)
https://github.com/python/cpython/issues/81057
2022-12-08 17:50:58 -07:00
Eric Snow bc8cdf8c3d
gh-81057: Move Ceval Trampoline Globals to _PyRuntimeState (gh-100083)
https://github.com/python/cpython/issues/81057
2022-12-08 17:17:20 -07:00
Guido van Rossum 1cfa704f64
GH-98831: Generate things in the input order (#100123)
This makes it easier to see what changed in the generated code
when converting an instruction to super or macro.
2022-12-08 15:54:07 -08:00
Eric Snow 8a3f06c54b
gh-81057: Move time Globals to _PyRuntimeState (gh-100122)
https://github.com/python/cpython/issues/81057
2022-12-08 16:46:09 -07:00
Guido van Rossum c85be734d1
GH-98831: Typed stack effects, and more instructions converted (#99764)
Stack effects can now have a type, e.g. `inst(X, (left, right -- jump/uint64_t)) { ... }`.

Instructions converted to the non-legacy format:

* COMPARE_OP
* COMPARE_OP_FLOAT_JUMP
* COMPARE_OP_INT_JUMP
* COMPARE_OP_STR_JUMP
* STORE_ATTR
* DELETE_ATTR
* STORE_GLOBAL
* STORE_ATTR_INSTANCE_VALUE
* STORE_ATTR_WITH_HINT
* STORE_ATTR_SLOT, and complete the store_attr family
* Complete the store_subscr family: STORE_SUBSCR{,DICT,LIST_INT}
  (STORE_SUBSCR was alread half converted,
  but wasn't using cache effects yet.)
* DELETE_SUBSCR
* PRINT_EXPR
* INTERPRETER_EXIT (a bit weird, ends in return)
* RETURN_VALUE
* GET_AITER (had to restructure it some)
  The original had mysterious `SET_TOP(NULL)` before `goto error`.
  I assume those just account for `obj` having been decref'ed,
  so I got rid of them in favor of the cleanup implied by `ERROR_IF()`.
* LIST_APPEND (a bit unhappy with it)
* SET_ADD (also a bit unhappy with it)

Various other improvements/refactorings as well.
2022-12-08 13:31:27 -08:00
Eric Snow 91a8e002c2
gh-81057: Move More Globals to _PyRuntimeState (gh-100092)
https://github.com/python/cpython/issues/81057
2022-12-07 15:56:31 -07:00
Brandt Bucher b72014c783
GH-99729: Unlink frames before clearing them (GH-100030) 2022-12-06 14:01:38 +00:00
Eric Snow 530cc9dbb6
gh-99741: Implement Multi-Phase Init for the _xxsubinterpreters Module (gh-99742)
_xxsubinterpreters is an internal module used for testing.

https://github.com/python/cpython/issues/99741
2022-12-05 13:40:20 -07:00
Serhiy Storchaka a87c46eab3
bpo-15999: Accept arbitrary values for boolean parameters. (#15609)
builtins and extension module functions and methods that expect boolean values for parameters now accept any Python object rather than just a bool or int type. This is more consistent with how native Python code itself behaves.
2022-12-03 11:52:21 -08:00
Guido van Rossum acf9184e6b
GH-98831: Support cache effects in super- and macro instructions (#99601) 2022-12-02 19:57:30 -08:00
Eric Snow 0547a981ae
gh-99741: Clean Up the _xxsubinterpreters Module (gh-99940)
This cleanup up resolves a few subtle bugs and makes the implementation for multi-phase init much cleaner.

https://github.com/python/cpython/issues/99741
2022-12-02 11:36:57 -07:00
Irit Katriel ab02262cd0
gh-99955: standardize return values of functions in assembler and optimizer. (#99956) 2022-12-02 17:43:10 +00:00
Eric Snow b4f3505549
gh-99741: Fix the Cross-Interpreter Data API (gh-99939)
There were some minor issues that showed up while I was working on porting _xxsubinterpreters to multi-phase init. This fixes them.

https://github.com/python/cpython/issues/99741
2022-12-02 10:39:17 -07:00
Itamar Ostricher 3c137dc613
GH-91054: Add code object watchers API (GH-99859)
* Add API to allow extensions to set callback function on creation and destruction of PyCodeObject

Co-authored-by: Ye11ow-Flash <janshah@cs.stonybrook.edu>
2022-12-02 17:28:27 +00:00
Serhiy Storchaka 787764219f
gh-89189: More compact range iterator (GH-27986) 2022-11-30 23:04:30 +02:00
Irit Katriel 18a6967544
GH-99877) 2022-11-30 16:16:54 +00:00
Irit Katriel ac12e3941f
gh-87092: move all localsplus preparation into separate function called from assembler stage (GH-99869) 2022-11-30 12:37:30 +00:00
Kumar Aditya fe17d35313
GH-81057: remove static state from suggestions.c (#99411) 2022-11-30 16:55:16 +05:30
colorfulappl 0da728387c
gh-64490: Fix bugs in argument clinic varargs processing (#32092) 2022-11-24 20:56:50 +01:00
Irit Katriel ae185fdcca
gh-99708: fix bug where compiler crashes on if expression with an empty body block (GH-99732) 2022-11-24 10:59:07 +00:00
Victor Stinner 81f7359f67
gh-99537: Use Py_SETREF(var, NULL) in C code (#99687)
Replace "Py_DECREF(var); var = NULL;" with "Py_SETREF(var, NULL);".
2022-11-23 14:57:50 +01:00
Guido van Rossum 8f18ac04d3
GH-98831: Add `macro` and `op` and their implementation to DSL (#99495)
Newly supported interpreter definition syntax:
- `op(NAME, (input_stack_effects -- output_stack_effects)) { ... }`
- `macro(NAME) = OP1 + OP2;`

Also some other random improvements:
- Convert `WITH_EXCEPT_START` to use stack effects
- Fix lexer to balk at unrecognized characters, e.g. `@`
- Fix moved output names; support object pointers in cache
- Introduce `error()` method to print errors
- Introduce read_uint16(p) as equivalent to `*p`

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2022-11-22 16:04:57 -08:00
Skip Montanaro d4cf192826
gh-88226: Emit TARGET labels in Python/ceval.c when debugging, even if computed gotos aren't enabled (GH-98265)
Keep target labels when debugging, but don't warn about lack of use.

Co-authored-by: Eryk Sun <eryksun@gmail.com>
2022-11-22 22:13:54 +02:00
Victor Stinner 135ec7cefb
gh-99537: Use Py_SETREF() function in C code (#99657)
Fix potential race condition in code patterns:

* Replace "Py_DECREF(var); var = new;" with "Py_SETREF(var, new);"
* Replace "Py_XDECREF(var); var = new;" with "Py_XSETREF(var, new);"
* Replace "Py_CLEAR(var); var = new;" with "Py_XSETREF(var, new);"

Other changes:

* Replace "old = var; var = new; Py_DECREF(var)"
  with "Py_SETREF(var, new);"
* Replace "old = var; var = new; Py_XDECREF(var)"
  with "Py_XSETREF(var, new);"
* And remove the "old" variable.
2022-11-22 13:39:11 +01:00
mpage 3db0a21f73
gh-91053: Add an optional callback that is invoked whenever a function is modified (#98175) 2022-11-22 13:06:44 +01:00
Victor Stinner cb2ef8b2ac
gh-99578: Fix refleak in _imp.create_builtin() (#99642)
Fix a reference bug in _imp.create_builtin() after the creation of
the first sub-interpreter for modules "builtins" and "sys".
2022-11-21 12:14:54 +01:00
Ronald Oussoren cdde29dde9
gh-99337: Fix compile errors with gcc 12 on macOS (#99470)
Fix a number of compile errors with GCC-12 on macOS:

1. In pylifecycle.c the compile rejects _Pragma within a declaration
2. posixmodule.c was missing a number of ..._RUNTIME macros for non-clang on macOS
3. _ctypes assumed that __builtin_available is always present on macOS
2022-11-21 10:50:20 +01:00
Guido van Rossum 4f5e1cb00a
GH-98831: Refactor and fix cases generator (#99526)
Also complete cache effects for BINARY_SUBSCR family.
2022-11-17 17:06:07 -08:00
Brandt Bucher b629fdd88a
GH-99298: Clean up attribute specializations (GH-99398) 2022-11-17 15:09:18 -08:00
Brandt Bucher 8555dee5ae
GH-98686: Get rid of BINARY_OP_GENERIC and COMPARE_OP_GENERIC (GH-99399) 2022-11-17 11:36:57 -08:00
Brandt Bucher 6f8b0e781c
Add a macro for "inlining" new frames (GH-99490) 2022-11-17 11:36:03 -08:00
Steve Dower 5fdd49dc65
gh-99377: Revert audit events for thread state creation and free, because the GIL is not properly held at these times (GH-99543) 2022-11-17 00:24:16 +00:00
Eric Snow 9db1e17c80
gh-81057: Move the global Dict-Related Versions to _PyRuntimeState (gh-99497)
We also move the global func version.

https://github.com/python/cpython/issues/81057
2022-11-16 10:37:29 -07:00
Victor Stinner 8211cf5d28
gh-99300: Replace Py_INCREF() with Py_NewRef() (#99530)
Replace Py_INCREF() and Py_XINCREF() using a cast with Py_NewRef()
and Py_XNewRef().
2022-11-16 18:34:24 +01:00
Steve Dower 19c1462e8d
gh-99377: Add audit events for thread creation and clear (GH-99378) 2022-11-16 17:15:52 +00:00
Eric Snow 01fa907aa8
gh-81057: Move contextvars-related Globals to _PyRuntimeState (gh-99400)
This is part of the effort to consolidate global variables, to make them easier to manage (and make it easier to later move some of them to PyInterpreterState).

https://github.com/python/cpython/issues/81057
2022-11-16 09:54:28 -07:00
Eric Snow 5f55067e23
gh-81057: Move More Globals in Core Code to _PyRuntimeState (gh-99516)
https://github.com/python/cpython/issues/81057
2022-11-16 09:37:14 -07:00
Guido van Rossum e37744f289
GH-98831: Implement basic cache effects (#99313) 2022-11-15 19:59:19 -08:00
Hood Chatham 86a49e0673
gh-99460 Emscripten trampolines on optimized METH_O and METH_NOARGS code paths (#99461) 2022-11-15 09:53:39 -08:00
Eric Snow 3c57971a2d
gh-81057: Move Globals in Core Code to _PyRuntimeState (gh-99496)
This is the first of several changes to consolidate non-object globals in core code.

https://github.com/python/cpython/issues/81057
2022-11-15 09:45:11 -07:00
Victor Stinner f13f466474
gh-99300: Use Py_NewRef() in Python/Python-ast.c (#99499)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in Python/Python-ast.c.

Update Parser/asdl_c.py to regenerate code.
2022-11-15 10:29:56 +01:00
Kumar Aditya dc3e4350a5
GH-99205: remove `_static` field from `PyThreadState` and `PyInterpreterState` (GH-99385) 2022-11-14 16:35:37 -08:00
Eric Snow e874c2f198
gh-81057: Move the Remaining Import State Globals to _PyRuntimeState (gh-99488)
https://github.com/python/cpython/issues/81057
2022-11-14 15:56:16 -07:00
Steve Dower 4e4b13e8f6
gh-87604: Avoid publishing list of active per-interpreter audit hooks via the gc module (GH-99373) 2022-11-14 21:39:18 +00:00
Eric Snow a088290f9d
gh-81057: Move Global Variables Holding Objects to _PyRuntimeState. (gh-99487)
This moves nearly all remaining object-holding globals in core code (other than static types).

https://github.com/python/cpython/issues/81057
2022-11-14 13:50:56 -07:00
Irit Katriel a3ac9232f8
gh-87092: expose the compiler's codegen to python for unit tests (GH-99111) 2022-11-14 13:56:40 +00:00
Batuhan Taskaya 57be545959
gh-99103: Normalize specialized traceback anchors against the current line (GH-99145)
Automerge-Triggered-By: GH:isidentical
2022-11-12 15:37:25 -08:00
Irit Katriel fb844e1931
gh-98762: Fix locations of match sub-patterns (GH-98775) 2022-11-12 18:42:17 +00:00
Eric Snow 7f3a4b967c
gh-81057: Move PyImport_Inittab to _PyRuntimeState (gh-99402)
We actually don't move PyImport_Inittab.  Instead, we make a copy that we keep on _PyRuntimeState and use only that after Py_Initialize().  We also prevent folks from modifying PyImport_Inittab (the best we can) after that point.

https://github.com/python/cpython/issues/81057
2022-11-11 17:06:05 -07:00
Eric Snow 67807cfc87
gh-81057: Move the Allocators to _PyRuntimeState (gh-99217)
The global allocators were stored in 3 static global variables: _PyMem_Raw, _PyMem, and _PyObject.  State for the "small block" allocator was stored in another 13.  That makes a total of 16 global variables. We are moving all 16 to the _PyRuntimeState struct as part of the work for gh-81057.  (If PEP 684 is accepted then we will follow up by moving them all to PyInterpreterState.)

https://github.com/python/cpython/issues/81057
2022-11-11 16:30:46 -07:00
Eric Snow f531b6879b
gh-81057: Add PyInterpreterState.static_objects (gh-99397)
As we consolidate global variables, we find some objects that are almost suitable to add to _PyRuntimeState.global_objects, but have some small/sneaky bit of per-interpreter state (e.g. a weakref list). We're adding PyInterpreterState.static_objects so we can move such objects there. (We'll removed the _not_used field once we've added others.)

https://github.com/python/cpython/issues/81057
2022-11-11 14:24:18 -07:00
Eric Snow dd36b71fa6
gh-81057: Move the Extension Modules Cache to _PyRuntimeState (gh-99355)
We also move the closely related max_module_number and add comments documenting the group of struct members.

https://github.com/python/cpython/issues/81057
2022-11-11 14:16:28 -07:00
Irit Katriel 3dd6ee2c00
gh-99254: remove all unused consts from code objects (GH-99255) 2022-11-11 10:53:43 +00:00
Guido van Rossum 694cdb24a6
GH-98831: Remove all remaining DISPATCH() calls from bytecodes.c (#99271)
Also mark those opcodes that have no stack effect as such.

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2022-11-10 10:50:57 -08:00
Brandt Bucher 00ee6d506e
GH-99298: Don't perform jumps before error handling (GH-99299) 2022-11-10 08:46:56 -08:00
Mark Shannon 1e197e63e2
GH-96421: Insert shim frame on entry to interpreter (GH-96319)
* Adds EXIT_INTERPRETER instruction to exit PyEval_EvalDefault()

* Simplifies RETURN_VALUE, YIELD_VALUE and RETURN_GENERATOR instructions as they no longer need to check for entry frames.
2022-11-10 12:34:57 +00:00
Brandt Bucher 9d69284169
GH-99257: Check the owner's type when specializing slots (GH-99258) 2022-11-10 11:50:34 +00:00
Victor Stinner 2e343fc465
gh-99300: Use Py_NewRef() in Python/ceval.c (#99318)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in Python/ceval.c and related files.
2022-11-10 11:25:33 +01:00
Victor Stinner 231d83b724
gh-99300: Use Py_NewRef() in Python/ directory (#99317)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in C files of the Python/ directory.

Update Parser/asdl_c.py to regenerate Python/Python-ast.c.
2022-11-10 11:23:36 +01:00
Victor Stinner d8f239d86e
gh-99300: Use Py_NewRef() in Python/ directory (#99302)
Replace Py_INCREF() and Py_XINCREF() with Py_NewRef() and
Py_XNewRef() in C files of the Python/ directory.
2022-11-10 09:03:39 +01:00
Brandt Bucher 283ab0e1c0
GH-99205: Mark new interpreters and threads as non-static (GH-99268) 2022-11-09 13:55:20 -08:00
Brandt Bucher c7f5708714
GH-98686: Get rid of "adaptive" and "quick" instructions (GH-99182) 2022-11-09 10:50:09 -08:00
Eric Snow 52f91c642b
gh-90868: Adjust the Generated Objects (gh-99223)
We do the following:

* move the generated _PyUnicode_InitStaticStrings() to its own file
* move the generated _PyStaticObjects_CheckRefcnt() to its own file
* include pycore_global_objects.h in extension modules instead of pycore_runtime_init.h

These changes help us avoid including things that aren't needed.

https://github.com/python/cpython/issues/90868
2022-11-08 10:03:03 -07:00
Guido van Rossum f1a654648b
GH-98831: Simple input-output stack effects for bytecodes.c (#99120) 2022-11-08 08:22:56 -08:00
Nikita Sobolev e56e33d271
gh-72719: Remove asyncore and asynchat modules (#96580)
Remove modules asyncore and asynchat, which were deprecated by PEP 594.

Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-11-08 16:48:58 +01:00
Mark Shannon 4a1c58d504
GH-96793: Specialize FOR_ITER for generators. (GH-98772) 2022-11-07 14:49:51 +00:00
Guido van Rossum 7dcd28eb41
GH-98831: Implement super-instruction generation (#99084)
Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
2022-11-06 09:40:47 -08:00
Pablo Galindo Salgado 99e2e60cb2
gh-99139: Improve NameError error suggestion for instances (#99140) 2022-11-06 13:52:06 +00:00
Guido van Rossum d04899abb0
GH-99104: Update headers for bytecodes.c and generate_cases.py (#99112)
Also tweak the labels near the end of bytecodes.c.
2022-11-04 17:40:43 -07:00
Brandt Bucher 6a8d3c57af
GH-98686: Fix compiler warning for HAS_ARG (GH-99106) 2022-11-04 16:54:32 -07:00
Mark Shannon 7a020b8e5b
GH-98831: Add some macros definitions to bytecodes.c to reduce IDE warnings. (#99093) 2022-11-04 16:37:53 -07:00
Guido van Rossum c885623e9f
GH-98831: Auto-generate PREDICTED() macro calls (#99102)
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2022-11-04 15:30:17 -07:00
Victor Stinner 0faa0ba240
gh-92584: Remove the distutils package (#99061)
Remove the distutils package. It was deprecated in Python 3.10 by PEP
632 "Deprecate distutils module". For projects still using distutils
and cannot be updated to something else, the setuptools project can
be installed: it still provides distutils.

* Remove Lib/distutils/ directory
* Remove test_distutils
* Remove references to distutils
* Skip test_check_c_globals and test_peg_generator since they use
  distutils
2022-11-03 19:27:27 +01:00
Victor Stinner b07f546ea3
gh-98978: Fix Py_SetPythonHome(NULL) (#99066)
Fix use-after-free in Py_SetPythonHome(NULL), Py_SetProgramName(NULL)
and _Py_SetProgramFullPath(NULL) function calls.

Issue reported by Benedikt Reinartz.
2022-11-03 18:34:32 +01:00
Mark Shannon f4adb97506
GH-96793: Implement PEP 479 in bytecode. (GH-99006)
* Handle converting StopIteration to RuntimeError in bytecode.

* Add custom instruction for converting StopIteration into RuntimeError.
2022-11-03 11:38:51 +00:00
Guido van Rossum 41bc101dd6
GH-98831: "Generate" the interpreter (#98830)
The switch cases (really TARGET(opcode) macros) have been moved from ceval.c to generated_cases.c.h. That file is generated from instruction definitions in bytecodes.c (which impersonates a C file so the C code it contains can be edited without custom support in e.g. VS Code).

The code generator lives in Tools/cases_generator (it has a README.md explaining how it works). The DSL used to describe the instructions is a work in progress, described in https://github.com/faster-cpython/ideas/blob/main/3.12/interpreter_definition.md.

This is surely a work-in-progress. An easy next step could be auto-generating super-instructions.

**IMPORTANT: Merge Conflicts**

If you get a merge conflict for instruction implementations in ceval.c, your best bet is to port your changes to bytecodes.c. That file looks almost the same as the original cases, except instead of `TARGET(NAME)` it uses `inst(NAME)`, and the trailing `DISPATCH()` call is omitted (the code generator adds it automatically).
2022-11-02 21:31:26 -07:00
Brandt Bucher 276d77724f
GH-98686: Quicken everything (GH-98687) 2022-11-02 10:42:57 -07:00
Irit Katriel df84b7b0bc
gh-87092: remove unused SET_LOC/UNSET_LOC macros (GH-98914) 2022-11-02 15:52:11 +00:00
Irit Katriel 6d683d8525
gh-87092: do not allocate PyFutureFeatures dynamically (GH-98913) 2022-11-02 15:13:07 +00:00
Brett Cannon 9711265182
gh-98925: Lower marshal recursion depth for WASI (GH-98938)
For wasmtime 2.0, the stack depth cost is 6% higher. This causes the default max `marshal` recursion depth to blow the stack.

As the default marshal depth is 2000 and Windows is set to 1000, split the difference and choose 1500 for WASI to be safe.
2022-11-01 15:51:05 -07:00
Guido van Rossum 7640ede177
Rename JUMP_TO_INSTRUCTION to GO_TO_INSTRUCTION (#98934)
This reduces confusion between jumps at the bytecode level
(e.g. JUMPTO(), JUMPBY(), and various JUMP_*() opcodes)
and jumps in the C code (which are 'goto' statements).
2022-10-31 19:53:32 -07:00
Eric Snow 4702552885
gh-98610: Adjust the Optional Restrictions on Subinterpreters (GH-98618)
Previously, the optional restrictions on subinterpreters were: disallow fork, subprocess, and threads.  By default, we were disallowing all three for "isolated" interpreters.  We always allowed all three for the main interpreter and those created through the legacy `Py_NewInterpreter()` API.

Those settings were a bit conservative, so here we've adjusted the optional restrictions to: fork, exec, threads, and daemon threads.  The default for "isolated" interpreters disables fork, exec, and daemon threads.  Regular threads are allowed by default.  We continue always allowing everything For the main interpreter and the legacy API.

In the code, we add `_PyInterpreterConfig.allow_exec` and  `_PyInterpreterConfig.allow_daemon_threads`.  We also add `Py_RTFLAGS_DAEMON_THREADS` and `Py_RTFLAGS_EXEC`.
2022-10-31 12:35:54 -07:00
Irit Katriel 39448adc9d
gh-98811: use full source location to simplify __future__ imports error checking. This also fixes an incorrect error offset. (GH-98812) 2022-10-31 13:08:03 +00:00
Dennis Sweeney 87b5fd9d9d
gh-98879: Remove unreachable error case from COMPARE_OP_STR_JUMP (GH-98882)
Thanks to PEP 623 changes, the comparison cannot fail.
2022-10-30 19:07:11 -04:00
Nick Coghlan 05e48865be
gh-96853: Restore test coverage for Py_Initialize(Ex) (GH-98212)
* As most of `test_embed` now uses `Py_InitializeFromConfig`, add
  a specific test case to cover `Py_Initialize` (and `Py_InitializeEx`)
* Rename `_testembed` init helper to clarify the API used
* Add a `PyConfig_Clear` call in `Py_InitializeEx` to make
  the code more obviously correct (it already didn't leak as
  none of the dynamically allocated config fields were being
  populated, but it's clearer if the wrappers follow the
  documented API usage guidelines)
2022-10-30 22:01:30 +10:00
Dennis Sweeney fbcafa6eee
gh-98789: Fix FOR_ITER assert on big-endian (GH-98792)
Fix FOR_ITER assertion syntax
2022-10-28 14:42:39 +08:00
Erlend E. Aasland 723ebe76e7
gh-96143: Improve perf profiler docs (#96445) 2022-10-27 14:06:48 +01:00
Mark Shannon 22863df7ca
GH-96793: Change `FOR_ITER` to not pop the iterator on exhaustion. (GH-96801)
Change FOR_ITER to have the same stack effect regardless of whether it branches or not.
Performance is unchanged as FOR_ITER (and specialized forms jump over the cleanup code).
2022-10-27 11:55:03 +01:00
Eric Snow f32369480d
gh-98608: Change _Py_NewInterpreter() to _Py_NewInterpreterFromConfig() (gh-98609)
(see https://github.com/python/cpython/issues/98608)

This change does the following:

1. change the argument to a new `_PyInterpreterConfig` struct
2. rename the function to `_Py_NewInterpreterFromConfig()`, inspired by `Py_InitializeFromConfig()` (takes a `_PyInterpreterConfig`  instead of `isolated_subinterpreter`)
3. split up the boolean `isolated_subinterpreter` into the corresponding multiple granular settings
   * allow_fork
   * allow_subprocess
   * allow_threads
4. add `PyInterpreterState.feature_flags` to store those settings
5. add a function for checking if a feature is enabled on an opaque `PyInterpreterState *`
6. drop `PyConfig._isolated_interpreter`

The existing default (see `Py_NewInterpeter()` and `Py_Initialize*()`) allows fork, subprocess, and threads and the optional "isolated" interpreter (see the `_xxsubinterpreters` module) disables all three.  None of that changes here; the defaults are preserved.

Note that the given `_PyInterpreterConfig` will not be used outside `_Py_NewInterpreterFromConfig()`, nor preserved.  This contrasts with how `PyConfig` is currently preserved, used, and even modified outside `Py_InitializeFromConfig()`.  I'd rather just avoid that mess from the start for `_PyInterpreterConfig`.  We can preserve it later if we find an actual need.

This change allows us to follow up with a number of improvements (e.g. stop disallowing subprocess and support disallowing exec instead).

(Note that this PR adds "private" symbols.  We'll probably make them public, and add docs, in a separate change.)
2022-10-26 11:16:30 -06:00
Neil Schemenauer de6981680b
gh-90716: add _pylong.py module (#96673)
Add Python implementations of certain longobject.c functions. These use
asymptotically faster algorithms that can be used for operations on
integers with many digits. In those cases, the performance overhead of
the Python implementation is not significant since the asymptotic
behavior is what dominates runtime. Functions provided by this module
should be considered private and not part of any public API.

Co-author: Tim Peters <tim.peters@gmail.com>
Co-author: Mark Dickinson <dickinsm@gmail.com>
Co-author: Bjorn Martinsson
2022-10-25 22:00:50 -07:00
Pablo Galindo Salgado 7cfbb49fcd
gh-91058: Add error suggestions to 'import from' import errors (#98305) 2022-10-25 23:56:59 +01:00
Pablo Galindo Salgado 1f737edb67
gh-96143: Move the perf trampoline files to the Python directory (#98675) 2022-10-25 23:34:22 +01:00
Irit Katriel a1a8828e42
gh-98461: Fix location of RETURN_VALUE in async generator bytecode. compiler_jump_if no longer needs a pointer to the loc. (GH-98494) 2022-10-25 12:26:26 +01:00
Dennis Sweeney 39bc70e267
gh-97912: Avoid quadratic behavior when adding LOAD_FAST_CHECK (GH-97952)
* The compiler analyzes the usage of the first 64 local variables all at once using bit masks.

* Local variables beyond the first 64 are only partially analyzed, achieving linear time.
2022-10-20 18:27:41 -04:00
Irit Katriel 4ec9ed8fde
gh-98461: Fix source location in comprehensions bytecode (GH-98464) 2022-10-20 16:58:37 +01:00
chgnrdv 1f369ad07f
gh-98354: Add unicode check for 'name' attribute in _imp_create_builtin (GH-98412)
Fixes #98354
2022-10-19 17:25:10 -07:00
Victor Stinner a8fe4bbd6b
gh-98257: Make _PyEval_SetTrace() reentrant (#98258)
Make sys.setprofile() and sys.settrace() functions reentrant.  They
can no long fail with: RuntimeError("Cannot install a trace function
while another trace function is being installed").

Make _PyEval_SetTrace() and _PyEval_SetProfile() functions reentrant,
rather than detecting and rejecting reentrant calls. Only delete the
reference to function arguments once the new function is fully set,
when a reentrant call is safe. Call also _PySys_Audit() earlier.
2022-10-20 00:31:47 +02:00
Eric Snow 9c8dde0fa5
gh-98417: Store int_max_str_digits on the Interpreter State (GH-98418) 2022-10-19 13:27:46 -07:00
Irit Katriel 9be05df399
gh-98398: Fix source locations for 'assert' bytecode (GH-98405) 2022-10-19 10:48:50 +01:00
Irit Katriel c051d55ddb
gh-98390: Fix source locations of boolean sub-expressions (GH-98396) 2022-10-18 17:18:38 +01:00
Irit Katriel 6da1a2e993
gh-93691: Compiler's code-gen passes location around instead of holding it on the global compiler state (GH-98001) 2022-10-17 14:28:51 +01:00
Victor Stinner 1863302d61
gh-97669: Create Tools/build/ directory (#97963)
Create Tools/build/ directory. Move the following scripts from
Tools/scripts/ to Tools/build/:

* check_extension_modules.py
* deepfreeze.py
* freeze_modules.py
* generate_global_objects.py
* generate_levenshtein_examples.py
* generate_opcode_h.py
* generate_re_casefix.py
* generate_sre_constants.py
* generate_stdlib_module_names.py
* generate_token.py
* parse_html5_entities.py
* smelly.py
* stable_abi.py
* umarshal.py
* update_file.py
* verify_ensurepip_wheels.py

Update references to these scripts.
2022-10-17 12:01:00 +02:00
Pablo Galindo Salgado bb56dead33
gh-98254: Include stdlib module names in error messages for NameErrors (#98255) 2022-10-15 23:13:33 +01:00
Guido van Rossum f5d71073e6
Fix some incorrect indentation around the main switch (#98177)
The `}` marked with `/* End instructions */` is the end of the switch.
There is another pair of `{}` around the switch, which is vestigial
from ancient times when it was `for (;;) { switch (opcode) { ... } }`.
All `DISPATCH` macro calls should be inside that pair.
2022-10-11 16:45:53 -07:00
Dong-hee Na a04656ec32
gh-97841: Add methoddef for _filters_mutated (gh-98115) 2022-10-09 20:16:33 +09:00
Pablo Galindo Salgado 83eb827247
gh-97922: Run the GC only on eval breaker (#97920) 2022-10-08 07:57:09 -07:00
Eric Wieser c7b2204996
Add a warning message about PyOS_snprintf (#95993) 2022-10-07 11:49:53 -07:00
Barry Warsaw 13d4489142
gh-86298: Ensure that __loader__ and __spec__.loader agree in warnings.warn_explicit() (GH-97803)
In `_warnings.c`, in the C equivalent of `warnings.warn_explicit()`, if the module globals are given (and not None), the warning will attempt to get the source line for the issued warning.  To do this, it needs the module's loader.

Previously, it would only look up `__loader__` in the module globals.  In https://github.com/python/cpython/issues/86298 we want to defer to the `__spec__.loader` if available.

The first step on this journey is to check that `loader == __spec__.loader` and issue another warning if it is not.  This commit does that.

Since this is a PoC, only manual testing for now.

```python
# /tmp/foo.py
import warnings

import bar

warnings.warn_explicit(
    'warning!',
    RuntimeWarning,
    'bar.py', 2,
    module='bar knee',
    module_globals=bar.__dict__,
    )
```

```python
# /tmp/bar.py
import sys
import os
import pathlib

# __loader__ = pathlib.Path()
```

Then running this: `./python.exe -Wdefault /tmp/foo.py`

Produces:

```
bar.py:2: RuntimeWarning: warning!
  import os
```

Uncomment the `__loader__ = ` line in `bar.py` and try it again:

```
sys:1: ImportWarning: Module bar; __loader__ != __spec__.loader (<_frozen_importlib_external.SourceFileLoader object at 0x109f7dfa0> != PosixPath('.'))
bar.py:2: RuntimeWarning: warning!
  import os
```

Automerge-Triggered-By: GH:warsaw
2022-10-06 19:32:53 -07:00
Carl Meyer a4b7794887
GH-91052: Add C API for watching dictionaries (GH-31787) 2022-10-07 01:08:00 +01:00
Gregory P. Smith 683ab85955
bpo-38693: Use f-strings instead of str.format() within importlib (#17058)
This is a small performance improvement, especially for one or two hot
places such as _handle_fromlist() that are called a lot and the
.format() method was being used just to join two strings with a dot.

Otherwise it is merely a readability improvement.

We keep `_ERR_MSG` and `_ERR_MSG_PREFIX` as those may be used elsewhere for canonical looking error messages.
2022-10-06 16:43:16 -07:00
Brandt Bucher 21a2d9ff55
GH-97002: Prevent `_PyInterpreterFrame`s from backing more than one `PyFrameObject` (GH-97996) 2022-10-07 00:20:01 +01:00
Lysandros Nikolaou cbf0afd8a1
gh-97973: Return all necessary information from the tokenizer (GH-97984)
Right now, the tokenizer only returns type and two pointers to the start and end of the token.
This PR modifies the tokenizer to return the type and set all of the necessary information,
so that the parser does not have to this.
2022-10-06 16:07:17 -07:00
Brett Cannon c206e53bb7
gh-65961: Raise `DeprecationWarning` when `__package__` differs from `__spec__.parent` (#97879)
Also remove `importlib.util.set_package()` which was already slated for removal.

Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
2022-10-05 15:00:45 -07:00
Irit Katriel c529b45122
gh-87092: bring compiler code closer to a preprocessing-opt-assembler organisation (GH-97644) 2022-10-05 08:52:35 +01:00
Mark Shannon 76449350b3
GH-91079: Decouple C stack overflow checks from Python recursion checks. (GH-96510) 2022-10-05 01:34:03 +01:00
Brandt Bucher 0ff8fd6583
GH-97779: Ensure that *all* frame objects are backed by "complete" frames (GH-97845) 2022-10-04 17:30:03 -07:00
Victor Stinner 116fa62c6e
gh-97670: Remove sys.getdxp() and analyze_dxp.py script (#97671)
Remove the sys.getdxp() function and the Tools/scripts/analyze_dxp.py
script. DXP stands for "dynamic execution pairs". They were related
to DYNAMIC_EXECUTION_PROFILE and DXPAIRS macros which have been
removed in Python 3.11. Python can now be built with "./configure
--enable-pystats" to gather statistics on Python opcodes.
2022-10-04 15:28:57 +02:00
Brandt Bucher 93fcc1f413
GH-97752: Clear the `previous` member of newly-created generator/coroutine frames (GH-97795) 2022-10-04 00:36:52 +01:00
Michael 07b8e85d0e
gh-96526: Clarify format and __format__ docstrings (gh-96648) 2022-10-03 15:28:02 -07:00
Gregory P. Smith b0f89cb431
gh-96512: Move int_max_str_digits setting to PyConfig (#96944)
It had to live as a global outside of PyConfig for stable ABI reasons in
the pre-3.12 backports.

This removes the `_Py_global_config_int_max_str_digits` and gets rid of
the equivalent field in the internal `struct _is PyInterpreterState` as
code can just use the existing nested config struct within that.

Adds tests to verify unique settings and configs in subinterpreters.
2022-10-03 13:55:45 -07:00
Nikita Sobolev 873a2f2527
Fix typos in `bltinmodule.c`. (GH-97766) 2022-10-03 12:09:03 -07:00
Pablo Galindo Salgado aab01e3524
gh-96670: Raise SyntaxError when parsing NULL bytes (#97594) 2022-09-27 23:23:42 +01:00
Victor Stinner 41351662bc
gh-96848: Fix -X int_max_str_digits option parsing (#96988)
Fix command line parsing: reject "-X int_max_str_digits" option with
no value (invalid) when the PYTHONINTMAXSTRDIGITS environment
variable is set to a valid limit.
2022-09-26 17:20:08 +02:00
Brandt Bucher 8fd2c3b75b
GH-96975: Skip incomplete frames in PyEval_GetFrame (GH-97003) 2022-09-22 09:16:52 -07:00
Brandt Bucher dfc73b5724
GH-95921: Fix positions for some chained comparisons (GH-96968) 2022-09-20 12:22:24 -07:00
Irit Katriel 98e785d364
gh-87092: in compiler, move the detection of exception handlers before the CFG optimization stage (GH-96935) 2022-09-20 13:14:07 +01:00
serge-sans-paille fc05107af9
gh-96711: Enhance SystemError message upon Invalid opcode (#96712)
Raise verbose SystemError instead of printing debug information
upon Invalid opcode.

Fix #96711
2022-09-20 12:00:34 +02:00
Victor Stinner 04f4977f50
gh-96387: take_gil() resets drop request before exit (#96869)
At Python exit, sometimes a thread holding the GIL can wait forever
for a thread (usually a daemon thread) which requested to drop the
GIL, whereas the thread already exited. To fix the race condition,
the thread which requested the GIL drop now resets its request before
exiting.

take_gil() now calls RESET_GIL_DROP_REQUEST() before
PyThread_exit_thread() if it called SET_GIL_DROP_REQUEST to fix a
race condition with drop_gil().

Issue discovered and analyzed by Mingliang ZHAO.
2022-09-20 00:13:56 +02:00
Brandt Bucher c10e33ac11
GH-96864: Check for error between line and opcode events (GH-96880) 2022-09-19 14:02:24 -07:00
adphrost a41ed975e8
GH-91049: Introduce set vectorcall field API for PyFunctionObject (GH-92257)
Co-authored-by: Andrew Frost <adfrost@fb.com>
Co-authored-by: Itamar Ostricher <itamarost@gmail.com>
2022-09-15 16:42:37 +01:00
Nikita Sobolev e37ac5fbb6
gh-96751: Remove dead code from `CALL_FUNCTION_EX` opcode (GH-96752) 2022-09-15 10:33:13 +01:00
Kevin Modzelewski 4781535a57
ceval: Use _PyTuple_FromArraySteal in BUILD_TUPLE (GH-96516) 2022-09-14 12:44:32 +09:00
Irit Katriel 6d7a0e0dd7
gh-87092: reduce redundancy and repetition in compiler's optimization stage (GH-96713) 2022-09-13 13:03:41 +01:00
Mark Shannon 12c5f328d2
GH-96754: Check whether the interpreter frame is complete before creating frame object. (GH-96776) 2022-09-13 09:25:16 +01:00
Mark Shannon 50a70a083d
GH-96678: Fix undefined behavior in ceval.c (#96708) 2022-09-10 01:12:06 +01:00
Brandt Bucher 72b29b2611
Replace ad-hoc labels with JUMP_TO_INSTRUCTION (GH-96634) 2022-09-09 14:43:10 -07:00
Brandt Bucher b4954b1a9e
GH-90230: Fix warnings and failures with --enable-pystats (GH-96622) 2022-09-09 14:42:29 -07:00
Kumar Aditya 88a7f661ca
Fix possible NULL pointer dereference in _PyThread_CurrentFrames (GH-96584) 2022-09-09 12:33:40 +01:00
Mark Shannon aa3b4cf779
GH-96636: Remove all uses of NOTRACE_DISPATCH (GH-96643)
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2022-09-08 17:16:48 +01:00
Itamar Ostricher 4f523a7054
gh-96653: Remove duplicate CALL_STAT_INC(inlined_py_calls) in BINARY_SUBSCR_GETITEM (GH-96654) 2022-09-08 16:22:32 +08:00
Kumar Aditya 4e4bfffe2d
GH-90699: use statically allocated interned strings in typeobject's slotdefs (GH-94706) 2022-09-07 15:02:08 -07:00
Steve Dower de33df27aa
gh-89545: Updates platform module to use new internal _wmi module on Windows to directly query OS properties (GH-96289) 2022-09-07 21:09:20 +01:00
Irit Katriel 0cd992c000
gh-88057: in compile.c, assertion that stackdepth is alway >=0 is missing in one place (GH-96513) 2022-09-07 14:01:57 +01:00
Brandt Bucher 0d04b8d9e1
GH-91432: Remove the iterator_exhausted_no_error label (GH-96517) 2022-09-06 21:07:20 -07:00
Kumar Aditya 67444902a0
GH-96572: fix use after free in trace refs build mode (#96618) 2022-09-06 11:02:41 -07:00
Mark Shannon 222f10ca2d
GH-96569: Add two NULL checks to avoid undefined behavior. (GH-96585) 2022-09-06 16:45:43 +01:00
Brandt Bucher cd0ff9bd14
GH-93911: Fix `LOAD_ATTR_PROPERTY` caches (GH-96519) 2022-09-06 12:11:38 +01:00
Dennis Sweeney ac18665472
ceval.c's GETITEM should have asserts, not set exceptions (GH-96518) 2022-09-04 19:00:24 -04:00
Gregory P. Smith 511ca94520
gh-95778: CVE-2020-10735: Prevent DoS by very large int() (#96499)
Integer to and from text conversions via CPython's bignum `int` type is not safe against denial of service attacks due to malicious input. Very large input strings with hundred thousands of digits can consume several CPU seconds.

This PR comes fresh from a pile of work done in our private PSRT security response team repo.

Signed-off-by: Christian Heimes [Red Hat] <christian@python.org>
Tons-of-polishing-up-by: Gregory P. Smith [Google] <greg@krypto.org>
Reviews via the private PSRT repo via many others (see the NEWS entry in the PR).

<!-- gh-issue-number: gh-95778 -->
* Issue: gh-95778
<!-- /gh-issue-number -->

I wrote up [a one pager for the release managers](https://docs.google.com/document/d/1KjuF_aXlzPUxTK4BMgezGJ2Pn7uevfX7g0_mvgHlL7Y/edit#). Much of that text wound up in the Issue. Backports PRs already exist. See the issue for links.
2022-09-02 09:35:08 -07:00
Irit Katriel 4c72517cad
gh-93554: Conditional jump opcodes only jump forward (GH-96318) 2022-09-01 21:36:47 +01:00
Irit Katriel 894cafd9a5
gh-93678: apply remove_redundant_jumps in optimize_cfg (GH-96274) 2022-09-01 11:03:52 +01:00
Brandt Bucher 88671a9d69
Remove the binary_subscr_dict_error label (GH-96443) 2022-08-30 15:45:24 -07:00
Pablo Galindo Salgado 6d791a9736
gh-96143: Allow Linux perf profiler to see Python calls (GH-96123)
⚠️  ⚠️ Note for reviewers, hackers and fellow systems/low-level/compiler engineers ⚠️ ⚠️ 

If you have a lot of experience with this kind of shenanigans and want to improve the **first** version, **please make a PR against my branch** or **reach out by email** or **suggest code changes directly on GitHub**. 

If you have any **refinements or optimizations** please, wait until the first version is merged before starting hacking or proposing those so we can keep this PR productive.
2022-08-30 10:11:18 -07:00
Mark Shannon c09fa7542c
GH-96237: Allow non-functions as reference-holder in frames. (GH-96238) 2022-08-25 10:16:55 +01:00
Pablo Galindo Salgado e34c82abeb
GH-93503: Add thread-specific APIs to set profiling and tracing functions in the C-API (#93504)
* gh-93503: Add APIs to set profiling and tracing functions in all threads in the C-API

* Use a separate API

* Fix NEWS entry

* Add locks around the loop

* Document ignoring exceptions

* Use the new APIs in the sys module

* Update docs
2022-08-24 23:21:39 +01:00
Irit Katriel fba3b67af4
gh-87092: use basicblock_last_instr consistently in the compiler (GH-96243) 2022-08-24 16:58:42 +01:00
Mark Shannon a4a9f2e879
GH-96177: Move GIL and eval breaker code out of ceval.c into ceval_gil.c. (GH-96204) 2022-08-24 14:21:01 +01:00
Irit Katriel 420f39f457
gh-93678: add _testinternalcapi.optimize_cfg() and test utils for compiler optimization unit tests (GH-96007) 2022-08-24 11:02:53 +01:00
Kumar Aditya e0d54a4a79
GH-96071: fix deadlock in PyGILState_Ensure (GH-96124)
Alternative of #96107
2022-08-19 12:43:00 -07:00
Christian Heimes 822955c166
gh-96125: Fix sys.thread_info.name on pthread platforms (GH-96126)
Automerge-Triggered-By: GH:tiran
2022-08-19 12:41:25 -07:00
Brandt Bucher 5bfb3c372b
GH-90997: Wrap yield from/await in a virtual try/except StopIteration (GH-96010) 2022-08-19 12:33:44 -07:00
Christian Heimes d9c1b746b5
gh-96017: Fix some compiler warnings (GH-96018)
- "comparison of integers of different signs" in typeobject.c
- only define static_builtin_index_is_set in DEBUG builds
- only define recreate_gil with ifdef HAVE_FORK
2022-08-19 08:08:43 +02:00
Matthias Görgens 4a6fa89465
Remove dead code in _PyDict_GetItemHint and rename to _PyDict_LookupIndex (GH-95948) 2022-08-18 10:19:21 +01:00
Ken Jin 7276ca25f5
GH-93911: Specialize `LOAD_ATTR` for custom `__getattribute__` (GH-93988) 2022-08-17 12:37:07 +01:00
Kumar Aditya 9b30b965f0
GH-95909: Make `_PyArg_Parser` initialization thread safe (GH-95958) 2022-08-16 11:22:14 -07:00
Christian Heimes 32ac98e899
gh-95853: Add script to automate WASM build (GH-95828)
Automate WASM build with a new Python script. The script provides
several build profiles with configure flags for Emscripten flavors
and WASI. The script can detect and use Emscripten SDK and WASI SDK from
default locations or env vars.

``configure`` now detects Node arguments and creates HOSTRUNNER
arguments for Node 16. It also sets correct arguments for
``wasm64-emscripten``.

Co-authored-by: Brett Cannon <brett@python.org>
2022-08-13 21:56:08 +02:00
Erlend E. Aasland f07adf82f3
gh-90928: Improve static initialization of keywords tuple in AC (#95907) 2022-08-13 12:09:40 +02:00
Irit Katriel 41757bfabd
gh-95922: compiler's eliminate_empty_basic_blocks ignores the last block of the compilation unit (GH-95924) 2022-08-12 16:35:09 +01:00
Eric Snow 6f6a4e6cc5
gh-90928: Statically Initialize the Keywords Tuple in Clinic-Generated Code (gh-95860)
We only statically initialize for core code and builtin modules.  Extension modules still create
the tuple at runtime.  We'll solve that part of interpreter isolation separately.

This change includes generated code. The non-generated changes are in:

* Tools/clinic/clinic.py
* Python/getargs.c
* Include/cpython/modsupport.h
* Makefile.pre.in (re-generate global strings after running clinic)
* very minor tweaks to Modules/_codecsmodule.c and Python/Python-tokenize.c

All other changes are generated code (clinic, global strings).
2022-08-11 15:25:49 -06:00
Irit Katriel 9533b40cce
gh-87092: compiler's codegen stage uses int jump target labels, and the target pointer is only calculated just before optimization stage (GH-95655) 2022-08-11 17:40:49 +01:00
Mark Shannon 1b46d118e6
GH-95818: Skip incomplete frames in `PyThreadState_GetFrame` (GH-95886) 2022-08-11 14:06:32 +01:00
zhanpon 23a757f44f
Update _PyEval_AddPendingCall comment (#95817) 2022-08-11 13:32:52 +02:00
Oleg Iarygin 56d16e8cb4
gh-93243: Make smtpd private before porting its users (GH-93246)
gh-93243

This PR is required to reduce diffs of the following porting (no need to either maintain documentation and tests consistent with each porting step, or try to port everything and remove smtpd in a single PR).

Automerge-Triggered-By: GH:warsaw
2022-08-05 17:41:29 -07:00
Irit Katriel 000c3874bf
gh-87092: create a 'jump target label' abstraction so that the compiler's codegen stage does not work directly with basic blocks (GH-95398) 2022-08-04 14:59:32 +01:00
Mark Shannon de388c0a7b
GH-95245: Store object values and dict pointers in single tagged pointer. (GH-95278) 2022-08-01 14:34:54 +01:00
Pablo Galindo Salgado aa37ffda29
bpo-45445: Revert "bpo-45445: Fail if an invalid X-option is provided in the command line (GH-28823)" (#94745) 2022-07-31 16:33:56 +01:00
Mark Shannon b8b2990fb3
GH-90081: Run python tracers at full speed (GH-95328) 2022-07-28 10:17:22 +01:00