Commit Graph

7203 Commits

Author SHA1 Message Date
Gabriele N. Tornetta 50669083fe
bpo-43931: Export Python version as API data (GH-25577)
When Python is embedded in other applications, it is not easy to determine which version of Python is being used. This change exposes the Python version as part of the API data. Tools like Austin (https://github.com/P403n1x87/austin) can benefit from this data when targeting applications like uWSGI, as the Python version can then be inferred systematically by looking at the exported symbols rather than relying on unreliable pattern matching or other hacks (like remote code execution etc...).

Automerge-Triggered-By: GH:pablogsal
2021-12-09 17:52:05 -08:00
Eric Snow c8749b5783
bpo-46008: Make runtime-global object/type lifecycle functions and state consistent. (gh-29998)
This change is strictly renames and moving code around.  It helps in the following ways:

* ensures type-related init functions focus strictly on one of the three aspects (state, objects, types)
* passes in PyInterpreterState * to all those functions, simplifying work on moving types/objects/state to the interpreter
* consistent naming conventions help make what's going on more clear
* keeping API related to a type in the corresponding header file makes it more obvious where to look for it

https://bugs.python.org/issue46008
2021-12-09 12:59:26 -07:00
Kumar Aditya 44b0e76f2a
bpo-45654: Freeze the runpy module and stuff it imports (GH-29903) 2021-12-09 08:51:09 -08:00
Irit Katriel dc4a212bd3
bpo-45635: continue refactor of print_exception() to standardize error handling (GH-29996)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-12-09 14:38:00 +00:00
Brandt Bucher 5de39f4b41
bpo-45510: Check both types when specializing subtraction (GH-29995) 2021-12-09 10:17:16 +09:00
Irit Katriel 2109f7880b
bpo-45711: Remove unnecessary normalization of exc_info (GH-29922) 2021-12-08 20:08:06 +00:00
Irit Katriel f893bb2e01
bpo-45635: refactor print_exception() into smaller functions (GH-29981)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-12-08 18:47:27 +00:00
Mark Shannon 69806b9516
bpo-46009: Do not exhaust generator when send() method raises (GH-29986) 2021-12-08 12:09:26 +00:00
Eric Snow 32a67246b0
bpo-46008: Move Py*State init into distinct functions. (gh-29977)
Previously, basic initialization of PyInterprterState happened in PyInterpreterState_New() (along with allocation and adding the new interpreter to the runtime state). This prevented us from initializing interpreter states that were allocated separately (e.g. statically or in a free list). We've addressed that here by factoring out a separate function just for initialization. We've done the same for PyThreadState. _PyRuntimeState was sorted out when we added it since _PyRuntime is statically allocated. However, here we update the existing init code to line up with the functions for PyInterpreterState and PyThreadState.

https://bugs.python.org/issue46008
2021-12-07 18:59:49 -07:00
Eric Snow 758b74e71e
bpo-46008: Add _PyInterpreterState_Main(). (gh-29978)
PyInterpreterState_Main() is a plain function exposed in the public C-API. For internal usage we can take the more efficient approach in this PR.

https://bugs.python.org/issue46008
2021-12-07 18:56:06 -07:00
Eric Snow 1f384e3184
bpo-46008: Stop calling _PyThreadState_Init() in new_threadstate(). (gh-29973)
This simplifies new_threadstate().  We also rename _PyThreadState_Init() to _PyThreadState_SetCurrent() to reflect what it actually does.

https://bugs.python.org/issue46008
2021-12-07 17:26:29 -07:00
Eric Snow 9b577cd01f
bpo-46008: Use PyMem_RawCalloc() to allocate PyThreadState. (GH-29972)
Doing so allows us to stop assigning various fields to `NULL` and 0.  It also more closely matches the behavior of a static initializer.

Automerge-Triggered-By: GH:ericsnowcurrently
2021-12-07 13:37:51 -08:00
Eric Snow 313f92a57b
bpo-46008: Move thread-related interpreter state into a sub-struct. (gh-29971)
This parallels _PyRuntimeState.interpreters.  Doing this helps make it more clear what part of PyInterpreterState relates to its threads.

https://bugs.python.org/issue46008
2021-12-07 14:03:47 -07:00
Eric Snow 8262c96bcc
bpo-46008: Return void from _PyEval_InitState(). (gh-29970)
This falls into the category of keep-allocation-and-initialization separate. It also allows us to use _PyEval_InitState() safely in functions that return void.

https://bugs.python.org/issue46008
2021-12-07 14:02:17 -07:00
Mark Shannon 064e53d19a
Fix leak when an exception is raised during generator creation. (GH-29960) 2021-12-07 18:05:48 +00:00
Irit Katriel d596acbd3b
bpo-45635: standardize error handling in traceback.c (GH-29905)
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@innova.no>
2021-12-07 16:17:22 +00:00
Mark Shannon 8319114fee
bpo-45947: Place dict and values pointer at fixed (negative) offset just before GC header. (GH-29879)
* Place __dict__ immediately before GC header for plain Python objects.

* Fix up lazy dict creation logic to use managed dict pointers.

* Manage values pointer, placing them directly before managed dict pointers.

* Convert hint-based load/store attr specialization target managed dict classes.

* Specialize LOAD_METHOD for managed dict objects.

* Remove unsafe _PyObject_GC_Calloc function.

* Remove unsafe _PyObject_GC_Malloc() function.

* Add comment explaning use of Py_TPFLAGS_MANAGED_DICT.
2021-12-07 16:02:53 +00:00
Victor Stinner cf7eaa4617
Revert "bpo-28533: Remove asyncore, asynchat, smtpd modules (GH-29521)" (GH-29951)
This reverts commit 9bf2cbc4c4.
2021-12-07 12:31:04 +01:00
Mark Shannon 299483c95d
bpo-45963: Make space for the InterpreterFrame of a generator in that generator. (GH-29891)
* Make generator, coroutine and async gen structs all the same size.

* Store interpreter frame in generator (and coroutine). Reduces the number of allocations neeeded for a generator from two to one.
2021-12-06 10:13:49 +00:00
Guido van Rossum 605f4bf783
Delete orphaned comment (#29917)
(The function this described was deleted by PR #23743, the comment was accidentally retained.)
2021-12-05 09:03:58 -08:00
Irit Katriel 5bb7ef2768
bpo-45607: Make it possible to enrich exception displays via setting their __note__ field (GH-29880) 2021-12-03 22:01:15 +00:00
Christian Heimes 84ca1232b0
bpo-45950: Introduce Bootstrap Python again (#29859)
The build system now uses a :program:`_bootstrap_python` interpreter for
freezing and deepfreezing again. To speed up build process the build tools
:program:`_bootstrap_python` and :program:`_freeze_module` are no longer
build with LTO.

Cross building depends on a build Python interpreter, which must have same
version and bytecode as target host Python.
2021-12-03 16:01:11 +01:00
Dennis Sweeney 03768c4d13
bpo-45885: Specialize COMPARE_OP (GH-29734)
* Add COMPARE_OP_ADAPTIVE adaptive instruction.

* Add COMPARE_OP_FLOAT_JUMP, COMPARE_OP_INT_JUMP and COMPARE_OP_STR_JUMP specialized instructions.

* Introduce and use _PyUnicode_Equal
2021-12-03 11:29:12 +00:00
Steve Dower 99fcf15052
bpo-45582: Port getpath[p].c to Python (GH-29041)
The getpath.py file is frozen at build time and executed as code over a namespace. It is never imported, nor is it meant to be importable or reusable. However, it should be easier to read, modify, and patch than the previous code.

This commit attempts to preserve every previously tested quirk, but these may be changed in the future to better align platforms.
2021-12-03 00:08:42 +00:00
Christian Heimes cb8f491f46
bpo-40280: Optimize ints and and startup on wasm (GH-29887) 2021-12-02 12:19:30 +01:00
Victor Stinner a6c3b0faa1
bpo-45954: Rename PyConfig.no_debug_ranges to code_debug_ranges (GH-29886)
Rename PyConfig.no_debug_ranges to PyConfig.code_debug_ranges and
invert the value.

Document -X no_debug_ranges and PYTHONNODEBUGRANGES env var in
PyConfig.code_debug_ranges documentation.
2021-12-02 11:43:37 +01:00
Mark Shannon 49444fb807
bpo-45753: Interpreter internal tweaks (GH-29575)
* Split exit paths into exceptional and non-exceptional.

* Move exit tracing code to individual bytecodes.

* Wrap all trace entry and exit events in macros to make them clearer and easier to enhance.

* Move return sequence into RETURN_VALUE, YIELD_VALUE and YIELD_FROM. Distinguish between normal trace events and dtrace events.
2021-12-01 12:09:36 +00:00
Irit Katriel 8a45ca542a
bpo-45711: Change exc_info related APIs to derive type and traceback from the exception instance (GH-29780) 2021-11-30 22:37:04 +00:00
Mark Shannon 60929576e4
bpo-45786: Allocate space for frame in frame object. (GH-29729) 2021-11-29 12:34:59 +00:00
Serhiy Storchaka 87d5180b7a
Remove misleading comment in the AST optimizer (#29825) 2021-11-29 11:19:59 +03:00
Christian Heimes f87ea03502
bpo-45915: use fcntl(fd, F_GETFD) in is_valid_fd() (GH-29821) 2021-11-28 19:40:27 +01:00
Irit Katriel 4dfae6f38e
bpo-45614: Fix traceback display for exceptions with invalid module name (GH-29726) 2021-11-27 22:00:10 +00:00
Irit Katriel c456dfafe9
bpo-45711: use exc_value instead of exc_type to determine if exc_info is valid. Add more assertions. (GH-29627) 2021-11-25 09:41:28 +00:00
Christian Heimes 5c4b19ec49
bpo-45020: Fix strict-prototypes warning (GH-29755) 2021-11-24 20:01:39 +01:00
Mark Shannon 135cabd328
bpo-44525: Copy free variables in bytecode to allow calls to inner functions to be specialized (GH-29595)
* Make internal APIs that take PyFrameConstructor take a PyFunctionObject instead.

* Add reference to function to frame, borrow references to builtins and globals.

* Add COPY_FREE_VARS instruction to allow specialization of calls to inner functions.
2021-11-23 09:53:24 +00:00
Guido van Rossum 1037ca5a8e
bpo-45850: Implement deep-freeze on Windows (#29648)
Implement changes to build with deep-frozen modules on Windows.
Note that we now require Python 3.10 as the "bootstrap" or "host" Python.
This causes a modest startup speed (around 7%) on Windows.
2021-11-22 10:09:48 -08:00
Irit Katriel 4d6c0c0cce
bpo-45871: Refactor except matcher validation into a separate function so that it can be reused. Add missing unit test. (GH-29711) 2021-11-22 16:56:23 +00:00
Mark Shannon 7fd92a8b7e
bpo-45813: Make sure that frame->generator is NULLed when generator is deallocated. (GH-29700) 2021-11-22 14:01:23 +00:00
Pablo Galindo Salgado fdcc46d955
bpo-45848: Allow the parser to get error lines from encoded files (GH-29646) 2021-11-20 15:36:07 +01:00
Irit Katriel d64659d8cd
Fix link to exception handling notes (GH-29617) 2021-11-19 15:40:18 +00:00
Mark Shannon 337cb480e9
bpo-45709: Fix tracing when exception is handled. (GH-29638) 2021-11-19 15:16:49 +00:00
Dennis Sweeney 036fead695
bpo-45609: Specialize STORE_SUBSCR (GH-29242)
* Specialize STORE_SUBSCR for list[int], and dict[object]

* Adds _PyDict_SetItem_Take2 which consumes references to the key and values.
2021-11-19 10:30:37 +00:00
Mark Shannon 21fa7a3e8f
bpo-45829: Specialize BINARY_SUBSCR for __getitem__ implemented in Python. (GH-29592) 2021-11-18 11:02:14 +00:00
Dong-hee Na 345ba3f080
bpo-45510: Specialize BINARY_SUBTRACT (GH-29523) 2021-11-18 09:19:58 +00:00
Dennis Sweeney 5d90c467c0
bpo-45826: Fix a crash in suggestions.c by checking for `traceback is None` (GH-29590) 2021-11-18 00:03:52 +01:00
Sam Gross 736684b1bb
bpo-42540: reallocation of id_mutex should not force default allocator (GH-29564)
Unlike the other locks reinitialized by _PyRuntimeState_ReInitThreads,
the "interpreters.main->id_mutex" is not freed by _PyRuntimeState_Fini
and should not force the default raw allocator.
2021-11-17 21:51:03 +01:00
Victor Stinner b919d8105c
bpo-45831: _Py_DumpASCII() uses a single write() call if possible (GH-29596)
If the string is ASCII only and doesn't need to escape characters,
write the whole string with a single write() syscall.
2021-11-17 21:12:20 +01:00
Brandt Bucher 6a84d61c55
bpo-45636: Simplify BINARY_OP (GH-29565) 2021-11-16 05:53:57 -08:00
Mark Shannon b931077375
bpo-45753: Make recursion checks more efficient. (GH-29524)
* Uses recursion remaining, instead of recursion depth to speed up check against recursion limit.
2021-11-16 11:01:57 +00:00
Victor Stinner 9bf2cbc4c4
bpo-28533: Remove asyncore, asynchat, smtpd modules (GH-29521)
Remove the asyncore and asynchat modules, deprecated in Python
3.6: use the asyncio module instead.

Remove the smtpd module, deprecated in Python 3.6: the aiosmtpd
module can be used instead, it is based on asyncio.

* Remove asyncore, asynchat and smtpd documentation
* Remove test_asyncore, test_asynchat and test_smtpd
* Rename Lib/asynchat.py to Lib/test/support/_asynchat.py
* Rename Lib/asyncore.py to Lib/test/support/_asyncore.py
* Rename Lib/smtpd.py to Lib/test/support/_smtpd.py
* Remove DeprecationWarning from private _asyncore, _asynchat and
  _smtpd modules
* _smtpd: remove deprecated properties
2021-11-16 00:29:17 +01:00