Commit Graph

112 Commits

Author SHA1 Message Date
Mark Shannon 15d4c9fabc
GH-108716: Turn off deep-freezing of code objects. (GH-108722) 2023-09-08 10:34:40 +01:00
Dong-hee Na 3bfa24e29f
gh-107265: Remove all ENTER_EXECUTOR when execute _Py_Instrument (gh-108539) 2023-09-07 09:53:54 +09:00
Victor Stinner a0773b89df
gh-108753: Enhance pystats (#108754)
Statistics gathering is now off by default. Use the "-X pystats"
command line option or set the new PYTHONSTATS environment variable
to 1 to turn statistics gathering on at Python startup.

Statistics are no longer dumped at exit if statistics gathering was
off or statistics have been cleared.

Changes:

* Add PYTHONSTATS environment variable.
* sys._stats_dump() now returns False if statistics are not dumped
  because they are all equal to zero.
* Add PyConfig._pystats member.
* Add tests on sys functions and on setting PyConfig._pystats to 1.
* Add Include/cpython/pystats.h and Include/internal/pycore_pystats.h
  header files.
* Rename '_py_stats' variable to '_Py_stats'.
* Exclude Include/cpython/pystats.h from the Py_LIMITED_API.
* Move pystats.h include from object.h to Python.h.
* Add _Py_StatsOn() and _Py_StatsOff() functions. Remove
  '_py_stats_struct' variable from the API: make it static in
  specialize.c.
* Document API in Include/pystats.h and Include/cpython/pystats.h.
* Complete pystats documentation in Doc/using/configure.rst.
* Don't write "all zeros" stats: if _stats_off() and _stats_clear()
  or _stats_dump() were called.
* _PyEval_Fini() now always call _Py_PrintSpecializationStats() which
  does nothing if stats are all zeros.

Co-authored-by: Michael Droettboom <mdboom@gmail.com>
2023-09-06 15:54:59 +00:00
Victor Stinner 21c0844742
gh-108220: Internal header files require Py_BUILD_CORE to be defined (#108221)
* pycore_intrinsics.h does nothing if included twice
  (add #ifndef and #define).
* Update Tools/cases_generator/generate_cases.py to generate the
  Py_BUILD_CORE test.
* _bz2, _lzma, _opcode and zlib extensions now define the
  Py_BUILD_CORE_MODULE macro to use internal headers
  (pycore_code.h, pycore_intrinsics.h and pycore_blocks_output_buffer.h).
2023-08-21 19:15:52 +02:00
Mark Shannon 2ba7c7f7b1
Add some GC stats to Py_STATS (GH-107581) 2023-08-04 10:34:23 +01:00
Irit Katriel 6ef8f8ca88
gh-105481: the ENABLE_SPECIALIZATION flag does not need to be generated by the build script, or exposed in opcode.py (#107534) 2023-08-01 17:05:00 +00:00
Victor Stinner c5b13d6f80
gh-107211: No longer export internal functions (4) (#107217)
No longer export these 2 internal C API functions:

* _PyEval_SignalAsyncExc()
* _PyEval_SignalReceived()

Add also comments explaining why some internal functions have to be
exported, and update existing comments.
2023-07-25 03:16:28 +00:00
Victor Stinner 11306a91bd
gh-107211: No longer export internal functions (1) (#107213)
No longer export these 49 internal C API functions:

* _PyArgv_AsWstrList()
* _PyCode_New()
* _PyCode_Validate()
* _PyFloat_DebugMallocStats()
* _PyFloat_FormatAdvancedWriter()
* _PyImport_CheckSubinterpIncompatibleExtensionAllowed()
* _PyImport_ClearExtension()
* _PyImport_GetModuleId()
* _PyImport_SetModuleString()
* _PyInterpreterState_IDDecref()
* _PyInterpreterState_IDIncref()
* _PyInterpreterState_IDInitref()
* _PyInterpreterState_LookUpID()
* _PyWideStringList_AsList()
* _PyWideStringList_CheckConsistency()
* _PyWideStringList_Clear()
* _PyWideStringList_Copy()
* _PyWideStringList_Extend()
* _Py_ClearArgcArgv()
* _Py_DecodeUTF8Ex()
* _Py_DecodeUTF8_surrogateescape()
* _Py_EncodeLocaleRaw()
* _Py_EncodeUTF8Ex()
* _Py_GetEnv()
* _Py_GetForceASCII()
* _Py_GetLocaleEncoding()
* _Py_GetLocaleEncodingObject()
* _Py_GetLocaleconvNumeric()
* _Py_ResetForceASCII()
* _Py_device_encoding()
* _Py_dg_dtoa()
* _Py_dg_freedtoa()
* _Py_dg_strtod()
* _Py_get_blocking()
* _Py_get_env_flag()
* _Py_get_inheritable()
* _Py_get_osfhandle_noraise()
* _Py_get_xoption()
* _Py_open()
* _Py_open_osfhandle()
* _Py_open_osfhandle_noraise()
* _Py_read()
* _Py_set_blocking()
* _Py_str_to_int()
* _Py_wfopen()
* _Py_wgetcwd()
* _Py_wreadlink()
* _Py_wrealpath()
* _Py_write()
2023-07-25 03:44:11 +02:00
Mark Shannon b03755a234
GH-104909: Break LOAD_GLOBAL specializations in micro-ops. (GH-106677) 2023-07-12 14:34:14 +01:00
Brandt Bucher 7b2d94d875
GH-106008: Make implicit boolean conversions explicit (GH-106003) 2023-06-29 13:49:54 -07:00
Mark Shannon 7199584ac8
GH-100987: Allow objects other than code objects as the "executable" of an internal frame. (GH-105727)
* Add table describing possible executable classes for out-of-process debuggers.

* Remove shim code object creation code as it is no longer needed.

* Make lltrace a bit more robust w.r.t. non-standard frames.
2023-06-14 13:46:37 +01:00
Mark Shannon 4bfa01b9d9
GH-104584: Plugin optimizer API (GH-105100) 2023-06-02 11:46:18 +01:00
Carl Meyer 77262458fe
gh-87729: improve hit rate of LOAD_SUPER_ATTR specialization (#104270) 2023-05-11 08:08:13 -06:00
Carl Meyer c3b595e73e
gh-97933: (PEP 709) inline list/dict/set comprehensions (#101441)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
2023-05-09 11:02:14 -06:00
Carl Meyer ebf97c50f2
gh-103978: avoid using 'class' as an identifier (#103979) 2023-04-28 19:20:50 +00:00
Carl Meyer ef25febcf2
gh-87729: specialize LOAD_SUPER_ATTR_METHOD (#103809) 2023-04-25 17:45:51 +00:00
Mark Shannon 411b169281
GH-103082: Implementation of PEP 669: Low Impact Monitoring for CPython (GH-103083)
* The majority of the monitoring code is in instrumentation.c

* The new instrumentation bytecodes are in bytecodes.c

* legacy_tracing.c adapts the new API to the old sys.setrace and sys.setprofile APIs
2023-04-12 12:04:55 +01:00
Brandt Bucher b4978ff872
GH-88691: Shrink the CALL caches (GH-103230) 2023-04-05 14:15:49 -07:00
Brandt Bucher 121057aa36
GH-89987: Shrink the BINARY_SUBSCR caches (GH-103022) 2023-03-29 15:53:30 -07:00
Brandt Bucher 0444ae2487
GH-100982: Break up COMPARE_AND_BRANCH (GH-102801) 2023-03-23 15:25:09 -07:00
Brandt Bucher 08b67fb34f
GH-90997: Shrink the LOAD_GLOBAL caches (#102569) 2023-03-10 17:01:16 -08:00
Mark Shannon 160f2fe2b9
GH-87849: Simplify stack effect of SEND and specialize it for generators and coroutines. (GH-101788) 2023-02-13 11:24:55 +00:00
Mark Shannon 7b14c2ef19
GH-100982: Add `COMPARE_AND_BRANCH` instruction (GH-100983) 2023-01-16 12:35:21 +00:00
Mark Shannon 6e4e14d98f
GH-100923: Embed jump mask in `COMPARE_OP` oparg (GH-100924) 2023-01-11 20:40:43 +00:00
Mark Shannon 6997e77bdf
GH-100222: Redefine _Py_CODEUNIT as a union to clarify structure of code unit. (GH-100223) 2022-12-14 11:12:53 +00:00
Mark Shannon fb713b2183
GH-98522: Add version number to code objects. (GH-98525)
* Add version number to code object for better versioning of functions.

* Improves specialization for closures and list comprehensions.
2022-12-09 12:18:45 +00:00
Itamar Ostricher 3c137dc613
GH-91054: Add code object watchers API (GH-99859)
* Add API to allow extensions to set callback function on creation and destruction of PyCodeObject

Co-authored-by: Ye11ow-Flash <janshah@cs.stonybrook.edu>
2022-12-02 17:28:27 +00:00
Guido van Rossum 8f18ac04d3
GH-98831: Add `macro` and `op` and their implementation to DSL (#99495)
Newly supported interpreter definition syntax:
- `op(NAME, (input_stack_effects -- output_stack_effects)) { ... }`
- `macro(NAME) = OP1 + OP2;`

Also some other random improvements:
- Convert `WITH_EXCEPT_START` to use stack effects
- Fix lexer to balk at unrecognized characters, e.g. `@`
- Fix moved output names; support object pointers in cache
- Introduce `error()` method to print errors
- Introduce read_uint16(p) as equivalent to `*p`

Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2022-11-22 16:04:57 -08:00
Brandt Bucher b629fdd88a
GH-99298: Clean up attribute specializations (GH-99398) 2022-11-17 15:09:18 -08:00
Mark Shannon 1e197e63e2
GH-96421: Insert shim frame on entry to interpreter (GH-96319)
* Adds EXIT_INTERPRETER instruction to exit PyEval_EvalDefault()

* Simplifies RETURN_VALUE, YIELD_VALUE and RETURN_GENERATOR instructions as they no longer need to check for entry frames.
2022-11-10 12:34:57 +00:00
Brandt Bucher c7f5708714
GH-98686: Get rid of "adaptive" and "quick" instructions (GH-99182) 2022-11-09 10:50:09 -08:00
Mark Shannon 4a1c58d504
GH-96793: Specialize FOR_ITER for generators. (GH-98772) 2022-11-07 14:49:51 +00:00
Brandt Bucher 276d77724f
GH-98686: Quicken everything (GH-98687) 2022-11-02 10:42:57 -07:00
Brandt Bucher a83fdf2563
GH-90997: Improve inline cache performance for MSVC (GH-96781) 2022-09-14 17:05:04 -07:00
Ken Jin 7276ca25f5
GH-93911: Specialize `LOAD_ATTR` for custom `__getattribute__` (GH-93988) 2022-08-17 12:37:07 +01:00
Mark Shannon 2f8bff6879
GH-94739: Mark stacks of exception handling blocks for setting frame.f_lineno in the debugger. (GH-94958) 2022-07-18 16:06:42 +01:00
Mark Shannon 6f8875eba3
GH-93841: Allow stats to be turned on and off, cleared and dumped at runtime. (GH-93843) 2022-06-21 15:40:54 +01:00
Dennis Sweeney 5fcfdd87c9
GH-91432: Specialize FOR_ITER (GH-91713)
* Adds FOR_ITER_LIST and FOR_ITER_RANGE specializations.

* Adds _PyLong_AssignValue() internal function to avoid temporary boxing of ints.
2022-06-21 11:19:26 +01:00
Victor Stinner 7ad6f74fcf
gh-87347: Add parenthesis around macro arguments (#93915)
Add unit test on Py_MEMBER_SIZE() and some other macros.
2022-06-20 16:04:52 +02:00
Mark Shannon ab0e601016
GH-93516: Speedup line number checks when tracing. (GH-93763)
* Use a lookup table to reduce overhead of getting line numbers during tracing.
2022-06-20 13:00:42 +01:00
Ken Jin b083450f88
GH-93429: Merge `LOAD_METHOD` back into `LOAD_ATTR` (GH-93430) 2022-06-14 11:36:22 +01:00
Mark Shannon f012df706c
Shrink the LOAD_METHOD cache by one codeunit. (#93537) 2022-06-07 10:28:53 +01:00
Mark Shannon eb618d5ff0
GH-93354: Use exponential backoff to avoid excessive specialization attempts. (GH-93355) 2022-05-31 11:58:26 +01:00
Mark Shannon bbcf42449e
GH-90230: Add stats to breakdown the origin of calls to `PyEval_EvalFrame` (GH-93284) 2022-05-27 16:31:41 +01:00
Mark Shannon e48ac9c100
GH-90690: Remove `PRECALL` instruction (GH-92925) 2022-05-19 11:05:26 +01:00
Mark Shannon fa2b8b75eb
Improve object stats (#92845)
* Add incref/decref stats

* Show ratios for allocation in summary
2022-05-16 14:35:11 +01:00
Mark Shannon 836b17c9c3
Add more stats for freelist use and allocations. (GH-92211) 2022-05-03 16:40:24 -06:00
Victor Stinner 87c6cf9aa7
gh-89479: Export _Py_GetSpecializationStats() internal function (#92011)
When Python is built with "./configure --enable-pystats" (if the
Py_STATS macro is defined), the _Py_GetSpecializationStats() function
must be exported, since it's used by the _opcode extension which is
built as a shared library.
2022-04-28 14:02:45 +02:00
Mark Shannon 944fffee89
GH-88116: Use a compact format to represent end line and column offsets. (GH-91666)
* Stores all location info in linetable to conform to PEP 626.

* Remove column table from code objects.

* Remove end-line table from code objects.

* Document new location table format
2022-04-21 16:10:37 +01:00
Mark Shannon 04acfa94bb
Merge deoptimization blocks in interpreter (GH-32155) 2022-03-30 13:11:33 +01:00