Commit Graph

3960 Commits

Author SHA1 Message Date
Victor Stinner 86bc9e35c4
gh-108494: AC supports pos-only args in limited C API (#108498)
AC now checks for "#define Py_LIMITED_API" pattern to use the limited
C API.
2023-08-26 00:39:24 +02:00
Victor Stinner 73d33c1a30
gh-107603: Argument Clinic can emit includes (#108486)
* Add Clinic.add_include() method
* Add CConverter.include and CConverter.add_include()
* Printer.print_block() gets a second parameter: clinic.
* Remove duplicated declaration of "clinic" global variable.
2023-08-26 00:01:10 +02:00
Victor Stinner 1dd9510977
gh-108494: Argument Clinic partial supports of Limited C API (#108495)
Argument Clinic now has a partial support of the
Limited API:

* Add --limited option to clinic.c.
* Add '_testclinic_limited' extension which is built with
  the limited C API version 3.13.
* For now, hardcode in clinic.py that "_testclinic_limited.c" targets
  the limited C API.
2023-08-25 23:22:08 +02:00
Irit Katriel 5f41376e93
gh-107901: add the HAS_EVAL_BREAK instruction flag (#108375) 2023-08-25 18:33:59 +00:00
Alex Waygood 5a25daa512
gh-104504: Cases generator: enable mypy's `possibly-undefined` error code (#108454) 2023-08-25 18:08:29 +01:00
Guido van Rossum ddf66b54ed
gh-106581: Split CALL_BOUND_METHOD_EXACT_ARGS into uops (#108462)
Instead of using `GO_TO_INSTRUCTION(CALL_PY_EXACT_ARGS)` we just add the macro elements of the latter to the macro for the former. This requires lengthening the uops array in struct opcode_macro_expansion. (It also required changes to stacking.py that were merged already.)
2023-08-24 17:36:00 -07:00
Victor Stinner 4e5a7284ee
gh-108444: Argument Clinic uses PyLong_AsInt() (#108458)
Argument Clinic now uses the new public PyLong_AsInt(), rather than
the old name _PyLong_AsInt().
2023-08-25 00:51:22 +02:00
Guido van Rossum 88941d665f
gh-106581: Fix two bugs in the code generator's copy optimization (#108380)
I was comparing the last preceding poke with the *last* peek,
rather than the *first* peek.

Unfortunately this bug obscured another bug:
When the last preceding poke is UNUSED, the first peek disappears,
leaving the variable unassigned. This is how I fixed it:

- Rename CopyEffect to CopyItem.
- Change CopyItem to contain StackItems instead of StackEffects.
- Update those StackItems when adjusting the manager higher or lower.
- Assert that those StackItems' offsets are equivalent.
- Other clever things.

---------

Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2023-08-24 19:10:51 +00:00
Victor Stinner f1ae706ca5
gh-107211: No longer export internal functions (7) (#108425)
No longer export _PyUnicode_FromId() internal C API function.

Change comment style to "// comment" and add comment explaining why
other functions have to be exported.

Update Tools/build/generate_token.py to update Include/internal/pycore_token.h
comments.
2023-08-24 17:40:56 +02:00
Irit Katriel 72119d16a5
gh-105481: remove regen-opcode. Generated _PyOpcode_Caches in regen-cases. (#108367) 2023-08-23 18:39:00 +01:00
Alex Waygood a0bb4a39d1
Fix spurious diff if the cases generator is run on Windows (#108319) 2023-08-22 20:49:39 +01:00
Dennis Sweeney 86617518c4
gh-108179: Add error message for parser stack overflows (#108256) 2023-08-22 08:41:50 +01:00
Victor Stinner 0dd3fc2a64
gh-108216: Cleanup #include in internal header files (#108228)
* Add missing includes.
* Remove unused includes.
* Update old include/symbol names to newer names.
* Mention at least one included symbol.
* Sort includes.
* Update Tools/cases_generator/generate_cases.py used to generated
  pycore_opcode_metadata.h.
* Update Parser/asdl_c.py used to generate pycore_ast.h.
* Cleanup also includes in _testcapimodule.c and _testinternalcapi.c.
2023-08-21 18:05:59 +00:00
Victor Stinner 21c0844742
gh-108220: Internal header files require Py_BUILD_CORE to be defined (#108221)
* pycore_intrinsics.h does nothing if included twice
  (add #ifndef and #define).
* Update Tools/cases_generator/generate_cases.py to generate the
  Py_BUILD_CORE test.
* _bz2, _lzma, _opcode and zlib extensions now define the
  Py_BUILD_CORE_MODULE macro to use internal headers
  (pycore_code.h, pycore_intrinsics.h and pycore_blocks_output_buffer.h).
2023-08-21 19:15:52 +02:00
Serhiy Storchaka 60942cccb1
gh-95065, gh-107704: Argument Clinic: support multiple '/ [from ...]' and '* [from ...]' markers (GH-108132) 2023-08-21 13:59:58 +00:00
Alex Waygood 05ef4ca94c
gh-104504: cases generator: Add `--warn-unreachable` to the mypy config (#108112) 2023-08-21 00:40:41 +01:00
Serhiy Storchaka 2f311437cd
gh-107704: Argument Clinic: add support for deprecating keyword use of parameters (GH-107984)
It is now possible to deprecate passing keyword arguments for
keyword-or-positional parameters with Argument Clinic, using the new
'/ [from X.Y]' syntax.
(To be read as "positional-only from Python version X.Y")

Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-19 10:13:35 +03:00
Ned Deily ed25f09716
gh-107565: Update multissltests and GitHub CI workflows to use OpenSSL 1.1.1v, 3.0.10, and 3.1.2. (GH-107896) 2023-08-18 15:48:20 -04:00
Dong-hee Na 28cab71f95
gh-104504: Run mypy on cases_generator in CI (and blacken the code) (gh-108090)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-18 22:42:45 +09:00
Guido van Rossum 61c7249759
gh-106581: Project through calls (#108067)
This finishes the work begun in gh-107760. When, while projecting a superblock, we encounter a call to a short, simple function, the superblock will now enter the function using `_PUSH_FRAME`, continue through it, and leave it using `_POP_FRAME`, and then continue through the original code. Multiple frame pushes and pops are even possible. It is also possible to stop appending to the superblock in the middle of a called function, when running out of space or encountering an unsupported bytecode.
2023-08-17 11:29:58 -07:00
Erlend E. Aasland 292a22bdc2
gh-104683: Argument Clinic: Remove unreachable code from _module_and_class() (#108092)
'not hasattr(parent, "classes")' is always false, since 'parent' is an
instance of either the Module, Class, or Clinic classes, and all of
them has a "classes" attribute.
2023-08-17 18:16:08 +00:00
Irit Katriel 0b243c2f66
gh-105481: opcode.h is no longer generated during the build (#108080) 2023-08-17 17:07:58 +01:00
Irit Katriel 2da67a5789
gh-105481: fix out of date comment (#108079) 2023-08-17 10:25:28 +00:00
Guido van Rossum dc8fdf5fd5
gh-106581: Split `CALL_PY_EXACT_ARGS` into uops (#107760)
* Split `CALL_PY_EXACT_ARGS` into uops

This is only the first step for doing `CALL` in Tier 2.
The next step involves tracing into the called code object and back.
After that we'll have to do the remaining `CALL` specialization.
Finally we'll have to deal with `KW_NAMES`.

Note: this moves setting `frame->return_offset` directly in front of
`DISPATCH_INLINED()`, to make it easier to move it into `_PUSH_FRAME`.
2023-08-16 16:26:43 -07:00
Irit Katriel 665a4391e1
gh-105481: generate op IDs from bytecode.c instead of hard coding them in opcode.py (#107971) 2023-08-16 22:25:18 +00:00
Erlend E. Aasland 42429d3b9a
gh-104683: Argument Clinic: Extract parse function name helper (#107964)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-16 13:47:15 +00:00
Ken Jin e28b0dc86d
gh-107557: Setup abstract interpretation (#107847)
Co-authored-by: Guido van Rossum <gvanrossum@users.noreply.github.com>
Co-authored-by: Jules <57632293+juliapoo@users.noreply.github.com>
2023-08-15 18:04:17 +00:00
Erlend E. Aasland 607f18c894
gh-107972: Argument Clinic: Ensure a C basename is provided after 'as' (#107973)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-08-15 14:41:40 +00:00
Erlend E. Aasland e90036c9bd
gh-107880: Argument Clinic: Fix regression in gh-107885 (#107974)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-15 14:00:52 +00:00
Guido van Rossum a2a4b9f1ec
Attempt to speed up deepfreeze.py (#107887)
* Instead of calling get_identifiers_and_strings(), extract identifiers and strings from pycore_global_strings.h.
* Avoid ast.literal_eval(), it's very slow.
2023-08-14 14:41:27 -07:00
Irit Katriel 39745347f6
gh-105481: reduce repetition in opcode metadata generation code (#107942) 2023-08-14 18:36:29 +00:00
Erlend E. Aasland 9b75ada6e4
gh-107880: Teach Argument Clinic to clone __init__ and __new__ methods (#107885) 2023-08-13 12:13:11 +02:00
Erlend E. Aasland ee40b3e20d
gh-107883: Argument Clinic: Handle full module/class path in Function.fulldisplayname (#107884)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-12 23:46:00 +00:00
Dong-hee Na bf707749e8
gh-106797: Remove warning logs from Python/generated_cases.c.h and executor_cases.c.h (gh-107889)
gh-106797: Remove warning logs from Python/generated_cases.c.h
2023-08-13 04:36:46 +09:00
Irit Katriel caa41a4f1d
gh-105481: split opcode_ids.h out of opcode.h so that it can be generated separately (#107866) 2023-08-11 17:42:01 +01:00
Serhiy Storchaka 3901c991e1
gh-84805: Autogenerate signature for METH_NOARGS and METH_O extension functions (GH-107794) 2023-08-11 18:08:38 +03:00
Martin DeMello 50bbc56009
Fix the long64 reader in umarshal.py (GH-107828) 2023-08-10 23:43:14 +03:00
Erlend E. Aasland 39ef93edb9
gh-95065: Argument Clinic: Add functional tests of deprecated positionals (#107768)
Move the "deprecated positinal" tests from clinic.test.c to
_testclinic.c. Mock PY_VERSION_HEX in order to prevent generated
compiler warnings/errors to trigger. Put clinic code for deprecated
positionals in Modules/clinic/_testclinic_depr_star.c.h for easy
inspection of the generated code.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-10 07:19:05 +00:00
Guido van Rossum 4890bfe1f9
Update README for the cases generator (#107826) 2023-08-10 01:05:51 +00:00
Carl Meyer 2ec16fed14
gh-91054: make code watcher tests resilient to other watchers (#107821) 2023-08-09 16:42:32 -06:00
Erlend E. Aasland 0a7f48b9a8
gh-95065: Produce nicer deprecation messages in Argument Clinic (#107808) 2023-08-09 13:28:18 +00:00
Erlend E. Aasland 1b3f5f24af
gh-104683: Argument Clinic: Params now render their own docstrings (#107790)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-09 12:44:26 +00:00
Alex Waygood 65ce3652fa
gh-104683: Remove unused variables from `Tools/clinic` and tests for `Tools/clinic` (#107771) 2023-08-09 11:24:05 +01:00
Erlend E. Aasland 925bbc2166
gh-80282: Argument Clinic: Add clarifying comment about ASCII docstring limitation (#107764)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-09 07:55:34 +02:00
Erlend E. Aasland 73507382ac
gh-104683: Argument Clinic: refactor format_docstring() (#107623)
Extract helper methods for formatting the signature and parameter
sections, and clean up the remaining function body.

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-08-08 21:12:35 +00:00
Erlend E. Aasland 0be3743f54
gh-104683: Add --exclude option to Argument Clinic CLI (#107770)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-08-08 20:50:54 +00:00
Alex Waygood 7c5153de5a
gh-106368: Argument clinic: add tests for more failure paths (#107731) 2023-08-08 12:12:49 +00:00
Erlend E. Aasland 5df8b0d5c7
gh-95065: Argument Clinic: Add comment to preprocessor warning code (#107766) 2023-08-08 08:43:41 +00:00
Erlend E. Aasland 0db043dd5a
gh-95065: Make Argument Clinic append deprecation warnings to docstrings (#107745) 2023-08-08 08:41:09 +02:00
Guido van Rossum 2df58dcd50
gh-106812: Small stack effect fixes (#107759)
- Generalize the syntax for the type of a stack effect to allow a trailing `*`,
  so we can declare something as e.g. `PyCodeObject *`.

- When generating assignments for stack effects,
  the type of the value on the stack should be the default (i.e., `PyObject *`)
  even when the variable copied to/from it has a different type,
  so that an appropriate cast is generated
  However, not when the variable is an array --
  then the type is taken from the variable (as it is always `PyObject **`).
2023-08-07 21:32:42 -07:00