Commit Graph

3975 Commits

Author SHA1 Message Date
Alex Waygood 77e8f233ac
gh-108455: peg_generator: install two stubs packages before running mypy (#108637) 2023-08-29 20:14:08 +01:00
Guido van Rossum 4f22152713
gh-107557: Remove unnecessary SAVE_IP instructions (#108583)
Also remove NOP instructions.

The "stubs" are not optimized in this fashion (their SAVE_IP should always be preserved since it's where to jump next, and they don't contain NOPs by their nature).
2023-08-29 16:51:51 +00:00
Nikita Sobolev f8be2e262c
gh-108455: peg generator: Use `strict_optional=True` for `grammar_parser` (#108629) 2023-08-29 13:02:52 +01:00
Victor Stinner e675e515ae
gh-108494: Argument Clinic: fix option group for Limited C API (#108574)
Use PyTuple_Size() instead of PyTuple_GET_SIZE().
2023-08-29 13:33:28 +02:00
Alex Waygood 0d140b8c5e
gh-108455: peg_generator: make the mypy config slightly stricter (#108627)
* Enable `--no-implicit-reexport`

* Enable the `truthy-bool` error code

* Enable the `ignore-without-code` error code

* Explicitly note that `--warn-unreachable` cannot yet be enabled
2023-08-29 11:23:22 +00:00
Alex Waygood c879b9fddf
gh-108455: Simplify the peg_generator mypy config (#108620)
make it easier to see exactly which options from mypy's `--strict` mode can't currently be enabled
2023-08-29 10:24:06 +00:00
Victor Stinner 301eb7e607
gh-106320: Remove _PyAnextAwaitable_Type from the public C API (#108597)
It's not needed to declare it in Include/iterobject.h: just use
"extern" where it's used (only in object.c).
2023-08-29 02:05:11 +00:00
Adam Turner 0bd2ba553d
GH-107603: Clinic: Pass specific attributes to ``print_block()`` (#108581) 2023-08-28 21:31:45 +00:00
Nikita Sobolev cf7ba83eb2
gh-108455: Run `mypy` on `Tools/peg_generator` (#108456)
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2023-08-28 21:04:12 +01:00
Alex Waygood 242bef459b
gh-108494: Argument clinic: Improve the `parse_file()` API (#108575)
Co-authored-by: Victor Stinner <vstinner@python.org>
2023-08-28 18:25:16 +00:00
Serhiy Storchaka bc5356bb5d
gh-108494: Argument Clinic: fix support of Limited C API (GH-108536) 2023-08-28 16:04:27 +03:00
Erlend E. Aasland d90973340b
gh-104683: Argument Clinic: Refactor the module and class resolver (#108552) 2023-08-28 14:41:05 +02:00
Victor Stinner 8ba4714611
gh-106320: Remove private AC converter functions (#108505)
Move these private functions to the internal C API
(pycore_abstract.h):

* _Py_convert_optional_to_ssize_t()
* _PyNumber_Index()

Argument Clinic now emits #include "pycore_abstract.h" when these
functions are used.

The parser of the c-analyzer tool now uses a list of files which use
the limited C API, rather than a list of files using the internal C
API.
2023-08-26 04:05:17 +02:00
Victor Stinner 6353c21b78
gh-106320: Remove private _PyLong_FileDescriptor_Converter() (#108503)
Move the private _PyLong converter functions to the internal C API

* _PyLong_FileDescriptor_Converter(): moved to pycore_fileutils.h
* _PyLong_Size_t_Converter(): moved to pycore_long.h

Argument Clinic now emits includes for pycore_fileutils.h and
pycore_long.h when these functions are used.
2023-08-26 03:18:09 +02:00
Victor Stinner 713afb8804
gh-106320: Remove private _PyLong converter functions (#108499)
Move these private functions to the internal C API (pycore_long.h):

* _PyLong_UnsignedInt_Converter()
* _PyLong_UnsignedLongLong_Converter()
* _PyLong_UnsignedLong_Converter()
* _PyLong_UnsignedShort_Converter()

Argument Clinic now emits #include "pycore_long.h" when these
functions are used.
2023-08-26 02:24:27 +02:00
Victor Stinner 86bc9e35c4
gh-108494: AC supports pos-only args in limited C API (#108498)
AC now checks for "#define Py_LIMITED_API" pattern to use the limited
C API.
2023-08-26 00:39:24 +02:00
Victor Stinner 73d33c1a30
gh-107603: Argument Clinic can emit includes (#108486)
* Add Clinic.add_include() method
* Add CConverter.include and CConverter.add_include()
* Printer.print_block() gets a second parameter: clinic.
* Remove duplicated declaration of "clinic" global variable.
2023-08-26 00:01:10 +02:00
Victor Stinner 1dd9510977
gh-108494: Argument Clinic partial supports of Limited C API (#108495)
Argument Clinic now has a partial support of the
Limited API:

* Add --limited option to clinic.c.
* Add '_testclinic_limited' extension which is built with
  the limited C API version 3.13.
* For now, hardcode in clinic.py that "_testclinic_limited.c" targets
  the limited C API.
2023-08-25 23:22:08 +02:00
Irit Katriel 5f41376e93
gh-107901: add the HAS_EVAL_BREAK instruction flag (#108375) 2023-08-25 18:33:59 +00:00
Alex Waygood 5a25daa512
gh-104504: Cases generator: enable mypy's `possibly-undefined` error code (#108454) 2023-08-25 18:08:29 +01:00
Guido van Rossum ddf66b54ed
gh-106581: Split CALL_BOUND_METHOD_EXACT_ARGS into uops (#108462)
Instead of using `GO_TO_INSTRUCTION(CALL_PY_EXACT_ARGS)` we just add the macro elements of the latter to the macro for the former. This requires lengthening the uops array in struct opcode_macro_expansion. (It also required changes to stacking.py that were merged already.)
2023-08-24 17:36:00 -07:00
Victor Stinner 4e5a7284ee
gh-108444: Argument Clinic uses PyLong_AsInt() (#108458)
Argument Clinic now uses the new public PyLong_AsInt(), rather than
the old name _PyLong_AsInt().
2023-08-25 00:51:22 +02:00
Guido van Rossum 88941d665f
gh-106581: Fix two bugs in the code generator's copy optimization (#108380)
I was comparing the last preceding poke with the *last* peek,
rather than the *first* peek.

Unfortunately this bug obscured another bug:
When the last preceding poke is UNUSED, the first peek disappears,
leaving the variable unassigned. This is how I fixed it:

- Rename CopyEffect to CopyItem.
- Change CopyItem to contain StackItems instead of StackEffects.
- Update those StackItems when adjusting the manager higher or lower.
- Assert that those StackItems' offsets are equivalent.
- Other clever things.

---------

Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2023-08-24 19:10:51 +00:00
Victor Stinner f1ae706ca5
gh-107211: No longer export internal functions (7) (#108425)
No longer export _PyUnicode_FromId() internal C API function.

Change comment style to "// comment" and add comment explaining why
other functions have to be exported.

Update Tools/build/generate_token.py to update Include/internal/pycore_token.h
comments.
2023-08-24 17:40:56 +02:00
Irit Katriel 72119d16a5
gh-105481: remove regen-opcode. Generated _PyOpcode_Caches in regen-cases. (#108367) 2023-08-23 18:39:00 +01:00
Alex Waygood a0bb4a39d1
Fix spurious diff if the cases generator is run on Windows (#108319) 2023-08-22 20:49:39 +01:00
Dennis Sweeney 86617518c4
gh-108179: Add error message for parser stack overflows (#108256) 2023-08-22 08:41:50 +01:00
Victor Stinner 0dd3fc2a64
gh-108216: Cleanup #include in internal header files (#108228)
* Add missing includes.
* Remove unused includes.
* Update old include/symbol names to newer names.
* Mention at least one included symbol.
* Sort includes.
* Update Tools/cases_generator/generate_cases.py used to generated
  pycore_opcode_metadata.h.
* Update Parser/asdl_c.py used to generate pycore_ast.h.
* Cleanup also includes in _testcapimodule.c and _testinternalcapi.c.
2023-08-21 18:05:59 +00:00
Victor Stinner 21c0844742
gh-108220: Internal header files require Py_BUILD_CORE to be defined (#108221)
* pycore_intrinsics.h does nothing if included twice
  (add #ifndef and #define).
* Update Tools/cases_generator/generate_cases.py to generate the
  Py_BUILD_CORE test.
* _bz2, _lzma, _opcode and zlib extensions now define the
  Py_BUILD_CORE_MODULE macro to use internal headers
  (pycore_code.h, pycore_intrinsics.h and pycore_blocks_output_buffer.h).
2023-08-21 19:15:52 +02:00
Serhiy Storchaka 60942cccb1
gh-95065, gh-107704: Argument Clinic: support multiple '/ [from ...]' and '* [from ...]' markers (GH-108132) 2023-08-21 13:59:58 +00:00
Alex Waygood 05ef4ca94c
gh-104504: cases generator: Add `--warn-unreachable` to the mypy config (#108112) 2023-08-21 00:40:41 +01:00
Serhiy Storchaka 2f311437cd
gh-107704: Argument Clinic: add support for deprecating keyword use of parameters (GH-107984)
It is now possible to deprecate passing keyword arguments for
keyword-or-positional parameters with Argument Clinic, using the new
'/ [from X.Y]' syntax.
(To be read as "positional-only from Python version X.Y")

Co-authored-by: Erlend E. Aasland <erlend@python.org>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-19 10:13:35 +03:00
Ned Deily ed25f09716
gh-107565: Update multissltests and GitHub CI workflows to use OpenSSL 1.1.1v, 3.0.10, and 3.1.2. (GH-107896) 2023-08-18 15:48:20 -04:00
Dong-hee Na 28cab71f95
gh-104504: Run mypy on cases_generator in CI (and blacken the code) (gh-108090)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-18 22:42:45 +09:00
Guido van Rossum 61c7249759
gh-106581: Project through calls (#108067)
This finishes the work begun in gh-107760. When, while projecting a superblock, we encounter a call to a short, simple function, the superblock will now enter the function using `_PUSH_FRAME`, continue through it, and leave it using `_POP_FRAME`, and then continue through the original code. Multiple frame pushes and pops are even possible. It is also possible to stop appending to the superblock in the middle of a called function, when running out of space or encountering an unsupported bytecode.
2023-08-17 11:29:58 -07:00
Erlend E. Aasland 292a22bdc2
gh-104683: Argument Clinic: Remove unreachable code from _module_and_class() (#108092)
'not hasattr(parent, "classes")' is always false, since 'parent' is an
instance of either the Module, Class, or Clinic classes, and all of
them has a "classes" attribute.
2023-08-17 18:16:08 +00:00
Irit Katriel 0b243c2f66
gh-105481: opcode.h is no longer generated during the build (#108080) 2023-08-17 17:07:58 +01:00
Irit Katriel 2da67a5789
gh-105481: fix out of date comment (#108079) 2023-08-17 10:25:28 +00:00
Guido van Rossum dc8fdf5fd5
gh-106581: Split `CALL_PY_EXACT_ARGS` into uops (#107760)
* Split `CALL_PY_EXACT_ARGS` into uops

This is only the first step for doing `CALL` in Tier 2.
The next step involves tracing into the called code object and back.
After that we'll have to do the remaining `CALL` specialization.
Finally we'll have to deal with `KW_NAMES`.

Note: this moves setting `frame->return_offset` directly in front of
`DISPATCH_INLINED()`, to make it easier to move it into `_PUSH_FRAME`.
2023-08-16 16:26:43 -07:00
Irit Katriel 665a4391e1
gh-105481: generate op IDs from bytecode.c instead of hard coding them in opcode.py (#107971) 2023-08-16 22:25:18 +00:00
Erlend E. Aasland 42429d3b9a
gh-104683: Argument Clinic: Extract parse function name helper (#107964)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-16 13:47:15 +00:00
Ken Jin e28b0dc86d
gh-107557: Setup abstract interpretation (#107847)
Co-authored-by: Guido van Rossum <gvanrossum@users.noreply.github.com>
Co-authored-by: Jules <57632293+juliapoo@users.noreply.github.com>
2023-08-15 18:04:17 +00:00
Erlend E. Aasland 607f18c894
gh-107972: Argument Clinic: Ensure a C basename is provided after 'as' (#107973)
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2023-08-15 14:41:40 +00:00
Erlend E. Aasland e90036c9bd
gh-107880: Argument Clinic: Fix regression in gh-107885 (#107974)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-15 14:00:52 +00:00
Guido van Rossum a2a4b9f1ec
Attempt to speed up deepfreeze.py (#107887)
* Instead of calling get_identifiers_and_strings(), extract identifiers and strings from pycore_global_strings.h.
* Avoid ast.literal_eval(), it's very slow.
2023-08-14 14:41:27 -07:00
Irit Katriel 39745347f6
gh-105481: reduce repetition in opcode metadata generation code (#107942) 2023-08-14 18:36:29 +00:00
Erlend E. Aasland 9b75ada6e4
gh-107880: Teach Argument Clinic to clone __init__ and __new__ methods (#107885) 2023-08-13 12:13:11 +02:00
Erlend E. Aasland ee40b3e20d
gh-107883: Argument Clinic: Handle full module/class path in Function.fulldisplayname (#107884)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-08-12 23:46:00 +00:00
Dong-hee Na bf707749e8
gh-106797: Remove warning logs from Python/generated_cases.c.h and executor_cases.c.h (gh-107889)
gh-106797: Remove warning logs from Python/generated_cases.c.h
2023-08-13 04:36:46 +09:00
Irit Katriel caa41a4f1d
gh-105481: split opcode_ids.h out of opcode.h so that it can be generated separately (#107866) 2023-08-11 17:42:01 +01:00