9b280ab0ab
gh-116988: Remove duplicates of `annotated_rhs` in the Grammar ( #117004 )
2024-04-24 18:16:06 +01:00
5f1997896d
gh-112984: Fix link error on free-threaded Windows build (GH-114455)
...
The test_peg_generator test tried to link the python313_d.lib library,
which failed because the library is now named python313t_d.lib. The
underlying problem is that the "compiler" attribute was not set when
we call get_libraries() from distutils.
2024-01-23 18:05:15 +00:00
b3d2427f22
gh-58032: Do not use argparse.FileType in module CLIs and scripts (GH-113649)
...
Open and close files manually. It prevents from leaking files,
preliminary creation of output files, and accidental closing of stdin
and stdout.
2024-01-10 15:07:19 +02:00
e5cce70df7
gh-112532: Fix peg generator build.py for mimalloc build (gh-113492)
...
gh-112532: Fix peg generator for mimalloc build
2023-12-27 02:54:16 +09:00
79dad03747
gh-111650: Ensure pyconfig.h includes Py_GIL_DISABLED on Windows (GH-112778)
2023-12-13 15:38:45 +00:00
56e59a49ae
GH-111807: Lower the parser stack depth under WASI debug builds ( #112225 )
2023-11-20 13:27:33 +00:00
8316769d83
gh-110805: Fix test_peg_generator after the change in the parser for REPL tracebacks ( #110814 )
...
Fix test_peg_generator after the change in the parser for REPL tracebacks
2023-10-13 13:04:23 +02:00
17d65547df
gh-104169: Fix test_peg_generator after tokenizer refactoring ( #110727 )
...
* Fix test_peg_generator after tokenizer refactoring
* Remove references to tokenizer.c in comments etc.
2023-10-12 09:34:35 +02:00
b28ffaa193
gh-109596: Ensure repeated rules in the grammar are not allowed and fix incorrect soft keywords ( #109606 )
2023-09-22 19:03:23 +01:00
697c9dcf8f
gh-108455: peg_generator: enable mypy's `--warn-unreachable` setting and `redundant-expr` error code ( #109160 )
2023-09-08 22:05:40 +01:00
b89b838ebc
gh-108455: peg_generator: use `types-setuptools==68.1.0.1` in CI ( #108697 )
2023-08-31 08:41:25 +01:00
77e8f233ac
gh-108455: peg_generator: install two stubs packages before running mypy ( #108637 )
2023-08-29 20:14:08 +01:00
f8be2e262c
gh-108455: peg generator: Use `strict_optional=True` for `grammar_parser` ( #108629 )
2023-08-29 13:02:52 +01:00
0d140b8c5e
gh-108455: peg_generator: make the mypy config slightly stricter ( #108627 )
...
* Enable `--no-implicit-reexport`
* Enable the `truthy-bool` error code
* Enable the `ignore-without-code` error code
* Explicitly note that `--warn-unreachable` cannot yet be enabled
2023-08-29 11:23:22 +00:00
c879b9fddf
gh-108455: Simplify the peg_generator mypy config ( #108620 )
...
make it easier to see exactly which options from mypy's `--strict` mode can't currently be enabled
2023-08-29 10:24:06 +00:00
cf7ba83eb2
gh-108455: Run `mypy` on `Tools/peg_generator` ( #108456 )
...
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2023-08-28 21:04:12 +01:00
86617518c4
gh-108179: Add error message for parser stack overflows ( #108256 )
2023-08-22 08:41:50 +01:00
da8f87b7ea
gh-107015: Remove async_hacks from the tokenizer ( #107018 )
2023-07-26 16:34:15 +01:00
a241003d04
gh-105191: Cleanup peg generator; keep only necessary files ( #105197 )
2023-06-01 17:24:15 +02:00
afa759fb80
gh-100238: Use setuptools in peg-generator and reenable tests ( #104798 )
2023-05-26 15:32:03 +02:00
ae00b810d1
gh-104780: Remove 2to3 program and lib2to3 module ( #104781 )
...
* Remove the Tools/scripts/2to3 script.
* Remove the Lib/test/test_lib2to3/ directory.
* Doc/tools/extensions/pyspecific.py: remove the "2to3fixer" object
type.
* Makefile and PC/layout/main.py no longer compile lib2to3 grammar
files.
* Update Makefile for 2to3 removal.
2023-05-23 19:40:02 +02:00
1ef61cf71a
gh-102856: Initial implementation of PEP 701 ( #102855 )
...
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
Co-authored-by: Marta Gómez Macías <mgmacias@google.com>
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
2023-04-19 11:18:16 -05:00
f533f216e6
gh-102416: Do not memoize incorrectly loop rules in the parser ( #102467 )
2023-03-06 14:41:53 +01:00
1de4395f62
gh-101046: Fix a potential memory leak in the parser when raising MemoryError ( #101051 )
2023-01-16 18:45:37 +00:00
4ce2a202c7
gh-99300: Use Py_NewRef() in Parser/ directory ( #99330 )
...
Replace Py_INCREF() with Py_NewRef() in C files of the Parser/
directory and in the PEG generator.
2022-11-10 15:30:05 +01:00
47e35625ff
gh-84623: Remove unused imports ( #94132 )
2022-06-22 19:14:27 +02:00
616fa3465d
gh-54781: Move Lib/lib2to3/tests/ to Lib/test/test_lib2to3/ ( #94049 )
...
* Move Lib/lib2to3/tests/ to Lib/test/test_lib2to3/.
* Remove Lib/test/test_lib2to3.py.
* Update imports.
* all_project_files(): use different paths and sort files
to make the tests more reproducible.
* Update references to tests.
2022-06-21 15:21:22 +02:00
5115a16831
gh-93103: Parser uses PyConfig.parser_debug instead of Py_DebugFlag ( #93106 )
...
* Replace deprecated Py_DebugFlag with PyConfig.parser_debug in the
parser.
* Add Parser.debug member.
* Add tok_state.debug member.
* Py_FrozenMain(): Replace Py_VerboseFlag with PyConfig.verbose.
2022-05-24 22:35:08 +02:00
137fd3d88a
gh-90473: Decrease recursion limit and skip tests on WASI (GH-92803)
2022-05-19 12:43:16 +02:00
804f2529d8
gh-91320: Use _PyCFunction_CAST() ( #92251 )
...
Replace "(PyCFunction)(void(*)(void))func" cast with
_PyCFunction_CAST(func).
Change generated by the command:
sed -i -e \
's!(PyCFunction)(void(\*)(void)) *\([A-Za-z0-9_]\+\)!_PyCFunction_CAST(\1)!g' \
$(find -name "*.c")
2022-05-03 21:42:14 +02:00
612e422c6e
bpo-46576: Speed up test_peg_generator by using a static library for shared sources (GH-32338)
...
Speed up test_peg_generator by using a static library for shared sources to avoid recompiling as much code.
2022-04-06 14:55:58 -07:00
7f07b5ee9c
Update grammar_grapher with the new forced (&&) directive ( #31704 )
2022-03-06 15:26:28 +01:00
390459de6d
Allow the parser to avoid nested processing of invalid rules (GH-31252)
2022-02-10 13:12:14 +00:00
164a017e13
bpo-46576: bpo-46524: Disable compiler optimization within test_peg_generator. ( #31015 )
...
Disable compiler optimization within test_peg_generator.
This speed up test_peg_generator by always disabling compiler
optimizations by using -O0 or equivalent when the test is building its
own C extensions.
A build not using --with-pydebug in order to speed up test execution
winds up with this test taking a very long time as it would do
repeated compilation of parser C code using the same optimization
flags as CPython was built with.
This speeds the test up 6-8x on gps-raspbian.
Also incorporate's #31017 's win32 conditional and flags.
Co-authored-by: Kumar Aditya kumaraditya303
2022-02-02 12:15:16 -08:00
dd6c35761a
bpo-46110: Restore commit e9898bf153
...
This restores commit e9898bf153
2022-01-03 19:54:06 +00:00
9d35dedc5e
Revert "bpo-46110: Add a recursion check to avoid stack overflow in the PEG parser (GH-30177)" (GH-30363)
...
This reverts commit e9898bf153
2022-01-03 18:29:18 +00:00
e9898bf153
bpo-46110: Add a recursion check to avoid stack overflow in the PEG parser (GH-30177)
...
Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
2021-12-20 15:43:26 +00:00
253b7a0a9f
bpo-45866: pegen strips directory of "generated from" header (GH-29777)
...
"make regen-all" now produces the same output when run from a
directory other than the source tree: when building Python out of the
source tree.
2021-11-26 11:50:34 +01:00
c9c4444d9f
Refactor parser compilation units into specific components (GH-29676)
2021-11-21 01:08:50 +00:00
762173c670
bpo-44257: fix "assigment_expr" typo + regenerate the grammar, and remove unused imports (GH-29393)
...
Co-authored-by: Wim Glenn <wglenn@jumptrading.com>
2021-11-03 20:57:40 +00:00
a106343f63
bpo-45494: Fix parser crash when reporting errors involving invalid continuation characters (GH-28993)
...
There are two errors that this commit fixes:
* The parser was not correctly computing the offset and the string
source for E_LINECONT errors due to the incorrect usage of strtok().
* The parser was not correctly unwinding the call stack when a tokenizer
exception happened in rules involving optionals ('?', [...]) as we
always make them return valid results by using the comma operator. We
need to check first if we don't have an error before continuing.
2021-10-19 21:24:12 +02:00
682aecfdeb
Fix typos in the Tools directory (GH-28769)
...
Like #28744 but for the Tools directory.
[skip issue] Opening a related issue is pending python/psf-infra-meta#130
Automerge-Triggered-By: GH:pablogsal
2021-10-06 10:55:16 -07:00
48fadb1f19
[Tools/peg_generator/pegen/parser.py] Fix typo: s/wether/whether/ (GH-28739)
2021-10-05 23:06:19 +02:00
b01fd533fe
Extract visitors from the grammar nodes and call makers in the peg generator (GH-28172)
...
Simplify the peg generator logic by extracting as much visitors as possible to disentangle the flow and separate concerns.
2021-09-05 14:58:52 +01:00
953d27261e
Update pegen to use the latest upstream developments (GH-27586)
2021-08-12 17:37:30 +01:00
aef1b58dc8
bpo-44345: Fix 'generated by' comment in parser.c (GH-26615)
2021-06-09 16:38:53 +02:00
c878a97968
bpo-44180: Fix edge cases in invalid assigment rules in the parser (GH-26283)
...
The invalid assignment rules are very delicate since the parser can
easily raise an invalid assignment when a keyword argument is provided.
As they are very deep into the grammar tree, is very difficult to
specify in which contexts these rules can be used and in which don't.
For that, we need to use a different version of the rule that doesn't do
error checking in those situations where we don't want the rule to raise
(keyword arguments and generator expressions).
We also need to check if we are in left-recursive rule, as those can try
to eagerly advance the parser even if the parse will fail at the end of
the expression. Failing to do this allows the parser to start parsing a
call as a tuple and incorrectly identify a keyword argument as an
invalid assignment, before it realizes that it was not a tuple after all.
2021-05-21 18:34:54 +01:00
b0544ba77c
bpo-38605: Revert making 'from __future__ import annotations' the default (GH-25490)
...
This reverts commits 044a1048ca1be456ae9d
2021-04-21 12:41:19 +01:00
b280248be8
bpo-43822: Improve syntax errors for missing commas (GH-25377)
2021-04-15 21:38:45 +01:00
58bafe42ab
Sanitize macros and debug functions in pegen.c (GH-25291)
2021-04-09 01:17:31 +01:00
David Rubin
Sam Gross
Serhiy Storchaka
Donghee Na
Steve Dower
Brett Cannon
Pablo Galindo Salgado
Lysandros Nikolaou
Alex Waygood
Nikita Sobolev
Dennis Sweeney
Victor Stinner
Christian Heimes
Jeremy Kloth
Luca Chiodini
Gregory P. Smith
wim glenn
Christian Clauss
Ikko Ashimine
Akira Nonaka