348184845a
gh-120956: Avoid comparison of int to Py_ssize_t in parser ( #120959 )
2024-06-24 18:13:02 +02:00
ace2045ea6
Fix types in pegen parser generator (GH-120720)
2024-06-19 16:12:40 +02:00
1e5f615086
gh-116991: Improve `pygen --help` for `python` subparser ( #116992 )
2024-06-03 10:52:35 +03:00
48f21b3631
gh-118235: Move RAISE_SYNTAX_ERROR actions to invalid rules and make sure they stay there (GH-119731)
...
The Full Grammar specification in the docs omits rule actions, so grammar rules that raise a syntax error looked like valid syntax.
This was solved in ef940de
2024-05-30 09:27:32 +02:00
bf08f0a5fe
Fix typos in comments ( #119645 )
2024-05-28 09:53:32 +02:00
c4f9823be2
gh-118671: Updated dead ActiveState links ( #118730 )
...
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
2024-05-08 01:06:38 -06:00
9b280ab0ab
gh-116988: Remove duplicates of `annotated_rhs` in the Grammar ( #117004 )
2024-04-24 18:16:06 +01:00
5f1997896d
gh-112984: Fix link error on free-threaded Windows build (GH-114455)
...
The test_peg_generator test tried to link the python313_d.lib library,
which failed because the library is now named python313t_d.lib. The
underlying problem is that the "compiler" attribute was not set when
we call get_libraries() from distutils.
2024-01-23 18:05:15 +00:00
b3d2427f22
gh-58032: Do not use argparse.FileType in module CLIs and scripts (GH-113649)
...
Open and close files manually. It prevents from leaking files,
preliminary creation of output files, and accidental closing of stdin
and stdout.
2024-01-10 15:07:19 +02:00
e5cce70df7
gh-112532: Fix peg generator build.py for mimalloc build (gh-113492)
...
gh-112532: Fix peg generator for mimalloc build
2023-12-27 02:54:16 +09:00
79dad03747
gh-111650: Ensure pyconfig.h includes Py_GIL_DISABLED on Windows (GH-112778)
2023-12-13 15:38:45 +00:00
56e59a49ae
GH-111807: Lower the parser stack depth under WASI debug builds ( #112225 )
2023-11-20 13:27:33 +00:00
8316769d83
gh-110805: Fix test_peg_generator after the change in the parser for REPL tracebacks ( #110814 )
...
Fix test_peg_generator after the change in the parser for REPL tracebacks
2023-10-13 13:04:23 +02:00
17d65547df
gh-104169: Fix test_peg_generator after tokenizer refactoring ( #110727 )
...
* Fix test_peg_generator after tokenizer refactoring
* Remove references to tokenizer.c in comments etc.
2023-10-12 09:34:35 +02:00
b28ffaa193
gh-109596: Ensure repeated rules in the grammar are not allowed and fix incorrect soft keywords ( #109606 )
2023-09-22 19:03:23 +01:00
697c9dcf8f
gh-108455: peg_generator: enable mypy's `--warn-unreachable` setting and `redundant-expr` error code ( #109160 )
2023-09-08 22:05:40 +01:00
b89b838ebc
gh-108455: peg_generator: use `types-setuptools==68.1.0.1` in CI ( #108697 )
2023-08-31 08:41:25 +01:00
77e8f233ac
gh-108455: peg_generator: install two stubs packages before running mypy ( #108637 )
2023-08-29 20:14:08 +01:00
f8be2e262c
gh-108455: peg generator: Use `strict_optional=True` for `grammar_parser` ( #108629 )
2023-08-29 13:02:52 +01:00
0d140b8c5e
gh-108455: peg_generator: make the mypy config slightly stricter ( #108627 )
...
* Enable `--no-implicit-reexport`
* Enable the `truthy-bool` error code
* Enable the `ignore-without-code` error code
* Explicitly note that `--warn-unreachable` cannot yet be enabled
2023-08-29 11:23:22 +00:00
c879b9fddf
gh-108455: Simplify the peg_generator mypy config ( #108620 )
...
make it easier to see exactly which options from mypy's `--strict` mode can't currently be enabled
2023-08-29 10:24:06 +00:00
cf7ba83eb2
gh-108455: Run `mypy` on `Tools/peg_generator` ( #108456 )
...
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2023-08-28 21:04:12 +01:00
86617518c4
gh-108179: Add error message for parser stack overflows ( #108256 )
2023-08-22 08:41:50 +01:00
da8f87b7ea
gh-107015: Remove async_hacks from the tokenizer ( #107018 )
2023-07-26 16:34:15 +01:00
a241003d04
gh-105191: Cleanup peg generator; keep only necessary files ( #105197 )
2023-06-01 17:24:15 +02:00
afa759fb80
gh-100238: Use setuptools in peg-generator and reenable tests ( #104798 )
2023-05-26 15:32:03 +02:00
ae00b810d1
gh-104780: Remove 2to3 program and lib2to3 module ( #104781 )
...
* Remove the Tools/scripts/2to3 script.
* Remove the Lib/test/test_lib2to3/ directory.
* Doc/tools/extensions/pyspecific.py: remove the "2to3fixer" object
type.
* Makefile and PC/layout/main.py no longer compile lib2to3 grammar
files.
* Update Makefile for 2to3 removal.
2023-05-23 19:40:02 +02:00
1ef61cf71a
gh-102856: Initial implementation of PEP 701 ( #102855 )
...
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
Co-authored-by: Marta Gómez Macías <mgmacias@google.com>
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
2023-04-19 11:18:16 -05:00
f533f216e6
gh-102416: Do not memoize incorrectly loop rules in the parser ( #102467 )
2023-03-06 14:41:53 +01:00
1de4395f62
gh-101046: Fix a potential memory leak in the parser when raising MemoryError ( #101051 )
2023-01-16 18:45:37 +00:00
4ce2a202c7
gh-99300: Use Py_NewRef() in Parser/ directory ( #99330 )
...
Replace Py_INCREF() with Py_NewRef() in C files of the Parser/
directory and in the PEG generator.
2022-11-10 15:30:05 +01:00
47e35625ff
gh-84623: Remove unused imports ( #94132 )
2022-06-22 19:14:27 +02:00
616fa3465d
gh-54781: Move Lib/lib2to3/tests/ to Lib/test/test_lib2to3/ ( #94049 )
...
* Move Lib/lib2to3/tests/ to Lib/test/test_lib2to3/.
* Remove Lib/test/test_lib2to3.py.
* Update imports.
* all_project_files(): use different paths and sort files
to make the tests more reproducible.
* Update references to tests.
2022-06-21 15:21:22 +02:00
5115a16831
gh-93103: Parser uses PyConfig.parser_debug instead of Py_DebugFlag ( #93106 )
...
* Replace deprecated Py_DebugFlag with PyConfig.parser_debug in the
parser.
* Add Parser.debug member.
* Add tok_state.debug member.
* Py_FrozenMain(): Replace Py_VerboseFlag with PyConfig.verbose.
2022-05-24 22:35:08 +02:00
137fd3d88a
gh-90473: Decrease recursion limit and skip tests on WASI (GH-92803)
2022-05-19 12:43:16 +02:00
804f2529d8
gh-91320: Use _PyCFunction_CAST() ( #92251 )
...
Replace "(PyCFunction)(void(*)(void))func" cast with
_PyCFunction_CAST(func).
Change generated by the command:
sed -i -e \
's!(PyCFunction)(void(\*)(void)) *\([A-Za-z0-9_]\+\)!_PyCFunction_CAST(\1)!g' \
$(find -name "*.c")
2022-05-03 21:42:14 +02:00
612e422c6e
bpo-46576: Speed up test_peg_generator by using a static library for shared sources (GH-32338)
...
Speed up test_peg_generator by using a static library for shared sources to avoid recompiling as much code.
2022-04-06 14:55:58 -07:00
7f07b5ee9c
Update grammar_grapher with the new forced (&&) directive ( #31704 )
2022-03-06 15:26:28 +01:00
390459de6d
Allow the parser to avoid nested processing of invalid rules (GH-31252)
2022-02-10 13:12:14 +00:00
164a017e13
bpo-46576: bpo-46524: Disable compiler optimization within test_peg_generator. ( #31015 )
...
Disable compiler optimization within test_peg_generator.
This speed up test_peg_generator by always disabling compiler
optimizations by using -O0 or equivalent when the test is building its
own C extensions.
A build not using --with-pydebug in order to speed up test execution
winds up with this test taking a very long time as it would do
repeated compilation of parser C code using the same optimization
flags as CPython was built with.
This speeds the test up 6-8x on gps-raspbian.
Also incorporate's #31017 's win32 conditional and flags.
Co-authored-by: Kumar Aditya kumaraditya303
2022-02-02 12:15:16 -08:00
dd6c35761a
bpo-46110: Restore commit e9898bf153
...
This restores commit e9898bf153
2022-01-03 19:54:06 +00:00
9d35dedc5e
Revert "bpo-46110: Add a recursion check to avoid stack overflow in the PEG parser (GH-30177)" (GH-30363)
...
This reverts commit e9898bf153
2022-01-03 18:29:18 +00:00
e9898bf153
bpo-46110: Add a recursion check to avoid stack overflow in the PEG parser (GH-30177)
...
Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
2021-12-20 15:43:26 +00:00
253b7a0a9f
bpo-45866: pegen strips directory of "generated from" header (GH-29777)
...
"make regen-all" now produces the same output when run from a
directory other than the source tree: when building Python out of the
source tree.
2021-11-26 11:50:34 +01:00
c9c4444d9f
Refactor parser compilation units into specific components (GH-29676)
2021-11-21 01:08:50 +00:00
762173c670
bpo-44257: fix "assigment_expr" typo + regenerate the grammar, and remove unused imports (GH-29393)
...
Co-authored-by: Wim Glenn <wglenn@jumptrading.com>
2021-11-03 20:57:40 +00:00
a106343f63
bpo-45494: Fix parser crash when reporting errors involving invalid continuation characters (GH-28993)
...
There are two errors that this commit fixes:
* The parser was not correctly computing the offset and the string
source for E_LINECONT errors due to the incorrect usage of strtok().
* The parser was not correctly unwinding the call stack when a tokenizer
exception happened in rules involving optionals ('?', [...]) as we
always make them return valid results by using the comma operator. We
need to check first if we don't have an error before continuing.
2021-10-19 21:24:12 +02:00
682aecfdeb
Fix typos in the Tools directory (GH-28769)
...
Like #28744 but for the Tools directory.
[skip issue] Opening a related issue is pending python/psf-infra-meta#130
Automerge-Triggered-By: GH:pablogsal
2021-10-06 10:55:16 -07:00
48fadb1f19
[Tools/peg_generator/pegen/parser.py] Fix typo: s/wether/whether/ (GH-28739)
2021-10-05 23:06:19 +02:00
b01fd533fe
Extract visitors from the grammar nodes and call makers in the peg generator (GH-28172)
...
Simplify the peg generator logic by extracting as much visitors as possible to disentangle the flow and separate concerns.
2021-09-05 14:58:52 +01:00
Lysandros Nikolaou
yf-yang
Nikita Sobolev
Petr Viktorin
Xie Yanbo
trag1c
David Rubin
Sam Gross
Serhiy Storchaka
Donghee Na
Steve Dower
Brett Cannon
Pablo Galindo Salgado
Alex Waygood
Dennis Sweeney
Victor Stinner
Christian Heimes
Jeremy Kloth
Luca Chiodini
Gregory P. Smith
wim glenn
Christian Clauss
Ikko Ashimine