da8f87b7ea
gh-107015: Remove async_hacks from the tokenizer ( #107018 )
2023-07-26 16:34:15 +01:00
13237a2da8
gh-98931: Add custom error messages to invalid import/from with multiple targets ( #105985 )
...
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-06-22 15:56:40 +00:00
41de54378d
gh-105194: Fix format specifier escaped characters in f-strings ( #105231 )
2023-06-02 13:33:26 +02:00
ba73473f4c
gh-104799: Move location of type_params AST fields ( #104828 )
...
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-26 05:54:37 -07:00
a5f244d627
gh-104656: Rename typeparams AST node to type_params ( #104657 )
2023-05-21 21:25:09 -07:00
6715f91edc
gh-102856: Python tokenizer implementation for PEP 701 ( #104323 )
...
This commit replaces the Python implementation of the tokenize module with an implementation
that reuses the real C tokenizer via a private extension module. The tokenize module now implements
a compatibility layer that transforms tokens from the C tokenizer into Python tokenize tokens for backward
compatibility.
As the C tokenizer does not emit some tokens that the Python tokenizer provides (such as comments and non-semantic newlines), a new special mode has been added to the C tokenizer mode that currently is only used via
the extension module that exposes it to the Python layer. This new mode forces the C tokenizer to emit these new extra tokens and add the appropriate metadata that is needed to match the old Python implementation.
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2023-05-21 01:03:02 +01:00
24d8b88420
gh-103763: Implement PEP 695 ( #103764 )
...
This implements PEP 695, Type Parameter Syntax. It adds support for:
- Generic functions (def func[T](): ...)
- Generic classes (class X[T](): ...)
- Type aliases (type X = ...)
- New scoping when the new syntax is used within a class body
- Compiler and interpreter changes to support the new syntax and scoping rules
Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
Co-authored-by: Eric Traut <eric@traut.com>
Co-authored-by: Larry Hastings <larry@hastings.org>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-15 20:36:23 -07:00
9169a56fad
gh-103656: Transfer f-string buffers to parser to avoid use-after-free (GH-103896)
...
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2023-04-27 01:33:31 +00:00
1ef61cf71a
gh-102856: Initial implementation of PEP 701 ( #102855 )
...
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
Co-authored-by: Marta Gómez Macías <mgmacias@google.com>
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
2023-04-19 11:18:16 -05:00
6d8da238cc
gh-90994: Improve error messages upon call arguments syntax errors (GH-96893)
2022-11-21 00:15:05 +01:00
9c4232ae89
gh-99211: Point to except/except* on syntax errors when mixing them (GH-99215)
...
Automerge-Triggered-By: GH:lysnikolaou
2022-11-20 09:11:02 -08:00
61b6c40b64
gh-99153: set location on SyntaxError for try with both except and except* (GH-99160)
2022-11-06 15:36:19 +00:00
395d4285bf
gh-98931: Improve error message when the user types 'import x from y' instead of 'from y import x' ( #98932 )
2022-11-01 13:01:20 +00:00
7e36abbb78
gh-91210: Improve error message when non-default param follows default (GH-95933)
...
- Improve error message when parameter without a default follows one with a default
- Show same error message when positional-only params precede the default/non-default sequence
2022-09-17 10:09:28 -07:00
2c7d2e8d46
gh-96587: Raise `SyntaxError` for PEP654 on older `feature_version` ( #96588 )
2022-09-05 17:54:09 +01:00
a965db37f2
gh-94996: Disallow lambda pos only params with feature_version < (3, 8) (GH-95934)
2022-08-12 20:41:02 +02:00
b5e3ea2862
gh-94996: Disallow parsing pos only params with feature_version < (3, 8) (GH-94997)
2022-08-12 19:27:50 +02:00
0daba82221
gh-94949: Disallow parsing parenthesised ctx mgr with old feature_version ( #94950 )
...
* gh-94949: Disallow parsing parenthesised ctx manager with old feature_version
* 📜 🤖 Added by blurb_it.
* Allow it with feature_version=(3, 9) as well
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2022-07-18 22:10:49 +01:00
ae0be5a53b
gh-94947: Disallow parsing walrus with feature_version < (3, 8) ( #94948 )
...
* gh-94947: Disallow parsing walrus with feature_version < (3, 8)
* oops, commit the parser
* 📜 🤖 Added by blurb_it.
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2022-07-18 10:20:12 +01:00
8c237a7a71
gh-94192: Fix error for dictionary literals with invalid expression as value. ( #94304 )
...
* Fix error for dictionary literals with invalid expression as value.
* Remove trailing whitespace
2022-06-26 19:42:47 +01:00
2fc83ac3af
gh-92858: Improve error message for some suites with syntax error before ':' ( #92894 )
2022-06-23 17:31:09 +01:00
53a8b17895
gh-93671: Avoid exponential backtracking in deeply nested sequence patterns in match statements (GH-93680)
...
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2022-06-10 17:56:45 +02:00
aa0f056a00
bpo-47212: Improve error messages for un-parenthesized generator expressions (GH-32302)
2022-04-05 14:47:13 +01:00
e8e737bcf6
bpo-43224: Implement PEP 646 grammar changes (GH-31018)
...
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-03-26 09:55:35 -07:00
7d810b6a4e
bpo-46838: Syntax error improvements for function definitions (GH-31590)
2022-03-22 11:38:41 +00:00
390459de6d
Allow the parser to avoid nested processing of invalid rules (GH-31252)
2022-02-10 13:12:14 +00:00
b71dc71905
bpo-46707: Avoid potential exponential backtracking in some syntax errors (GH-31241)
2022-02-10 03:37:17 +00:00
1bee9a4625
bpo-46272: Fix two heading comments in python.gram (GH-30499)
...
One typo fix and one heading change, both in comments. No functional changes.
2022-01-09 08:22:54 -08:00
d60457a667
bpo-45292: [PEP-654] add except* (GH-29581)
2021-12-14 16:48:15 +00:00
24c10d2943
bpo-45727: Only trigger the 'did you forgot a comma' error suggestion if inside parentheses (GH-29757)
2021-11-24 22:21:23 +00:00
7a1d932528
bpo-45450: Improve syntax error for parenthesized arguments (GH-28906)
2021-11-20 18:27:40 +00:00
546cefcda7
bpo-45727: Make the syntax error for missing comma more consistent (GH-29427)
2021-11-19 23:11:57 +00:00
2819e98d10
bpo-45764: improve error message when missing '(' after 'def' (GH-29484)
...
to achieve this, change the grammar to expect the '(' token after 'def' NAME.
Automerge-Triggered-By: GH:pablogsal
2021-11-09 06:03:32 -08:00
e2d65630f3
bpo-45716: Improve the error message when using True/False/None as keywords in a call (GH-29413)
2021-11-05 13:54:55 +00:00
762173c670
bpo-44257: fix "assigment_expr" typo + regenerate the grammar, and remove unused imports (GH-29393)
...
Co-authored-by: Wim Glenn <wglenn@jumptrading.com>
2021-11-03 20:57:40 +00:00
e5f13ce5b4
bpo-43914: Correctly highlight SyntaxError exceptions for invalid generator expression in function calls (GH-28576)
2021-09-27 14:37:43 +01:00
b2f68b1900
bpo-44947: Refine the syntax error for trailing commas in import statements (GH-27814)
2021-08-18 22:09:21 +02:00
f5cbea6b1b
bpo-44838: Refine the custom syntax errors for invalid 'if' expressions (GH-27615)
2021-08-05 18:28:57 +01:00
28b6dc9dd5
bpo-44792: Improve syntax errors for if expressions (GH-27506)
2021-08-02 18:11:37 +01:00
208a7e957b
bpo-34013: Don't consider a grouped expression when reporting legacy print syntax errors (GH-27521)
2021-08-01 02:10:50 +01:00
302cf35505
Divide the grammar into sections to improve readability (GH-27502)
2021-07-31 13:04:37 +01:00
ecc3c8e421
bpo-34013: Move the Python 2 hints from the exception constructor to the parser (GH-27392)
2021-07-27 21:30:32 +01:00
6948964ecf
bpo-34013: Generalize the invalid legacy statement error message (GH-27389)
2021-07-27 17:19:22 +01:00
a283ef116b
Remove unnecessary 'invalid_primary' rule in the parser (GH-27186)
2021-07-16 13:20:09 +01:00
0acc258fe6
bpo-44456: Improve the syntax error when mixing keyword and positional patterns (GH-26793)
2021-06-24 16:09:57 +01:00
05073036dc
bpo-44368: Improve syntax errors with invalid as pattern targets (GH-26632)
2021-06-10 23:50:32 +01:00
e7b4644607
bpo-44385: Remove unused grammar rules (GH-26655)
...
Automerge-Triggered-By: GH:lysnikolaou
2021-06-10 15:05:06 -07:00
b250f89bb7
bpo-44305: Improve syntax error for try blocks without except or finally (GH-26523)
2021-06-03 23:52:12 +01:00
c878a97968
bpo-44180: Fix edge cases in invalid assigment rules in the parser (GH-26283)
...
The invalid assignment rules are very delicate since the parser can
easily raise an invalid assignment when a keyword argument is provided.
As they are very deep into the grammar tree, is very difficult to
specify in which contexts these rules can be used and in which don't.
For that, we need to use a different version of the rule that doesn't do
error checking in those situations where we don't want the rule to raise
(keyword arguments and generator expressions).
We also need to check if we are in left-recursive rule, as those can try
to eagerly advance the parser even if the parse will fail at the end of
the expression. Failing to do this allows the parser to start parsing a
call as a tuple and incorrectly identify a keyword argument as an
invalid assignment, before it realizes that it was not a tuple after all.
2021-05-21 18:34:54 +01:00
33c0c90dea
bpo-44168: Fix error message in the parser for keyword arguments for invalid expressions (GH-26210)
2021-05-19 19:03:04 +01:00
Pablo Galindo Salgado
Jelle Zijlstra
Marta Gómez Macías
Lysandros Nikolaou
Irit Katriel
Nikita Sobolev
Shantanu
wookie184
Matthieu Dartiailh
Matthew Rahtz
Mark Dickinson
Carl Friedrich Bolz-Tereick
wim glenn
Miguel Brito