Commit Graph

255 Commits

Author SHA1 Message Date
Pablo Galindo Salgado 3d180347ae
gh-110696: Fix incorrect syntax error message for incorrect argument unpacking (#110706) 2023-10-12 09:02:02 +00:00
Pablo Galindo Salgado b28ffaa193
gh-109596: Ensure repeated rules in the grammar are not allowed and fix incorrect soft keywords (#109606) 2023-09-22 19:03:23 +01:00
Pablo Galindo Salgado 5bda2f637e
gh-109114: Relax the check for invalid lambdas inside f-strings to avoid false positives (#109121) 2023-09-08 17:00:23 +00:00
Pablo Galindo Salgado da8f87b7ea
gh-107015: Remove async_hacks from the tokenizer (#107018) 2023-07-26 16:34:15 +01:00
Pablo Galindo Salgado 13237a2da8
gh-98931: Add custom error messages to invalid import/from with multiple targets (#105985)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-06-22 15:56:40 +00:00
Pablo Galindo Salgado 41de54378d
gh-105194: Fix format specifier escaped characters in f-strings (#105231) 2023-06-02 13:33:26 +02:00
Jelle Zijlstra ba73473f4c
gh-104799: Move location of type_params AST fields (#104828)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-26 05:54:37 -07:00
Jelle Zijlstra a5f244d627
gh-104656: Rename typeparams AST node to type_params (#104657) 2023-05-21 21:25:09 -07:00
Marta Gómez Macías 6715f91edc
gh-102856: Python tokenizer implementation for PEP 701 (#104323)
This commit replaces the Python implementation of the tokenize module with an implementation
that reuses the real C tokenizer via a private extension module. The tokenize module now implements
a compatibility layer that transforms tokens from the C tokenizer into Python tokenize tokens for backward
compatibility.

As the C tokenizer does not emit some tokens that the Python tokenizer provides (such as comments and non-semantic newlines), a new special mode has been added to the C tokenizer mode that currently is only used via
the extension module that exposes it to the Python layer. This new mode forces the C tokenizer to emit these new extra tokens and add the appropriate metadata that is needed to match the old Python implementation.

Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2023-05-21 01:03:02 +01:00
Jelle Zijlstra 24d8b88420
gh-103763: Implement PEP 695 (#103764)
This implements PEP 695, Type Parameter Syntax. It adds support for:

- Generic functions (def func[T](): ...)
- Generic classes (class X[T](): ...)
- Type aliases (type X = ...)
- New scoping when the new syntax is used within a class body
- Compiler and interpreter changes to support the new syntax and scoping rules 

Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
Co-authored-by: Eric Traut <eric@traut.com>
Co-authored-by: Larry Hastings <larry@hastings.org>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-15 20:36:23 -07:00
Lysandros Nikolaou 9169a56fad
gh-103656: Transfer f-string buffers to parser to avoid use-after-free (GH-103896)
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2023-04-27 01:33:31 +00:00
Pablo Galindo Salgado 1ef61cf71a
gh-102856: Initial implementation of PEP 701 (#102855)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
Co-authored-by: Marta Gómez Macías <mgmacias@google.com>
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
2023-04-19 11:18:16 -05:00
Lysandros Nikolaou 6d8da238cc
gh-90994: Improve error messages upon call arguments syntax errors (GH-96893) 2022-11-21 00:15:05 +01:00
Lysandros Nikolaou 9c4232ae89
gh-99211: Point to except/except* on syntax errors when mixing them (GH-99215)
Automerge-Triggered-By: GH:lysnikolaou
2022-11-20 09:11:02 -08:00
Irit Katriel 61b6c40b64
gh-99153: set location on SyntaxError for try with both except and except* (GH-99160) 2022-11-06 15:36:19 +00:00
Pablo Galindo Salgado 395d4285bf
gh-98931: Improve error message when the user types 'import x from y' instead of 'from y import x' (#98932) 2022-11-01 13:01:20 +00:00
Lysandros Nikolaou 7e36abbb78
gh-91210: Improve error message when non-default param follows default (GH-95933)
- Improve error message when parameter without a default follows one with a default
- Show same error message when positional-only params precede the default/non-default sequence
2022-09-17 10:09:28 -07:00
Nikita Sobolev 2c7d2e8d46
gh-96587: Raise `SyntaxError` for PEP654 on older `feature_version` (#96588) 2022-09-05 17:54:09 +01:00
Shantanu a965db37f2
gh-94996: Disallow lambda pos only params with feature_version < (3, 8) (GH-95934) 2022-08-12 20:41:02 +02:00
Shantanu b5e3ea2862
gh-94996: Disallow parsing pos only params with feature_version < (3, 8) (GH-94997) 2022-08-12 19:27:50 +02:00
Shantanu 0daba82221
gh-94949: Disallow parsing parenthesised ctx mgr with old feature_version (#94950)
* gh-94949: Disallow parsing parenthesised ctx manager with old feature_version

* 📜🤖 Added by blurb_it.

* Allow it with feature_version=(3, 9) as well

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2022-07-18 22:10:49 +01:00
Shantanu ae0be5a53b
gh-94947: Disallow parsing walrus with feature_version < (3, 8) (#94948)
* gh-94947: Disallow parsing walrus with feature_version < (3, 8)

* oops, commit the parser

* 📜🤖 Added by blurb_it.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2022-07-18 10:20:12 +01:00
wookie184 8c237a7a71
gh-94192: Fix error for dictionary literals with invalid expression as value. (#94304)
* Fix error for dictionary literals with invalid expression as value.

* Remove trailing whitespace
2022-06-26 19:42:47 +01:00
wookie184 2fc83ac3af
gh-92858: Improve error message for some suites with syntax error before ':' (#92894) 2022-06-23 17:31:09 +01:00
Pablo Galindo Salgado 53a8b17895
gh-93671: Avoid exponential backtracking in deeply nested sequence patterns in match statements (GH-93680)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2022-06-10 17:56:45 +02:00
Matthieu Dartiailh aa0f056a00
bpo-47212: Improve error messages for un-parenthesized generator expressions (GH-32302) 2022-04-05 14:47:13 +01:00
Matthew Rahtz e8e737bcf6
bpo-43224: Implement PEP 646 grammar changes (GH-31018)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-03-26 09:55:35 -07:00
Pablo Galindo Salgado 7d810b6a4e
bpo-46838: Syntax error improvements for function definitions (GH-31590) 2022-03-22 11:38:41 +00:00
Pablo Galindo Salgado 390459de6d
Allow the parser to avoid nested processing of invalid rules (GH-31252) 2022-02-10 13:12:14 +00:00
Pablo Galindo Salgado b71dc71905
bpo-46707: Avoid potential exponential backtracking in some syntax errors (GH-31241) 2022-02-10 03:37:17 +00:00
Mark Dickinson 1bee9a4625
bpo-46272: Fix two heading comments in python.gram (GH-30499)
One typo fix and one heading change, both in comments. No functional changes.
2022-01-09 08:22:54 -08:00
Irit Katriel d60457a667
bpo-45292: [PEP-654] add except* (GH-29581) 2021-12-14 16:48:15 +00:00
Pablo Galindo Salgado 24c10d2943
bpo-45727: Only trigger the 'did you forgot a comma' error suggestion if inside parentheses (GH-29757) 2021-11-24 22:21:23 +00:00
Pablo Galindo Salgado 7a1d932528
bpo-45450: Improve syntax error for parenthesized arguments (GH-28906) 2021-11-20 18:27:40 +00:00
Pablo Galindo Salgado 546cefcda7
bpo-45727: Make the syntax error for missing comma more consistent (GH-29427) 2021-11-19 23:11:57 +00:00
Carl Friedrich Bolz-Tereick 2819e98d10
bpo-45764: improve error message when missing '(' after 'def' (GH-29484)
to achieve this, change the grammar to expect the '(' token after 'def' NAME.

Automerge-Triggered-By: GH:pablogsal
2021-11-09 06:03:32 -08:00
Pablo Galindo Salgado e2d65630f3
bpo-45716: Improve the error message when using True/False/None as keywords in a call (GH-29413) 2021-11-05 13:54:55 +00:00
wim glenn 762173c670
bpo-44257: fix "assigment_expr" typo + regenerate the grammar, and remove unused imports (GH-29393)
Co-authored-by: Wim Glenn <wglenn@jumptrading.com>
2021-11-03 20:57:40 +00:00
Pablo Galindo Salgado e5f13ce5b4
bpo-43914: Correctly highlight SyntaxError exceptions for invalid generator expression in function calls (GH-28576) 2021-09-27 14:37:43 +01:00
Pablo Galindo Salgado b2f68b1900
bpo-44947: Refine the syntax error for trailing commas in import statements (GH-27814) 2021-08-18 22:09:21 +02:00
Pablo Galindo Salgado f5cbea6b1b
bpo-44838: Refine the custom syntax errors for invalid 'if' expressions (GH-27615) 2021-08-05 18:28:57 +01:00
Miguel Brito 28b6dc9dd5
bpo-44792: Improve syntax errors for if expressions (GH-27506) 2021-08-02 18:11:37 +01:00
Pablo Galindo Salgado 208a7e957b
bpo-34013: Don't consider a grouped expression when reporting legacy print syntax errors (GH-27521) 2021-08-01 02:10:50 +01:00
Pablo Galindo Salgado 302cf35505
Divide the grammar into sections to improve readability (GH-27502) 2021-07-31 13:04:37 +01:00
Pablo Galindo Salgado ecc3c8e421
bpo-34013: Move the Python 2 hints from the exception constructor to the parser (GH-27392) 2021-07-27 21:30:32 +01:00
Pablo Galindo Salgado 6948964ecf
bpo-34013: Generalize the invalid legacy statement error message (GH-27389) 2021-07-27 17:19:22 +01:00
Pablo Galindo Salgado a283ef116b
Remove unnecessary 'invalid_primary' rule in the parser (GH-27186) 2021-07-16 13:20:09 +01:00
Pablo Galindo 0acc258fe6
bpo-44456: Improve the syntax error when mixing keyword and positional patterns (GH-26793) 2021-06-24 16:09:57 +01:00
Pablo Galindo 05073036dc
bpo-44368: Improve syntax errors with invalid as pattern targets (GH-26632) 2021-06-10 23:50:32 +01:00
Lysandros Nikolaou e7b4644607
bpo-44385: Remove unused grammar rules (GH-26655)
Automerge-Triggered-By: GH:lysnikolaou
2021-06-10 15:05:06 -07:00