Commit Graph

217 Commits

Author SHA1 Message Date
Pablo Galindo Salgado e5f13ce5b4
bpo-43914: Correctly highlight SyntaxError exceptions for invalid generator expression in function calls (GH-28576) 2021-09-27 14:37:43 +01:00
Pablo Galindo Salgado b2f68b1900
bpo-44947: Refine the syntax error for trailing commas in import statements (GH-27814) 2021-08-18 22:09:21 +02:00
Pablo Galindo Salgado f5cbea6b1b
bpo-44838: Refine the custom syntax errors for invalid 'if' expressions (GH-27615) 2021-08-05 18:28:57 +01:00
Miguel Brito 28b6dc9dd5
bpo-44792: Improve syntax errors for if expressions (GH-27506) 2021-08-02 18:11:37 +01:00
Pablo Galindo Salgado 208a7e957b
bpo-34013: Don't consider a grouped expression when reporting legacy print syntax errors (GH-27521) 2021-08-01 02:10:50 +01:00
Pablo Galindo Salgado 302cf35505
Divide the grammar into sections to improve readability (GH-27502) 2021-07-31 13:04:37 +01:00
Pablo Galindo Salgado ecc3c8e421
bpo-34013: Move the Python 2 hints from the exception constructor to the parser (GH-27392) 2021-07-27 21:30:32 +01:00
Pablo Galindo Salgado 6948964ecf
bpo-34013: Generalize the invalid legacy statement error message (GH-27389) 2021-07-27 17:19:22 +01:00
Pablo Galindo Salgado a283ef116b
Remove unnecessary 'invalid_primary' rule in the parser (GH-27186) 2021-07-16 13:20:09 +01:00
Pablo Galindo 0acc258fe6
bpo-44456: Improve the syntax error when mixing keyword and positional patterns (GH-26793) 2021-06-24 16:09:57 +01:00
Pablo Galindo 05073036dc
bpo-44368: Improve syntax errors with invalid as pattern targets (GH-26632) 2021-06-10 23:50:32 +01:00
Lysandros Nikolaou e7b4644607
bpo-44385: Remove unused grammar rules (GH-26655)
Automerge-Triggered-By: GH:lysnikolaou
2021-06-10 15:05:06 -07:00
Pablo Galindo b250f89bb7
bpo-44305: Improve syntax error for try blocks without except or finally (GH-26523) 2021-06-03 23:52:12 +01:00
Pablo Galindo c878a97968
bpo-44180: Fix edge cases in invalid assigment rules in the parser (GH-26283)
The invalid assignment rules are very delicate since the parser can
easily raise an invalid assignment when a keyword argument is provided.
As they are very deep into the grammar tree, is very difficult to
specify in which contexts these rules can be used and in which don't.
For that, we need to use a different version of the rule that doesn't do
error checking in those situations where we don't want the rule to raise
(keyword arguments and generator expressions).

We also need to check if we are in left-recursive rule, as those can try
to eagerly advance the parser even if the parse will fail at the end of
the expression. Failing to do this allows the parser to start parsing a
call as a tuple and incorrectly identify a keyword argument as an
invalid assignment, before it realizes that it was not a tuple after all.
2021-05-21 18:34:54 +01:00
Pablo Galindo 33c0c90dea
bpo-44168: Fix error message in the parser for keyword arguments for invalid expressions (GH-26210) 2021-05-19 19:03:04 +01:00
Pablo Galindo 6692dc1ca9
bpo-43149: Correct the syntax error message for multiple exception types (GH-25996)
Automerge-Triggered-By: GH:pablogsal
2021-05-08 11:24:41 -07:00
Brandt Bucher dbe60ee09d
bpo-43892: Validate the first term of complex literal value patterns (GH-25735) 2021-04-29 17:19:28 -07:00
Nick Coghlan 1e7b858575
bpo-43892: Make match patterns explicit in the AST (GH-25585)
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2021-04-28 22:58:44 -07:00
Pablo Galindo a77aac4fca
bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525)
To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way:

>>> foo(x, z for z in range(10), t, w)
  File "<stdin>", line 1
    foo(x, z for z in range(10), t, w)
           ^
SyntaxError: Generator expression must be parenthesized

becomes

>>> foo(x, z for z in range(10), t, w)
  File "<stdin>", line 1
    foo(x, z for z in range(10), t, w)
           ^^^^^^^^^^^^^^^^^^^^
SyntaxError: Generator expression must be parenthesized
2021-04-23 14:27:05 +01:00
Pablo Galindo 56c95dfe27
bpo-43859: Improve the error message for IndentationError exceptions (GH-25431) 2021-04-21 15:28:21 +01:00
Pablo Galindo b5b98bd8f8
bpo-43823: Fix location of one of the errors for invalid dictionary literals (GH-25427) 2021-04-16 00:45:42 +01:00
Pablo Galindo b280248be8
bpo-43822: Improve syntax errors for missing commas (GH-25377) 2021-04-15 21:38:45 +01:00
Pablo Galindo da74350174
bpo-43823: Improve syntax errors for invalid dictionary literals (GH-25378) 2021-04-15 14:06:39 +01:00
Pablo Galindo 30ed93bfec
bpo-43797: Handle correctly invalid assignments inside function calls and generators (GH-25390) 2021-04-13 17:51:21 +01:00
Pablo Galindo d9151cb453
Ensure that early = are not matched by the parser as invalid comparisons (GH-25375) 2021-04-13 02:32:33 +01:00
Pablo Galindo b86ed8e3bb
bpo-43797: Improve syntax error for invalid comparisons (#25317)
* bpo-43797: Improve syntax error for invalid comparisons

* Update Lib/test/test_fstring.py

Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>

* Apply review comments

* can't -> cannot

Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
2021-04-12 16:59:30 +01:00
Matthew Suozzo 75a06f067b
bpo-43798: Add source location attributes to alias (GH-25324)
* Add source location attributes to alias.
* Move alias star construction to pegen helper.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-04-10 22:56:28 +02:00
Victor Stinner d27f8d2e07
bpo-43244: Rename pycore_ast.h functions to _PyAST_xxx() (GH-25252)
Rename AST functions of pycore_ast.h to use the "_PyAST_" prefix.
Remove macros creating aliases without prefix. For example, Module()
becomes _PyAST_Module(). Update Grammar/python.gram to use
_PyAST_xxx() functions.
2021-04-07 21:34:22 +02:00
Pablo Galindo 8efad61963
bpo-41064: Improve syntax error for invalid usage of '**' in f-strings (GH-25006) 2021-03-24 19:34:17 +00:00
Pablo Galindo 08fb8ac99a
bpo-42128: Add 'missing :' syntax error message to match statements (GH-24733) 2021-03-18 01:03:11 +00:00
Brandt Bucher 145bf269df
bpo-42128: Structural Pattern Matching (PEP 634) (GH-22917)
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Talin <viridia@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-02-26 14:51:55 -08:00
Pablo Galindo 206cbdab16
bpo-43149: Improve error message for exception group without parentheses (GH-24467) 2021-02-07 18:42:21 +00:00
Pablo Galindo d4e6ed7e5f
bpo-43121: Fix incorrect SyntaxError message for missing comma (GH-24436) 2021-02-03 23:29:26 +00:00
Pablo Galindo 58fb156edd
bpo-42997: Improve error message for missing : before suites (GH-24292)
* Add to the peg generator a new directive ('&&') that allows to expect
  a token and hard fail the parsing if the token is not found. This
  allows to quickly emmit syntax errors for missing tokens.

* Use the new grammar element to hard-fail if the ':' is missing before
  suites.
2021-02-02 19:54:22 +00:00
Pablo Galindo 835f14ff8e
bpo-43017: Improve error message for unparenthesised tuples in comprehensions (GH24314) 2021-01-31 22:52:56 +00:00
Lysandros Nikolaou 07dcd86cee
bpo-42860: Remove type error from grammar (GH-24156)
This is only there so that alternative implementations written in statically-typed languages can use this grammar without
having type errors in the way.

Automerge-Triggered-By: GH:lysnikolaou
2021-01-07 14:31:25 -08:00
Lysandros Nikolaou 2ea320dddd
bpo-40631: Disallow single parenthesized star target (GH-24027) 2021-01-03 01:14:21 +02:00
Pablo Galindo 43c4fb6c90
bpo-30858: Improve error location for expressions with assignments (GH-23753)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-12-13 16:46:48 +00:00
Pablo Galindo 9bdc40ee3e
Refactor the grammar to match the language specification docs (GH-23574) 2020-11-30 19:42:38 +00:00
Pablo Galindo b0aba1fcdc
bpo-42381: Allow walrus in set literals and set comprehensions (GH-23332)
Currently walruses are not allowerd in set literals and set comprehensions:

>>> {y := 4, 4**2, 3**3}
  File "<stdin>", line 1
    {y := 4, 4**2, 3**3}
       ^
SyntaxError: invalid syntax

but they should be allowed as well per PEP 572
2020-11-17 01:17:12 +00:00
Lysandros Nikolaou cae60187cf
bpo-42316: Allow unparenthesized walrus operator in indexes (GH-23317) 2020-11-17 01:09:35 +02:00
Lysandros Nikolaou cb3e5ed071
bpo-42374: Allow unparenthesized walrus in genexps (GH-23319)
This fixes a regression that was introduced by the new parser.

Automerge-Triggered-By: GH:lysnikolaou
2020-11-16 15:08:35 -08:00
Pablo Galindo 06f8c3328d
bpo-42214: Fix check for NOTEQUAL token in the PEG parser for the barry_as_flufl rule (GH-23048) 2020-10-30 23:48:42 +00:00
Lysandros Nikolaou 15acc4eaba
bpo-41659: Disallow curly brace directly after primary (GH-22996) 2020-10-27 20:54:20 +02:00
Lysandros Nikolaou bca7014032
bpo-42123: Run the parser two times and only enable invalid rules on the second run (GH-22111)
* Implement running the parser a second time for the errors messages

The first parser run is only responsible for detecting whether
there is a `SyntaxError` or not. If there isn't the AST gets returned.
Otherwise, the parser is run a second time with all the `invalid_*`
rules enabled so that all the customized error messages get produced.
2020-10-27 00:42:04 +02:00
Lysandros Nikolaou 2e5ca9e3f6
bpo-41746: Cast to typed seqs in CHECK macros to avoid type erasure (GH-22864) 2020-10-21 22:53:14 +03:00
Batuhan Taskaya 48f305fd12
bpo-41979: Accept star-unpacking on with-item targets (GH-22611)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-10-09 10:56:48 +01:00
Pablo Galindo a5634c4067
bpo-41746: Add type information to asdl_seq objects (GH-22223)
* Add new capability to the PEG parser to type variable assignments. For instance:
```
       | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a }
```

* Add new sequence types from the asdl definition (automatically generated)
* Make `asdl_seq` type a generic aliasing pointer type.
* Create a new `asdl_generic_seq` for the generic case using `void*`.
* The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed.
* New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences.
* Changes all possible `asdl_seq` types to use specific versions everywhere.
2020-09-16 19:42:00 +01:00
Pablo Galindo 315a61f7a9
bpo-41697: Correctly handle KeywordOrStarred when parsing arguments in the parser (GH-22077) 2020-09-03 15:29:32 +01:00
Pablo Galindo 4a97b1517a
bpo-41690: Use a loop to collect args in the parser instead of recursion (GH-22053)
This program can segfault the parser by stack overflow:

```
import ast

code = "f(" + ",".join(['a' for _ in range(100000)]) + ")"
print("Ready!")
ast.parse(code)
```

the reason is that the rule for arguments has a simple recursion when collecting args:

args[expr_ty]:
    [...]
    | a=named_expression b=[',' c=args { c }] {
        [...] }
2020-09-02 17:44:19 +01:00