Commit Graph

357 Commits

Author SHA1 Message Date
Batuhan Taskaya 02a1603f91
bpo-42000: Cleanup the AST related C-code (GH-22641)
- Use the proper asdl sequence when creating empty arguments
- Remove reduntant casts (thanks to new typed asdl_sequences)
- Remove MarshalPrototypeVisitor and some utilities from asdl generator
- Fix the header of `Python/ast.c` (kept from pgen times)

Automerge-Triggered-By: @pablogsal
2020-10-10 10:14:59 -07:00
Pablo Galindo a5634c4067
bpo-41746: Add type information to asdl_seq objects (GH-22223)
* Add new capability to the PEG parser to type variable assignments. For instance:
```
       | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a }
```

* Add new sequence types from the asdl definition (automatically generated)
* Make `asdl_seq` type a generic aliasing pointer type.
* Create a new `asdl_generic_seq` for the generic case using `void*`.
* The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed.
* New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences.
* Changes all possible `asdl_seq` types to use specific versions everywhere.
2020-09-16 19:42:00 +01:00
Lysandros Nikolaou 314858e276
bpo-40939: Remove the old parser (Part 2) (GH-21005)
Remove some remaining files and Makefile targets for the old parser
2020-06-20 19:07:25 +01:00
Pablo Galindo 1ed83adb0e
bpo-40939: Remove the old parser (GH-20768)
This commit removes the old parser, the deprecated parser module, the old parser compatibility flags and environment variables and all associated support code and documentation.
2020-06-11 17:30:46 +01:00
Batuhan Taskaya 68874a8502
bpo-40870: Invalidate usage of some constants with ast.Name (GH-20649) 2020-06-06 05:44:16 -07:00
Shantanu c116c94ff1
bpo-40614: Respect feature version for f-string debug expressions (GH-20196)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2020-05-27 21:30:38 +01:00
Pablo Galindo 16ab07063c
bpo-40334: Correctly identify invalid target in assignment errors (GH-20076)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-05-15 02:04:52 +01:00
Serhiy Storchaka cd8295ff75
bpo-39943: Add the const qualifier to pointers on non-mutable PyUnicode data. (GH-19345) 2020-04-11 10:48:40 +03:00
Pablo Galindo 40cf35c5b0
bpo-40141: Include the value in the column position for keyword AST nodes (GH-19348) 2020-04-03 21:02:26 +01:00
Pablo Galindo 254ec78341
bpo-40147: Move the check for duplicate keywords to the compiler (GH-19289) 2020-04-03 20:37:13 +01:00
Pablo Galindo 168660b547
bpo-40141: Add line and column information to ast.keyword nodes (GH-19283) 2020-04-02 00:47:39 +01:00
Victor Stinner 87d3b9db4a
bpo-39882: Add _Py_FatalErrorFormat() function (GH-19157) 2020-03-25 19:27:36 +01:00
Batuhan Taşkaya 0ac59f93c0
bpo-40000: Improve error messages when validating invalid ast.Constant nodes (GH-19055)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-03-19 11:32:28 +00:00
Serhiy Storchaka 6b97598fb6
bpo-39988: Remove ast.AugLoad and ast.AugStore node classes. (GH-19038) 2020-03-17 23:41:08 +02:00
Batuhan Taşkaya 8689209e03
bpo-39969: Remove ast.Param node class as is no longer used (GH-19020) 2020-03-15 19:32:17 +00:00
Serhiy Storchaka 13d52c2686
bpo-34822: Simplify AST for subscription. (GH-9605)
* Remove the slice type.
* Make Slice a kind of the expr type instead of the slice type.
* Replace ExtSlice(slices) with Tuple(slices, Load()).
* Replace Index(value) with a value itself.

All non-terminal nodes in AST for expressions are now of the expr type.
2020-03-10 18:52:34 +02:00
Batuhan Taşkaya d82e469048
bpo-39639: Remove the AST "Suite" node and associated code (GH-18513)
The AST "Suite" node is no longer used and it can be removed from the ASDL definition and related structures (compiler, visitors, ...).

Co-Authored-By: Victor Stinner <vstinner@python.org>
Co-authored-by: Brett Cannon <54418+brettcannon@users.noreply.github.com>
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-03-04 16:16:46 +00:00
Brandt Bucher be501ca241
bpo-39702: Relax grammar restrictions on decorators (PEP 614) (GH-18570) 2020-03-03 14:25:44 -08:00
Serhiy Storchaka 6e619c48b8
bpo-39474: Fix AST pos for expressions like (a)(b), (a)[b] and (a).b. (GH-18477) 2020-02-12 22:37:49 +02:00
Lysandros Nikolaou d2e1098641
bpo-39579: Fix Attribute end_col_offset to point at the current node (GH-18405) 2020-02-07 15:36:32 -08:00
Guido van Rossum a796d8ef9d bpo-39235: Fix end location for genexp in call args (GH-17925)
The fix changes copy_location() to require an extra node from which to extract the end location, and fixing all 5 call sites.


https://bugs.python.org/issue39235
2020-01-09 11:18:47 -08:00
Ned Batchelder 37143a8e3b bpo-39176: Improve error message for 'named assignment' (GH-17777) 2019-12-31 20:40:58 -06:00
Lysandros Nikolaou 50d4f12958 bpo-39080: Starred Expression's column offset fix when inside a CALL (GH-17645)
Co-Authored-By: Pablo Galindo <Pablogsal@gmail.com>
2019-12-18 00:20:55 +00:00
Guido van Rossum b08d3f71be The comment in ast_for_namedexpr shouldn't include if_stmt (GH-17586)
Automerge-Triggered-By: @gvanrossum
2019-12-15 10:00:33 -08:00
Lysandros Nikolaou 5936a4ce91 Fix elif start column offset when there is an else following (GH-17596) 2019-12-14 10:24:57 +00:00
Lysandros Nikolaou 025a602af7 bpo-39031: Include elif keyword when producing lineno/col-offset info for if_stmt (GH-17582)
When parsing an "elif" node, lineno and col_offset of the node now point to the "elif" keyword and not to its condition, making it consistent with the "if" node.


https://bugs.python.org/issue39031



Automerge-Triggered-By: @pablogsal
2019-12-12 13:40:21 -08:00
Serhiy Storchaka 26ae9f6d3d
bpo-38535: Fix positions for AST nodes for calls without arguments in decorators. (GH-16861) 2019-10-26 16:46:05 +03:00
Greg Price 3a4f66707e Cut disused recode_encoding logic in _PyBytes_DecodeEscape. (GH-16013)
All call sites pass NULL for `recode_encoding`, so this path is
completely untested.  That's been true since before Python 3.0.
It adds significant complexity to this logic, so it's best to
take it out.

All call sites now have a literal NULL, and that's been true since
commit 768921cf3 eliminated a conditional (`foo ? bar : NULL`) at
the call site in Python/ast.c where we're parsing a bytes literal.
But even before then, that condition `foo` had been a constant
since unadorned string literals started meaning Unicode, in commit
572dbf8f1 aka v3.0a1~1035 .

The `unicode` parameter is already unused, so mark it as unused too.
The code that acted on it was also taken out before Python 3.0, in
commit 8d30cc014 aka v3.0a1~1031 .

The function (PyBytes_DecodeEscape) is exposed in the API, but it's
never been documented.
2019-09-12 19:12:22 +01:00
Dino Viehland 8d88e8c662 bpo-38138: Fix memory leak introduced by interned strings (GH-16053)
Interned string needs to be decref'd



https://bugs.python.org/issue38138



Automerge-Triggered-By: @matrixise
2019-09-12 07:38:13 -07:00
Dino Viehland 5b172c27f7 bpo-38113: Update Python/ast.c to PEP-384 (GH-15975)
Removes statics for better subinterpreter support and moves to _PyType_Name


https://bugs.python.org/issue38113



Automerge-Triggered-By: @tiran
2019-09-11 08:47:16 -07:00
Min ho Kim 39d87b5471 Fix typos mostly in comments, docs and test names (GH-15209) 2019-08-30 16:21:19 -04:00
Gregory P. Smith b4be87a04a bpo-32912: Revert SyntaxWarning on invalid escape sequences. (GH-15195)
DeprecationWarning will continue to be emitted for invalid escape
sequences in string and bytes literals just as it did in 3.7.

SyntaxWarning may be emitted in the future. But per mailing list
discussion, we don't yet know when because we haven't settled on how to
do so in a non-disruptive manner.

(Applies 4c5b6bac24 to the master branch).
(This is https://github.com/python/cpython/pull/15142 for master/3.9)


https://bugs.python.org/issue32912



Automerge-Triggered-By: @gpshead
2019-08-10 00:19:07 -07:00
Min ho Kim c4cacc8c5e Fix typos in comments, docs and test names (#15018)
* Fix typos in comments, docs and test names

* Update test_pyparse.py

account for change in string length

* Apply suggestion: splitable -> splittable

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Apply suggestion: splitable -> splittable

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Apply suggestion: Dealloccte -> Deallocate

Co-Authored-By: Terry Jan Reedy <tjreedy@udel.edu>

* Update posixmodule checksum.

* Reverse idlelib changes.
2019-07-30 18:16:13 -04:00
Pablo Galindo cd6e83b481 bpo-37593: Swap the positions of posonlyargs and args in the constructor of ast.parameters nodes (GH-14778)
https://bugs.python.org/issue37593
2019-07-14 16:32:18 -07:00
Carl Friedrich Bolz-Tereick 110a47c4f4 bpo-18374: fix wrong col_offset of some ast.BinOp instances (GH-14607)
Nested BinOp instances (e.g. a+b+c) had a wrong col_offset for the
second BinOp (e.g. 2 instead of 0 in the example). Fix it by using the
correct st node to copy the line and col_offset from in ast.c.
2019-07-08 22:17:56 +01:00
David Carlier 9bbece9d7a remove dead code (GH-14104)
default case ought to handle the "unexpected".
2019-06-18 15:36:34 +09:00
Victor Stinner 37d66d7d4b
bpo-37253: Add _PyCompilerFlags_INIT macro (GH-14018)
Add a new _PyCompilerFlags_INIT macro to initialize PyCompilerFlags
variables, rather than initializing cf_flags and cf_feature_version
explicitly in each variable.
2019-06-13 02:16:41 +02:00
Pablo Galindo 8565f6b6db
bpo-35814: Allow unpacking in r.h.s of annotated assignment expressions (GH-13760) 2019-06-03 08:34:20 +01:00
Pablo Galindo 2f58a84104
bpo-37112: Allow compile to work on AST with positional only arguments with defaults (GH-13697) 2019-05-31 14:09:49 +01:00
Eric V. Smith f83d1dbd3b
bpo-37070: Cleanup fstring debug handling (GH-13607)
* Clean up some comments, fix potential memory leaks, clarify literal and expr_text.
2019-05-29 03:55:44 -04:00
Guido van Rossum 77f0ed7a42
bpo-37072: Fix crash in PyAST_FromNodeObject() when flags is NULL (#13634)
I'm confident that this fixes the reported crash. flags=NULL is treated as using the latest minor version.

https://bugs.python.org/issue37072
2019-05-28 16:44:58 -07:00
Eric V. Smith 6f6ff8a565
bpo-37050: Remove expr_text from FormattedValue ast node, use Constant node instead (GH-13597)
When using the "=" debug functionality of f-strings, use another Constant node (or a merged constant node) instead of adding expr_text to the FormattedValue node.
2019-05-27 15:31:52 -04:00
Michael J. Sullivan 933e1509ec bpo-36878: Track extra text added to 'type: ignore' in the AST (GH-13479)
GH-13238 made extra text after a # type: ignore accepted by the parser.
This finishes the job and actually plumbs the extra text through the
parser and makes it available in the AST.
2019-05-22 15:54:20 +01:00
David Carlier 27ee0f8551 Fix couple of dead code paths (GH-7418) 2019-05-17 19:46:22 -04:00
Pablo Galindo 26f55c29f2
bpo-36817: Do not decrement reference for expr_text on fstring = parsing failure (GH-13256) 2019-05-12 01:43:04 +01:00
Pablo Galindo 5833e94d86
bpo-36817: Fix reference leak for expr_text in f-string = parsing (GH-13249) 2019-05-11 20:54:37 +01:00
Eric V. Smith 9a4135e939
bpo-36817: Add f-string debugging using '='. (GH-13123)
If a "=" is specified a the end of an f-string expression, the f-string will evaluate to the text of the expression, followed by '=', followed by the repr of the value of the expression.
2019-05-08 16:28:48 -04:00
Pablo Galindo 8c77b8cb91
bpo-36540: PEP 570 -- Implementation (GH-12701)
This commit contains the implementation of PEP570: Python positional-only parameters.

* Update Grammar/Grammar with new typedarglist and varargslist

* Regenerate grammar files

* Update and regenerate AST related files

* Update code object

* Update marshal.c

* Update compiler and symtable

* Regenerate importlib files

* Update callable objects

* Implement positional-only args logic in ceval.c

* Regenerate frozen data

* Update standard library to account for positional-only args

* Add test file for positional-only args

* Update other test files to account for positional-only args

* Add News entry

* Update inspect module and related tests
2019-04-29 13:36:57 +01:00
Brad Larsen a4d7836239 bpo-36495: Fix two out-of-bounds array reads (GH-12641)
Research and fix by @bradlarsen.
2019-04-01 07:36:05 -07:00
Pablo Galindo 0c9258a6d2
bpo-36332: Allow compile() to handle AST objects with assignment expressions (GH-12398) 2019-03-18 13:51:53 +00:00