Commit Graph

92 Commits

Author SHA1 Message Date
Irit Katriel 10a91d7e98
gh-108113: Make it possible to create an optimized AST (#108154) 2023-08-21 16:31:30 +00:00
Jelle Zijlstra 957a974d4f
gh-104799: PEP 695 backward compatibility for ast.unparse (#105846) 2023-06-16 09:31:23 -07:00
Jelle Zijlstra a5f244d627
gh-104656: Rename typeparams AST node to type_params (#104657) 2023-05-21 21:25:09 -07:00
Jelle Zijlstra 24d8b88420
gh-103763: Implement PEP 695 (#103764)
This implements PEP 695, Type Parameter Syntax. It adds support for:

- Generic functions (def func[T](): ...)
- Generic classes (class X[T](): ...)
- Type aliases (type X = ...)
- New scoping when the new syntax is used within a class body
- Compiler and interpreter changes to support the new syntax and scoping rules 

Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
Co-authored-by: Eric Traut <eric@traut.com>
Co-authored-by: Larry Hastings <larry@hastings.org>
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2023-05-15 20:36:23 -07:00
Alex Waygood 376137f6ec
gh-90953: Emit deprecation warnings for `ast` features deprecated in Python 3.8 (#104199)
`ast.Num`, `ast.Str`, `ast.Bytes`, `ast.Ellipsis` and `ast.NameConstant` now all emit deprecation warnings on import, access, instantation or `isinstance()` checks.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-05-06 16:48:07 +00:00
Tian Gao 36860134a9
gh-103285: Rewrite _splitlines_no_ff to improve performance (#103307) 2023-04-23 23:03:49 -06:00
Batuhan Taskaya 1acdfec359
gh-99341: Cover type ignore nodes when incrementing line numbers (GH-99422) 2022-11-22 02:41:14 -08:00
Gregory P. Smith 8baef8ae36
gh-95588: Drop the safety claim from `ast.literal_eval` docs. (#95919)
It was never really safe and this claim conflicts directly with the big warning in the docs about it being able to crash the interpreter.
2022-10-01 17:55:40 -07:00
Shantanu 200c9a8da0
gh-92986: Fix ast.unparse when ImportFrom.level is None (#92992)
This doesn't happen naturally, but is allowed by the ASDL and compiler.
We don't want to change ASDL for backward compatibility reasons
(#57645, #92987)
2022-09-05 20:14:50 +03:00
Anthony Sottile 9c2b9348e2
ast.parse: check `feature_version` common case first (GH-94640) 2022-08-29 17:05:24 +03:00
Shantanu a5dde0fe4f
gh-95066: ast: Replace assert with ValueError (GH-95072)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2022-07-26 11:43:09 +02:00
Batuhan Taskaya f6fd8aac13
gh-92671: Don't omit parentheses when unparsing empty tuples (GH-92673) 2022-05-16 13:38:06 +01:00
Matthew Rahtz e8e737bcf6
bpo-43224: Implement PEP 646 grammar changes (GH-31018)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2022-03-26 09:55:35 -07:00
Irit Katriel d60457a667
bpo-45292: [PEP-654] add except* (GH-29581) 2021-12-14 16:48:15 +00:00
Binbin 17b16e13bb
Fix typos in multiple files (GH-26689)
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
2021-06-12 22:47:44 -04:00
Batuhan Taskaya 51cef8be8c
bpo-44142: drop redundant parantheses when unparsing tuples as assignment targets (GH-26156) 2021-05-16 16:33:22 +03:00
Batuhan Taskaya e4e931a67e
bpo-44081: improve ast.unparse() for lambdas with no parameters (GH-26000) 2021-05-15 15:55:53 +03:00
Batuhan Taskaya 3d98ececda
bpo-43417: Better buffer handling for ast.unparse (GH-24772) 2021-05-09 02:32:04 +03:00
Ikko Ashimine 9ee8448243
Fix typo in ast.py (GH-25740)
parantheses -> parentheses
2021-05-04 06:18:58 -07:00
Nick Coghlan 1e7b858575
bpo-43892: Make match patterns explicit in the AST (GH-25585)
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2021-04-28 22:58:44 -07:00
Ethan Furman a02cb474f9
bpo-38659: [Enum] add _simple_enum decorator (GH-25497)
add:

* `_simple_enum` decorator to transform a normal class into an enum
* `_test_simple_enum` function to compare
* `_old_convert_` to enable checking `_convert_` generated enums

`_simple_enum` takes a normal class and converts it into an enum:

    @simple_enum(Enum)
    class Color:
        RED = 1
        GREEN = 2
        BLUE = 3

`_old_convert_` works much like` _convert_` does, using the original logic:

    # in a test file
    import socket, enum
    CheckedAddressFamily = enum._old_convert_(
            enum.IntEnum, 'AddressFamily', 'socket',
            lambda C: C.isupper() and C.startswith('AF_'),
            source=_socket,
            )

`_test_simple_enum` takes a traditional enum and a simple enum and
compares the two:

    # in the REPL or the same module as Color
    class CheckedColor(Enum):
        RED = 1
        GREEN = 2
        BLUE = 3

    _test_simple_enum(CheckedColor, Color)

    _test_simple_enum(CheckedAddressFamily, socket.AddressFamily)

Any important differences will raise a TypeError
2021-04-21 10:20:44 -07:00
Ethan Furman 503cdc7c12
Revert "bpo-38659: [Enum] add _simple_enum decorator (GH-25285)" (GH-25476)
This reverts commit dbac8f40e8.
2021-04-19 19:12:24 -07:00
Ethan Furman dbac8f40e8
bpo-38659: [Enum] add _simple_enum decorator (GH-25285)
add:

_simple_enum decorator to transform a normal class into an enum
_test_simple_enum function to compare
_old_convert_ to enable checking _convert_ generated enums
_simple_enum takes a normal class and converts it into an enum:

@simple_enum(Enum)
class Color:
    RED = 1
    GREEN = 2
    BLUE = 3

_old_convert_ works much like _convert_ does, using the original logic:

# in a test file
import socket, enum
CheckedAddressFamily = enum._old_convert_(
        enum.IntEnum, 'AddressFamily', 'socket',
        lambda C: C.isupper() and C.startswith('AF_'),
        source=_socket,
        )

test_simple_enum takes a traditional enum and a simple enum and
compares the two:

# in the REPL or the same module as Color
class CheckedColor(Enum):
    RED = 1
    GREEN = 2
    BLUE = 3

_test_simple_enum(CheckedColor, Color)

_test_simple_enum(CheckedAddressFamily, socket.AddressFamily)

Any important differences will raise a TypeError
2021-04-19 18:04:53 -07:00
Kodi Arfer 08ff4369af
bpo-43521: Allow ast.unparse with empty sets and NaN (GH-24897)
Automerge-Triggered-By: GH:pablogsal
2021-03-18 10:36:06 -07:00
Brandt Bucher 145bf269df
bpo-42128: Structural Pattern Matching (PEP 634) (GH-22917)
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Talin <viridia@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-02-26 14:51:55 -08:00
Irit Katriel 586f3dbe15
bpo-28964: add line number of node (if available) to ast.literal_eval error messages (GH-23677) 2020-12-25 20:04:31 +03:00
Shantanu a993e901eb
bpo-28002: Roundtrip f-strings with ast.unparse better (#19612)
By attempting to avoid backslashes in f-string expressions.
We also now proactively raise errors for some backslashes we can't
avoid while unparsing FormattedValues

Co-authored-by: hauntsaninja <>
Co-authored-by: Shantanu <hauntsaninja@users.noreply.github.com>
Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
2020-11-21 00:16:42 +03:00
Batuhan Taskaya e799aa8b92
bpo-41887: omit leading spaces/tabs on ast.literal_eval (#22469)
Also document that eval() does this (the same way).
2020-10-03 17:46:44 -07:00
Victor Stinner e5fbe0cbd4
bpo-41631: _ast module uses again a global state (#21961)
Partially revert commit ac46eb4ad6662cf6d771b20d8963658b2186c48c:
"bpo-38113: Update the Python-ast.c generator to PEP384 (gh-15957)".

Using a module state per module instance is causing subtle practical
problems.

For example, the Mercurial project replaces the __import__() function
to implement lazy import, whereas Python expected that "import _ast"
always return a fully initialized _ast module.

Add _PyAST_Fini() to clear the state at exit.

The _ast module has no state (set _astmodule.m_size to 0). Remove
astmodule_traverse(), astmodule_clear() and astmodule_free()
functions.
2020-09-15 18:03:34 +02:00
Batuhan Taskaya 8f4380d2f5
bpo-40726: handle uninitalized end_lineno on ast.increment_lineno (GH-20312) 2020-08-05 14:32:32 +01:00
Rémi Lapeyre c73914a562
bpo-36290: Fix keytword collision handling in AST node constructors (GH-12382) 2020-05-24 22:12:57 +01:00
Batuhan Taskaya c102a14825
bpo-38870: Don't omit parenthesis when unparsing a slice in ast.unparse
When unparsing a non-empty tuple, the parentheses can be safely
omitted if there aren't any elements that explicitly require them (such as starred expressions).
2020-05-18 21:48:49 +01:00
CyberSaxosTiGER d71a6492db
bpo-38870: correctly escape unprintable characters on ast.unparse (GH-20166)
Unprintable characters such as `\x00` weren't correctly roundtripped
due to not using default string repr when generating docstrings. This
patch correctly encodes all unprintable characters (except `\n` and `\t`, which
are commonly used for formatting, and found unescaped).

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
2020-05-18 19:41:35 +01:00
Irit Katriel e6578a226d
bpo-40662: Fixed ast.get_source_segment for ast nodes that have incomplete location information (GH-20157)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-18 19:14:12 +01:00
Batuhan Taskaya dff92bb31f
bpo-38870: Implement round tripping support for typed AST in ast.unparse (GH-17797) 2020-05-17 00:04:12 +01:00
Batuhan Taskaya e966af7cff
bpo-38870: Correctly handle empty docstrings in ast.unparse (GH-18768)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-16 23:49:07 +01:00
Batuhan Taskaya 25160cdc47
bpo-38870: Don't put unnecessary parentheses on class declarations in ast.parse (GH-20134) 2020-05-16 22:53:25 +01:00
Batuhan Taskaya ce4a753dcb
bpo-38870: Do not separate factor prefixes in ast.unparse (GH-20133) 2020-05-16 22:46:11 +01:00
Curtis Bucher c21c51235a
bpo-40355: Improve error messages in ast.literal_eval with malformed Dict nodes (GH-19868)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-05 20:40:56 +01:00
Batuhan Taskaya 493bf1cc31
bpo-38870: Don't start generated output with newlines in ast.unparse (GH-19636) 2020-05-03 18:11:51 +01:00
Shantanu 01508dcde4
Fix typo in exception thrown by ast.unparse (GH-19534) 2020-04-16 11:10:12 +01:00
Serhiy Storchaka bace59d8b8
bpo-39999: Improve compatibility of the ast module. (GH-19056)
* Re-add removed classes Suite, slice, Param, AugLoad and AugStore.
* Add docstrings for dummy classes.
* Add docstrings for attribute aliases.
* Set __module__ to "ast" instead of "_ast".
2020-03-22 20:33:34 +02:00
Batuhan Taşkaya 5b66ec166b
bpo-38870: Implement support for ast.FunctionType in ast.unparse (GH-19016) 2020-03-15 19:56:57 +00:00
Pablo Galindo c00c86b904
Fix syntax error in an example in the ast documentation and sync docstrings (GH-18946) 2020-03-12 00:48:19 +00:00
Serhiy Storchaka 13d52c2686
bpo-34822: Simplify AST for subscription. (GH-9605)
* Remove the slice type.
* Make Slice a kind of the expr type instead of the slice type.
* Replace ExtSlice(slices) with Tuple(slices, Load()).
* Replace Index(value) with a value itself.

All non-terminal nodes in AST for expressions are now of the expr type.
2020-03-10 18:52:34 +02:00
Serhiy Storchaka b7e9525f9c
bpo-36287: Make ast.dump() not output optional fields and attributes with default values. (GH-18843)
The default values for optional fields and attributes of AST nodes are now set
as class attributes (e.g. Constant.kind is set to None).
2020-03-10 00:07:47 +02:00
Batuhan Taşkaya e7cab7f780
bpo-38870: Simplify sequence interleaves in ast.unparse (GH-17892) 2020-03-09 20:27:03 +00:00
Serhiy Storchaka c4928fc1a8
bpo-39889: Fix ast.unparse() for subscript. (GH-18824) 2020-03-07 17:25:32 +02:00
Batuhan Taşkaya 89aa4694fc
bpo-38870: Add docstring support to ast.unparse (GH-17760)
Allow ast.unparse to detect docstrings in functions, modules and classes and produce
nicely formatted unparsed output for said docstrings.

Co-Authored-By: Pablo Galindo <Pablogsal@gmail.com>
2020-03-02 18:59:01 +00:00
Batuhan Taşkaya 397b96f6d7
bpo-38870: Implement a precedence algorithm in ast.unparse (GH-17377)
Implement a simple precedence algorithm for ast.unparse in order to avoid redundant
parenthesis for nested structures in the final output.
2020-03-01 20:12:17 +00:00