Commit Graph

779 Commits

Author SHA1 Message Date
Mark Shannon 3bd6035b6b
bpo-42908: Mark cleanup code at end of try-except and with artificial (#24202)
* Mark bytecodes at end of try-except as artificial.

* Make sure that the CFG is consistent throughout optimiization. 

* Extend line-number propagation logic so that implicit returns after 'try-except' or 'with' have the correct line numbers.

* Update importlib
2021-01-13 12:05:43 +00:00
Mark Shannon ee9f98d9f4
bpo-42823: Fix frame lineno when frame.f_trace is set (GH-24099)
* Add test for frame.f_lineno with/without tracing.

* Make sure that frame.f_lineno is correct regardless of whether frame.f_trace is set.

* Update importlib

* Add NEWS
2021-01-05 12:04:10 +00:00
Mark Shannon 127dde5916
bpo-42810: Mark jumps at end of if and try statements as artificial. (GH-24091)
* Mark jumps at end of if and try statements as artificial.

* Update importlib

* Add comment explaining the purpose of ADDOP_JUMP_NOLINE.
2021-01-04 18:06:55 +00:00
Mark Shannon bf06b209da
Delete the now unused c_do_not_emit_bytecode field. (#24094) 2021-01-04 13:51:17 +00:00
Mark Shannon 28b75c80dc
bpo-42246: Don't eliminate jumps to jumps, if it will break PEP 626. (GH-23896) 2020-12-23 11:43:10 +00:00
Mark Shannon f2dbfd7e20
bpo-42634: Mark reraise after except blocks as artificial. (GH-23877)
* Mark reraise after except blocks as artificial.

* Update importlib

* Update dis test.
2020-12-21 13:53:50 +00:00
Mark Shannon bf353f3c2d
bpo-42246: Make sure that `f_lasti`, and thus `f_lineno`, is set correctly after raising or reraising an exception (GH-23803)
* Ensure that f_lasti is set correctly after an exception is raised to conform to PEP 626.

* Update importlib

* Add NEWS.
2020-12-17 13:55:28 +00:00
Mark Shannon 5274b682bc
bpo-42645: Make sure that return/break/continue are only traced once when exiting via a finally block. (GH-23780)
* Make sure that return/break/continue are only traced once when exiting via a finally block.

* Add test for return in try-finally.

* Update importlib
2020-12-16 13:07:01 +00:00
Om G c71581c7a4
bpo-42615: Delete redundant jump instructions that only bypass empty blocks (GH-23733)
* Delete jump instructions that bypass empty blocks

* Add news entry

* Explicitly check for unconditional jump opcodes

Using the is_jump function results in the inclusion of instructions like
returns for which this optimization is not really valid. So, instead
explicitly check that the instruction is an unconditional jump.

* Handle conditional jumps, delete jumps gracefully

* Ensure b_nofallthrough and b_reachable are valid

* Add test for redundant jumps

* Regenerate importlib.h and edit Misc/ACKS

* Fix bad whitespace
2020-12-16 12:18:05 +00:00
Mark Shannon 8473cf89bd
bpo-42246: Remove DO_NOT_EMIT_BYTECODE macros, so that while loops and if statements conform to PEP 626. (GH-23743) 2020-12-15 11:07:50 +00:00
Mark Shannon f5e97b72fe
bpo-42635: Mark JUMP_ABSOLUTE at end of 'for' loop as artificial to avoid spurious line events. (GH-23761) 2020-12-14 11:28:39 +00:00
Mark Shannon 56aa20f9eb
Don't generate spurious line number in try-except-finally. (#23760) 2020-12-14 10:19:10 +00:00
Mark Shannon eaccc12aa9
bpo-42246: Don't forget the entry block when ensuring that all exits have a line number (GH-23636)
Don't forget the entry block when ensuring that all exits have a line number.
2020-12-04 15:22:12 +00:00
Mark Shannon 5977a7989d
bpo-42246: Make sure that line number is correct after a return, as required by PEP 626 (GH-23495)
Make sure that line number is correct after a return, as defined by PEP 626.
2020-12-02 13:31:40 +00:00
Yurii Karabas 7301979b23
bpo-42202: Store func annotations as a tuple (GH-23316)
Reduce memory footprint and improve performance of loading modules having many func annotations.

  >>> sys.getsizeof({"a":"int","b":"int","return":"int"})
  232
  >>> sys.getsizeof(("a","int","b","int","return","int"))
  88

The tuple is converted into dict on the fly when `func.__annotations__` is accessed first.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Co-authored-by: Inada Naoki <songofacandy@gmail.com>
2020-11-25 19:43:18 +09:00
Mark Shannon 266b462238
bpo-42349: Compiler clean up. More yak-shaving for PEP 626. (GH-23267)
Make sure that CFG from compiler front-end is correct. Be a bit more aggressive in the compiler back-end.
2020-11-17 19:30:14 +00:00
Mark Shannon fd009e606a
bpo-42246: Fix memory leak in compiler (GH-23256)
* Fix potential memory leak in assembler init.

* Fix reference leak when encountering error during compilation of function body.
2020-11-13 12:53:53 +00:00
Mark Shannon cc75ab791d
bpo-42246: Eliminate jumps to exit blocks by copying those blocks. (#23251)
* Compiler: eliminate jumps to short exit blocks by copying.
2020-11-12 19:49:33 +00:00
Mark Shannon 877df851c3
bpo-42246: Partial implementation of PEP 626. (GH-23113)
* Implement new line number table format, as defined in PEP 626.
2020-11-12 09:43:29 +00:00
Victor Stinner c9bc290dd6
bpo-42161: Use _PyLong_GetZero() and _PyLong_GetOne() (GH-22995)
Use _PyLong_GetZero() and _PyLong_GetOne()
in Objects/ and Python/ directories.
2020-10-27 02:24:34 +01:00
Serhiy Storchaka fb5db7ec58
bpo-42006: Stop using PyDict_GetItem, PyDict_GetItemString and _PyDict_GetItemId. (GH-22648)
These functions are considered not safe because they suppress all internal errors
and can return wrong result.  PyDict_GetItemString and _PyDict_GetItemId can
also silence current exception in rare cases.

Remove no longer used _PyDict_GetItemId.
Add _PyDict_ContainsId and rename _PyDict_Contains into
_PyDict_Contains_KnownHash.
2020-10-26 08:43:39 +02:00
Batuhan Taskaya 044a1048ca
bpo-38605: Make 'from __future__ import annotations' the default (GH-20434)
The hard part was making all the tests pass; there are some subtle issues here, because apparently the future import wasn't tested very thoroughly in previous Python versions.

For example, `inspect.signature()` returned type objects normally (except for forward references), but strings with the future import. We changed it to try and return type objects by calling `typing.get_type_hints()`, but fall back on returning strings if that function fails (which it may do if there are future references in the annotations that require passing in a specific namespace to resolve).
2020-10-06 13:03:02 -07:00
Mark Shannon 02d126aa09
bpo-39934: Account for control blocks in 'except' in compiler. (GH-22395)
* Account for control blocks in 'except' in compiler. Fixes #39934.
2020-09-25 14:04:19 +01:00
Victor Stinner 71f2ff4ccf
bpo-40941: Fix fold_tuple_on_constants() compiler warnings (GH-22378)
Add explicit casts to fix compiler warnings in
fold_tuple_on_constants().

The limit of constants per code is now INT_MAX, rather than UINT_MAX.
2020-09-23 14:06:55 +02:00
Pablo Galindo a5634c4067
bpo-41746: Add type information to asdl_seq objects (GH-22223)
* Add new capability to the PEG parser to type variable assignments. For instance:
```
       | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a }
```

* Add new sequence types from the asdl definition (automatically generated)
* Make `asdl_seq` type a generic aliasing pointer type.
* Create a new `asdl_generic_seq` for the generic case using `void*`.
* The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed.
* New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences.
* Changes all possible `asdl_seq` types to use specific versions everywhere.
2020-09-16 19:42:00 +01:00
Pablo Galindo c51db0ea40
bpo-41531: Fix compilation of dict literals with more than 0xFFFF elements (GH-21850) 2020-08-13 09:48:41 +01:00
Mark Shannon 582aaf19e8
bpo-41463: Generate information about jumps from 'opcode.py' rather than duplicating it in 'compile.c' (GH-21714)
Generate information about jumps from 'opcode.py' rather than duplicate it in 'compile.c'
2020-08-04 17:30:11 +01:00
Mark Shannon 6e8128f02e
bpo-41323: Perform 'peephole' optimizations directly on the CFG. (GH-21517)
* Move 'peephole' optimizations into compile.c and perform them directly on the CFG.
2020-07-30 10:03:00 +01:00
Matthias Bussonnier bd46174a5a
bpo-41218: Only mark async code with CO_COROUTINE. (#21357)
3.8.3 had a regression where compiling with
ast.PyCF_ALLOW_TOP_LEVEL_AWAIT woudl agressively mark things are
coroutine even if there were not.
2020-07-06 23:26:52 +02:00
Pablo Galindo 60eb9f1ab5
bpo-39151: Simplify DFS in the assembler (GH-17733) 2020-06-28 01:55:47 +01:00
Pablo Galindo 1ed83adb0e
bpo-40939: Remove the old parser (GH-20768)
This commit removes the old parser, the deprecated parser module, the old parser compatibility flags and environment variables and all associated support code and documentation.
2020-06-11 17:30:46 +01:00
Mark Shannon db64f12e4d
Make sure that keyword arguments are merged into the arguments dictionary when dict unpacking and keyword arguments are interleaved. (GH-20553) 2020-06-01 10:42:42 +01:00
Pablo Galindo ee40e4b856
bpo-40334: Don't downcast from Py_ssize_t to int (GH-19671) 2020-04-23 03:43:08 +01:00
Pablo Galindo c5fc156852
bpo-40334: PEP 617 implementation: New PEG parser for CPython (GH-19503)
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-04-22 23:29:27 +01:00
Victor Stinner da7933ecc3
bpo-40268: Add _PyInterpreterState_GetConfig() (GH-19492)
Don't access PyInterpreterState.config member directly anymore, but
use new functions:

* _PyInterpreterState_GetConfig()
* _PyInterpreterState_SetConfig()
* _Py_GetConfig()
2020-04-13 03:04:28 +02:00
Zackery Spytz 08050e959e
bpo-40147: Fix a compiler warning on Windows in Python/compile.c (GH-19389)
Change the type of nkeywords to Py_ssize_t.
2020-04-06 07:47:47 +01:00
Pablo Galindo 254ec78341
bpo-40147: Move the check for duplicate keywords to the compiler (GH-19289) 2020-04-03 20:37:13 +01:00
Furkan Önder cb6534e1a8
bpo-40067: Improve error messages for multiple star expressions in assignments (GH-19168)
Co-Authored-By: Batuhan Taşkaya <isidentical@gmail.com>
Co-Authored-By: Pablo Galindo <Pablogsal@gmail.com>
2020-03-26 01:54:31 +00:00
Victor Stinner 87d3b9db4a
bpo-39882: Add _Py_FatalErrorFormat() function (GH-19157) 2020-03-25 19:27:36 +01:00
Andy Lester 7668a8bc93
Use calloc-based functions, not malloc. (GH-19152) 2020-03-24 23:26:44 -05:00
Batuhan Taşkaya 9052f7a41b
bpo-39562: Allow executing asynchronous comprehensions in the asyncio REPL (GH-18968)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-03-19 11:35:44 +00:00
Pablo Galindo d112c600ab
bpo-39220: Do not optimise annotation if 'from __future__ import annotations' is used (GH-17866)
Do not apply AST-based optimizations if 'from __future__ import annotations' is used in order to
prevent information lost in the final version of the annotations.
2020-03-18 23:02:09 +00:00
Serhiy Storchaka 6b97598fb6
bpo-39988: Remove ast.AugLoad and ast.AugStore node classes. (GH-19038) 2020-03-17 23:41:08 +02:00
Serhiy Storchaka 61cb3d02b8
bpo-39987: Simplify setting lineno in the compiler. (GH-19037) 2020-03-17 18:07:30 +02:00
Batuhan Taşkaya 8689209e03
bpo-39969: Remove ast.Param node class as is no longer used (GH-19020) 2020-03-15 19:32:17 +00:00
Pablo Galindo 90235810ec
bpo-39965: Correctly raise SyntaxError if await is used outside async functions when PyCF_ALLOW_TOP_LEVEL_AWAIT is set (GH-19010) 2020-03-15 04:29:22 +00:00
Andy Lester 76d5877b72
closes bpo-39922: Remove unused args from four functions. (GH-18893) 2020-03-10 19:18:12 -07:00
Serhiy Storchaka 13d52c2686
bpo-34822: Simplify AST for subscription. (GH-9605)
* Remove the slice type.
* Make Slice a kind of the expr type instead of the slice type.
* Replace ExtSlice(slices) with Tuple(slices, Load()).
* Replace Index(value) with a value itself.

All non-terminal nodes in AST for expressions are now of the expr type.
2020-03-10 18:52:34 +02:00
Brandt Bucher d5aa2e941c
bpo-39890: Don't mutate the AST when compiling starred assignments (GH-18833) 2020-03-08 03:44:18 +00:00
Batuhan Taşkaya d82e469048
bpo-39639: Remove the AST "Suite" node and associated code (GH-18513)
The AST "Suite" node is no longer used and it can be removed from the ASDL definition and related structures (compiler, visitors, ...).

Co-Authored-By: Victor Stinner <vstinner@python.org>
Co-authored-by: Brett Cannon <54418+brettcannon@users.noreply.github.com>
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-03-04 16:16:46 +00:00