Commit Graph

132 Commits

Author SHA1 Message Date
Mark Shannon 89fd7c3452
bpo-46329: Split calls into precall and call instructions. (GH-30855)
* Add PRECALL_FUNCTION opcode.

* Move 'call shape' varaibles into struct.

* Replace CALL_NO_KW and CALL_KW with KW_NAMES and CALL instructions.

* Specialize for builtin methods taking using the METH_FASTCALL | METH_KEYWORDS protocol.

* Allow kwnames for specialized calls to builtin types.

* Specialize calls to tuple(arg) and str(arg).
2022-01-28 12:42:30 +00:00
Mark Shannon b04dfbbe4b
bpo-46409: Make generators in bytecode (GH-30633)
* Add RETURN_GENERATOR and JUMP_NO_INTERRUPT opcodes.

* Trim frame and generator by word each.

* Minor refactor of frame.c

* Update test.test_sys to account for smaller frames.

* Treat generator functions as normal functions when evaluating and specializing.
2022-01-20 11:46:39 +00:00
Mark Shannon e028ae99ec
bpo-45923: Handle call events in bytecode (GH-30364)
* Add a RESUME instruction to handle "call" events.
2022-01-06 13:09:25 +00:00
Mark Shannon 9f8f45144b
bpo-44525: Split calls into PRECALL and CALL (GH-30011)
* Add 3 new opcodes for calls: PRECALL_METHOD, CALL_NO_KW, CALL_KW.

* Update specialization to handle new CALL opcodes.

* Specialize call to method descriptors.

* Remove old CALL opcodes: CALL_FUNCTION, CALL_METHOD, CALL_METHOD_KW, CALL_FUNCTION_KW.
2021-12-14 18:22:44 +00:00
Irit Katriel d60457a667
bpo-45292: [PEP-654] add except* (GH-29581) 2021-12-14 16:48:15 +00:00
Weipeng Hong 28179aac79
bpo-42918: Improve build-in function compile() in mode 'single' (GH-29934)
Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
2021-12-11 00:44:26 +01:00
Brandt Bucher 9178f533ff
bpo-45636: Merge all numeric operators (GH-29482) 2021-11-10 22:56:22 -08:00
Inada Naoki 55c4a92fc1
bpo-45056: Remove trailing unused constants from co_consts (GH-28109) 2021-09-02 13:02:06 +02:00
Mark Shannon f333ab0f2e
bpo-44622: Set line number of END_ASYNC_FOR to match that of iterator. (GH-27160) 2021-07-15 14:37:57 +01:00
Ammar Askar 4823d9a512
bpo-43950: Add option to opt-out of PEP-657 (GH-27023)
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
Co-authored-by: Ammar Askar <ammar@ammaraskar.com>
2021-07-07 20:07:12 +01:00
Batuhan Taskaya 44f91fc802
bpo-43950: use 0-indexed column offsets for bytecode positions (GH-27011) 2021-07-04 11:02:16 -07:00
Pablo Galindo 98eee94421
bpo-43950: Add code.co_positions (PEP 657) (GH-26955)
This PR is part of PEP 657 and augments the compiler to emit ending
line numbers as well as starting and ending columns from the AST
into compiled code objects. This allows bytecodes to be correlated
to the exact source code ranges that generated them.

This information is made available through the following public APIs:

* The `co_positions` method on code objects.
* The C API function `PyCode_Addr2Location`.

Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
Co-authored-by: Ammar Askar <ammar@ammaraskar.com>
2021-07-02 15:10:11 +01:00
Batuhan Taskaya 1b28187a0e
bpo-44313: generate LOAD_ATTR/CALL_FUNCTION for top-level imported objects (GH-26677) 2021-06-30 23:53:36 +01:00
Mark Shannon 82e5c28af7
bpo-44297: Fix missing line number in generator expressions (GH-26801)
* Make sure that line number is set when entering comprehension scope in compiler.
2021-06-21 10:55:15 +01:00
Serhiy Storchaka 2ea6d89028
bpo-43833: Emit warnings for numeric literals followed by keyword (GH-25466)
Emit a deprecation warning if the numeric literal is immediately followed by
one of keywords: and, else, for, if, in, is, or. Raise a syntax error with
more informative message if it is immediately followed by other keyword or
identifier.

Automerge-Triggered-By: GH:pablogsal
2021-06-08 16:31:10 -07:00
Serhiy Storchaka face87c94e
bpo-42609: Check recursion depth in the AST validator and optimizer (GH-23744) 2021-04-25 13:38:00 +03:00
Mark Shannon 11e0b295de
bpo-43846: Use less stack for large literals and calls (GH-25403)
* Modify compiler to reduce stack consumption for large expressions.

* Add more tests for stack usage.

* Add NEWS item.

* Raise SystemError for truly excessive stack use.
2021-04-15 14:28:56 +01:00
Inada Naoki 35715d1e72
bpo-43651: PEP 597: Fix EncodingWarning in some tests (GH-25145)
* test_asyncio
* test_bz2
* test_math
* test_cmath
* test_cmd_line
* test_cmd_line_script
* test_compile
* test_contextlib
* test_profile
* ctypes/test/test_find
* test_multiprocessing
* test_configparser
* test_csv
* test_dbm_dumb
* test_decimal
* test_difflib
* os.fdopen() calls io.text_encoding() to emit EncodingWarning for right place.
2021-04-04 09:01:23 +09:00
Mark Shannon c5440937ef
Mark POP_TOP at end of expression statement as artificial, to conform to PEP 626. (GH-24860) 2021-03-15 14:24:25 +00:00
Mark Shannon d48848c83e
bpo-39316: Make sure that attribute accesses and stores, including method calls, conform to PEP 626. (GH-24859) 2021-03-14 18:01:30 +00:00
Inada Naoki bdb941be42
bpo-42217: compiler: merge same co_code and co_linetable objects (GH-23056) 2021-02-10 09:20:42 +09:00
Om G c71581c7a4
bpo-42615: Delete redundant jump instructions that only bypass empty blocks (GH-23733)
* Delete jump instructions that bypass empty blocks

* Add news entry

* Explicitly check for unconditional jump opcodes

Using the is_jump function results in the inclusion of instructions like
returns for which this optimization is not really valid. So, instead
explicitly check that the instruction is an unconditional jump.

* Handle conditional jumps, delete jumps gracefully

* Ensure b_nofallthrough and b_reachable are valid

* Add test for redundant jumps

* Regenerate importlib.h and edit Misc/ACKS

* Fix bad whitespace
2020-12-16 12:18:05 +00:00
Mark Shannon 8473cf89bd
bpo-42246: Remove DO_NOT_EMIT_BYTECODE macros, so that while loops and if statements conform to PEP 626. (GH-23743) 2020-12-15 11:07:50 +00:00
Mark Shannon eaccc12aa9
bpo-42246: Don't forget the entry block when ensuring that all exits have a line number (GH-23636)
Don't forget the entry block when ensuring that all exits have a line number.
2020-12-04 15:22:12 +00:00
Mark Shannon 5977a7989d
bpo-42246: Make sure that line number is correct after a return, as required by PEP 626 (GH-23495)
Make sure that line number is correct after a return, as defined by PEP 626.
2020-12-02 13:31:40 +00:00
Mark Shannon 266b462238
bpo-42349: Compiler clean up. More yak-shaving for PEP 626. (GH-23267)
Make sure that CFG from compiler front-end is correct. Be a bit more aggressive in the compiler back-end.
2020-11-17 19:30:14 +00:00
Mark Shannon 877df851c3
bpo-42246: Partial implementation of PEP 626. (GH-23113)
* Implement new line number table format, as defined in PEP 626.
2020-11-12 09:43:29 +00:00
Pablo Galindo c51db0ea40
bpo-41531: Fix compilation of dict literals with more than 0xFFFF elements (GH-21850) 2020-08-13 09:48:41 +01:00
Hai Shi 847f94f47b
bpo-40275: Use new test.support helper submodules in tests (GH-21151)
Use new test.support helper submodules in tests:

* distutils tests
* test_buffer
* test_compile
* test_filecmp
* test_fileinput
* test_readline
* test_smtpnet
* test_structmembers
* test_tools
2020-06-25 19:17:57 +02:00
Lysandros Nikolaou 6d65087655
bpo-40334: Disallow invalid single statements in the new parser (GH-19774)
After parsing is done in single statement mode, the tokenizer buffer has to be checked for additional lines and a `SyntaxError` must be raised, in case there are any.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-04-29 02:42:27 +01:00
Victor Stinner 1def7754b7
bpo-40334: Rename PyConfig.use_peg to _use_peg_parser (GH-19670)
* Rename PyConfig.use_peg to _use_peg_parser
* Document PyConfig._use_peg_parser and mark it a deprecated
* Mark -X oldparser option and PYTHONOLDPARSER env var as deprecated
  in the documentation.
* Add use_old_parser() and skip_if_new_parser() to test.support
* Remove sys.flags.use_peg: use_old_parser() uses
  _testinternalcapi.get_configs() instead.
* Enhance test_embed tests
* subprocess._args_from_interpreter_flags() copies -X oldparser
2020-04-23 03:03:24 +02:00
Pablo Galindo c5fc156852
bpo-40334: PEP 617 implementation: New PEG parser for CPython (GH-19503)
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-04-22 23:29:27 +01:00
Pablo Galindo 6c3e66a34b
bpo-38640: Allow break and continue in always false while loops (GH-16992) 2019-10-30 11:53:26 +00:00
Pablo Galindo 18c5f9d44d bpo-37500: Make sure dead code does not generate bytecode but also detect syntax errors (GH-14612)
https://bugs.python.org/issue37500

Add a new field to the compiler structure that allows to be configured
so no bytecode is emitted. In this way is possible to detect errors by
walking the nodes while preserving optimizations.


https://bugs.python.org/issue37500
2019-07-15 02:15:01 -07:00
INADA Naoki f7e4d3642f bpo-34100: compile: Re-enable frozenset merging (GH-10760)
This reverts commit 1005c84535.
2018-11-28 16:58:46 +01:00
Victor Stinner 1005c84535
bpo-34100: Partially revert merge_consts_recursive() (GH-10743)
Partically revert commit c2e1607a51 to
fix a reference leak.
2018-11-27 15:12:47 +01:00
INADA Naoki c2e1607a51 bpo-34100: Merge constants recursively (GH-8341)
There are some same consts in a module.  This commit merges them into
single instance.  It reduces number of objects in memory after loading modules.


https://bugs.python.org/issue34100
2018-11-26 04:23:22 -08:00
Gregory P. Smith 49fa4a9f1e
bpo-35193: Fix an off by one error in the RETURN_VALUE case. (GH-10418)
Fix an off by one error in the peephole optimizer when checking for unreachable code beyond a return.

Do a bounds check within find_op so it can return before going past the end as a safety measure.

7db3c48833 (diff-a33329ae6ae0bb295d742f0caf93c137)
introduced this off by one error while fixing another one nearby.

This bug was shipped in all Python 3.6 and 3.7 releases.

The included unittest won't fail unless you do a clang msan build.
2018-11-08 17:55:07 -08:00
Serhiy Storchaka 34fd4c2019
bpo-35133: Fix mistakes when concatenate string literals on different lines. (GH-10284)
Two kind of mistakes:

1. Missed space. After concatenating there is no space between words.

2. Missed comma. Causes unintentional concatenating in a list of strings.
2018-11-05 16:20:25 +02:00
Serhiy Storchaka fe2bbb1869
bpo-32489: Allow 'continue' in 'finally' clause. (GH-5822) 2018-03-18 09:56:52 +02:00
Serhiy Storchaka b21d155f57
bpo-32964: Reuse a testing implementation of the path protocol in tests. (#5930) 2018-03-02 11:53:51 +02:00
Serhiy Storchaka 782d6fe443
bpo-31113: Get rid of recursion in the compiler for normal control flow. (#3015) 2018-01-11 20:20:13 +02:00
Serhiy Storchaka d4864c61e3
bpo-24340: Fix estimation of the code stack size. (#5076) 2018-01-09 21:54:52 +02:00
Serhiy Storchaka 3325a6780c
bpo-27169: The __debug__ constant is now optimized out at compile time. (#4880)
This fixes also bpo-22091.
2017-12-15 12:35:48 +02:00
Serhiy Storchaka 067cbd02bf Issue #29337: Fixed possible BytesWarning when compare the code objects.
Warnings could be emitted at compile time.
2017-01-24 20:54:07 +02:00
Serhiy Storchaka 4102d25b7e Issue #29337: Fixed possible BytesWarning when compare the code objects.
Warnings could be emitted at compile time.
2017-01-24 20:52:43 +02:00
Serhiy Storchaka 713640c4c9 Issue #29337: Fixed possible BytesWarning when compare the code objects.
Warnings could be emitted at compile time.
2017-01-24 20:49:26 +02:00
Serhiy Storchaka 5bb8b9134b Issue #18896: Python function can now have more than 255 parameters.
collections.namedtuple() now supports tuples with more than 255 elements.
2016-12-16 19:19:02 +02:00
Brett Cannon a571120410 Issue #27182: Add support for path-like objects to PyUnicode_FSDecoder(). 2016-09-06 19:36:01 -07:00
Serhiy Storchaka febc332056 Issue #26754: Undocumented support of general bytes-like objects
as path in compile() and similar functions is now deprecated.
2016-08-06 23:29:29 +03:00