Commit Graph

264 Commits

Author SHA1 Message Date
Mark Shannon 6e8128f02e
bpo-41323: Perform 'peephole' optimizations directly on the CFG. (GH-21517)
* Move 'peephole' optimizations into compile.c and perform them directly on the CFG.
2020-07-30 10:03:00 +01:00
Lysandros Nikolaou 1f0f4abb11
bpo-41076: Pre-feed the parser with the f-string expression location (GH-21054)
This commit changes the parsing of f-string expressions with the new parser. The parser gets pre-fed with the location of the expression itself (not the f-string, which was what we were doing before). This allows us to completely skip the shifting of the AST nodes after the parsing is completed.
2020-06-28 00:41:48 +01:00
Pablo Galindo c5fc156852
bpo-40334: PEP 617 implementation: New PEG parser for CPython (GH-19503)
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-04-22 23:29:27 +01:00
Serhiy Storchaka 61cb3d02b8
bpo-39987: Simplify setting lineno in the compiler. (GH-19037) 2020-03-17 18:07:30 +02:00
Armin Rigo 6daa37fd42
bpo-38091: Import deadlock detection causes deadlock (GH-17518)
Automerge-Triggered-By: @brettcannon
2020-03-02 17:37:25 -08:00
Mark Shannon 8a4cd700a7
bpo-39320: Handle unpacking of **values in compiler (GH-18141)
* Add DICT_UPDATE and DICT_MERGE bytecodes. Use them for ** unpacking.

* Remove BUILD_MAP_UNPACK and BUILD_MAP_UNPACK_WITH_CALL, as they are now unused.

* Update magic number for ** unpacking opcodes.

* Update dis.rst to incorporate new bytecodes.

* Add blurb entry.
2020-01-27 09:57:45 +00:00
Dino Viehland 9b6fec4651
bpo-39336: Allow packages to not let their child modules be set on them (#18006)
* bpo-39336: Allow setattr to fail on modules which aren't assignable

When attaching a child module to a package if the object in sys.modules raises an AttributeError (e.g. because it is immutable) it causes the whole import to fail.  This now allows immutable packages to exist and an ImportWarning is reported and the AttributeError exception is ignored.
2020-01-22 16:42:38 -08:00
Mark Shannon 9af0e47b17
bpo-39156: Break up COMPARE_OP into four logically distinct opcodes. (GH-17754)
Break up COMPARE_OP into four logically distinct opcodes:
* COMPARE_OP for rich comparisons
* IS_OP for 'is' and 'is not' tests
* CONTAINS_OP for 'in' and 'is not' tests
* JUMP_IF_NOT_EXC_MATCH for checking exceptions in 'try-except' statements.
2020-01-14 10:12:45 +00:00
Mark Shannon 88dce26da6
Fix handling of line numbers around finally-blocks. (#17737) 2019-12-30 09:53:36 +00:00
Mark Shannon fee552669f
Produce cleaner bytecode for 'with' and 'async with' by generating separate code for normal and exceptional paths. (#6641)
Remove BEGIN_FINALLY, END_FINALLY, CALL_FINALLY and POP_FINALLY bytecodes. Implement finally blocks by code duplication.
Reimplement frame.lineno setter using line numbers rather than bytecode offsets.
2019-11-21 09:11:43 +00:00
Dong-hee Na 145cf1f50c bpo-35923: Update the BuiltinImporter to use loader._ORIGIN instead of a hard-coded value (GH-15651) 2019-09-11 17:00:02 +01:00
Zackery Spytz ce6a070414 bpo-34880: Add the LOAD_ASSERTION_ERROR opcode. (GH-15073)
Fix assert statement misbehavior if AssertionError is shadowed.
2019-08-25 12:44:09 +03:00
Serhiy Storchaka ef61c524dd
bpo-37830: Fix compilation of break and continue in finally. (GH-15320)
Fix compilation of "break" and "continue" in the
"finally" block when the corresponding "try" block
contains "return" with a non-constant value.
2019-08-24 13:11:52 +03:00
Serhiy Storchaka 662db125cd
bpo-37685: Fixed __eq__, __lt__ etc implementations in some classes. (GH-14952)
They now return NotImplemented for unsupported type of the other operand.
2019-08-08 08:42:54 +03:00
Ngalim Siregar c5fa44944e bpo-37444: Update differing exception between builtins and importlib (GH-14869)
Imports now raise `TypeError` instead of `ValueError` for relative import failures. This makes things consistent between `builtins.__import__` and `importlib.__import__` as well as using a more natural import for the failure.


https://bugs.python.org/issue37444



Automerge-Triggered-By: @brettcannon
2019-08-02 22:46:02 -07:00
Pablo Galindo 3498c642f4
bpo-37213: Handle negative line deltas correctly in the peephole optimizer (GH-13969)
The peephole optimizer was not optimizing correctly bytecode after negative deltas were introduced. This is due to the fact that some special values (255) were being searched for in both instruction pointer delta and line number deltas.
2019-06-13 19:16:22 +01:00
Victor Stinner ef9d9b6312
bpo-36829: Add sys.unraisablehook() (GH-13187)
Add new sys.unraisablehook() function which can be overridden to
control how "unraisable exceptions" are handled. It is called when an
exception has occurred but there is no way for Python to handle it.
For example, when a destructor raises an exception or during garbage
collection (gc.collect()).

Changes:

* Add an internal UnraisableHookArgs type used to pass arguments to
  sys.unraisablehook.
* Add _PyErr_WriteUnraisableDefaultHook().
* The default hook now ignores exception on writing the traceback.
* test_sys now uses unittest.main() to automatically discover tests:
  remove test_main().
* Add _PyErr_Init().
* Fix PyErr_WriteUnraisable(): hold a strong reference to sys.stderr
  while using it
2019-05-22 11:28:22 +02:00
Pablo Galindo 8c77b8cb91
bpo-36540: PEP 570 -- Implementation (GH-12701)
This commit contains the implementation of PEP570: Python positional-only parameters.

* Update Grammar/Grammar with new typedarglist and varargslist

* Regenerate grammar files

* Update and regenerate AST related files

* Update code object

* Update marshal.c

* Update compiler and symtable

* Regenerate importlib files

* Update callable objects

* Implement positional-only args logic in ceval.c

* Regenerate frozen data

* Update standard library to account for positional-only args

* Add test file for positional-only args

* Update other test files to account for positional-only args

* Add News entry

* Update inspect module and related tests
2019-04-29 13:36:57 +01:00
Nina Zakharenko 69091cb497 bpo-35321: Set the spec origin to frozen in frozen modules (#11732)
* bpo-35321: Set the spec origin to frozen in frozen modules

This fix correctly sets the spec origin to
"frozen" for the _frozen_importlib module. Note that the
origin was already correctly set in _frozen_importlib_external.

* 📜🤖 Added by blurb_it.
2019-02-04 16:56:26 -08:00
Anthony Sottile 995d9b9297 bpo-16806: Fix `lineno` and `col_offset` for multi-line string tokens (GH-10021) 2019-01-13 13:05:13 +09:00
INADA Naoki c2e1607a51 bpo-34100: Merge constants recursively (GH-8341)
There are some same consts in a module.  This commit merges them into
single instance.  It reduces number of objects in memory after loading modules.


https://bugs.python.org/issue34100
2018-11-26 04:23:22 -08:00
Serhiy Storchaka 95b6acf951
bpo-34876: Change the lineno of the AST for decorated function and class. (GH-9731)
It was overridden by the lineno of the first decorator. Now it is
the lineno of 'def' or 'class'.
2018-10-30 13:16:02 +02:00
Serhiy Storchaka c93c58b5d5
bpo-33331: Clean modules in the reversed order in PyImport_Cleanup(). (GH-6565)
Modules imported last are now cleared first at interpreter shutdown.

A newly imported module is moved to the end of sys.modules, behind
modules on which it depends.
2018-10-29 19:30:16 +02:00
Serhiy Storchaka 79d1c2e6c9
bpo-25711: Rewrite zipimport in pure Python. (GH-6809) 2018-09-18 22:22:29 +03:00
Serhiy Storchaka da8d72c953
bpo-12458: Fix line numbers for multiline expressions. (GH-8774) 2018-09-17 15:17:29 +03:00
Serhiy Storchaka 73cbe7a01a
bpo-32911: Revert bpo-29463. (GH-7121) (GH-7197)
Remove the docstring attribute of AST types and restore docstring
expression as a first stmt in their body.

Co-authored-by: INADA Naoki <methane@users.noreply.github.com>
2018-05-29 12:04:55 +03:00
Zackery Spytz f303639e3a bpo-33270: Intern names for all anonymous code objects (#6472) 2018-04-16 01:12:29 +03:00
Serhiy Storchaka 4e2442505c
bpo-32946: Speed up "from ... import ..." from non-packages. (GH-5873) 2018-03-11 10:52:37 +02:00
Serhiy Storchaka 520b7ae27e
bpo-17611. Move unwinding of stack for "pseudo exceptions" from interpreter to compiler. (GH-5006)
Co-authored-by: Mark Shannon <mark@hotpy.org>
Co-authored-by: Antoine Pitrou <antoine@python.org>
2018-02-22 23:33:30 +02:00
Barry Warsaw bbbcf8693b
bpo-32303 - Consistency fixes for namespace loaders (#5481)
* Make sure ``__spec__.loader`` matches ``__loader__`` for namespace packages.
* Make sure ``__spec__.origin` matches ``__file__`` for namespace packages.

https://bugs.python.org/issue32303
https://bugs.python.org/issue32305
2018-02-02 15:15:58 -05:00
Serhiy Storchaka d4864c61e3
bpo-24340: Fix estimation of the code stack size. (#5076) 2018-01-09 21:54:52 +02:00
Serhiy Storchaka 15a8728415
bpo-29469: Optimize literal lists and sets iterating on the AST level. (#4866) 2017-12-14 20:24:31 +02:00
INADA Naoki 7ea143ae79
bpo-29469: Move constant folding to AST optimizer (GH-2858) 2017-12-14 16:47:20 +09:00
Serhiy Storchaka 41c56940c6 bpo-21720: Restore the Python 2.7 logic in handling a fromlist. (#4118)
BytesWarning no longer emitted when the fromlist argument of
__import__() or the __all__ attribute of the module contain bytes
instances.
2017-10-26 10:41:59 +03:00
Serhiy Storchaka f07e2b64df bpo-31642: Restore blocking "from" import by setting None in sys.modules. (#3834) 2017-10-08 10:44:10 +03:00
Antoine Pitrou 88c60c9668 Trivial cleanups following bpo-31370 (#3649)
* Trivial cleanups following bpo-31370

* Also cleanup the "importlib._bootstrap_external" module
2017-09-18 23:50:44 +02:00
Serhiy Storchaka 9b0d1d647e bpo-31070: Fix a race condition in importlib _get_module_lock(). (#3033) 2017-08-09 14:29:12 +03:00
Victor Stinner e72b1359f8 bpo-30891: Fix again importlib _find_and_load() (#2665)
Use sys.modules.get() in the "with _ModuleLockManager(name):" block
to protect the dictionary key with the module lock and use an atomic
get to prevent race condition.

Remove also _bootstrap._POPULATE since it was unused
(_bootstrap_external now has its own _POPULATE object), add a new
_SENTINEL object instead.
2017-07-21 13:00:46 +02:00
Serhiy Storchaka 8a9cd20edc bpo-30876: Relative import from unloaded package now reimports the package (#2639)
instead of failing with SystemError.

Relative import from non-package now fails with ImportError rather than
SystemError.
2017-07-12 06:50:03 +03:00
Victor Stinner 4f9a446f3f bpo-30891: Fix importlib _find_and_load() race condition (#2646)
* Rewrite importlib _get_module_lock(): it is now responsible to hold
  the imp lock directly.
* _find_and_load() now holds the module lock to check if name is in
  sys.modules to prevent a race condition
2017-07-10 22:52:32 +02:00
Serhiy Storchaka b4baacee1a bpo-30814: Fixed a race condition when import a submodule from a package. (#2580) 2017-07-06 08:09:03 +03:00
Serhiy Storchaka 36ff451eba bpo-30501: Make the compiler producing optimized code for condition expressions. (#1851) 2017-06-11 14:50:22 +03:00
Zachary Ware 84c6607a81 Update importlib.h (GH-1762) 2017-05-23 12:23:05 -05:00
Eric Snow 6b4be195cd bpo-22257: Small changes for PEP 432. (#1728)
PEP 432 specifies a number of large changes to interpreter startup code, including exposing a cleaner C-API. The major changes depend on a number of smaller changes. This patch includes all those smaller changes.
2017-05-22 21:36:03 -07:00
INADA Naoki cb41b2766d bpo-29463: Add docstring field to some AST nodes. (#46)
* bpo-29463: Add docstring field to some AST nodes.

ClassDef, ModuleDef, FunctionDef, and AsyncFunctionDef has docstring
field for now.  It was first statement of there body.

* fix document.  thanks travis!

* doc fixes
2017-02-22 16:31:59 +01:00
Yury Selivanov f2392133eb Issue #26110: Add LOAD_METHOD/CALL_METHOD opcodes.
Special thanks to INADA Naoki for pushing the patch through
the last mile, Serhiy Storchaka for reviewing the code, and to
Victor Stinner for suggesting the idea (originally implemented
in the PyPy project).
2016-12-13 19:03:51 -05:00
Serhiy Storchaka 7db3c48833 Issue #28517: Fixed of-by-one error in the peephole optimizer that caused
keeping unreachable code.
2016-10-25 09:30:43 +03:00
Serhiy Storchaka 3738c2d8ae Issue #27942: String constants now interned recursively in tuples and frozensets. 2016-09-30 10:23:01 +03:00
Serhiy Storchaka 00a0fc1144 Issue #27942: String constants now interned recursively in tuples and frozensets. 2016-09-30 10:07:26 +03:00
Serhiy Storchaka b72810583e Issue #27213: Fixed different issues with reworked CALL_FUNCTION* opcodes.
* BUILD_TUPLE_UNPACK and BUILD_MAP_UNPACK_WITH_CALL no longer generated with
  single tuple or dict.
* Restored more informative error messages for incorrect var-positional and
  var-keyword arguments.
* Removed code duplications in _PyEval_EvalCodeWithName().
* Removed redundant runtime checks and parameters in _PyStack_AsDict().
* Added a workaround and enabled previously disabled test in test_traceback.
* Removed dead code from the dis module.
2016-09-12 00:52:40 +03:00