* Added new opcode END_ASYNC_FOR.
* Setting global StopAsyncIteration no longer breaks "async for" loops.
* Jumping into an "async for" loop is now disabled.
* Jumping out of an "async for" loop no longer corrupts the stack.
* Simplify the compiler.
* Document `from __future__ import annotations`
* Provide plumbing and tests for `from __future__ import annotations`
* Implement unparsing the AST back to string form
This is required for PEP 563 and as such only implements a part of the
unparsing process that covers expressions.
Previously, CO_NOFREE was set in the compiler, which meant
it could end up being set incorrectly when code objects
were created directly. Setting it in the constructor based
on freevars and cellvars ensures it is always accurate,
regardless of how the code object is defined.
* Add Py_UNREACHABLE() as an alias to abort().
* Use Py_UNREACHABLE() instead of assert(0)
* Convert more unreachable code to use Py_UNREACHABLE()
* Document Py_UNREACHABLE() and a few other macros.
* bpo-29463: Add docstring field to some AST nodes.
ClassDef, ModuleDef, FunctionDef, and AsyncFunctionDef has docstring
field for now. It was first statement of there body.
* fix document. thanks travis!
* doc fixes
Special thanks to INADA Naoki for pushing the patch through
the last mile, Serhiy Storchaka for reviewing the code, and to
Victor Stinner for suggesting the idea (originally implemented
in the PyPy project).
Handling zero-argument super() in __init_subclass__ and
__set_name__ involved moving __class__ initialisation to
type.__new__. This requires cooperation from custom
metaclasses to ensure that the new __classcell__ entry
is passed along appropriately.
The initial implementation of that change resulted in abruptly
broken zero-argument super() support in metaclasses that didn't
adhere to the new requirements (such as Django's metaclass for
Model definitions).
The updated approach adopted here instead emits a deprecation
warning for those cases, and makes them work the same way they
did in Python 3.5.
This patch also improves the related class machinery documentation
to cover these details and to include more reader-friendly
cross-references and index entries.
* BUILD_TUPLE_UNPACK and BUILD_MAP_UNPACK_WITH_CALL no longer generated with
single tuple or dict.
* Restored more informative error messages for incorrect var-positional and
var-keyword arguments.
* Removed code duplications in _PyEval_EvalCodeWithName().
* Removed redundant runtime checks and parameters in _PyStack_AsDict().
* Added a workaround and enabled previously disabled test in test_traceback.
* Removed dead code from the dis module.
The __class__ cell used by zero-argument super() is now initialized
from type.__new__ rather than __build_class__, so class methods
relying on that will now work correctly when called from metaclass
methods during class creation.
Patch by Martin Teichmann.
Issue #27213: Rework CALL_FUNCTION* opcodes to produce shorter and more
efficient bytecode:
* CALL_FUNCTION now only accepts position arguments
* CALL_FUNCTION_KW accepts position arguments and keyword arguments, but keys
of keyword arguments are packed into a constant tuple.
* CALL_FUNCTION_EX is the most generic, it expects a tuple and a dict for
positional and keyword arguments.
CALL_FUNCTION_VAR and CALL_FUNCTION_VAR_KW opcodes have been removed.
2 tests of test_traceback are currently broken: skip test, the issue #28050 was
created to track the issue.
Patch by Demur Rumed, design by Serhiy Storchaka, reviewed by Serhiy Storchaka
and Victor Stinner.
In practice, bytecode instruction arguments are unsigned. Update the assertion
to make it more explicit that argument must be greater or equal than 0.
Rewrite also the comment.
The compile ignores constant statements and emit a SyntaxWarning warning.
Don't emit the warning for string statement because triple quoted string is a
common syntax for multiline comments.
Don't emit the warning on ellipis neither: 'def f(): ...' is a legit syntax for
abstract functions.
Changes:
* test_ast: ignore SyntaxWarning when compiling test statements. Modify
test_load_const() to use assignment expressions rather than constant
expression.
* test_code: add more kinds of constant statements, ignore SyntaxWarning when
testing that the compiler removes constant statements.
* test_grammar: ignore SyntaxWarning on the statement "1"
Issue #26146: Add a new kind of AST node: ast.Constant. It can be used by
external AST optimizers, but the compiler does not emit directly such node.
An optimizer can replace the following AST nodes with ast.Constant:
* ast.NameConstant: None, False, True
* ast.Num: int, float, complex
* ast.Str: str
* ast.Bytes: bytes
* ast.Tuple if items are constants too: tuple
* frozenset
Update code to accept ast.Constant instead of ast.Num and/or ast.Str:
* compiler
* docstrings
* ast.literal_eval()
* Tools/parser/unparse.py
Issue #25843: When compiling code, don't merge constants if they are equal but
have a different types. For example, "f1, f2 = lambda: 1, lambda: 1.0" is now
correctly compiled to two different functions: f1() returns 1 (int) and f2()
returns 1.0 (int), even if 1 and 1.0 are equal.
Add a new _PyCode_ConstantKey() private function.
Issue #26107: The format of the co_lnotab attribute of code objects changes to
support negative line number delta.
Changes:
* assemble_lnotab(): if line number delta is less than -128 or greater than
127, emit multiple (offset_delta, lineno_delta) in co_lnotab
* update functions decoding co_lnotab to use signed 8-bit integers
- dis.findlinestarts()
- PyCode_Addr2Line()
- _PyCode_CheckLineNumber()
- frame_setlineno()
* update lnotab_notes.txt
* increase importlib MAGIC_NUMBER to 3361
* document the change in What's New in Python 3.6
* cleanup also PyCode_Optimize() to use better variable names
This changes the main documentation, doc strings, source code comments, and a
couple error messages in the test suite. In some cases the word was removed
or edited some other way to fix the grammar.
Summary of changes:
1. Coroutines now have a distinct, separate from generators
type at the C level: PyGen_Type, and a new typedef PyCoroObject.
PyCoroObject shares the initial segment of struct layout with
PyGenObject, making it possible to reuse existing generators
machinery. The new type is exposed as 'types.CoroutineType'.
As a consequence of having a new type, CO_GENERATOR flag is
no longer applied to coroutines.
2. Having a separate type for coroutines made it possible to add
an __await__ method to the type. Although it is not used by the
interpreter (see details on that below), it makes coroutines
naturally (without using __instancecheck__) conform to
collections.abc.Coroutine and collections.abc.Awaitable ABCs.
[The __instancecheck__ is still used for generator-based
coroutines, as we don't want to add __await__ for generators.]
3. Add new opcode: GET_YIELD_FROM_ITER. The opcode is needed to
allow passing native coroutines to the YIELD_FROM opcode.
Before this change, 'yield from o' expression was compiled to:
(o)
GET_ITER
LOAD_CONST
YIELD_FROM
Now, we use GET_YIELD_FROM_ITER instead of GET_ITER.
The reason for adding a new opcode is that GET_ITER is used
in some contexts (such as 'for .. in' loops) where passing
a coroutine object is invalid.
4. Add two new introspection functions to the inspec module:
getcoroutinestate(c) and getcoroutinelocals(c).
5. inspect.iscoroutine(o) is updated to test if 'o' is a native
coroutine object. Before this commit it used abc.Coroutine,
and it was requested to update inspect.isgenerator(o) to use
abc.Generator; it was decided, however, that inspect functions
should really be tailored for checking for native types.
6. sys.set_coroutine_wrapper(w) API is updated to work with only
native coroutines. Since types.coroutine decorator supports
any type of callables now, it would be confusing that it does
not work for all types of coroutines.
7. Exceptions logic in generators C implementation was updated
to raise clearer messages for coroutines:
Before: TypeError("generator raised StopIteration")
After: TypeError("coroutine raised StopIteration")
compiler warnings on Windows 64-bit. Use Py_SAFE_DOWNCAST() where the final
downcast is needed.
The bytecode doesn't support integer parameters larger than 32-bit yet.
Previously, excessive nesting in expressions would blow the
stack and segfault the interpreter. Now, a hard limit based
on the configured recursion limit and a hardcoded scaling
factor is applied.
* In debug mode, fill the string data with invalid characters
* Simplify also reference counting in PyCodec_BackslashReplaceErrors()
and PyCodec_XMLCharRefReplaceError()
This allows generators that are using yield from to be seen by debuggers. It
also kills the f_yieldfrom field on frame objects.
Patch mostly from Mark Shannon with a few tweaks by me.
Issue #25843: When compiling code, don't merge constants if they are equal but
have a different types. For example, "f1, f2 = lambda: 1, lambda: 1.0" is now
correctly compiled to two different functions: f1() returns 1 (int) and f2()
returns 1.0 (int), even if 1 and 1.0 are equal.
Add a new _PyCode_ConstantKey() private function.
namespace if it occurs as a free variable in a nested block. This limitation
of the compiler has been lifted, and a new opcode introduced (DELETE_DEREF).
This sample was valid in 2.6, but fails to compile in 3.x without this change::
>>> def f():
... def print_error():
... print(e)
... try:
... something
... except Exception as e:
... print_error()
... # implicit "del e" here
This sample has always been invalid in Python, and now works::
>>> def outer(x):
... def inner():
... return x
... inner()
... del x
There is no need to bump the PYC magic number: the new opcode is used
for code that did not compile before.
svn+ssh://pythondev@svn.python.org/python/trunk
........
r82169 | antoine.pitrou | 2010-06-22 23:42:05 +0200 (mar., 22 juin 2010) | 4 lines
Fix misindents in compile.c (for Benjamin).
Of course, whoever used the wrong indentation rules needs to be spanked.
........
svn+ssh://pythondev@svn.python.org/python/branches/py3k
........
r82059 | victor.stinner | 2010-06-18 01:08:50 +0200 (ven., 18 juin 2010) | 5 lines
Issue #6543: Write the traceback in the terminal encoding instead of utf-8.
Fix the encoding of the modules filename.
Reindent also traceback.h, just because I hate tabs :-)
........
r82061 | victor.stinner | 2010-06-18 01:17:37 +0200 (ven., 18 juin 2010) | 2 lines
Issue #6543: Mention the author of the patch, Amaury Forgeot d'Arc
........
svn+ssh://pythondev@svn.python.org/python/branches/py3k
................
r76577 | mark.dickinson | 2009-11-28 16:38:16 +0000 (Sat, 28 Nov 2009) | 12 lines
Merged revisions 76575 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r76575 | mark.dickinson | 2009-11-28 16:32:27 +0000 (Sat, 28 Nov 2009) | 5 lines
Issue #1678380: When distinguishing between -0.0 and 0.0 in
compiler_add_o, use copysign instead of examining the first and last
bytes of the double. The latter method fails for little-endian
ARM, OABI, where doubles are little-endian but with the words swapped.
........
................
svn+ssh://pythondev@svn.python.org/python/trunk
........
r76575 | mark.dickinson | 2009-11-28 16:32:27 +0000 (Sat, 28 Nov 2009) | 5 lines
Issue #1678380: When distinguishing between -0.0 and 0.0 in
compiler_add_o, use copysign instead of examining the first and last
bytes of the double. The latter method fails for little-endian
ARM, OABI, where doubles are little-endian but with the words swapped.
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r75264 | andrew.kuchling | 2009-10-05 17:30:22 -0500 (Mon, 05 Oct 2009) | 1 line
Add various items
........
r75268 | andrew.kuchling | 2009-10-05 17:45:39 -0500 (Mon, 05 Oct 2009) | 1 line
Remove two notes
........
r75293 | kristjan.jonsson | 2009-10-09 09:32:19 -0500 (Fri, 09 Oct 2009) | 2 lines
http://bugs.python.org/issue7029
a non-default timer wasn't actually used by the individual Tests.
........
r75318 | benjamin.peterson | 2009-10-10 16:15:58 -0500 (Sat, 10 Oct 2009) | 1 line
remove script which uses long gone module
........
r75391 | andrew.kuchling | 2009-10-13 10:49:33 -0500 (Tue, 13 Oct 2009) | 1 line
Link to PEP
........
r75392 | andrew.kuchling | 2009-10-13 11:11:49 -0500 (Tue, 13 Oct 2009) | 1 line
Various link, textual, and markup fixes
........
r75436 | benjamin.peterson | 2009-10-15 10:39:15 -0500 (Thu, 15 Oct 2009) | 1 line
don't need to mess up sys.path
........
r75478 | senthil.kumaran | 2009-10-17 20:58:45 -0500 (Sat, 17 Oct 2009) | 3 lines
Fix a typo.
........
r75971 | benjamin.peterson | 2009-10-30 22:56:15 -0500 (Fri, 30 Oct 2009) | 1 line
add some checks for evaluation order with parenthesis #7210
........
r76003 | antoine.pitrou | 2009-10-31 19:30:13 -0500 (Sat, 31 Oct 2009) | 6 lines
Hopefully fix the buildbot problems on test_mailbox, by computing
the maildir toc cache refresh date before actually refreshing the cache.
(see #6896)
........
r76058 | benjamin.peterson | 2009-11-02 10:14:19 -0600 (Mon, 02 Nov 2009) | 1 line
grant list.index() a more informative error message #7252
........
r76140 | nick.coghlan | 2009-11-07 02:13:55 -0600 (Sat, 07 Nov 2009) | 1 line
Add test for runpy.run_module package execution and use something other than logging as the example of a non-executable package
........
r76141 | nick.coghlan | 2009-11-07 02:15:01 -0600 (Sat, 07 Nov 2009) | 1 line
Some minor cleanups to private runpy code and docstrings
........
r76231 | benjamin.peterson | 2009-11-12 17:42:23 -0600 (Thu, 12 Nov 2009) | 1 line
this main is much more useful
........
r76380 | antoine.pitrou | 2009-11-18 14:20:46 -0600 (Wed, 18 Nov 2009) | 3 lines
Mention Giampolo R's new FTP TLS support in the what's new file
........
r76428 | benjamin.peterson | 2009-11-19 20:15:50 -0600 (Thu, 19 Nov 2009) | 1 line
turn goto into do while loop
........
r76429 | benjamin.peterson | 2009-11-19 20:56:43 -0600 (Thu, 19 Nov 2009) | 2 lines
avoid doing an uneeded import in a function
........
svn+ssh://pythondev@svn.python.org/python/trunk
........
r72487 | jeffrey.yasskin | 2009-05-08 17:51:06 -0400 (Fri, 08 May 2009) | 7 lines
PyCode_NewEmpty:
Most uses of PyCode_New found by http://www.google.com/codesearch?q=PyCode_New
are trying to build an empty code object, usually to put it in a dummy frame
object. This patch adds a PyCode_NewEmpty wrapper which lets the user specify
just the filename, function name, and first line number, instead of also
requiring lots of code internals.
........
r72488 | jeffrey.yasskin | 2009-05-08 18:23:21 -0400 (Fri, 08 May 2009) | 13 lines
Issue 5954, PyFrame_GetLineNumber:
Most uses of PyCode_Addr2Line
(http://www.google.com/codesearch?q=PyCode_Addr2Line) are just trying to get
the line number of a specified frame, but there's no way to do that directly.
Forcing people to go through the code object makes them know more about the
guts of the interpreter than they should need.
The remaining uses of PyCode_Addr2Line seem to be getting the line from a
traceback (for example,
http://www.google.com/codesearch/p?hl=en#u_9_nDrchrw/pygame-1.7.1release/src/base.c&q=PyCode_Addr2Line),
which is replaced by the tb_lineno field. So we may be able to deprecate
PyCode_Addr2Line entirely for external use.
........
r72879 | jeffrey.yasskin | 2009-05-23 19:23:01 -0400 (Sat, 23 May 2009) | 14 lines
Issue #6042:
lnotab-based tracing is very complicated and isn't documented very well. There
were at least 3 comment blocks purporting to document co_lnotab, and none did a
very good job. This patch unifies them into Objects/lnotab_notes.txt which
tries to completely capture the current state of affairs.
I also discovered that we've attached 2 layers of patches to the basic tracing
scheme. The first layer avoids jumping to instructions that don't start a line,
to avoid problems in if statements and while loops. The second layer
discovered that jumps backward do need to trace at instructions that don't
start a line, so it added extra lnotab entries for 'while' and 'for' loops, and
added a special case for backward jumps within the same line. I replaced these
patches by just treating forward and backward jumps differently.
........
Serhiy Storchaka
Mark Shannon
Guido van Rossum
INADA Naoki
Nick Coghlan
Yury Selivanov
Barry Warsaw
Stefan Krah
Matthieu Dartiailh
Guido van Rossum
Victor Stinner
Benjamin Peterson
Ned Deily
Eric V. Smith
Martin Panter
Yury Selivanov
Antoine Pitrou
Christian Heimes
Larry Hastings
Ezio Melotti
Mark Dickinson
Meador Inge
Victor Stinner
Martin v. Löwis
Georg Brandl
Amaury Forgeot d'Arc
Mark Dickinson
Neil Schemenauer
Alexandre Vassalotti