traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

bpo-17611. Move unwinding of stack for "pseudo exceptions" from interpreter to compiler. (GH-5006) 

Co-authored-by: Mark Shannon <mark@hotpy.org>
Co-authored-by: Antoine Pitrou <antoine@python.org>
This commit is contained in:
Serhiy Storchaka 2018-02-22 23:33:30 +02:00 committed by GitHub
parent 4af8fd5614
commit 520b7ae27e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
19 changed files with 4506 additions and 4392 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

126
Doc/library/dis.rst
View File

@ -335,6 +335,14 @@ The Python compiler currently generates the following bytecode instructions.
three.
.. opcode:: ROT_FOUR
Lifts second, third and forth stack items one position up, moves top down
to position four.
.. versionadded:: 3.8
.. opcode:: DUP_TOP
Duplicates the reference on top of the stack.
@ -605,17 +613,6 @@ the original TOS1.
is terminated with :opcode:`POP_TOP`.
.. opcode:: BREAK_LOOP
Terminates a loop due to a :keyword:`break` statement.
.. opcode:: CONTINUE_LOOP (target)
Continues a loop due to a :keyword:`continue` statement. *target* is the
address to jump to (which should be a :opcode:`FOR_ITER` instruction).
.. opcode:: SET_ADD (i)
Calls ``set.add(TOS1[-i], TOS)``. Used to implement set comprehensions.
@ -676,7 +673,7 @@ iterations of the loop.
.. opcode:: POP_BLOCK
Removes one block from the block stack. Per frame, there is a stack of
blocks, denoting nested loops, try statements, and such.
blocks, denoting :keyword:`try` statements, and such.
.. opcode:: POP_EXCEPT
@ -687,11 +684,50 @@ iterations of the loop.
popped values are used to restore the exception state.
.. opcode:: POP_FINALLY (preserve_tos)
Cleans up the value stack and the block stack. If *preserve_tos* is not
``0`` TOS first is popped from the stack and pushed on the stack after
perfoming other stack operations:
* If TOS is ``NULL`` or an integer (pushed by :opcode:`BEGIN_FINALLY`
or :opcode:`CALL_FINALLY`) it is popped from the stack.
* If TOS is an exception type (pushed when an exception has been raised)
6 values are popped from the stack, the last three popped values are
used to restore the exception state. An exception handler block is
removed from the block stack.
It is similar to :opcode:`END_FINALLY`, but doesn't change the bytecode
counter nor raise an exception. Used for implementing :keyword:`break`
and :keyword:`return` in the :keyword:`finally` block.
.. versionadded:: 3.8
.. opcode:: BEGIN_FINALLY
Pushes ``NULL`` onto the stack for using it in :opcode:`END_FINALLY`,
:opcode:`POP_FINALLY`, :opcode:`WITH_CLEANUP_START` and
:opcode:`WITH_CLEANUP_FINISH`. Starts the :keyword:`finally` block.
.. versionadded:: 3.8
.. opcode:: END_FINALLY
Terminates a :keyword:`finally` clause. The interpreter recalls whether the
exception has to be re-raised, or whether the function returns, and continues
with the outer-next block.
exception has to be re-raised or execution has to be continued depending on
the value of TOS.
* If TOS is ``NULL`` (pushed by :opcode:`BEGIN_FINALLY`) continue from
the next instruction. TOS is popped.
* If TOS is an integer (pushed by :opcode:`CALL_FINALLY`), sets the
bytecode counter to TOS. TOS is popped.
* If TOS is an exception type (pushed when an exception has been raised)
6 values are popped from the stack, the first three popped values are
used to re-raise the exception and the last three popped values are used
to restore the exception state. An exception handler block is removed
from the block stack.
.. opcode:: LOAD_BUILD_CLASS
@ -704,9 +740,9 @@ iterations of the loop.
This opcode performs several operations before a with block starts. First,
it loads :meth:`~object.__exit__` from the context manager and pushes it onto
the stack for later use by :opcode:`WITH_CLEANUP`. Then,
the stack for later use by :opcode:`WITH_CLEANUP_START`. Then,
:meth:`~object.__enter__` is called, and a finally block pointing to *delta*
is pushed. Finally, the result of calling the enter method is pushed onto
is pushed. Finally, the result of calling the ``__enter__()`` method is pushed onto
the stack. The next opcode will either ignore it (:opcode:`POP_TOP`), or
store it in (a) variable(s) (:opcode:`STORE_FAST`, :opcode:`STORE_NAME`, or
:opcode:`UNPACK_SEQUENCE`).
@ -716,30 +752,31 @@ iterations of the loop.
.. opcode:: WITH_CLEANUP_START
Cleans up the stack when a :keyword:`with` statement block exits. TOS is the
context manager's :meth:`__exit__` bound method. Below TOS are 1--3 values
indicating how/why the finally clause was entered:
Starts cleaning up the stack when a :keyword:`with` statement block exits.
* SECOND = ``None``
* (SECOND, THIRD) = (``WHY_{RETURN,CONTINUE}``), retval
* SECOND = ``WHY_*``; no retval below it
* (SECOND, THIRD, FOURTH) = exc_info()
At the top of the stack are either ``NULL`` (pushed by
:opcode:`BEGIN_FINALLY`) or 6 values pushed if an exception has been
raised in the with block. Below is the context manager's
:meth:`~object.__exit__` or :meth:`~object.__aexit__` bound method.
In the last case, ``TOS(SECOND, THIRD, FOURTH)`` is called, otherwise
``TOS(None, None, None)``. Pushes SECOND and result of the call
to the stack.
If TOS is ``NULL``, calls ``SECOND(None, None, None)``,
removes the function from the stack, leaving TOS, and pushes ``None``
to the stack. Otherwise calls ``SEVENTH(TOP, SECOND, THIRD)``,
shifts the bottom 3 values of the stack down, replaces the empty spot
with ``NULL`` and pushes TOS. Finally pushes the result of the call.
.. opcode:: WITH_CLEANUP_FINISH
Pops exception type and result of 'exit' function call from the stack.
Finishes cleaning up the stack when a :keyword:`with` statement block exits.
If the stack represents an exception, *and* the function call returns a
'true' value, this information is "zapped" and replaced with a single
``WHY_SILENCED`` to prevent :opcode:`END_FINALLY` from re-raising the
exception. (But non-local gotos will still be resumed.)
TOS is result of ``__exit__()`` or ``__aexit__()`` function call pushed
by :opcode:`WITH_CLEANUP_START`. SECOND is ``None`` or an exception type
(pushed when an exception has been raised).
.. XXX explain the WHY stuff!
Pops two values from the stack. If SECOND is not None and TOS is true
unwinds the EXCEPT_HANDLER block which was created when the exception
was caught and pushes ``NULL`` to the stack.
All of the following opcodes use their arguments.
@ -987,22 +1024,19 @@ All of the following opcodes use their arguments.
Loads the global named ``co_names[namei]`` onto the stack.
.. opcode:: SETUP_LOOP (delta)
Pushes a block for a loop onto the block stack. The block spans from the
current instruction with a size of *delta* bytes.
.. opcode:: SETUP_EXCEPT (delta)
Pushes a try block from a try-except clause onto the block stack. *delta*
points to the first except block.
.. opcode:: SETUP_FINALLY (delta)
Pushes a try block from a try-except clause onto the block stack. *delta*
points to the finally block.
Pushes a try block from a try-finally or try-except clause onto the block
stack. *delta* points to the finally block or the first except block.
.. opcode:: CALL_FINALLY (delta)
Pushes the address of the next instruction onto the stack and increments
bytecode counter by *delta*. Used for calling the finally block as a
"subroutine".
.. versionadded:: 3.8
.. opcode:: LOAD_FAST (var_num)

18
Doc/whatsnew/3.8.rst
View File

@ -137,3 +137,21 @@ Changes in the Python API
:func:`dbm.dumb.open` with flags ``'r'`` and ``'w'`` no longer creates
a database if it does not exist.
(Contributed by Serhiy Storchaka in :issue:`32749`.)
CPython bytecode changes
------------------------
* The interpreter loop has been simplified by moving the logic of unrolling
the stack of blocks into the compiler. The compiler emits now explicit
instructions for adjusting the stack of values and calling the cleaning
up code for :keyword:`break`, :keyword:`continue` and :keyword:`return`.
Removed opcodes :opcode:`BREAK_LOOP`, :opcode:`CONTINUE_LOOP`,
:opcode:`SETUP_LOOP` and :opcode:`SETUP_EXCEPT`. Added new opcodes
:opcode:`ROT_FOUR`, :opcode:`BEGIN_FINALLY`, :opcode:`CALL_FINALLY` and
:opcode:`POP_FINALLY`. Changed the behavior of :opcode:`END_FINALLY`
and :opcode:`WITH_CLEANUP_START`.
(Contributed by Mark Shannon, Antoine Pitrou and Serhiy Storchaka in
:issue:`17611`.)

8
Include/opcode.h
View File

@ -12,6 +12,7 @@ extern "C" {
#define ROT_THREE 3
#define DUP_TOP 4
#define DUP_TOP_TWO 5
#define ROT_FOUR 6
#define NOP 9
#define UNARY_POSITIVE 10
#define UNARY_NEGATIVE 11
@ -32,6 +33,7 @@ extern "C" {
#define GET_AITER 50
#define GET_ANEXT 51
#define BEFORE_ASYNC_WITH 52
#define BEGIN_FINALLY 53
#define INPLACE_ADD 55
#define INPLACE_SUBTRACT 56
#define INPLACE_MULTIPLY 57
@ -55,7 +57,6 @@ extern "C" {
#define INPLACE_AND 77
#define INPLACE_XOR 78
#define INPLACE_OR 79
#define BREAK_LOOP 80
#define WITH_CLEANUP_START 81
#define WITH_CLEANUP_FINISH 82
#define RETURN_VALUE 83
@ -92,9 +93,6 @@ extern "C" {
#define POP_JUMP_IF_FALSE 114
#define POP_JUMP_IF_TRUE 115
#define LOAD_GLOBAL 116
#define CONTINUE_LOOP 119
#define SETUP_LOOP 120
#define SETUP_EXCEPT 121
#define SETUP_FINALLY 122
#define LOAD_FAST 124
#define STORE_FAST 125
@ -127,6 +125,8 @@ extern "C" {
#define BUILD_TUPLE_UNPACK_WITH_CALL 158
#define LOAD_METHOD 160
#define CALL_METHOD 161
#define CALL_FINALLY 162
#define POP_FINALLY 163
/* EXCEPT_HANDLER is a special, implicit block type which is created when
entering an except handler. It is not an opcode but we define it here

3
Lib/importlib/_bootstrap_external.py
View File

@ -246,6 +246,7 @@ _code_type = type(_write_atomic.__code__)
# Python 3.7a2 3391 (update GET_AITER #31709)
# Python 3.7a4 3392 (PEP 552: Deterministic pycs #31650)
# Python 3.7b1 3393 (remove STORE_ANNOTATION opcode #32550)
# Python 3.8a1 3400 (move frame block handling to compiler #17611)
#
# MAGIC must change whenever the bytecode emitted by the compiler may no
# longer be understood by older implementations of the eval loop (usually
@ -254,7 +255,7 @@ _code_type = type(_write_atomic.__code__)
# Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array
# in PC/launcher.c must also be updated.
MAGIC_NUMBER = (3393).to_bytes(2, 'little') + b'\r\n'
MAGIC_NUMBER = (3400).to_bytes(2, 'little') + b'\r\n'
_RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c
_PYCACHE = '__pycache__'

11
Lib/opcode.py
View File

@ -60,6 +60,7 @@ def_op('ROT_TWO', 2)
def_op('ROT_THREE', 3)
def_op('DUP_TOP', 4)
def_op('DUP_TOP_TWO', 5)
def_op('ROT_FOUR', 6)
def_op('NOP', 9)
def_op('UNARY_POSITIVE', 10)
@ -86,6 +87,7 @@ def_op('INPLACE_TRUE_DIVIDE', 29)
def_op('GET_AITER', 50)
def_op('GET_ANEXT', 51)
def_op('BEFORE_ASYNC_WITH', 52)
def_op('BEGIN_FINALLY', 53)
def_op('INPLACE_ADD', 55)
def_op('INPLACE_SUBTRACT', 56)
@ -113,10 +115,8 @@ def_op('INPLACE_RSHIFT', 76)
def_op('INPLACE_AND', 77)
def_op('INPLACE_XOR', 78)
def_op('INPLACE_OR', 79)
def_op('BREAK_LOOP', 80)
def_op('WITH_CLEANUP_START', 81)
def_op('WITH_CLEANUP_FINISH', 82)
def_op('RETURN_VALUE', 83)
def_op('IMPORT_STAR', 84)
def_op('SETUP_ANNOTATIONS', 85)
@ -158,10 +158,7 @@ jabs_op('POP_JUMP_IF_TRUE', 115) # ""
name_op('LOAD_GLOBAL', 116) # Index in name list
jabs_op('CONTINUE_LOOP', 119) # Target address
jrel_op('SETUP_LOOP', 120) # Distance to target address
jrel_op('SETUP_EXCEPT', 121) # ""
jrel_op('SETUP_FINALLY', 122) # ""
jrel_op('SETUP_FINALLY', 122) # Distance to target address
def_op('LOAD_FAST', 124) # Local variable number
haslocal.append(124)
@ -213,5 +210,7 @@ def_op('BUILD_TUPLE_UNPACK_WITH_CALL', 158)
name_op('LOAD_METHOD', 160)
def_op('CALL_METHOD', 161)
jrel_op('CALL_FINALLY', 162)
def_op('POP_FINALLY', 163)
del def_op, name_op, jrel_op, jabs_op

243
Lib/test/test_dis.py
View File

@ -120,20 +120,18 @@ def bug708901():
pass
dis_bug708901 = """\
%3d 0 SETUP_LOOP 18 (to 20)
2 LOAD_GLOBAL 0 (range)
4 LOAD_CONST 1 (1)
%3d 0 LOAD_GLOBAL 0 (range)
2 LOAD_CONST 1 (1)
%3d 6 LOAD_CONST 2 (10)
8 CALL_FUNCTION 2
10 GET_ITER
>> 12 FOR_ITER 4 (to 18)
14 STORE_FAST 0 (res)
%3d 4 LOAD_CONST 2 (10)
6 CALL_FUNCTION 2
8 GET_ITER
>> 10 FOR_ITER 4 (to 16)
12 STORE_FAST 0 (res)
%3d 16 JUMP_ABSOLUTE 12
>> 18 POP_BLOCK
>> 20 LOAD_CONST 0 (None)
22 RETURN_VALUE
%3d 14 JUMP_ABSOLUTE 10
>> 16 LOAD_CONST 0 (None)
18 RETURN_VALUE
""" % (bug708901.__code__.co_firstlineno + 1,
bug708901.__code__.co_firstlineno + 2,
bug708901.__code__.co_firstlineno + 3)
@ -259,20 +257,17 @@ dis_compound_stmt_str = """\
1 0 LOAD_CONST 0 (0)
2 STORE_NAME 0 (x)
2 4 SETUP_LOOP 12 (to 18)
3 >> 6 LOAD_NAME 0 (x)
8 LOAD_CONST 1 (1)
10 INPLACE_ADD
12 STORE_NAME 0 (x)
14 JUMP_ABSOLUTE 6
16 POP_BLOCK
>> 18 LOAD_CONST 2 (None)
20 RETURN_VALUE
3 >> 4 LOAD_NAME 0 (x)
6 LOAD_CONST 1 (1)
8 INPLACE_ADD
10 STORE_NAME 0 (x)
12 JUMP_ABSOLUTE 4
14 LOAD_CONST 2 (None)
16 RETURN_VALUE
"""
dis_traceback = """\
%3d 0 SETUP_EXCEPT 12 (to 14)
%3d 0 SETUP_FINALLY 12 (to 14)
%3d 2 LOAD_CONST 1 (1)
4 LOAD_CONST 2 (0)
@ -294,7 +289,7 @@ dis_traceback = """\
32 LOAD_ATTR 1 (__traceback__)
34 STORE_FAST 1 (tb)
36 POP_BLOCK
38 LOAD_CONST 0 (None)
38 BEGIN_FINALLY
>> 40 LOAD_CONST 0 (None)
42 STORE_FAST 0 (e)
44 DELETE_FAST 0 (e)
@ -749,7 +744,14 @@ Stack size: 10
Flags: OPTIMIZED, NEWLOCALS, NOFREE, COROUTINE
Constants:
0: None
1: 1"""
1: 1
Names:
0: b
1: StopAsyncIteration
2: c
Variable names:
0: a
1: d"""
class CodeInfoTests(unittest.TestCase):
test_pairs = [
@ -913,103 +915,100 @@ expected_opinfo_inner = [
]
expected_opinfo_jumpy = [
Instruction(opname='SETUP_LOOP', opcode=120, arg=52, argval=54, argrepr='to 54', offset=0, starts_line=3, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=0, argval='range', argrepr='range', offset=2, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=1, argval=10, argrepr='10', offset=4, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=6, starts_line=None, is_jump_target=False),
Instruction(opname='GET_ITER', opcode=68, arg=None, argval=None, argrepr='', offset=8, starts_line=None, is_jump_target=False),
Instruction(opname='FOR_ITER', opcode=93, arg=32, argval=44, argrepr='to 44', offset=10, starts_line=None, is_jump_target=True),
Instruction(opname='STORE_FAST', opcode=125, arg=0, argval='i', argrepr='i', offset=12, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=14, starts_line=4, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=16, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=18, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=20, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=22, starts_line=5, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=24, starts_line=None, is_jump_target=False),
Instruction(opname='COMPARE_OP', opcode=107, arg=0, argval='<', argrepr='<', offset=26, starts_line=None, is_jump_target=False),
Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=32, argval=32, argrepr='', offset=28, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=10, argval=10, argrepr='', offset=30, starts_line=6, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=32, starts_line=7, is_jump_target=True),
Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=6, argrepr='6', offset=34, starts_line=None, is_jump_target=False),
Instruction(opname='COMPARE_OP', opcode=107, arg=4, argval='>', argrepr='>', offset=36, starts_line=None, is_jump_target=False),
Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=10, argval=10, argrepr='', offset=38, starts_line=None, is_jump_target=False),
Instruction(opname='BREAK_LOOP', opcode=80, arg=None, argval=None, argrepr='', offset=40, starts_line=8, is_jump_target=False),
Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=10, argval=10, argrepr='', offset=42, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=44, starts_line=None, is_jump_target=True),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=46, starts_line=10, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=4, argval='I can haz else clause?', argrepr="'I can haz else clause?'", offset=48, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=50, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=52, starts_line=None, is_jump_target=False),
Instruction(opname='SETUP_LOOP', opcode=120, arg=52, argval=108, argrepr='to 108', offset=54, starts_line=11, is_jump_target=True),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=56, starts_line=None, is_jump_target=True),
Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=98, argval=98, argrepr='', offset=58, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=60, starts_line=12, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=62, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=64, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=66, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=68, starts_line=13, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=70, starts_line=None, is_jump_target=False),
Instruction(opname='INPLACE_SUBTRACT', opcode=56, arg=None, argval=None, argrepr='', offset=72, starts_line=None, is_jump_target=False),
Instruction(opname='STORE_FAST', opcode=125, arg=0, argval='i', argrepr='i', offset=74, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=76, starts_line=14, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=6, argrepr='6', offset=78, starts_line=None, is_jump_target=False),
Instruction(opname='COMPARE_OP', opcode=107, arg=4, argval='>', argrepr='>', offset=80, starts_line=None, is_jump_target=False),
Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=86, argval=86, argrepr='', offset=82, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=56, argval=56, argrepr='', offset=84, starts_line=15, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=86, starts_line=16, is_jump_target=True),
Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=88, starts_line=None, is_jump_target=False),
Instruction(opname='COMPARE_OP', opcode=107, arg=0, argval='<', argrepr='<', offset=90, starts_line=None, is_jump_target=False),
Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=56, argval=56, argrepr='', offset=92, starts_line=None, is_jump_target=False),
Instruction(opname='BREAK_LOOP', opcode=80, arg=None, argval=None, argrepr='', offset=94, starts_line=17, is_jump_target=False),
Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=56, argval=56, argrepr='', offset=96, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=98, starts_line=None, is_jump_target=True),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=100, starts_line=19, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Who let lolcatz into this test suite?', argrepr="'Who let lolcatz into this test suite?'", offset=102, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=104, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=106, starts_line=None, is_jump_target=False),
Instruction(opname='SETUP_FINALLY', opcode=122, arg=70, argval=180, argrepr='to 180', offset=108, starts_line=20, is_jump_target=True),
Instruction(opname='SETUP_EXCEPT', opcode=121, arg=12, argval=124, argrepr='to 124', offset=110, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=112, starts_line=21, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=7, argval=0, argrepr='0', offset=114, starts_line=None, is_jump_target=False),
Instruction(opname='BINARY_TRUE_DIVIDE', opcode=27, arg=None, argval=None, argrepr='', offset=116, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=118, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=120, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=28, argval=152, argrepr='to 152', offset=122, starts_line=None, is_jump_target=False),
Instruction(opname='DUP_TOP', opcode=4, arg=None, argval=None, argrepr='', offset=124, starts_line=22, is_jump_target=True),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=2, argval='ZeroDivisionError', argrepr='ZeroDivisionError', offset=126, starts_line=None, is_jump_target=False),
Instruction(opname='COMPARE_OP', opcode=107, arg=10, argval='exception match', argrepr='exception match', offset=128, starts_line=None, is_jump_target=False),
Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=150, argval=150, argrepr='', offset=130, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=132, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=134, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=136, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=138, starts_line=23, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=8, argval='Here we go, here we go, here we go...', argrepr="'Here we go, here we go, here we go...'", offset=140, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=142, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=144, starts_line=None, is_jump_target=False),
Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=146, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=26, argval=176, argrepr='to 176', offset=148, starts_line=None, is_jump_target=False),
Instruction(opname='END_FINALLY', opcode=88, arg=None, argval=None, argrepr='', offset=150, starts_line=None, is_jump_target=True),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=152, starts_line=25, is_jump_target=True),
Instruction(opname='SETUP_WITH', opcode=143, arg=14, argval=170, argrepr='to 170', offset=154, starts_line=None, is_jump_target=False),
Instruction(opname='STORE_FAST', opcode=125, arg=1, argval='dodgy', argrepr='dodgy', offset=156, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=158, starts_line=26, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=9, argval='Never reach this', argrepr="'Never reach this'", offset=160, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=162, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=164, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=166, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=168, starts_line=None, is_jump_target=False),
Instruction(opname='WITH_CLEANUP_START', opcode=81, arg=None, argval=None, argrepr='', offset=170, starts_line=None, is_jump_target=True),
Instruction(opname='WITH_CLEANUP_FINISH', opcode=82, arg=None, argval=None, argrepr='', offset=172, starts_line=None, is_jump_target=False),
Instruction(opname='END_FINALLY', opcode=88, arg=None, argval=None, argrepr='', offset=174, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=176, starts_line=None, is_jump_target=True),
Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=178, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=180, starts_line=28, is_jump_target=True),
Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=182, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=184, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=186, starts_line=None, is_jump_target=False),
Instruction(opname='END_FINALLY', opcode=88, arg=None, argval=None, argrepr='', offset=188, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=190, starts_line=None, is_jump_target=False),
Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=192, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=0, argval='range', argrepr='range', offset=0, starts_line=3, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=1, argval=10, argrepr='10', offset=2, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=4, starts_line=None, is_jump_target=False),
Instruction(opname='GET_ITER', opcode=68, arg=None, argval=None, argrepr='', offset=6, starts_line=None, is_jump_target=False),
Instruction(opname='FOR_ITER', opcode=93, arg=34, argval=44, argrepr='to 44', offset=8, starts_line=None, is_jump_target=True),
Instruction(opname='STORE_FAST', opcode=125, arg=0, argval='i', argrepr='i', offset=10, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=12, starts_line=4, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=14, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=16, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=18, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=20, starts_line=5, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=22, starts_line=None, is_jump_target=False),
Instruction(opname='COMPARE_OP', opcode=107, arg=0, argval='<', argrepr='<', offset=24, starts_line=None, is_jump_target=False),
Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=30, argval=30, argrepr='', offset=26, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=8, argval=8, argrepr='', offset=28, starts_line=6, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=30, starts_line=7, is_jump_target=True),
Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=6, argrepr='6', offset=32, starts_line=None, is_jump_target=False),
Instruction(opname='COMPARE_OP', opcode=107, arg=4, argval='>', argrepr='>', offset=34, starts_line=None, is_jump_target=False),
Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=8, argval=8, argrepr='', offset=36, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=38, starts_line=8, is_jump_target=False),
Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=52, argval=52, argrepr='', offset=40, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=8, argval=8, argrepr='', offset=42, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=44, starts_line=10, is_jump_target=True),
Instruction(opname='LOAD_CONST', opcode=100, arg=4, argval='I can haz else clause?', argrepr="'I can haz else clause?'", offset=46, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=48, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=50, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=52, starts_line=11, is_jump_target=True),
Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=94, argval=94, argrepr='', offset=54, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=56, starts_line=12, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=58, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=60, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=62, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=64, starts_line=13, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=66, starts_line=None, is_jump_target=False),
Instruction(opname='INPLACE_SUBTRACT', opcode=56, arg=None, argval=None, argrepr='', offset=68, starts_line=None, is_jump_target=False),
Instruction(opname='STORE_FAST', opcode=125, arg=0, argval='i', argrepr='i', offset=70, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=72, starts_line=14, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=3, argval=6, argrepr='6', offset=74, starts_line=None, is_jump_target=False),
Instruction(opname='COMPARE_OP', opcode=107, arg=4, argval='>', argrepr='>', offset=76, starts_line=None, is_jump_target=False),
Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=82, argval=82, argrepr='', offset=78, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=52, argval=52, argrepr='', offset=80, starts_line=15, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=82, starts_line=16, is_jump_target=True),
Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=4, argrepr='4', offset=84, starts_line=None, is_jump_target=False),
Instruction(opname='COMPARE_OP', opcode=107, arg=0, argval='<', argrepr='<', offset=86, starts_line=None, is_jump_target=False),
Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=52, argval=52, argrepr='', offset=88, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=102, argval=102, argrepr='', offset=90, starts_line=17, is_jump_target=False),
Instruction(opname='JUMP_ABSOLUTE', opcode=113, arg=52, argval=52, argrepr='', offset=92, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=94, starts_line=19, is_jump_target=True),
Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Who let lolcatz into this test suite?', argrepr="'Who let lolcatz into this test suite?'", offset=96, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=98, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=100, starts_line=None, is_jump_target=False),
Instruction(opname='SETUP_FINALLY', opcode=122, arg=70, argval=174, argrepr='to 174', offset=102, starts_line=20, is_jump_target=True),
Instruction(opname='SETUP_FINALLY', opcode=122, arg=12, argval=118, argrepr='to 118', offset=104, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=106, starts_line=21, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=7, argval=0, argrepr='0', offset=108, starts_line=None, is_jump_target=False),
Instruction(opname='BINARY_TRUE_DIVIDE', opcode=27, arg=None, argval=None, argrepr='', offset=110, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=112, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=114, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=28, argval=146, argrepr='to 146', offset=116, starts_line=None, is_jump_target=False),
Instruction(opname='DUP_TOP', opcode=4, arg=None, argval=None, argrepr='', offset=118, starts_line=22, is_jump_target=True),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=2, argval='ZeroDivisionError', argrepr='ZeroDivisionError', offset=120, starts_line=None, is_jump_target=False),
Instruction(opname='COMPARE_OP', opcode=107, arg=10, argval='exception match', argrepr='exception match', offset=122, starts_line=None, is_jump_target=False),
Instruction(opname='POP_JUMP_IF_FALSE', opcode=114, arg=144, argval=144, argrepr='', offset=124, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=126, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=128, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=130, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=132, starts_line=23, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=8, argval='Here we go, here we go, here we go...', argrepr="'Here we go, here we go, here we go...'", offset=134, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=136, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=138, starts_line=None, is_jump_target=False),
Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=140, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=26, argval=170, argrepr='to 170', offset=142, starts_line=None, is_jump_target=False),
Instruction(opname='END_FINALLY', opcode=88, arg=None, argval=None, argrepr='', offset=144, starts_line=None, is_jump_target=True),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=146, starts_line=25, is_jump_target=True),
Instruction(opname='SETUP_WITH', opcode=143, arg=14, argval=164, argrepr='to 164', offset=148, starts_line=None, is_jump_target=False),
Instruction(opname='STORE_FAST', opcode=125, arg=1, argval='dodgy', argrepr='dodgy', offset=150, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=152, starts_line=26, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=9, argval='Never reach this', argrepr="'Never reach this'", offset=154, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=156, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=158, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=160, starts_line=None, is_jump_target=False),
Instruction(opname='BEGIN_FINALLY', opcode=53, arg=None, argval=None, argrepr='', offset=162, starts_line=None, is_jump_target=False),
Instruction(opname='WITH_CLEANUP_START', opcode=81, arg=None, argval=None, argrepr='', offset=164, starts_line=None, is_jump_target=True),
Instruction(opname='WITH_CLEANUP_FINISH', opcode=82, arg=None, argval=None, argrepr='', offset=166, starts_line=None, is_jump_target=False),
Instruction(opname='END_FINALLY', opcode=88, arg=None, argval=None, argrepr='', offset=168, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=170, starts_line=None, is_jump_target=True),
Instruction(opname='BEGIN_FINALLY', opcode=53, arg=None, argval=None, argrepr='', offset=172, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=174, starts_line=28, is_jump_target=True),
Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=176, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=178, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=180, starts_line=None, is_jump_target=False),
Instruction(opname='END_FINALLY', opcode=88, arg=None, argval=None, argrepr='', offset=182, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=184, starts_line=None, is_jump_target=False),
Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=186, starts_line=None, is_jump_target=False),
]
# One last piece of inspect fodder to check the default line number handling

2
Lib/test/test_importlib/test_util.py
View File

@ -789,7 +789,7 @@ class MagicNumberTests(unittest.TestCase):
in advance. Such exceptional releases will then require an
adjustment to this test case.
"""
EXPECTED_MAGIC_NUMBER = 3379
EXPECTED_MAGIC_NUMBER = 3400
actual = int.from_bytes(importlib.util.MAGIC_NUMBER[:2], 'little')
msg = (

4
Lib/test/test_peepholer.py
View File

@ -268,7 +268,7 @@ class TestTranforms(BytecodeTestCase):
self.assertNotInBytecode(f, 'JUMP_ABSOLUTE')
returns = [instr for instr in dis.get_instructions(f)
if instr.opname == 'RETURN_VALUE']
self.assertEqual(len(returns), 6)
self.assertLessEqual(len(returns), 6)
def test_elim_jump_after_return2(self):
# Eliminate dead code: jumps immediately after returns can't be reached
@ -282,7 +282,7 @@ class TestTranforms(BytecodeTestCase):
self.assertEqual(len(returns), 1)
returns = [instr for instr in dis.get_instructions(f)
if instr.opname == 'RETURN_VALUE']
self.assertEqual(len(returns), 2)
self.assertLessEqual(len(returns), 2)
def test_make_function_doesnt_bail(self):
def f():

58
Lib/test/test_sys_settrace.py
View File

@ -187,7 +187,6 @@ tightloop_example.events = [(0, 'call'),
(1, 'line'),
(2, 'line'),
(3, 'line'),
(4, 'line'),
(5, 'line'),
(5, 'line'),
(5, 'line'),
@ -715,6 +714,21 @@ class JumpTestCase(unittest.TestCase):
output.append(11)
output.append(12)
@jump_test(5, 11, [2, 4, 12])
def test_jump_over_return_try_finally_in_finally_block(output):
try:
output.append(2)
finally:
output.append(4)
output.append(5)
return
try:
output.append(8)
finally:
output.append(10)
pass
output.append(12)
@jump_test(3, 4, [1, 4])
def test_jump_infinite_while_loop(output):
output.append(1)
@ -722,6 +736,22 @@ class JumpTestCase(unittest.TestCase):
output.append(3)
output.append(4)
@jump_test(2, 4, [4, 4])
def test_jump_forwards_into_while_block(output):
i = 1
output.append(2)
while i <= 2:
output.append(4)
i += 1
@jump_test(5, 3, [3, 3, 3, 5])
def test_jump_backwards_into_while_block(output):
i = 1
while i <= 2:
output.append(3)
i += 1
output.append(5)
@jump_test(2, 3, [1, 3])
def test_jump_forwards_out_of_with_block(output):
with tracecontext(output, 1):
@ -916,22 +946,6 @@ class JumpTestCase(unittest.TestCase):
output.append(2)
output.append(3)
@jump_test(2, 4, [], (ValueError, 'into'))
def test_no_jump_forwards_into_while_block(output):
i = 1
output.append(2)
while i <= 2:
output.append(4)
i += 1
@jump_test(5, 3, [3, 3], (ValueError, 'into'))
def test_no_jump_backwards_into_while_block(output):
i = 1
while i <= 2:
output.append(3)
i += 1
output.append(5)
@jump_test(1, 3, [], (ValueError, 'into'))
def test_no_jump_forwards_into_with_block(output):
output.append(1)
@ -1024,6 +1038,16 @@ class JumpTestCase(unittest.TestCase):
with tracecontext(output, 4):
output.append(5)
@jump_test(5, 7, [2, 4], (ValueError, 'finally'))
def test_no_jump_over_return_out_of_finally_block(output):
try:
output.append(2)
finally:
output.append(4)
output.append(5)
return
output.append(7)
@jump_test(7, 4, [1, 6], (ValueError, 'into'))
def test_no_jump_into_for_block_before_else(output):
output.append(1)

10
Misc/NEWS.d/next/Core and Builtins/2017-12-24-19-48-59.bpo-17611.P85kWL.rst Normal file
View File

@ -0,0 +1,10 @@
Simplified the interpreter loop by moving the logic of unrolling the stack
of blocks into the compiler. The compiler emits now explicit instructions
for adjusting the stack of values and calling the cleaning up code for
:keyword:`break`, :keyword:`continue` and :keyword:`return`.
Removed opcodes :opcode:`BREAK_LOOP`, :opcode:`CONTINUE_LOOP`,
:opcode:`SETUP_LOOP` and :opcode:`SETUP_EXCEPT`. Added new opcodes
:opcode:`ROT_FOUR`, :opcode:`BEGIN_FINALLY` and :opcode:`CALL_FINALLY` and
:opcode:`POP_FINALLY`. Changed the behavior of :opcode:`END_FINALLY` and
:opcode:`WITH_CLEANUP_START`.

203
Objects/frameobject.c
View File

@ -45,6 +45,27 @@ frame_getlineno(PyFrameObject *f, void *closure)
return PyLong_FromLong(PyFrame_GetLineNumber(f));
}
/* Given the index of the effective opcode,
scan back to construct the oparg with EXTENDED_ARG */
static unsigned int
get_arg(const _Py_CODEUNIT *codestr, Py_ssize_t i)
{
_Py_CODEUNIT word;
unsigned int oparg = _Py_OPARG(codestr[i]);
if (i >= 1 && _Py_OPCODE(word = codestr[i-1]) == EXTENDED_ARG) {
oparg |= _Py_OPARG(word) << 8;
if (i >= 2 && _Py_OPCODE(word = codestr[i-2]) == EXTENDED_ARG) {
oparg |= _Py_OPARG(word) << 16;
if (i >= 3 && _Py_OPCODE(word = codestr[i-3]) == EXTENDED_ARG) {
oparg |= _Py_OPARG(word) << 24;
}
}
}
return oparg;
}
/* Setter for f_lineno - you can set f_lineno from within a trace function in
* order to jump to a given line of code, subject to some restrictions. Most
* lines are OK to jump to because they don't make any assumptions about the
@ -56,8 +77,9 @@ frame_getlineno(PyFrameObject *f, void *closure)
* they expect an exception to be on the top of the stack.
* o Lines that live in a 'finally' block can't be jumped from or to, since
* the END_FINALLY expects to clean up the stack after the 'try' block.
* o 'try'/'for'/'while' blocks can't be jumped into because the blockstack
* needs to be set up before their code runs, and for 'for' loops the
* o 'try', 'with' and 'async with' blocks can't be jumped into because
* the blockstack needs to be set up before their code runs.
* o 'for' and 'async for' loops can't be jumped into because the
* iterator needs to be on the stack.
*/
static int
@ -75,17 +97,10 @@ frame_setlineno(PyFrameObject *f, PyObject* p_new_lineno)
int offset = 0; /* (ditto) */
int line = 0; /* (ditto) */
int addr = 0; /* (ditto) */
int min_addr = 0; /* Scanning the SETUPs and POPs */
int max_addr = 0; /* (ditto) */
int delta_iblock = 0; /* (ditto) */
int min_delta_iblock = 0; /* (ditto) */
int min_iblock = 0; /* (ditto) */
int f_lasti_setup_addr = 0; /* Policing no-jump-into-finally */
int new_lasti_setup_addr = 0; /* (ditto) */
int delta_iblock = 0; /* Scanning the SETUPs and POPs */
int for_loop_delta = 0; /* (ditto) */
int blockstack[CO_MAXBLOCKS]; /* Walking the 'finally' blocks */
int in_finally[CO_MAXBLOCKS]; /* (ditto) */
int blockstack_top = 0; /* (ditto) */
unsigned char setup_op = 0; /* (ditto) */
/* f_lineno must be an integer. */
if (!PyLong_CheckExact(p_new_lineno)) {
@ -159,8 +174,6 @@ frame_setlineno(PyFrameObject *f, PyObject* p_new_lineno)
/* We're now ready to look at the bytecode. */
PyBytes_AsStringAndSize(f->f_code->co_code, (char **)&code, &code_len);
min_addr = Py_MIN(new_lasti, f->f_lasti);
max_addr = Py_MAX(new_lasti, f->f_lasti);
/* You can't jump onto a line with an 'except' statement on it -
* they expect to have an exception on the top of the stack, which
@ -179,141 +192,73 @@ frame_setlineno(PyFrameObject *f, PyObject* p_new_lineno)
}
/* You can't jump into or out of a 'finally' block because the 'try'
* block leaves something on the stack for the END_FINALLY to clean
* up. So we walk the bytecode, maintaining a simulated blockstack.
* When we reach the old or new address and it's in a 'finally' block
* we note the address of the corresponding SETUP_FINALLY. The jump
* is only legal if neither address is in a 'finally' block or
* they're both in the same one. 'blockstack' is a stack of the
* bytecode addresses of the SETUP_X opcodes, and 'in_finally' tracks
* whether we're in a 'finally' block at each blockstack level. */
f_lasti_setup_addr = -1;
new_lasti_setup_addr = -1;
* block leaves something on the stack for the END_FINALLY to clean up.
* So we walk the bytecode, maintaining a simulated blockstack.
* 'blockstack' is a stack of the bytecode addresses of the starts of
* the 'finally' blocks. */
memset(blockstack, '\0', sizeof(blockstack));
memset(in_finally, '\0', sizeof(in_finally));
blockstack_top = 0;
for (addr = 0; addr < code_len; addr += sizeof(_Py_CODEUNIT)) {
unsigned char op = code[addr];
switch (op) {
case SETUP_LOOP:
case SETUP_EXCEPT:
case SETUP_FINALLY:
case SETUP_WITH:
case SETUP_ASYNC_WITH:
blockstack[blockstack_top++] = addr;
in_finally[blockstack_top-1] = 0;
break;
case POP_BLOCK:
assert(blockstack_top > 0);
setup_op = code[blockstack[blockstack_top-1]];
if (setup_op == SETUP_FINALLY || setup_op == SETUP_WITH
|| setup_op == SETUP_ASYNC_WITH) {
in_finally[blockstack_top-1] = 1;
case FOR_ITER: {
unsigned int oparg = get_arg((const _Py_CODEUNIT *)code,
addr / sizeof(_Py_CODEUNIT));
int target_addr = addr + oparg + sizeof(_Py_CODEUNIT);
assert(target_addr < code_len);
/* Police block-jumping (you can't jump into the middle of a block)
* and ensure that the blockstack finishes up in a sensible state (by
* popping any blocks we're jumping out of). We look at all the
* blockstack operations between the current position and the new
* one, and keep track of how many blocks we drop out of on the way.
* By also keeping track of the lowest blockstack position we see, we
* can tell whether the jump goes into any blocks without coming out
* again - in that case we raise an exception below. */
int first_in = addr < f->f_lasti && f->f_lasti < target_addr;
int second_in = addr < new_lasti && new_lasti < target_addr;
if (!first_in && second_in) {
PyErr_SetString(PyExc_ValueError,
"can't jump into the middle of a block");
return -1;
}
else {
blockstack_top--;
}
break;
case END_FINALLY:
/* Ignore END_FINALLYs for SETUP_EXCEPTs - they exist
* in the bytecode but don't correspond to an actual
* 'finally' block. (If blockstack_top is 0, we must
* be seeing such an END_FINALLY.) */
if (blockstack_top > 0) {
setup_op = code[blockstack[blockstack_top-1]];
if (setup_op == SETUP_FINALLY || setup_op == SETUP_WITH
|| setup_op == SETUP_ASYNC_WITH) {
blockstack_top--;
if (first_in && !second_in) {
if (op == FOR_ITER && !delta_iblock) {
for_loop_delta++;
}
if (op != FOR_ITER) {
delta_iblock++;
}
}
if (op != FOR_ITER) {
blockstack[blockstack_top++] = target_addr;
}
break;
}
/* For the addresses we're interested in, see whether they're
* within a 'finally' block and if so, remember the address
* of the SETUP_FINALLY. */
if (addr == new_lasti || addr == f->f_lasti) {
int i = 0;
int setup_addr = -1;
for (i = blockstack_top-1; i >= 0; i--) {
if (in_finally[i]) {
setup_addr = blockstack[i];
break;
}
}
if (setup_addr != -1) {
if (addr == new_lasti) {
new_lasti_setup_addr = setup_addr;
}
if (addr == f->f_lasti) {
f_lasti_setup_addr = setup_addr;
}
case END_FINALLY: {
assert(blockstack_top > 0);
int target_addr = blockstack[--blockstack_top];
assert(target_addr <= addr);
int first_in = target_addr <= f->f_lasti && f->f_lasti <= addr;
int second_in = target_addr <= new_lasti && new_lasti <= addr;
if (first_in != second_in) {
PyErr_SetString(PyExc_ValueError,
"can't jump into or out of a 'finally' block");
return -1;
}
break;
}
}
}
/* Verify that the blockstack tracking code didn't get lost. */
assert(blockstack_top == 0);
/* After all that, are we jumping into / out of a 'finally' block? */
if (new_lasti_setup_addr != f_lasti_setup_addr) {
PyErr_SetString(PyExc_ValueError,
"can't jump into or out of a 'finally' block");
return -1;
}
/* Police block-jumping (you can't jump into the middle of a block)
* and ensure that the blockstack finishes up in a sensible state (by
* popping any blocks we're jumping out of). We look at all the
* blockstack operations between the current position and the new
* one, and keep track of how many blocks we drop out of on the way.
* By also keeping track of the lowest blockstack position we see, we
* can tell whether the jump goes into any blocks without coming out
* again - in that case we raise an exception below. */
delta_iblock = 0;
for (addr = min_addr; addr < max_addr; addr += sizeof(_Py_CODEUNIT)) {
unsigned char op = code[addr];
switch (op) {
case SETUP_LOOP:
case SETUP_EXCEPT:
case SETUP_FINALLY:
case SETUP_WITH:
case SETUP_ASYNC_WITH:
delta_iblock++;
break;
case POP_BLOCK:
delta_iblock--;
break;
}
min_delta_iblock = Py_MIN(min_delta_iblock, delta_iblock);
}
/* Derive the absolute iblock values from the deltas. */
min_iblock = f->f_iblock + min_delta_iblock;
if (new_lasti > f->f_lasti) {
/* Forwards jump. */
new_iblock = f->f_iblock + delta_iblock;
}
else {
/* Backwards jump. */
new_iblock = f->f_iblock - delta_iblock;
}
/* Are we jumping into a block? */
if (new_iblock > min_iblock) {
PyErr_SetString(PyExc_ValueError,
"can't jump into the middle of a block");
return -1;
}
/* Pop any blocks that we're jumping out of. */
new_iblock = f->f_iblock - delta_iblock;
while (f->f_iblock > new_iblock) {
PyTryBlock *b = &f->f_blockstack[--f->f_iblock];
while ((f->f_stacktop - f->f_valuestack) > b->b_level) {
@ -321,6 +266,12 @@ frame_setlineno(PyFrameObject *f, PyObject* p_new_lineno)
Py_DECREF(v);
}
}
/* Pop the iterators of any 'for' loop we're jumping out of. */
while (for_loop_delta > 0) {
PyObject *v = (*--f->f_stacktop);
Py_DECREF(v);
for_loop_delta--;
}
/* Finally set the new f_lineno and f_lasti and return OK. */
f->f_lineno = new_lineno;

10
Objects/genobject.c
View File

@ -822,18 +822,16 @@ PyGen_New(PyFrameObject *f)
int
PyGen_NeedsFinalizing(PyGenObject *gen)
{
int i;
PyFrameObject *f = gen->gi_frame;
if (f == NULL || f->f_stacktop == NULL)
return 0; /* no frame or empty blockstack == no finalization */
/* Any block type besides a loop requires cleanup. */
for (i = 0; i < f->f_iblock; i++)
if (f->f_blockstack[i].b_type != SETUP_LOOP)
return 1;
/* Any (exception-handling) block type requires cleanup. */
if (f->f_iblock > 0)
return 1;
/* No blocks except loops, it's safe to skip finalization. */
/* No blocks, it's safe to skip finalization. */
return 0;
}

1
PC/launcher.c
View File

@ -1116,6 +1116,7 @@ static PYC_MAGIC magic_values[] = {
{ 3320, 3351, L"3.5" },
{ 3360, 3379, L"3.6" },
{ 3390, 3399, L"3.7" },
{ 3400, 3409, L"3.8" },
{ 0 }
};

369
Python/ceval.c
View File

@ -500,17 +500,6 @@ _Py_CheckRecursiveCall(const char *where)
return 0;
}
/* Status code for main loop (reason for stack unwind) */
enum why_code {
WHY_NOT = 0x0001, /* No error */
WHY_EXCEPTION = 0x0002, /* Exception occurred */
WHY_RETURN = 0x0008, /* 'return' statement */
WHY_BREAK = 0x0010, /* 'break' statement */
WHY_CONTINUE = 0x0020, /* 'continue' statement */
WHY_YIELD = 0x0040, /* 'yield' operator */
WHY_SILENCED = 0x0080 /* Exception silenced by 'with' */
};
static int do_raise(PyObject *, PyObject *);
static int unpack_iterable(PyObject *, int, int, PyObject **);
@ -556,7 +545,6 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
const _Py_CODEUNIT *next_instr;
int opcode; /* Current opcode */
int oparg; /* Current opcode argument, if any */
enum why_code why; /* Reason for block stack unwind */
PyObject **fastlocals, **freevars;
PyObject *retval = NULL; /* Return value */
PyThreadState *tstate = PyThreadState_GET();
@ -914,8 +902,6 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
lltrace = _PyDict_GetItemId(f->f_globals, &PyId___ltrace__) != NULL;
#endif
why = WHY_NOT;
if (throwflag) /* support for generator.throw() */
goto error;
@ -926,6 +912,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
assert(!PyErr_Occurred());
#endif
main_loop:
for (;;) {
assert(stack_pointer >= f->f_valuestack); /* else underflow */
assert(STACK_LEVEL() <= co->co_stacksize); /* else overflow */
@ -1056,9 +1043,8 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
switch (opcode) {
/* BEWARE!
It is essential that any operation that fails sets either
x to NULL, err to nonzero, or why to anything but WHY_NOT,
and that no operation that succeeds does this! */
It is essential that any operation that fails must goto error
and that all operation that succeed call [FAST_]DISPATCH() ! */
TARGET(NOP)
FAST_DISPATCH();
@ -1115,6 +1101,18 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
FAST_DISPATCH();
}
TARGET(ROT_FOUR) {
PyObject *top = TOP();
PyObject *second = SECOND();
PyObject *third = THIRD();
PyObject *fourth = FOURTH();
SET_TOP(second);
SET_SECOND(third);
SET_THIRD(fourth);
SET_FOURTH(top);
FAST_DISPATCH();
}
TARGET(DUP_TOP) {
PyObject *top = TOP();
Py_INCREF(top);
@ -1618,8 +1616,7 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
/* fall through */
case 0:
if (do_raise(exc, cause)) {
why = WHY_EXCEPTION;
goto fast_block_end;
goto exception_unwind;
}
break;
default:
@ -1632,8 +1629,8 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
TARGET(RETURN_VALUE) {
retval = POP();
why = WHY_RETURN;
goto fast_block_end;
assert(f->f_iblock == 0);
goto return_or_yield;
}
TARGET(GET_AITER) {
@ -1794,11 +1791,10 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
}
/* receiver remains on stack, retval is value to be yielded */
f->f_stacktop = stack_pointer;
why = WHY_YIELD;
/* and repeat... */
assert(f->f_lasti >= (int)sizeof(_Py_CODEUNIT));
f->f_lasti -= sizeof(_Py_CODEUNIT);
goto fast_yield;
goto return_or_yield;
}
TARGET(YIELD_VALUE) {
@ -1815,67 +1811,137 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
}
f->f_stacktop = stack_pointer;
why = WHY_YIELD;
goto fast_yield;
goto return_or_yield;
}
TARGET(POP_EXCEPT) {
PyObject *type, *value, *traceback;
_PyErr_StackItem *exc_info;
PyTryBlock *b = PyFrame_BlockPop(f);
if (b->b_type != EXCEPT_HANDLER) {
PyErr_SetString(PyExc_SystemError,
"popped block is not an except handler");
goto error;
}
UNWIND_EXCEPT_HANDLER(b);
assert(STACK_LEVEL() >= (b)->b_level + 3 &&
STACK_LEVEL() <= (b)->b_level + 4);
exc_info = tstate->exc_info;
type = exc_info->exc_type;
value = exc_info->exc_value;
traceback = exc_info->exc_traceback;
exc_info->exc_type = POP();
exc_info->exc_value = POP();
exc_info->exc_traceback = POP();
Py_XDECREF(type);
Py_XDECREF(value);
Py_XDECREF(traceback);
DISPATCH();
}
PREDICTED(POP_BLOCK);
TARGET(POP_BLOCK) {
PyTryBlock *b = PyFrame_BlockPop(f);
UNWIND_BLOCK(b);
PyFrame_BlockPop(f);
DISPATCH();
}
TARGET(POP_FINALLY) {
/* If oparg is 0 at the top of the stack are 1 or 6 values:
Either:
- TOP = NULL or an integer
or:
- (TOP, SECOND, THIRD) = exc_info()
- (FOURTH, FITH, SIXTH) = previous exception for EXCEPT_HANDLER
If oparg is 1 the value for 'return' was additionally pushed
at the top of the stack.
*/
PyObject *res = NULL;
if (oparg) {
res = POP();
}
PyObject *exc = POP();
if (exc == NULL || PyLong_CheckExact(exc)) {
Py_XDECREF(exc);
}
else {
Py_DECREF(exc);
Py_DECREF(POP());
Py_DECREF(POP());
PyObject *type, *value, *traceback;
_PyErr_StackItem *exc_info;
PyTryBlock *b = PyFrame_BlockPop(f);
if (b->b_type != EXCEPT_HANDLER) {
PyErr_SetString(PyExc_SystemError,
"popped block is not an except handler");
Py_XDECREF(res);
goto error;
}
assert(STACK_LEVEL() == (b)->b_level + 3);
exc_info = tstate->exc_info;
type = exc_info->exc_type;
value = exc_info->exc_value;
traceback = exc_info->exc_traceback;
exc_info->exc_type = POP();
exc_info->exc_value = POP();
exc_info->exc_traceback = POP();
Py_XDECREF(type);
Py_XDECREF(value);
Py_XDECREF(traceback);
}
if (oparg) {
PUSH(res);
}
DISPATCH();
}
TARGET(CALL_FINALLY) {
PyObject *ret = PyLong_FromLong(INSTR_OFFSET());
if (ret == NULL) {
goto error;
}
PUSH(ret);
JUMPBY(oparg);
FAST_DISPATCH();
}
TARGET(BEGIN_FINALLY) {
/* Push NULL onto the stack for using it in END_FINALLY,
POP_FINALLY, WITH_CLEANUP_START and WITH_CLEANUP_FINISH.
*/
PUSH(NULL);
FAST_DISPATCH();
}
PREDICTED(END_FINALLY);
TARGET(END_FINALLY) {
PyObject *status = POP();
if (PyLong_Check(status)) {
why = (enum why_code) PyLong_AS_LONG(status);
assert(why != WHY_YIELD && why != WHY_EXCEPTION);
if (why == WHY_RETURN ||
why == WHY_CONTINUE)
retval = POP();
if (why == WHY_SILENCED) {
/* An exception was silenced by 'with', we must
manually unwind the EXCEPT_HANDLER block which was
created when the exception was caught, otherwise
the stack will be in an inconsistent state. */
PyTryBlock *b = PyFrame_BlockPop(f);
assert(b->b_type == EXCEPT_HANDLER);
UNWIND_EXCEPT_HANDLER(b);
why = WHY_NOT;
Py_DECREF(status);
DISPATCH();
/* At the top of the stack are 1 or 6 values:
Either:
- TOP = NULL or an integer
or:
- (TOP, SECOND, THIRD) = exc_info()
- (FOURTH, FITH, SIXTH) = previous exception for EXCEPT_HANDLER
*/
PyObject *exc = POP();
if (exc == NULL) {
FAST_DISPATCH();
}
else if (PyLong_CheckExact(exc)) {
int ret = _PyLong_AsInt(exc);
Py_DECREF(exc);
if (ret == -1 && PyErr_Occurred()) {
goto error;
}
Py_DECREF(status);
goto fast_block_end;
JUMPTO(ret);
FAST_DISPATCH();
}
else if (PyExceptionClass_Check(status)) {
PyObject *exc = POP();
else {
assert(PyExceptionClass_Check(exc));
PyObject *val = POP();
PyObject *tb = POP();
PyErr_Restore(status, exc, tb);
why = WHY_EXCEPTION;
goto fast_block_end;
PyErr_Restore(exc, val, tb);
goto exception_unwind;
}
else if (status != Py_None) {
PyErr_SetString(PyExc_SystemError,
"'finally' pops bad exception");
Py_DECREF(status);
goto error;
}
Py_DECREF(status);
DISPATCH();
}
TARGET(LOAD_BUILD_CLASS) {
@ -2815,28 +2881,13 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
DISPATCH();
}
TARGET(BREAK_LOOP) {
why = WHY_BREAK;
goto fast_block_end;
}
TARGET(CONTINUE_LOOP) {
retval = PyLong_FromLong(oparg);
if (retval == NULL)
goto error;
why = WHY_CONTINUE;
goto fast_block_end;
}
TARGET(SETUP_LOOP)
TARGET(SETUP_EXCEPT)
TARGET(SETUP_FINALLY) {
/* NOTE: If you add any new block-setup opcodes that
are not try/except/finally handlers, you may need
to update the PyGen_NeedsFinalizing() function.
*/
PyFrame_BlockSetup(f, opcode, INSTR_OFFSET() + oparg,
PyFrame_BlockSetup(f, SETUP_FINALLY, INSTR_OFFSET() + oparg,
STACK_LEVEL());
DISPATCH();
}
@ -2904,60 +2955,40 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
}
TARGET(WITH_CLEANUP_START) {
/* At the top of the stack are 1-6 values indicating
/* At the top of the stack are 1 or 6 values indicating
how/why we entered the finally clause:
- TOP = None
- (TOP, SECOND) = (WHY_{RETURN,CONTINUE}), retval
- TOP = WHY_*; no retval below it
- TOP = NULL
- (TOP, SECOND, THIRD) = exc_info()
(FOURTH, FITH, SIXTH) = previous exception for EXCEPT_HANDLER
Below them is EXIT, the context.__exit__ bound method.
In the last case, we must call
EXIT(TOP, SECOND, THIRD)
otherwise we must call
Below them is EXIT, the context.__exit__ or context.__aexit__
bound method.
In the first case, we must call
EXIT(None, None, None)
otherwise we must call
EXIT(TOP, SECOND, THIRD)
In the first three cases, we remove EXIT from the
stack, leaving the rest in the same order. In the
fourth case, we shift the bottom 3 values of the
stack down, and replace the empty spot with NULL.
In the first case, we remove EXIT from the
stack, leaving TOP, and push TOP on the stack.
Otherwise we shift the bottom 3 values of the
stack down, replace the empty spot with NULL, and push
None on the stack.
In addition, if the stack represents an exception,
*and* the function call returns a 'true' value, we
push WHY_SILENCED onto the stack. END_FINALLY will
then not re-raise the exception. (But non-local
gotos should still be resumed.)
Finally we push the result of the call.
*/
PyObject* stack[3];
PyObject *stack[3];
PyObject *exit_func;
PyObject *exc, *val, *tb, *res;
val = tb = Py_None;
exc = TOP();
if (exc == Py_None) {
(void)POP();
if (exc == NULL) {
STACKADJ(-1);
exit_func = TOP();
SET_TOP(exc);
}
else if (PyLong_Check(exc)) {
STACKADJ(-1);
switch (PyLong_AsLong(exc)) {
case WHY_RETURN:
case WHY_CONTINUE:
/* Retval in TOP. */
exit_func = SECOND();
SET_SECOND(TOP());
SET_TOP(exc);
break;
default:
exit_func = TOP();
SET_TOP(exc);
break;
}
exc = Py_None;
}
else {
assert(PyExceptionClass_Check(exc));
PyObject *tp2, *exc2, *tb2;
PyTryBlock *block;
val = SECOND();
@ -2974,8 +3005,10 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
/* We just shifted the stack down, so we have
to tell the except handler block that the
values are lower than it expects. */
assert(f->f_iblock > 0);
block = &f->f_blockstack[f->f_iblock - 1];
assert(block->b_type == EXCEPT_HANDLER);
assert(block->b_level > 0);
block->b_level--;
}
@ -2996,6 +3029,12 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
PREDICTED(WITH_CLEANUP_FINISH);
TARGET(WITH_CLEANUP_FINISH) {
/* TOP = the result of calling the context.__exit__ bound method
SECOND = either None or exception type
If SECOND is None below is NULL or the return address,
otherwise below are 7 values representing an exception.
*/
PyObject *res = POP();
PyObject *exc = POP();
int err;
@ -3011,8 +3050,15 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
if (err < 0)
goto error;
else if (err > 0) {
/* There was an exception and a True return */
PUSH(PyLong_FromLong((long) WHY_SILENCED));
/* There was an exception and a True return.
* We must manually unwind the EXCEPT_HANDLER block
* which was created when the exception was caught,
* otherwise the stack will be in an inconsisten state.
*/
PyTryBlock *b = PyFrame_BlockPop(f);
assert(b->b_type == EXCEPT_HANDLER);
UNWIND_EXCEPT_HANDLER(b);
PUSH(NULL);
}
PREDICT(END_FINALLY);
DISPATCH();
@ -3322,10 +3368,6 @@ _PyEval_EvalFrameDefault(PyFrameObject *f, int throwflag)
Py_UNREACHABLE();
error:
assert(why == WHY_NOT);
why = WHY_EXCEPTION;
/* Double-check exception status. */
#ifdef NDEBUG
if (!PyErr_Occurred())
@ -3342,36 +3384,18 @@ error:
call_exc_trace(tstate->c_tracefunc, tstate->c_traceobj,
tstate, f);
fast_block_end:
assert(why != WHY_NOT);
/* Unwind stacks if a (pseudo) exception occurred */
while (why != WHY_NOT && f->f_iblock > 0) {
/* Peek at the current block. */
PyTryBlock *b = &f->f_blockstack[f->f_iblock - 1];
assert(why != WHY_YIELD);
if (b->b_type == SETUP_LOOP && why == WHY_CONTINUE) {
why = WHY_NOT;
JUMPTO(PyLong_AS_LONG(retval));
Py_DECREF(retval);
break;
}
/* Now we have to pop the block. */
f->f_iblock--;
exception_unwind:
/* Unwind stacks if an exception occurred */
while (f->f_iblock > 0) {
/* Pop the current block. */
PyTryBlock *b = &f->f_blockstack[--f->f_iblock];
if (b->b_type == EXCEPT_HANDLER) {
UNWIND_EXCEPT_HANDLER(b);
continue;
}
UNWIND_BLOCK(b);
if (b->b_type == SETUP_LOOP && why == WHY_BREAK) {
why = WHY_NOT;
JUMPTO(b->b_handler);
break;
}
if (why == WHY_EXCEPTION && (b->b_type == SETUP_EXCEPT
|| b->b_type == SETUP_FINALLY)) {
if (b->b_type == SETUP_FINALLY) {
PyObject *exc, *val, *tb;
int handler = b->b_handler;
_PyErr_StackItem *exc_info = tstate->exc_info;
@ -3408,70 +3432,37 @@ fast_block_end:
PUSH(tb);
PUSH(val);
PUSH(exc);
why = WHY_NOT;
JUMPTO(handler);
break;
}
if (b->b_type == SETUP_FINALLY) {
if (why & (WHY_RETURN | WHY_CONTINUE))
PUSH(retval);
PUSH(PyLong_FromLong((long)why));
why = WHY_NOT;
JUMPTO(b->b_handler);
break;
/* Resume normal execution */
goto main_loop;
}
} /* unwind stack */
/* End the loop if we still have an error (or return) */
if (why != WHY_NOT)
break;
assert(!PyErr_Occurred());
/* End the loop as we still have an error */
break;
} /* main loop */
assert(why != WHY_YIELD);
/* Pop remaining stack entries. */
while (!EMPTY()) {
PyObject *o = POP();
Py_XDECREF(o);
}
if (why != WHY_RETURN)
retval = NULL;
assert((retval != NULL) ^ (PyErr_Occurred() != NULL));
fast_yield:
assert(retval == NULL);
assert(PyErr_Occurred());
return_or_yield:
if (tstate->use_tracing) {
if (tstate->c_tracefunc) {
if (why == WHY_RETURN || why == WHY_YIELD) {
if (call_trace(tstate->c_tracefunc, tstate->c_traceobj,
tstate, f,
PyTrace_RETURN, retval)) {
Py_CLEAR(retval);
why = WHY_EXCEPTION;
}
}
else if (why == WHY_EXCEPTION) {
call_trace_protected(tstate->c_tracefunc, tstate->c_traceobj,
tstate, f,
PyTrace_RETURN, NULL);
if (call_trace_protected(tstate->c_tracefunc, tstate->c_traceobj,
tstate, f, PyTrace_RETURN, retval)) {
Py_CLEAR(retval);
}
}
if (tstate->c_profilefunc) {
if (why == WHY_EXCEPTION)
call_trace_protected(tstate->c_profilefunc,
tstate->c_profileobj,
tstate, f,
PyTrace_RETURN, NULL);
else if (call_trace(tstate->c_profilefunc, tstate->c_profileobj,
tstate, f,
PyTrace_RETURN, retval)) {
if (call_trace_protected(tstate->c_profilefunc, tstate->c_profileobj,
tstate, f, PyTrace_RETURN, retval)) {
Py_CLEAR(retval);
/* why = WHY_EXCEPTION; useless yet but cause compiler warnings */
}
}
}

411
Python/compile.c
View File

@ -81,11 +81,14 @@ It's called a frame block to distinguish it from a basic block in the
compiler IR.
*/
enum fblocktype { LOOP, EXCEPT, FINALLY_TRY, FINALLY_END };
enum fblocktype { WHILE_LOOP, FOR_LOOP, EXCEPT, FINALLY_TRY, FINALLY_END,
WITH, ASYNC_WITH, HANDLER_CLEANUP };
struct fblockinfo {
enum fblocktype fb_type;
basicblock *fb_block;
/* (optional) type-specific exit or cleanup block */
basicblock *fb_exit;
};
enum {
@ -183,13 +186,6 @@ static int compiler_annassign(struct compiler *, stmt_ty);
static int compiler_visit_slice(struct compiler *, slice_ty,
expr_context_ty);
static int compiler_push_fblock(struct compiler *, enum fblocktype,
basicblock *);
static void compiler_pop_fblock(struct compiler *, enum fblocktype,
basicblock *);
/* Returns true if there is a loop on the fblock stack. */
static int compiler_in_loop(struct compiler *);
static int inplace_binop(struct compiler *, operator_ty);
static int expr_constant(expr_ty);
@ -846,7 +842,7 @@ compiler_next_instr(struct compiler *c, basicblock *b)
- when entering a new scope
- on each statement
- on each expression that start a new line
- before the "except" clause
- before the "except" and "finally" clauses
- before the "for" and "while" expressions
*/
@ -881,6 +877,7 @@ stack_effect(int opcode, int oparg, int jump)
return -1;
case ROT_TWO:
case ROT_THREE:
case ROT_FOUR:
return 0;
case DUP_TOP:
return 1;
@ -947,8 +944,7 @@ stack_effect(int opcode, int oparg, int jump)
case INPLACE_XOR:
case INPLACE_OR:
return -1;
case BREAK_LOOP:
return 0;
case SETUP_WITH:
/* 1 in the normal flow.
* Restore the stack position and push 6 values before jumping to
@ -975,6 +971,7 @@ stack_effect(int opcode, int oparg, int jump)
case POP_EXCEPT:
return -3;
case END_FINALLY:
case POP_FINALLY:
/* Pop 6 values when an exception was raised. */
return -6;
@ -1043,16 +1040,20 @@ stack_effect(int opcode, int oparg, int jump)
case LOAD_GLOBAL:
return 1;
case CONTINUE_LOOP:
return 0;
case SETUP_LOOP:
return 0;
case SETUP_EXCEPT:
/* Exception handling */
case SETUP_FINALLY:
/* 0 in the normal flow.
* Restore the stack position and push 6 values before jumping to
* the handler if an exception be raised. */
return jump ? 6 : 0;
case BEGIN_FINALLY:
/* Actually pushes 1 value, but count 6 for balancing with
* END_FINALLY and POP_FINALLY.
* This is the main reason of using this opcode instead of
* "LOAD_CONST None". */
return 6;
case CALL_FINALLY:
return jump ? 1 : 0;
case LOAD_FAST:
return 1;
@ -1458,6 +1459,103 @@ find_ann(asdl_seq *stmts)
return res;
}
/*
* Frame block handling functions
*/
static int
compiler_push_fblock(struct compiler *c, enum fblocktype t, basicblock *b,
basicblock *exit)
{
struct fblockinfo *f;
if (c->u->u_nfblocks >= CO_MAXBLOCKS) {
PyErr_SetString(PyExc_SyntaxError,
"too many statically nested blocks");
return 0;
}
f = &c->u->u_fblock[c->u->u_nfblocks++];
f->fb_type = t;
f->fb_block = b;
f->fb_exit = exit;
return 1;
}
static void
compiler_pop_fblock(struct compiler *c, enum fblocktype t, basicblock *b)
{
struct compiler_unit *u = c->u;
assert(u->u_nfblocks > 0);
u->u_nfblocks--;
assert(u->u_fblock[u->u_nfblocks].fb_type == t);
assert(u->u_fblock[u->u_nfblocks].fb_block == b);
}
/* Unwind a frame block. If preserve_tos is true, the TOS before
* popping the blocks will be restored afterwards.
*/
static int
compiler_unwind_fblock(struct compiler *c, struct fblockinfo *info,
int preserve_tos)
{
switch (info->fb_type) {
case WHILE_LOOP:
return 1;
case FINALLY_END:
ADDOP_I(c, POP_FINALLY, preserve_tos);
return 1;
case FOR_LOOP:
/* Pop the iterator */
if (preserve_tos) {
ADDOP(c, ROT_TWO);
}
ADDOP(c, POP_TOP);
return 1;
case EXCEPT:
ADDOP(c, POP_BLOCK);
return 1;
case FINALLY_TRY:
ADDOP(c, POP_BLOCK);
ADDOP_JREL(c, CALL_FINALLY, info->fb_exit);
return 1;
case WITH:
case ASYNC_WITH:
ADDOP(c, POP_BLOCK);
if (preserve_tos) {
ADDOP(c, ROT_TWO);
}
ADDOP(c, BEGIN_FINALLY);
ADDOP(c, WITH_CLEANUP_START);
if (info->fb_type == ASYNC_WITH) {
ADDOP(c, GET_AWAITABLE);
ADDOP_O(c, LOAD_CONST, Py_None, consts);
ADDOP(c, YIELD_FROM);
}
ADDOP(c, WITH_CLEANUP_FINISH);
ADDOP_I(c, POP_FINALLY, 0);
return 1;
case HANDLER_CLEANUP:
if (preserve_tos) {
ADDOP(c, ROT_FOUR);
}
if (info->fb_exit) {
ADDOP(c, POP_BLOCK);
ADDOP(c, POP_EXCEPT);
ADDOP_JREL(c, CALL_FINALLY, info->fb_exit);
}
else {
ADDOP(c, POP_EXCEPT);
}
return 1;
}
Py_UNREACHABLE();
}
/* Compile a sequence of statements, checking for a docstring
and for annotations. */
@ -2312,9 +2410,10 @@ compiler_for(struct compiler *c, stmt_ty s)
end = compiler_new_block(c);
if (start == NULL || end == NULL || cleanup == NULL)
return 0;
ADDOP_JREL(c, SETUP_LOOP, end);
if (!compiler_push_fblock(c, LOOP, start))
if (!compiler_push_fblock(c, FOR_LOOP, start, end))
return 0;
VISIT(c, expr, s->v.For.iter);
ADDOP(c, GET_ITER);
compiler_use_next_block(c, start);
@ -2323,8 +2422,9 @@ compiler_for(struct compiler *c, stmt_ty s)
VISIT_SEQ(c, stmt, s->v.For.body);
ADDOP_JABS(c, JUMP_ABSOLUTE, start);
compiler_use_next_block(c, cleanup);
ADDOP(c, POP_BLOCK);
compiler_pop_fblock(c, LOOP, start);
compiler_pop_fblock(c, FOR_LOOP, start);
VISIT_SEQ(c, stmt, s->v.For.orelse);
compiler_use_next_block(c, end);
return 1;
@ -2356,8 +2456,7 @@ compiler_async_for(struct compiler *c, stmt_ty s)
|| after_try == NULL || try_cleanup == NULL)
return 0;
ADDOP_JREL(c, SETUP_LOOP, after_loop);
if (!compiler_push_fblock(c, LOOP, try))
if (!compiler_push_fblock(c, FOR_LOOP, try, after_loop))
return 0;
VISIT(c, expr, s->v.AsyncFor.iter);
@ -2366,19 +2465,21 @@ compiler_async_for(struct compiler *c, stmt_ty s)
compiler_use_next_block(c, try);
ADDOP_JREL(c, SETUP_EXCEPT, except);
if (!compiler_push_fblock(c, EXCEPT, try))
/* SETUP_FINALLY to guard the __anext__ call */
ADDOP_JREL(c, SETUP_FINALLY, except);
if (!compiler_push_fblock(c, EXCEPT, try, NULL))
return 0;
ADDOP(c, GET_ANEXT);
ADDOP_O(c, LOAD_CONST, Py_None, consts);
ADDOP(c, YIELD_FROM);
VISIT(c, expr, s->v.AsyncFor.target);
ADDOP(c, POP_BLOCK);
ADDOP(c, POP_BLOCK); /* for SETUP_FINALLY */
compiler_pop_fblock(c, EXCEPT, try);
ADDOP_JREL(c, JUMP_FORWARD, after_try);
/* Except block for __anext__ */
compiler_use_next_block(c, except);
ADDOP(c, DUP_TOP);
ADDOP_O(c, LOAD_GLOBAL, stop_aiter_error, names);
@ -2388,25 +2489,26 @@ compiler_async_for(struct compiler *c, stmt_ty s)
ADDOP(c, POP_TOP);
ADDOP(c, POP_TOP);
ADDOP(c, POP_TOP);
ADDOP(c, POP_EXCEPT); /* for SETUP_EXCEPT */
ADDOP(c, POP_TOP); /* for correct calculation of stack effect */
ADDOP(c, POP_BLOCK); /* for SETUP_LOOP */
ADDOP(c, POP_EXCEPT); /* for SETUP_FINALLY */
ADDOP(c, POP_TOP); /* pop iterator from stack */
ADDOP_JABS(c, JUMP_ABSOLUTE, after_loop_else);
compiler_use_next_block(c, try_cleanup);
ADDOP(c, END_FINALLY);
/* Success block for __anext__ */
compiler_use_next_block(c, after_try);
VISIT_SEQ(c, stmt, s->v.AsyncFor.body);
ADDOP_JABS(c, JUMP_ABSOLUTE, try);
ADDOP(c, POP_BLOCK); /* for SETUP_LOOP */
compiler_pop_fblock(c, LOOP, try);
compiler_pop_fblock(c, FOR_LOOP, try);
/* Block reached after `break`ing from loop */
compiler_use_next_block(c, after_loop);
ADDOP_JABS(c, JUMP_ABSOLUTE, end);
/* `else` block */
compiler_use_next_block(c, after_loop_else);
VISIT_SEQ(c, stmt, s->v.For.orelse);
@ -2443,9 +2545,8 @@ compiler_while(struct compiler *c, stmt_ty s)
else
orelse = NULL;
ADDOP_JREL(c, SETUP_LOOP, end);
compiler_use_next_block(c, loop);
if (!compiler_push_fblock(c, LOOP, loop))
if (!compiler_push_fblock(c, WHILE_LOOP, loop, end))
return 0;
if (constant == -1) {
if (!compiler_jump_if(c, s->v.While.test, anchor, 0))
@ -2460,8 +2561,8 @@ compiler_while(struct compiler *c, stmt_ty s)
if (constant == -1)
compiler_use_next_block(c, anchor);
ADDOP(c, POP_BLOCK);
compiler_pop_fblock(c, LOOP, loop);
compiler_pop_fblock(c, WHILE_LOOP, loop);
if (orelse != NULL) /* what if orelse is just pass? */
VISIT_SEQ(c, stmt, s->v.While.orelse);
compiler_use_next_block(c, end);
@ -2470,46 +2571,83 @@ compiler_while(struct compiler *c, stmt_ty s)
}
static int
compiler_continue(struct compiler *c)
compiler_return(struct compiler *c, stmt_ty s)
{
static const char LOOP_ERROR_MSG[] = "'continue' not properly in loop";
static const char IN_FINALLY_ERROR_MSG[] =
"'continue' not supported inside 'finally' clause";
int i;
if (!c->u->u_nfblocks)
return compiler_error(c, LOOP_ERROR_MSG);
i = c->u->u_nfblocks - 1;
switch (c->u->u_fblock[i].fb_type) {
case LOOP:
ADDOP_JABS(c, JUMP_ABSOLUTE, c->u->u_fblock[i].fb_block);
break;
case EXCEPT:
case FINALLY_TRY:
while (--i >= 0 && c->u->u_fblock[i].fb_type != LOOP) {
/* Prevent continue anywhere under a finally
even if hidden in a sub-try or except. */
if (c->u->u_fblock[i].fb_type == FINALLY_END)
return compiler_error(c, IN_FINALLY_ERROR_MSG);
}
if (i == -1)
return compiler_error(c, LOOP_ERROR_MSG);
ADDOP_JABS(c, CONTINUE_LOOP, c->u->u_fblock[i].fb_block);
break;
case FINALLY_END:
return compiler_error(c, IN_FINALLY_ERROR_MSG);
int preserve_tos = ((s->v.Return.value != NULL) &&
!is_const(s->v.Return.value));
if (c->u->u_ste->ste_type != FunctionBlock)
return compiler_error(c, "'return' outside function");
if (s->v.Return.value != NULL &&
c->u->u_ste->ste_coroutine && c->u->u_ste->ste_generator)
{
return compiler_error(
c, "'return' with value in async generator");
}
if (preserve_tos) {
VISIT(c, expr, s->v.Return.value);
}
for (int depth = c->u->u_nfblocks; depth--;) {
struct fblockinfo *info = &c->u->u_fblock[depth];
if (!compiler_unwind_fblock(c, info, preserve_tos))
return 0;
}
if (s->v.Return.value == NULL) {
ADDOP_O(c, LOAD_CONST, Py_None, consts);
}
else if (!preserve_tos) {
VISIT(c, expr, s->v.Return.value);
}
ADDOP(c, RETURN_VALUE);
return 1;
}
static int
compiler_break(struct compiler *c)
{
for (int depth = c->u->u_nfblocks; depth--;) {
struct fblockinfo *info = &c->u->u_fblock[depth];
if (!compiler_unwind_fblock(c, info, 0))
return 0;
if (info->fb_type == WHILE_LOOP || info->fb_type == FOR_LOOP) {
ADDOP_JABS(c, JUMP_ABSOLUTE, info->fb_exit);
return 1;
}
}
return compiler_error(c, "'break' outside loop");
}
static int
compiler_continue(struct compiler *c)
{
for (int depth = c->u->u_nfblocks; depth--;) {
struct fblockinfo *info = &c->u->u_fblock[depth];
if (info->fb_type == WHILE_LOOP || info->fb_type == FOR_LOOP) {
ADDOP_JABS(c, JUMP_ABSOLUTE, info->fb_block);
return 1;
}
if (info->fb_type == FINALLY_END) {
return compiler_error(c,
"'continue' not supported inside 'finally' clause");
}
if (!compiler_unwind_fblock(c, info, 0))
return 0;
}
return compiler_error(c, "'continue' not properly in loop");
}
/* Code generated for "try: <body> finally: <finalbody>" is as follows:
SETUP_FINALLY L
<code for body>
POP_BLOCK
LOAD_CONST <None>
L: <code for finalbody>
BEGIN_FINALLY
L:
<code for finalbody>
END_FINALLY
The special instructions use the block stack. Each block
@ -2521,33 +2659,34 @@ compiler_continue(struct compiler *c)
Pushes the current value stack level and the label
onto the block stack.
POP_BLOCK:
Pops en entry from the block stack, and pops the value
stack until its level is the same as indicated on the
block stack. (The label is ignored.)
Pops en entry from the block stack.
BEGIN_FINALLY
Pushes NULL onto the value stack.
END_FINALLY:
Pops a variable number of entries from the *value* stack
and re-raises the exception they specify. The number of
entries popped depends on the (pseudo) exception type.
Pops 1 (NULL or int) or 6 entries from the *value* stack and restore
the raised and the caught exceptions they specify.
The block stack is unwound when an exception is raised:
when a SETUP_FINALLY entry is found, the exception is pushed
onto the value stack (and the exception condition is cleared),
and the interpreter jumps to the label gotten from the block
stack.
when a SETUP_FINALLY entry is found, the raised and the caught
exceptions are pushed onto the value stack (and the exception
condition is cleared), and the interpreter jumps to the label
gotten from the block stack.
*/
static int
compiler_try_finally(struct compiler *c, stmt_ty s)
{
basicblock *body, *end;
body = compiler_new_block(c);
end = compiler_new_block(c);
if (body == NULL || end == NULL)
return 0;
/* `try` block */
ADDOP_JREL(c, SETUP_FINALLY, end);
compiler_use_next_block(c, body);
if (!compiler_push_fblock(c, FINALLY_TRY, body))
if (!compiler_push_fblock(c, FINALLY_TRY, body, end))
return 0;
if (s->v.Try.handlers && asdl_seq_LEN(s->v.Try.handlers)) {
if (!compiler_try_except(c, s))
@ -2557,16 +2696,16 @@ compiler_try_finally(struct compiler *c, stmt_ty s)
VISIT_SEQ(c, stmt, s->v.Try.body);
}
ADDOP(c, POP_BLOCK);
ADDOP(c, BEGIN_FINALLY);
compiler_pop_fblock(c, FINALLY_TRY, body);
ADDOP_O(c, LOAD_CONST, Py_None, consts);
/* `finally` block */
compiler_use_next_block(c, end);
if (!compiler_push_fblock(c, FINALLY_END, end))
if (!compiler_push_fblock(c, FINALLY_END, end, NULL))
return 0;
VISIT_SEQ(c, stmt, s->v.Try.finalbody);
ADDOP(c, END_FINALLY);
compiler_pop_fblock(c, FINALLY_END, end);
return 1;
}
@ -2577,7 +2716,7 @@ compiler_try_finally(struct compiler *c, stmt_ty s)
associated value, and 'exc' the exception.)
Value stack Label Instruction Argument
[] SETUP_EXCEPT L1
[] SETUP_FINALLY L1
[] <code for S>
[] POP_BLOCK
[] JUMP_FORWARD L0
@ -2613,9 +2752,9 @@ compiler_try_except(struct compiler *c, stmt_ty s)
end = compiler_new_block(c);
if (body == NULL || except == NULL || orelse == NULL || end == NULL)
return 0;
ADDOP_JREL(c, SETUP_EXCEPT, except);
ADDOP_JREL(c, SETUP_FINALLY, except);
compiler_use_next_block(c, body);
if (!compiler_push_fblock(c, EXCEPT, body))
if (!compiler_push_fblock(c, EXCEPT, body, NULL))
return 0;
VISIT_SEQ(c, stmt, s->v.Try.body);
ADDOP(c, POP_BLOCK);
@ -2666,25 +2805,23 @@ compiler_try_except(struct compiler *c, stmt_ty s)
/* second try: */
ADDOP_JREL(c, SETUP_FINALLY, cleanup_end);
compiler_use_next_block(c, cleanup_body);
if (!compiler_push_fblock(c, FINALLY_TRY, cleanup_body))
if (!compiler_push_fblock(c, HANDLER_CLEANUP, cleanup_body, cleanup_end))
return 0;
/* second # body */
VISIT_SEQ(c, stmt, handler->v.ExceptHandler.body);
ADDOP(c, POP_BLOCK);
compiler_pop_fblock(c, FINALLY_TRY, cleanup_body);
ADDOP(c, BEGIN_FINALLY);
compiler_pop_fblock(c, HANDLER_CLEANUP, cleanup_body);
/* finally: */
ADDOP_O(c, LOAD_CONST, Py_None, consts);
compiler_use_next_block(c, cleanup_end);
if (!compiler_push_fblock(c, FINALLY_END, cleanup_end))
if (!compiler_push_fblock(c, FINALLY_END, cleanup_end, NULL))
return 0;
/* name = None */
/* name = None; del name */
ADDOP_O(c, LOAD_CONST, Py_None, consts);
compiler_nameop(c, handler->v.ExceptHandler.name, Store);
/* del name */
compiler_nameop(c, handler->v.ExceptHandler.name, Del);
ADDOP(c, END_FINALLY);
@ -2701,11 +2838,11 @@ compiler_try_except(struct compiler *c, stmt_ty s)
ADDOP(c, POP_TOP);
ADDOP(c, POP_TOP);
compiler_use_next_block(c, cleanup_body);
if (!compiler_push_fblock(c, FINALLY_TRY, cleanup_body))
if (!compiler_push_fblock(c, HANDLER_CLEANUP, cleanup_body, NULL))
return 0;
VISIT_SEQ(c, stmt, handler->v.ExceptHandler.body);
ADDOP(c, POP_EXCEPT);
compiler_pop_fblock(c, FINALLY_TRY, cleanup_body);
compiler_pop_fblock(c, HANDLER_CLEANUP, cleanup_body);
}
ADDOP_JREL(c, JUMP_FORWARD, end);
compiler_use_next_block(c, except);
@ -2964,18 +3101,7 @@ compiler_visit_stmt(struct compiler *c, stmt_ty s)
case ClassDef_kind:
return compiler_class(c, s);
case Return_kind:
if (c->u->u_ste->ste_type != FunctionBlock)
return compiler_error(c, "'return' outside function");
if (s->v.Return.value) {
if (c->u->u_ste->ste_coroutine && c->u->u_ste->ste_generator)
return compiler_error(
c, "'return' with value in async generator");
VISIT(c, expr, s->v.Return.value);
}
else
ADDOP_O(c, LOAD_CONST, Py_None, consts);
ADDOP(c, RETURN_VALUE);
break;
return compiler_return(c, s);
case Delete_kind:
VISIT_SEQ(c, expr, s->v.Delete.targets)
break;
@ -3027,10 +3153,7 @@ compiler_visit_stmt(struct compiler *c, stmt_ty s)
case Pass_kind:
break;
case Break_kind:
if (!compiler_in_loop(c))
return compiler_error(c, "'break' outside loop");
ADDOP(c, BREAK_LOOP);
break;
return compiler_break(c);
case Continue_kind:
return compiler_continue(c);
case With_kind:
@ -3771,8 +3894,6 @@ compiler_call_helper(struct compiler *c,
The LC/SC version returns the populated container, while the GE version is
flagged in symtable.c as a generator, so it returns the generator object
when the function is called.
This code *knows* that the loop cannot contain break, continue, or return,
so it cheats and skips the SETUP_LOOP/POP_BLOCK steps used in normal loops.
Possible cleanups:
- iterate over the generator sequence instead of using recursion
@ -3932,8 +4053,8 @@ compiler_async_comprehension_generator(struct compiler *c,
compiler_use_next_block(c, try);
ADDOP_JREL(c, SETUP_EXCEPT, except);
if (!compiler_push_fblock(c, EXCEPT, try))
ADDOP_JREL(c, SETUP_FINALLY, except);
if (!compiler_push_fblock(c, EXCEPT, try, NULL))
return 0;
ADDOP(c, GET_ANEXT);
@ -3954,7 +4075,7 @@ compiler_async_comprehension_generator(struct compiler *c,
ADDOP(c, POP_TOP);
ADDOP(c, POP_TOP);
ADDOP(c, POP_TOP);
ADDOP(c, POP_EXCEPT); /* for SETUP_EXCEPT */
ADDOP(c, POP_EXCEPT); /* for SETUP_FINALLY */
ADDOP_JABS(c, JUMP_ABSOLUTE, anchor);
@ -4246,7 +4367,7 @@ compiler_async_with(struct compiler *c, stmt_ty s, int pos)
/* SETUP_ASYNC_WITH pushes a finally block. */
compiler_use_next_block(c, block);
if (!compiler_push_fblock(c, FINALLY_TRY, block)) {
if (!compiler_push_fblock(c, ASYNC_WITH, block, finally)) {
return 0;
}
@ -4267,11 +4388,11 @@ compiler_async_with(struct compiler *c, stmt_ty s, int pos)
/* End of try block; start the finally block */
ADDOP(c, POP_BLOCK);
compiler_pop_fblock(c, FINALLY_TRY, block);
ADDOP(c, BEGIN_FINALLY);
compiler_pop_fblock(c, ASYNC_WITH, block);
ADDOP_O(c, LOAD_CONST, Py_None, consts);
compiler_use_next_block(c, finally);
if (!compiler_push_fblock(c, FINALLY_END, finally))
if (!compiler_push_fblock(c, FINALLY_END, finally, NULL))
return 0;
/* Finally block starts; context.__exit__ is on the stack under
@ -4334,7 +4455,7 @@ compiler_with(struct compiler *c, stmt_ty s, int pos)
/* SETUP_WITH pushes a finally block. */
compiler_use_next_block(c, block);
if (!compiler_push_fblock(c, FINALLY_TRY, block)) {
if (!compiler_push_fblock(c, WITH, block, finally)) {
return 0;
}
@ -4355,11 +4476,11 @@ compiler_with(struct compiler *c, stmt_ty s, int pos)
/* End of try block; start the finally block */
ADDOP(c, POP_BLOCK);
compiler_pop_fblock(c, FINALLY_TRY, block);
ADDOP(c, BEGIN_FINALLY);
compiler_pop_fblock(c, WITH, block);
ADDOP_O(c, LOAD_CONST, Py_None, consts);
compiler_use_next_block(c, finally);
if (!compiler_push_fblock(c, FINALLY_END, finally))
if (!compiler_push_fblock(c, FINALLY_END, finally, NULL))
return 0;
/* Finally block starts; context.__exit__ is on the stack under
@ -4745,41 +4866,6 @@ compiler_annassign(struct compiler *c, stmt_ty s)
return 1;
}
static int
compiler_push_fblock(struct compiler *c, enum fblocktype t, basicblock *b)
{
struct fblockinfo *f;
if (c->u->u_nfblocks >= CO_MAXBLOCKS) {
PyErr_SetString(PyExc_SyntaxError,
"too many statically nested blocks");
return 0;
}
f = &c->u->u_fblock[c->u->u_nfblocks++];
f->fb_type = t;
f->fb_block = b;
return 1;
}
static void
compiler_pop_fblock(struct compiler *c, enum fblocktype t, basicblock *b)
{
struct compiler_unit *u = c->u;
assert(u->u_nfblocks > 0);
u->u_nfblocks--;
assert(u->u_fblock[u->u_nfblocks].fb_type == t);
assert(u->u_fblock[u->u_nfblocks].fb_block == b);
}
static int
compiler_in_loop(struct compiler *c) {
int i;
struct compiler_unit *u = c->u;
for (i = 0; i < u->u_nfblocks; ++i) {
if (u->u_fblock[i].fb_type == LOOP)
return 1;
}
return 0;
}
/* Raises a SyntaxError and returns 0.
If something goes wrong, a different exception may be raised.
*/
@ -4974,9 +5060,7 @@ dfs(struct compiler *c, basicblock *b, struct assembler *a, int end)
Py_LOCAL_INLINE(void)
stackdepth_push(basicblock ***sp, basicblock *b, int depth)
{
/* XXX b->b_startdepth > depth only for the target of SETUP_FINALLY,
* SETUP_WITH and SETUP_ASYNC_WITH. */
assert(b->b_startdepth < 0 || b->b_startdepth >= depth);
assert(b->b_startdepth < 0 || b->b_startdepth == depth);
if (b->b_startdepth < depth) {
assert(b->b_startdepth < 0);
b->b_startdepth = depth;
@ -5033,15 +5117,11 @@ stackdepth(struct compiler *c)
maxdepth = target_depth;
}
assert(target_depth >= 0); /* invalid code or bug in stackdepth() */
if (instr->i_opcode == CONTINUE_LOOP) {
/* Pops a variable number of values from the stack,
* but the target should be already proceeding.
*/
if (instr->i_opcode == CALL_FINALLY) {
assert(instr->i_target->b_startdepth >= 0);
assert(instr->i_target->b_startdepth <= depth);
/* remaining code is dead */
next = NULL;
break;
assert(instr->i_target->b_startdepth >= target_depth);
depth = new_depth;
continue;
}
stackdepth_push(&sp, instr->i_target, target_depth);
}
@ -5049,8 +5129,7 @@ stackdepth(struct compiler *c)
if (instr->i_opcode == JUMP_ABSOLUTE ||
instr->i_opcode == JUMP_FORWARD ||
instr->i_opcode == RETURN_VALUE ||
instr->i_opcode == RAISE_VARARGS ||
instr->i_opcode == BREAK_LOOP)
instr->i_opcode == RAISE_VARARGS)
{
/* remaining code is dead */
next = NULL;

3303
Python/importlib.h
View File

File diff suppressed because it is too large Load Diff

4069
Python/importlib_external.h
View File

File diff suppressed because it is too large Load Diff

16
Python/opcode_targets.h
View File

@ -5,7 +5,7 @@ static void *opcode_targets[256] = {
&&TARGET_ROT_THREE,
&&TARGET_DUP_TOP,
&&TARGET_DUP_TOP_TWO,
&&_unknown_opcode,
&&TARGET_ROT_FOUR,
&&_unknown_opcode,
&&_unknown_opcode,
&&TARGET_NOP,
@ -52,7 +52,7 @@ static void *opcode_targets[256] = {
&&TARGET_GET_AITER,
&&TARGET_GET_ANEXT,
&&TARGET_BEFORE_ASYNC_WITH,
&&_unknown_opcode,
&&TARGET_BEGIN_FINALLY,
&&_unknown_opcode,
&&TARGET_INPLACE_ADD,
&&TARGET_INPLACE_SUBTRACT,
@ -79,7 +79,7 @@ static void *opcode_targets[256] = {
&&TARGET_INPLACE_AND,
&&TARGET_INPLACE_XOR,
&&TARGET_INPLACE_OR,
&&TARGET_BREAK_LOOP,
&&_unknown_opcode,
&&TARGET_WITH_CLEANUP_START,
&&TARGET_WITH_CLEANUP_FINISH,
&&TARGET_RETURN_VALUE,
@ -118,9 +118,9 @@ static void *opcode_targets[256] = {
&&TARGET_LOAD_GLOBAL,
&&_unknown_opcode,
&&_unknown_opcode,
&&TARGET_CONTINUE_LOOP,
&&TARGET_SETUP_LOOP,
&&TARGET_SETUP_EXCEPT,
&&_unknown_opcode,
&&_unknown_opcode,
&&_unknown_opcode,
&&TARGET_SETUP_FINALLY,
&&_unknown_opcode,
&&TARGET_LOAD_FAST,
@ -161,8 +161,8 @@ static void *opcode_targets[256] = {
&&_unknown_opcode,
&&TARGET_LOAD_METHOD,
&&TARGET_CALL_METHOD,
&&_unknown_opcode,
&&_unknown_opcode,
&&TARGET_CALL_FINALLY,
&&TARGET_POP_FINALLY,
&&_unknown_opcode,
&&_unknown_opcode,
&&_unknown_opcode,

33
Python/peephole.c
View File

@ -13,7 +13,7 @@
#define UNCONDITIONAL_JUMP(op) (op==JUMP_ABSOLUTE || op==JUMP_FORWARD)
#define CONDITIONAL_JUMP(op) (op==POP_JUMP_IF_FALSE || op==POP_JUMP_IF_TRUE \
|| op==JUMP_IF_FALSE_OR_POP || op==JUMP_IF_TRUE_OR_POP)
#define ABSOLUTE_JUMP(op) (op==JUMP_ABSOLUTE || op==CONTINUE_LOOP \
#define ABSOLUTE_JUMP(op) (op==JUMP_ABSOLUTE \
|| op==POP_JUMP_IF_FALSE || op==POP_JUMP_IF_TRUE \
|| op==JUMP_IF_FALSE_OR_POP || op==JUMP_IF_TRUE_OR_POP)
#define JUMPS_ON_TRUE(op) (op==POP_JUMP_IF_TRUE || op==JUMP_IF_TRUE_OR_POP)
@ -185,12 +185,10 @@ markblocks(_Py_CODEUNIT *code, Py_ssize_t len)
case POP_JUMP_IF_FALSE:
case POP_JUMP_IF_TRUE:
case JUMP_ABSOLUTE:
case CONTINUE_LOOP:
case SETUP_LOOP:
case SETUP_EXCEPT:
case SETUP_FINALLY:
case SETUP_WITH:
case SETUP_ASYNC_WITH:
case CALL_FINALLY:
j = GETJUMPTGT(code, i);
assert(j < len);
blocks[j] = 1;
@ -373,15 +371,8 @@ PyCode_Optimize(PyObject *code, PyObject* consts, PyObject *names,
/* Replace jumps to unconditional jumps */
case POP_JUMP_IF_FALSE:
case POP_JUMP_IF_TRUE:
case FOR_ITER:
case JUMP_FORWARD:
case JUMP_ABSOLUTE:
case CONTINUE_LOOP:
case SETUP_LOOP:
case SETUP_EXCEPT:
case SETUP_FINALLY:
case SETUP_WITH:
case SETUP_ASYNC_WITH:
h = GETJUMPTGT(codestr, i);
tgt = find_op(codestr, h);
/* Replace JUMP_* to a RETURN into just a RETURN */
@ -407,7 +398,21 @@ PyCode_Optimize(PyObject *code, PyObject* consts, PyObject *names,
/* Remove unreachable ops after RETURN */
case RETURN_VALUE:
h = i + 1;
while (h < codelen && ISBASICBLOCK(blocks, i, h)) {
/* END_FINALLY should be kept since it denotes the end of
the 'finally' block in frame_setlineno() in frameobject.c.
SETUP_FINALLY should be kept for balancing.
*/
while (h < codelen && ISBASICBLOCK(blocks, i, h) &&
_Py_OPCODE(codestr[h]) != END_FINALLY)
{
if (_Py_OPCODE(codestr[h]) == SETUP_FINALLY) {
while (h > i + 1 &&
_Py_OPCODE(codestr[h - 1]) == EXTENDED_ARG)
{
h--;
}
break;
}
h++;
}
if (h > i + 1) {
@ -452,7 +457,6 @@ PyCode_Optimize(PyObject *code, PyObject* consts, PyObject *names,
case NOP:continue;
case JUMP_ABSOLUTE:
case CONTINUE_LOOP:
case POP_JUMP_IF_FALSE:
case POP_JUMP_IF_TRUE:
case JUMP_IF_FALSE_OR_POP:
@ -462,11 +466,10 @@ PyCode_Optimize(PyObject *code, PyObject* consts, PyObject *names,
case FOR_ITER:
case JUMP_FORWARD:
case SETUP_LOOP:
case SETUP_EXCEPT:
case SETUP_FINALLY:
case SETUP_WITH:
case SETUP_ASYNC_WITH:
case CALL_FINALLY:
j = blocks[j / sizeof(_Py_CODEUNIT) + i + 1] - blocks[i] - 1;
j *= sizeof(_Py_CODEUNIT);
break;