traverseda
/
cpython
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Produce cleaner bytecode for 'with' and 'async with' by generating separate code for normal and exceptional paths. (#6641) 

Remove BEGIN_FINALLY, END_FINALLY, CALL_FINALLY and POP_FINALLY bytecodes. Implement finally blocks by code duplication.
Reimplement frame.lineno setter using line numbers rather than bytecode offsets.
This commit is contained in:
Mark Shannon 2019-11-21 09:11:43 +00:00 committed by GitHub
parent 5dcc06f6e0
commit fee552669f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
15 changed files with 4789 additions and 4754 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

81
Doc/library/dis.rst
View File

@ -706,52 +706,23 @@ iterations of the loop.
popped values are used to restore the exception state.
.. opcode:: POP_FINALLY (preserve_tos)
.. opcode:: RERAISE
Cleans up the value stack and the block stack. If *preserve_tos* is not
``0`` TOS first is popped from the stack and pushed on the stack after
performing other stack operations:
* If TOS is ``NULL`` or an integer (pushed by :opcode:`BEGIN_FINALLY`
or :opcode:`CALL_FINALLY`) it is popped from the stack.
* If TOS is an exception type (pushed when an exception has been raised)
6 values are popped from the stack, the last three popped values are
used to restore the exception state. An exception handler block is
removed from the block stack.
It is similar to :opcode:`END_FINALLY`, but doesn't change the bytecode
counter nor raise an exception. Used for implementing :keyword:`break`,
:keyword:`continue` and :keyword:`return` in the :keyword:`finally` block.
Re-raises the exception currently on top of the stack.
.. versionadded:: 3.8
.. opcode:: BEGIN_FINALLY
.. opcode:: WITH_EXCEPT_START
Pushes ``NULL`` onto the stack for using it in :opcode:`END_FINALLY`,
:opcode:`POP_FINALLY`, :opcode:`WITH_CLEANUP_START` and
:opcode:`WITH_CLEANUP_FINISH`. Starts the :keyword:`finally` block.
Calls the function in position 7 on the stack with the top three
items on the stack as arguments.
Used to implement the call ``context_manager.__exit__(*exc_info())`` when an exception
has occurred in a :keyword:`with` statement.
.. versionadded:: 3.8
.. opcode:: END_FINALLY
Terminates a :keyword:`finally` clause. The interpreter recalls whether the
exception has to be re-raised or execution has to be continued depending on
the value of TOS.
* If TOS is ``NULL`` (pushed by :opcode:`BEGIN_FINALLY`) continue from
the next instruction. TOS is popped.
* If TOS is an integer (pushed by :opcode:`CALL_FINALLY`), sets the
bytecode counter to TOS. TOS is popped.
* If TOS is an exception type (pushed when an exception has been raised)
6 values are popped from the stack, the first three popped values are
used to re-raise the exception and the last three popped values are used
to restore the exception state. An exception handler block is removed
from the block stack.
.. opcode:: LOAD_ASSERTION_ERROR
Pushes :exc:`AssertionError` onto the stack. Used by the :keyword:`assert`
@ -780,35 +751,6 @@ iterations of the loop.
.. versionadded:: 3.2
.. opcode:: WITH_CLEANUP_START
Starts cleaning up the stack when a :keyword:`with` statement block exits.
At the top of the stack are either ``NULL`` (pushed by
:opcode:`BEGIN_FINALLY`) or 6 values pushed if an exception has been
raised in the with block. Below is the context manager's
:meth:`~object.__exit__` or :meth:`~object.__aexit__` bound method.
If TOS is ``NULL``, calls ``SECOND(None, None, None)``,
removes the function from the stack, leaving TOS, and pushes ``None``
to the stack. Otherwise calls ``SEVENTH(TOP, SECOND, THIRD)``,
shifts the bottom 3 values of the stack down, replaces the empty spot
with ``NULL`` and pushes TOS. Finally pushes the result of the call.
.. opcode:: WITH_CLEANUP_FINISH
Finishes cleaning up the stack when a :keyword:`with` statement block exits.
TOS is result of ``__exit__()`` or ``__aexit__()`` function call pushed
by :opcode:`WITH_CLEANUP_START`. SECOND is ``None`` or an exception type
(pushed when an exception has been raised).
Pops two values from the stack. If SECOND is not None and TOS is true
unwinds the EXCEPT_HANDLER block which was created when the exception
was caught and pushes ``NULL`` to the stack.
All of the following opcodes use their arguments.
.. opcode:: STORE_NAME (namei)
@ -1060,15 +1002,6 @@ All of the following opcodes use their arguments.
stack. *delta* points to the finally block or the first except block.
.. opcode:: CALL_FINALLY (delta)
Pushes the address of the next instruction onto the stack and increments
bytecode counter by *delta*. Used for calling the finally block as a
"subroutine".
.. versionadded:: 3.8
.. opcode:: LOAD_FAST (var_num)
Pushes a reference to the local ``co_varnames[var_num]`` onto the stack.

8
Include/opcode.h generated
View File

@ -30,10 +30,11 @@ extern "C" {
#define BINARY_TRUE_DIVIDE 27
#define INPLACE_FLOOR_DIVIDE 28
#define INPLACE_TRUE_DIVIDE 29
#define RERAISE 48
#define WITH_EXCEPT_START 49
#define GET_AITER 50
#define GET_ANEXT 51
#define BEFORE_ASYNC_WITH 52
#define BEGIN_FINALLY 53
#define END_ASYNC_FOR 54
#define INPLACE_ADD 55
#define INPLACE_SUBTRACT 56
@ -59,14 +60,11 @@ extern "C" {
#define INPLACE_AND 77
#define INPLACE_XOR 78
#define INPLACE_OR 79
#define WITH_CLEANUP_START 81
#define WITH_CLEANUP_FINISH 82
#define RETURN_VALUE 83
#define IMPORT_STAR 84
#define SETUP_ANNOTATIONS 85
#define YIELD_VALUE 86
#define POP_BLOCK 87
#define END_FINALLY 88
#define POP_EXCEPT 89
#define HAVE_ARGUMENT 90
#define STORE_NAME 90
@ -127,8 +125,6 @@ extern "C" {
#define BUILD_TUPLE_UNPACK_WITH_CALL 158
#define LOAD_METHOD 160
#define CALL_METHOD 161
#define CALL_FINALLY 162
#define POP_FINALLY 163
/* EXCEPT_HANDLER is a special, implicit block type which is created when
entering an except handler. It is not an opcode but we define it here

4
Lib/importlib/_bootstrap_external.py
View File

@ -272,6 +272,8 @@ _code_type = type(_write_atomic.__code__)
# only args in ast.arguments #37593)
# Python 3.8b4 3413 (Fix "break" and "continue" in "finally" #37830)
# Python 3.9a0 3420 (add LOAD_ASSERTION_ERROR #34880)
# Python 3.9a0 3421 (simplified bytecode for with blocks #32949)
# Python 3.9a0 3422 (remove BEGIN_FINALLY, END_FINALLY, CALL_FINALLY, POP_FINALLY bytecodes #33387)
#
# MAGIC must change whenever the bytecode emitted by the compiler may no
# longer be understood by older implementations of the eval loop (usually
@ -280,7 +282,7 @@ _code_type = type(_write_atomic.__code__)
# Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array
# in PC/launcher.c must also be updated.
MAGIC_NUMBER = (3420).to_bytes(2, 'little') + b'\r\n'
MAGIC_NUMBER = (3422).to_bytes(2, 'little') + b'\r\n'
_RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c
_PYCACHE = '__pycache__'

11
Lib/opcode.py
View File

@ -84,10 +84,12 @@ def_op('BINARY_TRUE_DIVIDE', 27)
def_op('INPLACE_FLOOR_DIVIDE', 28)
def_op('INPLACE_TRUE_DIVIDE', 29)
def_op('RERAISE', 48)
def_op('WITH_EXCEPT_START', 49)
def_op('GET_AITER', 50)
def_op('GET_ANEXT', 51)
def_op('BEFORE_ASYNC_WITH', 52)
def_op('BEGIN_FINALLY', 53)
def_op('END_ASYNC_FOR', 54)
def_op('INPLACE_ADD', 55)
def_op('INPLACE_SUBTRACT', 56)
@ -115,14 +117,13 @@ def_op('INPLACE_RSHIFT', 76)
def_op('INPLACE_AND', 77)
def_op('INPLACE_XOR', 78)
def_op('INPLACE_OR', 79)
def_op('WITH_CLEANUP_START', 81)
def_op('WITH_CLEANUP_FINISH', 82)
def_op('RETURN_VALUE', 83)
def_op('IMPORT_STAR', 84)
def_op('SETUP_ANNOTATIONS', 85)
def_op('YIELD_VALUE', 86)
def_op('POP_BLOCK', 87)
def_op('END_FINALLY', 88)
def_op('POP_EXCEPT', 89)
HAVE_ARGUMENT = 90 # Opcodes from here have an argument:
@ -210,7 +211,5 @@ def_op('BUILD_TUPLE_UNPACK_WITH_CALL', 158)
name_op('LOAD_METHOD', 160)
def_op('CALL_METHOD', 161)
jrel_op('CALL_FINALLY', 162)
def_op('POP_FINALLY', 163)
del def_op, name_op, jrel_op, jabs_op

85
Lib/test/test_dis.py
View File

@ -278,32 +278,34 @@ dis_traceback = """\
--> 6 BINARY_TRUE_DIVIDE
8 POP_TOP
10 POP_BLOCK
12 JUMP_FORWARD 40 (to 54)
12 JUMP_FORWARD 44 (to 58)
%3d >> 14 DUP_TOP
16 LOAD_GLOBAL 0 (Exception)
18 COMPARE_OP 10 (exception match)
20 POP_JUMP_IF_FALSE 52
20 POP_JUMP_IF_FALSE 56
22 POP_TOP
24 STORE_FAST 0 (e)
26 POP_TOP
28 SETUP_FINALLY 10 (to 40)
28 SETUP_FINALLY 18 (to 48)
%3d 30 LOAD_FAST 0 (e)
32 LOAD_ATTR 1 (__traceback__)
34 STORE_FAST 1 (tb)
36 POP_BLOCK
38 BEGIN_FINALLY
>> 40 LOAD_CONST 0 (None)
38 POP_EXCEPT
40 LOAD_CONST 0 (None)
42 STORE_FAST 0 (e)
44 DELETE_FAST 0 (e)
46 END_FINALLY
48 POP_EXCEPT
50 JUMP_FORWARD 2 (to 54)
>> 52 END_FINALLY
46 JUMP_FORWARD 10 (to 58)
>> 48 LOAD_CONST 0 (None)
50 STORE_FAST 0 (e)
52 DELETE_FAST 0 (e)
54 RERAISE
>> 56 RERAISE
%3d >> 54 LOAD_FAST 1 (tb)
56 RETURN_VALUE
%3d >> 58 LOAD_FAST 1 (tb)
60 RETURN_VALUE
""" % (TRACEBACK_CODE.co_firstlineno + 1,
TRACEBACK_CODE.co_firstlineno + 2,
TRACEBACK_CODE.co_firstlineno + 3,
@ -752,7 +754,7 @@ Argument count: 0
Positional-only arguments: 0
Kw-only arguments: 0
Number of locals: 2
Stack size: 10
Stack size: 9
Flags: OPTIMIZED, NEWLOCALS, NOFREE, COROUTINE
Constants:
0: None
@ -977,10 +979,10 @@ expected_opinfo_jumpy = [
Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Who let lolcatz into this test suite?', argrepr="'Who let lolcatz into this test suite?'", offset=96, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=98, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=100, starts_line=None, is_jump_target=False),
Instruction(opname='SETUP_FINALLY', opcode=122, arg=70, argval=174, argrepr='to 174', offset=102, starts_line=20, is_jump_target=True),
Instruction(opname='SETUP_FINALLY', opcode=122, arg=98, argval=202, argrepr='to 202', offset=102, starts_line=20, is_jump_target=True),
Instruction(opname='SETUP_FINALLY', opcode=122, arg=12, argval=118, argrepr='to 118', offset=104, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=106, starts_line=21, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=8, argval=0, argrepr='0', offset=108, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=7, argval=0, argrepr='0', offset=108, starts_line=None, is_jump_target=False),
Instruction(opname='BINARY_TRUE_DIVIDE', opcode=27, arg=None, argval=None, argrepr='', offset=110, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=112, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=114, starts_line=None, is_jump_target=False),
@ -993,33 +995,47 @@ expected_opinfo_jumpy = [
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=128, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=130, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=132, starts_line=23, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=9, argval='Here we go, here we go, here we go...', argrepr="'Here we go, here we go, here we go...'", offset=134, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=8, argval='Here we go, here we go, here we go...', argrepr="'Here we go, here we go, here we go...'", offset=134, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=136, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=138, starts_line=None, is_jump_target=False),
Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=140, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=26, argval=170, argrepr='to 170', offset=142, starts_line=None, is_jump_target=False),
Instruction(opname='END_FINALLY', opcode=88, arg=None, argval=None, argrepr='', offset=144, starts_line=None, is_jump_target=True),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=46, argval=190, argrepr='to 190', offset=142, starts_line=None, is_jump_target=False),
Instruction(opname='RERAISE', opcode=48, arg=None, argval=None, argrepr='', offset=144, starts_line=None, is_jump_target=True),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=146, starts_line=25, is_jump_target=True),
Instruction(opname='SETUP_WITH', opcode=143, arg=14, argval=164, argrepr='to 164', offset=148, starts_line=None, is_jump_target=False),
Instruction(opname='SETUP_WITH', opcode=143, arg=24, argval=174, argrepr='to 174', offset=148, starts_line=None, is_jump_target=False),
Instruction(opname='STORE_FAST', opcode=125, arg=1, argval='dodgy', argrepr='dodgy', offset=150, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=152, starts_line=26, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval='Never reach this', argrepr="'Never reach this'", offset=154, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=9, argval='Never reach this', argrepr="'Never reach this'", offset=154, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=156, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=158, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=160, starts_line=None, is_jump_target=False),
Instruction(opname='BEGIN_FINALLY', opcode=53, arg=None, argval=None, argrepr='', offset=162, starts_line=None, is_jump_target=False),
Instruction(opname='WITH_CLEANUP_START', opcode=81, arg=None, argval=None, argrepr='', offset=164, starts_line=None, is_jump_target=True),
Instruction(opname='WITH_CLEANUP_FINISH', opcode=82, arg=None, argval=None, argrepr='', offset=166, starts_line=None, is_jump_target=False),
Instruction(opname='END_FINALLY', opcode=88, arg=None, argval=None, argrepr='', offset=168, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=170, starts_line=None, is_jump_target=True),
Instruction(opname='BEGIN_FINALLY', opcode=53, arg=None, argval=None, argrepr='', offset=172, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=174, starts_line=28, is_jump_target=True),
Instruction(opname='LOAD_CONST', opcode=100, arg=7, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=176, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=178, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=180, starts_line=None, is_jump_target=False),
Instruction(opname='END_FINALLY', opcode=88, arg=None, argval=None, argrepr='', offset=182, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=184, starts_line=None, is_jump_target=False),
Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=186, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=162, starts_line=None, is_jump_target=False),
Instruction(opname='DUP_TOP', opcode=4, arg=None, argval=None, argrepr='', offset=164, starts_line=None, is_jump_target=False),
Instruction(opname='DUP_TOP', opcode=4, arg=None, argval=None, argrepr='', offset=166, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=3, argval=3, argrepr='', offset=168, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=170, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=16, argval=190, argrepr='to 190', offset=172, starts_line=None, is_jump_target=False),
Instruction(opname='WITH_EXCEPT_START', opcode=49, arg=None, argval=None, argrepr='', offset=174, starts_line=None, is_jump_target=True),
Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=180, argval=180, argrepr='', offset=176, starts_line=None, is_jump_target=False),
Instruction(opname='RERAISE', opcode=48, arg=None, argval=None, argrepr='', offset=178, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=180, starts_line=None, is_jump_target=True),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=182, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=184, starts_line=None, is_jump_target=False),
Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=186, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=188, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=190, starts_line=None, is_jump_target=True),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=192, starts_line=28, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=194, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=196, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=198, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=10, argval=212, argrepr='to 212', offset=200, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=202, starts_line=None, is_jump_target=True),
Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=204, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=206, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=208, starts_line=None, is_jump_target=False),
Instruction(opname='RERAISE', opcode=48, arg=None, argval=None, argrepr='', offset=210, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=212, starts_line=None, is_jump_target=True),
Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=214, starts_line=None, is_jump_target=False)
]
# One last piece of inspect fodder to check the default line number handling
@ -1032,6 +1048,10 @@ expected_opinfo_simple = [
class InstructionTests(BytecodeTestCase):
def __init__(self, *args):
super().__init__(*args)
self.maxDiff = None
def test_default_first_line(self):
actual = dis.get_instructions(simple)
self.assertEqual(list(actual), expected_opinfo_simple)
@ -1063,6 +1083,7 @@ class InstructionTests(BytecodeTestCase):
# get_instructions has its own tests above, so can rely on it to validate
# the object oriented API
class BytecodeTests(unittest.TestCase):
def test_instantiation(self):
# Test with function, method, code string and code object
for obj in [_f, _C(1).__init__, "a=1", _f.__code__]:

2
Lib/test/test_sys_settrace.py
View File

@ -1431,7 +1431,7 @@ output.append(4)
1 / 0
@jump_test(3, 2, [2], event='return', error=(ValueError,
"can't jump from a yield statement"))
"can't jump from a 'yield' statement"))
def test_no_jump_from_yield(output):
def gen():
output.append(2)

5
Misc/NEWS.d/next/Core and Builtins/2018-03-13-14-46-03.bpo-32949.v821M7.rst Normal file
View File

@ -0,0 +1,5 @@
Removed WITH_CLEANUP_START, WITH_CLEANUP_FINISH, BEGIN_FINALLY, END_FINALLY, CALL_FINALLY and POP_FINALLY bytecodes.
Replaced with RERAISE and WITH_EXCEPT_FINISH bytecodes.
The compiler now generates different code for exceptional and non-exceptional branches for 'with' and 'try-except'
statements. For 'try-finally' statements the 'finally' block is replicated for each exit from the 'try' body.

522
Objects/frameobject.c
View File

@ -66,6 +66,285 @@ get_arg(const _Py_CODEUNIT *codestr, Py_ssize_t i)
return oparg;
}
typedef struct _codetracker {
unsigned char *code;
Py_ssize_t code_len;
unsigned char *lnotab;
Py_ssize_t lnotab_len;
int start_line;
int offset;
int line;
int addr;
int line_addr;
} codetracker;
/* Reset the mutable parts of the tracker */
static void
reset(codetracker *tracker)
{
tracker->offset = 0;
tracker->addr = 0;
tracker->line_addr = 0;
tracker->line = tracker->start_line;
}
/* Initialise the tracker */
static void
init_codetracker(codetracker *tracker, PyCodeObject *code_obj)
{
PyBytes_AsStringAndSize(code_obj->co_code,
(char **)&tracker->code, &tracker->code_len);
PyBytes_AsStringAndSize(code_obj->co_lnotab,
(char **)&tracker->lnotab, &tracker->lnotab_len);
tracker->start_line = code_obj->co_firstlineno;
reset(tracker);
}
static void
advance_tracker(codetracker *tracker)
{
tracker->addr += sizeof(_Py_CODEUNIT);
if (tracker->offset >= tracker->lnotab_len) {
return;
}
while (tracker->offset < tracker->lnotab_len &&
tracker->addr >= tracker->line_addr + tracker->lnotab[tracker->offset]) {
tracker->line_addr += tracker->lnotab[tracker->offset];
tracker->line += (signed char)tracker->lnotab[tracker->offset+1];
tracker->offset += 2;
}
}
static void
retreat_tracker(codetracker *tracker)
{
tracker->addr -= sizeof(_Py_CODEUNIT);
while (tracker->addr < tracker->line_addr) {
tracker->offset -= 2;
tracker->line_addr -= tracker->lnotab[tracker->offset];
tracker->line -= (signed char)tracker->lnotab[tracker->offset+1];
}
}
static int
move_to_addr(codetracker *tracker, int addr)
{
while (addr > tracker->addr) {
advance_tracker(tracker);
if (tracker->addr >= tracker->code_len) {
return -1;
}
}
while (addr < tracker->addr) {
retreat_tracker(tracker);
if (tracker->addr < 0) {
return -1;
}
}
return 0;
}
static int
first_line_not_before(codetracker *tracker, int line)
{
int result = INT_MAX;
reset(tracker);
while (tracker->addr < tracker->code_len) {
if (tracker->line == line) {
return line;
}
if (tracker->line > line && tracker->line < result) {
result = tracker->line;
}
advance_tracker(tracker);
}
if (result == INT_MAX) {
return -1;
}
return result;
}
static int
move_to_nearest_start_of_line(codetracker *tracker, int line)
{
if (line > tracker->line) {
while (line != tracker->line) {
advance_tracker(tracker);
if (tracker->addr >= tracker->code_len) {
return -1;
}
}
}
else {
while (line != tracker->line) {
retreat_tracker(tracker);
if (tracker->addr < 0) {
return -1;
}
}
while (tracker->addr > tracker->line_addr) {
retreat_tracker(tracker);
}
}
return 0;
}
typedef struct _blockitem
{
unsigned char kind;
int end_addr;
int start_line;
} blockitem;
typedef struct _blockstack
{
blockitem stack[CO_MAXBLOCKS];
int depth;
} blockstack;
static void
init_blockstack(blockstack *blocks)
{
blocks->depth = 0;
}
static void
push_block(blockstack *blocks, unsigned char kind,
int end_addr, int start_line)
{
assert(blocks->depth < CO_MAXBLOCKS);
blocks->stack[blocks->depth].kind = kind;
blocks->stack[blocks->depth].end_addr = end_addr;
blocks->stack[blocks->depth].start_line = start_line;
blocks->depth++;
}
static unsigned char
pop_block(blockstack *blocks)
{
assert(blocks->depth > 0);
blocks->depth--;
return blocks->stack[blocks->depth].kind;
}
static blockitem *
top_block(blockstack *blocks)
{
assert(blocks->depth > 0);
return &blocks->stack[blocks->depth-1];
}
static inline int
is_try_except(unsigned char op, int target_op)
{
return op == SETUP_FINALLY && (target_op == DUP_TOP || target_op == POP_TOP);
}
static inline int
is_async_for(unsigned char op, int target_op)
{
return op == SETUP_FINALLY && target_op == END_ASYNC_FOR;
}
static inline int
is_try_finally(unsigned char op, int target_op)
{
return op == SETUP_FINALLY && !is_try_except(op, target_op) && !is_async_for(op, target_op);
}
/* Kind for finding except blocks in the jump to line code */
#define TRY_EXCEPT 250
static int
block_stack_for_line(codetracker *tracker, int line, blockstack *blocks)
{
if (line < tracker->start_line) {
return -1;
}
init_blockstack(blocks);
reset(tracker);
while (tracker->addr < tracker->code_len) {
if (tracker->line == line) {
return 0;
}
if (blocks->depth > 0 && tracker->addr == top_block(blocks)->end_addr) {
unsigned char kind = pop_block(blocks);
assert(kind != SETUP_FINALLY);
if (kind == TRY_EXCEPT) {
push_block(blocks, POP_EXCEPT, -1, tracker->line);
}
if (kind == SETUP_WITH || kind == SETUP_ASYNC_WITH) {
push_block(blocks, WITH_EXCEPT_START, -1, tracker->line);
}
}
unsigned char op = tracker->code[tracker->addr];
if (op == SETUP_FINALLY || op == SETUP_ASYNC_WITH || op == SETUP_WITH || op == FOR_ITER) {
unsigned int oparg = get_arg((const _Py_CODEUNIT *)tracker->code,
tracker->addr / sizeof(_Py_CODEUNIT));
int target_addr = tracker->addr + oparg + sizeof(_Py_CODEUNIT);
int target_op = tracker->code[target_addr];
if (is_async_for(op, target_op)) {
push_block(blocks, FOR_ITER, target_addr, tracker->line);
}
else if (op == FOR_ITER) {
push_block(blocks, FOR_ITER, target_addr-sizeof(_Py_CODEUNIT), tracker->line);
}
else if (is_try_except(op, target_op)) {
push_block(blocks, TRY_EXCEPT, target_addr-sizeof(_Py_CODEUNIT), tracker->line);
}
else if (is_try_finally(op, target_op)) {
int addr = tracker->addr;
// Skip over duplicate 'finally' blocks if line is after body.
move_to_addr(tracker, target_addr);
if (tracker->line > line) {
// Target is in body, rewind to start.
move_to_addr(tracker, addr);
push_block(blocks, op, target_addr, tracker->line);
}
else {
// Now in finally block.
push_block(blocks, RERAISE, -1, tracker->line);
}
}
else {
push_block(blocks, op, target_addr, tracker->line);
}
}
else if (op == RERAISE) {
assert(blocks->depth > 0);
unsigned char kind = top_block(blocks)->kind;
if (kind == RERAISE || kind == WITH_EXCEPT_START || kind == POP_EXCEPT) {
pop_block(blocks);
}
}
advance_tracker(tracker);
}
return -1;
}
static void
frame_stack_pop(PyFrameObject *f)
{
PyObject *v = (*--f->f_stacktop);
Py_DECREF(v);
}
static void
frame_block_unwind(PyFrameObject *f)
{
assert(f->f_iblock > 0);
f->f_iblock--;
PyTryBlock *b = &f->f_blockstack[f->f_iblock];
int delta = (f->f_stacktop - f->f_valuestack) - b->b_level;
while (delta > 0) {
frame_stack_pop(f);
delta--;
}
}
/* Setter for f_lineno - you can set f_lineno from within a trace function in
* order to jump to a given line of code, subject to some restrictions. Most
@ -77,7 +356,8 @@ get_arg(const _Py_CODEUNIT *codestr, Py_ssize_t i)
* o Lines with an 'except' statement on them can't be jumped to, because
* they expect an exception to be on the top of the stack.
* o Lines that live in a 'finally' block can't be jumped from or to, since
* the END_FINALLY expects to clean up the stack after the 'try' block.
* we cannot be sure which state the interpreter was in or would be in
* during execution of the finally block.
* o 'try', 'with' and 'async with' blocks can't be jumped into because
* the blockstack needs to be set up before their code runs.
* o 'for' and 'async for' loops can't be jumped into because the
@ -89,22 +369,6 @@ get_arg(const _Py_CODEUNIT *codestr, Py_ssize_t i)
static int
frame_setlineno(PyFrameObject *f, PyObject* p_new_lineno, void *Py_UNUSED(ignored))
{
int new_lineno = 0; /* The new value of f_lineno */
long l_new_lineno;
int overflow;
int new_lasti = 0; /* The new value of f_lasti */
unsigned char *code = NULL; /* The bytecode for the frame... */
Py_ssize_t code_len = 0; /* ...and its length */
unsigned char *lnotab = NULL; /* Iterating over co_lnotab */
Py_ssize_t lnotab_len = 0; /* (ditto) */
int offset = 0; /* (ditto) */
int line = 0; /* (ditto) */
int addr = 0; /* (ditto) */
int delta_iblock = 0; /* Scanning the SETUPs and POPs */
int delta = 0;
int blockstack[CO_MAXBLOCKS]; /* Walking the 'finally' blocks */
int blockstack_top = 0; /* (ditto) */
if (p_new_lineno == NULL) {
PyErr_SetString(PyExc_AttributeError, "cannot delete attribute");
return -1;
@ -145,8 +409,19 @@ frame_setlineno(PyFrameObject *f, PyObject* p_new_lineno, void *Py_UNUSED(ignore
return -1;
}
/* Fail if the line comes before the start of the code block. */
l_new_lineno = PyLong_AsLongAndOverflow(p_new_lineno, &overflow);
codetracker tracker;
init_codetracker(&tracker, f->f_code);
move_to_addr(&tracker, f->f_lasti);
int current_line = tracker.line;
assert(current_line >= 0);
int new_lineno;
{
/* Fail if the line falls outside the code block and
select first line with actual code. */
int overflow;
long l_new_lineno = PyLong_AsLongAndOverflow(p_new_lineno, &overflow);
if (overflow
#if SIZEOF_LONG > SIZEOF_INT
|| l_new_lineno > INT_MAX
@ -165,169 +440,100 @@ frame_setlineno(PyFrameObject *f, PyObject* p_new_lineno, void *Py_UNUSED(ignore
new_lineno);
return -1;
}
else if (new_lineno == f->f_code->co_firstlineno) {
new_lasti = 0;
new_lineno = f->f_code->co_firstlineno;
}
else {
/* Find the bytecode offset for the start of the given
* line, or the first code-owning line after it. */
char *tmp;
PyBytes_AsStringAndSize(f->f_code->co_lnotab,
&tmp, &lnotab_len);
lnotab = (unsigned char *) tmp;
addr = 0;
line = f->f_code->co_firstlineno;
new_lasti = -1;
for (offset = 0; offset < lnotab_len; offset += 2) {
addr += lnotab[offset];
line += (signed char)lnotab[offset+1];
if (line >= new_lineno) {
new_lasti = addr;
new_lineno = line;
break;
}
new_lineno = first_line_not_before(&tracker, new_lineno);
if (new_lineno < 0) {
PyErr_Format(PyExc_ValueError,
"line %d comes after the current code block",
(int)l_new_lineno);
return -1;
}
}
/* If we didn't reach the requested line, return an error. */
if (new_lasti == -1) {
PyErr_Format(PyExc_ValueError,
"line %d comes after the current code block",
new_lineno);
if (tracker.code[f->f_lasti] == YIELD_VALUE || tracker.code[f->f_lasti] == YIELD_FROM) {
PyErr_SetString(PyExc_ValueError,
"can't jump from a 'yield' statement");
return -1;
}
/* We're now ready to look at the bytecode. */
PyBytes_AsStringAndSize(f->f_code->co_code, (char **)&code, &code_len);
/* Find block stack for current line and target line. */
blockstack current_stack, new_stack;
block_stack_for_line(&tracker, new_lineno, &new_stack);
block_stack_for_line(&tracker, current_line, &current_stack);
/* The trace function is called with a 'return' trace event after the
* execution of a yield statement. */
assert(f->f_lasti != -1);
if (code[f->f_lasti] == YIELD_VALUE || code[f->f_lasti] == YIELD_FROM) {
PyErr_SetString(PyExc_ValueError,
"can't jump from a yield statement");
return -1;
}
/* You can't jump onto a line with an 'except' statement on it -
* they expect to have an exception on the top of the stack, which
* won't be true if you jump to them. They always start with code
* that either pops the exception using POP_TOP (plain 'except:'
* lines do this) or duplicates the exception on the stack using
* DUP_TOP (if there's an exception type specified). See compile.c,
* 'com_try_except' for the full details. There aren't any other
* cases (AFAIK) where a line's code can start with DUP_TOP or
* POP_TOP, but if any ever appear, they'll be subject to the same
* restriction (but with a different error message). */
if (code[new_lasti] == DUP_TOP || code[new_lasti] == POP_TOP) {
if (tracker.code[tracker.addr] == DUP_TOP || tracker.code[tracker.addr] == POP_TOP) {
PyErr_SetString(PyExc_ValueError,
"can't jump to 'except' line as there's no exception");
return -1;
}
/* You can't jump into or out of a 'finally' block because the 'try'
* block leaves something on the stack for the END_FINALLY to clean up.
* So we walk the bytecode, maintaining a simulated blockstack.
* 'blockstack' is a stack of the bytecode addresses of the starts of
* the 'finally' blocks. */
memset(blockstack, '\0', sizeof(blockstack));
blockstack_top = 0;
unsigned char prevop = NOP;
for (addr = 0; addr < code_len; addr += sizeof(_Py_CODEUNIT)) {
unsigned char op = code[addr];
switch (op) {
/* Validate change of block stack. */
if (new_stack.depth > 0) {
blockitem *current_block_at_new_depth = &(current_stack.stack[new_stack.depth-1]);
if (new_stack.depth > current_stack.depth ||
top_block(&new_stack)->start_line != current_block_at_new_depth->start_line) {
unsigned char target_kind = top_block(&new_stack)->kind;
char *msg;
if (target_kind == POP_EXCEPT) {
msg = "can't jump into an 'except' block as there's no exception";
}
else if (target_kind == RERAISE) {
msg = "can't jump into a 'finally' block";
}
else {
msg = "can't jump into the middle of a block";
}
PyErr_SetString(PyExc_ValueError, msg);
return -1;
}
}
/* Check for illegal jumps out of finally or except blocks. */
for (int depth = new_stack.depth; depth < current_stack.depth; depth++) {
switch(current_stack.stack[depth].kind) {
case RERAISE:
PyErr_SetString(PyExc_ValueError,
"can't jump out of a 'finally' block");
return -1;
case POP_EXCEPT:
PyErr_SetString(PyExc_ValueError,
"can't jump out of an 'except' block");
return -1;
}
}
/* Unwind block stack. */
while (current_stack.depth > new_stack.depth) {
unsigned char kind = pop_block(&current_stack);
switch(kind) {
case FOR_ITER:
frame_stack_pop(f);
break;
case SETUP_FINALLY:
case TRY_EXCEPT:
frame_block_unwind(f);
break;
case SETUP_WITH:
case SETUP_ASYNC_WITH:
case FOR_ITER: {
unsigned int oparg = get_arg((const _Py_CODEUNIT *)code,
addr / sizeof(_Py_CODEUNIT));
int target_addr = addr + oparg + sizeof(_Py_CODEUNIT);
assert(target_addr < code_len);
/* Police block-jumping (you can't jump into the middle of a block)
* and ensure that the blockstack finishes up in a sensible state (by
* popping any blocks we're jumping out of). We look at all the
* blockstack operations between the current position and the new
* one, and keep track of how many blocks we drop out of on the way.
* By also keeping track of the lowest blockstack position we see, we
* can tell whether the jump goes into any blocks without coming out
* again - in that case we raise an exception below. */
int first_in = addr < f->f_lasti && f->f_lasti < target_addr;
int second_in = addr < new_lasti && new_lasti < target_addr;
if (!first_in && second_in) {
PyErr_SetString(PyExc_ValueError,
"can't jump into the middle of a block");
frame_block_unwind(f);
// Pop the exit function
frame_stack_pop(f);
break;
default:
PyErr_SetString(PyExc_SystemError,
"unexpected block kind");
return -1;
}
int in_for_loop = op == FOR_ITER || code[target_addr] == END_ASYNC_FOR;
if (first_in && !second_in) {
if (!delta_iblock) {
if (in_for_loop) {
/* Pop the iterators of any 'for' and 'async for' loop
* we're jumping out of. */
delta++;
}
else if (prevop == LOAD_CONST) {
/* Pops None pushed before SETUP_FINALLY. */
delta++;
}
}
if (!in_for_loop) {
delta_iblock++;
}
}
if (!in_for_loop) {
blockstack[blockstack_top++] = target_addr;
}
break;
}
case END_FINALLY: {
assert(blockstack_top > 0);
int target_addr = blockstack[--blockstack_top];
assert(target_addr <= addr);
int first_in = target_addr <= f->f_lasti && f->f_lasti <= addr;
int second_in = target_addr <= new_lasti && new_lasti <= addr;
if (first_in != second_in) {
op = code[target_addr];
PyErr_Format(PyExc_ValueError,
"can't jump %s %s block",
second_in ? "into" : "out of",
(op == DUP_TOP || op == POP_TOP) ?
"an 'except'" : "a 'finally'");
return -1;
}
break;
}
}
prevop = op;
}
/* Verify that the blockstack tracking code didn't get lost. */
assert(blockstack_top == 0);
/* Pop any blocks that we're jumping out of. */
if (delta_iblock > 0) {
f->f_iblock -= delta_iblock;
PyTryBlock *b = &f->f_blockstack[f->f_iblock];
delta += (int)(f->f_stacktop - f->f_valuestack) - b->b_level;
if (b->b_type == SETUP_FINALLY &&
code[b->b_handler] == WITH_CLEANUP_START)
{
/* Pop the exit function. */
delta++;
}
}
while (delta > 0) {
PyObject *v = (*--f->f_stacktop);
Py_DECREF(v);
delta--;
}
move_to_addr(&tracker, f->f_lasti);
move_to_nearest_start_of_line(&tracker, new_lineno);
/* Finally set the new f_lineno and f_lasti and return OK. */
f->f_lineno = new_lineno;
f->f_lasti = new_lasti;
f->f_lasti = tracker.addr;
return 0;
}

210
Python/ceval.c
View File

@ -79,7 +79,7 @@ static PyObject * unicode_concatenate(PyThreadState *, PyObject *, PyObject *,
static PyObject * special_lookup(PyThreadState *, PyObject *, _Py_Identifier *);
static int check_args_iterable(PyThreadState *, PyObject *func, PyObject *vararg);
static void format_kwargs_error(PyThreadState *, PyObject *func, PyObject *kwargs);
static void format_awaitable_error(PyThreadState *, PyTypeObject *, int);
static void format_awaitable_error(PyThreadState *, PyTypeObject *, int, int);
#define NAME_ERROR_MSG \
"name '%.200s' is not defined"
@ -2017,7 +2017,12 @@ main_loop:
PyObject *iter = _PyCoro_GetAwaitableIter(iterable);
if (iter == NULL) {
int opcode_at_minus_3 = 0;
if ((next_instr - first_instr) > 2) {
opcode_at_minus_3 = _Py_OPCODE(next_instr[-3]);
}
format_awaitable_error(tstate, Py_TYPE(iterable),
opcode_at_minus_3,
_Py_OPCODE(next_instr[-2]));
}
@ -2128,105 +2133,14 @@ main_loop:
DISPATCH();
}
case TARGET(POP_FINALLY): {
/* If oparg is 0 at the top of the stack are 1 or 6 values:
Either:
- TOP = NULL or an integer
or:
- (TOP, SECOND, THIRD) = exc_info()
- (FOURTH, FITH, SIXTH) = previous exception for EXCEPT_HANDLER
If oparg is 1 the value for 'return' was additionally pushed
at the top of the stack.
*/
PyObject *res = NULL;
if (oparg) {
res = POP();
}
case TARGET(RERAISE): {
PyObject *exc = POP();
if (exc == NULL || PyLong_CheckExact(exc)) {
Py_XDECREF(exc);
}
else {
Py_DECREF(exc);
Py_DECREF(POP());
Py_DECREF(POP());
PyObject *type, *value, *traceback;
_PyErr_StackItem *exc_info;
PyTryBlock *b = PyFrame_BlockPop(f);
if (b->b_type != EXCEPT_HANDLER) {
_PyErr_SetString(tstate, PyExc_SystemError,
"popped block is not an except handler");
Py_XDECREF(res);
goto error;
}
assert(STACK_LEVEL() == (b)->b_level + 3);
exc_info = tstate->exc_info;
type = exc_info->exc_type;
value = exc_info->exc_value;
traceback = exc_info->exc_traceback;
exc_info->exc_type = POP();
exc_info->exc_value = POP();
exc_info->exc_traceback = POP();
Py_XDECREF(type);
Py_XDECREF(value);
Py_XDECREF(traceback);
}
if (oparg) {
PUSH(res);
}
DISPATCH();
}
case TARGET(CALL_FINALLY): {
PyObject *ret = PyLong_FromLong(INSTR_OFFSET());
if (ret == NULL) {
goto error;
}
PUSH(ret);
JUMPBY(oparg);
FAST_DISPATCH();
}
case TARGET(BEGIN_FINALLY): {
/* Push NULL onto the stack for using it in END_FINALLY,
POP_FINALLY, WITH_CLEANUP_START and WITH_CLEANUP_FINISH.
*/
PUSH(NULL);
FAST_DISPATCH();
}
case TARGET(END_FINALLY): {
PREDICTED(END_FINALLY);
/* At the top of the stack are 1 or 6 values:
Either:
- TOP = NULL or an integer
or:
- (TOP, SECOND, THIRD) = exc_info()
- (FOURTH, FITH, SIXTH) = previous exception for EXCEPT_HANDLER
*/
PyObject *exc = POP();
if (exc == NULL) {
FAST_DISPATCH();
}
else if (PyLong_CheckExact(exc)) {
int ret = _PyLong_AsInt(exc);
Py_DECREF(exc);
if (ret == -1 && _PyErr_Occurred(tstate)) {
goto error;
}
JUMPTO(ret);
FAST_DISPATCH();
}
else {
assert(PyExceptionClass_Check(exc));
PyObject *val = POP();
PyObject *tb = POP();
_PyErr_Restore(tstate, exc, val, tb);
assert(PyExceptionClass_Check(exc));
PyErr_Restore(exc, val, tb);
goto exception_unwind;
}
}
case TARGET(END_ASYNC_FOR): {
PyObject *exc = POP();
@ -3302,111 +3216,31 @@ main_loop:
DISPATCH();
}
case TARGET(WITH_CLEANUP_START): {
/* At the top of the stack are 1 or 6 values indicating
how/why we entered the finally clause:
- TOP = NULL
case TARGET(WITH_EXCEPT_START): {
/* At the top of the stack are 7 values:
- (TOP, SECOND, THIRD) = exc_info()
(FOURTH, FITH, SIXTH) = previous exception for EXCEPT_HANDLER
Below them is EXIT, the context.__exit__ or context.__aexit__
bound method.
In the first case, we must call
EXIT(None, None, None)
otherwise we must call
EXIT(TOP, SECOND, THIRD)
In the first case, we remove EXIT from the
stack, leaving TOP, and push TOP on the stack.
Otherwise we shift the bottom 3 values of the
stack down, replace the empty spot with NULL, and push
None on the stack.
Finally we push the result of the call.
- (FOURTH, FIFTH, SIXTH) = previous exception for EXCEPT_HANDLER
- SEVENTH: the context.__exit__ bound method
We call SEVENTH(TOP, SECOND, THIRD).
Then we push again the TOP exception and the __exit__
return value.
*/
PyObject *exit_func;
PyObject *exc, *val, *tb, *res;
val = tb = Py_None;
exc = TOP();
if (exc == NULL) {
STACK_SHRINK(1);
exit_func = TOP();
SET_TOP(exc);
exc = Py_None;
}
else {
assert(PyExceptionClass_Check(exc));
PyObject *tp2, *exc2, *tb2;
PyTryBlock *block;
val = SECOND();
tb = THIRD();
tp2 = FOURTH();
exc2 = PEEK(5);
tb2 = PEEK(6);
assert(exc != Py_None);
assert(!PyLong_Check(exc));
exit_func = PEEK(7);
SET_VALUE(7, tb2);
SET_VALUE(6, exc2);
SET_VALUE(5, tp2);
/* UNWIND_EXCEPT_HANDLER will pop this off. */
SET_FOURTH(NULL);
/* We just shifted the stack down, so we have
to tell the except handler block that the
values are lower than it expects. */
assert(f->f_iblock > 0);
block = &f->f_blockstack[f->f_iblock - 1];
assert(block->b_type == EXCEPT_HANDLER);
assert(block->b_level > 0);
block->b_level--;
}
PyObject *stack[4] = {NULL, exc, val, tb};
res = _PyObject_Vectorcall(exit_func, stack + 1,
3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL);
Py_DECREF(exit_func);
if (res == NULL)
goto error;
Py_INCREF(exc); /* Duplicating the exception on the stack */
PUSH(exc);
PUSH(res);
PREDICT(WITH_CLEANUP_FINISH);
DISPATCH();
}
case TARGET(WITH_CLEANUP_FINISH): {
PREDICTED(WITH_CLEANUP_FINISH);
/* TOP = the result of calling the context.__exit__ bound method
SECOND = either None or exception type
If SECOND is None below is NULL or the return address,
otherwise below are 7 values representing an exception.
*/
PyObject *res = POP();
PyObject *exc = POP();
int err;
if (exc != Py_None)
err = PyObject_IsTrue(res);
else
err = 0;
Py_DECREF(res);
Py_DECREF(exc);
if (err < 0)
goto error;
else if (err > 0) {
/* There was an exception and a True return.
* We must manually unwind the EXCEPT_HANDLER block
* which was created when the exception was caught,
* otherwise the stack will be in an inconsistent state.
*/
PyTryBlock *b = PyFrame_BlockPop(f);
assert(b->b_type == EXCEPT_HANDLER);
UNWIND_EXCEPT_HANDLER(b);
PUSH(NULL);
}
PREDICT(END_FINALLY);
DISPATCH();
}
@ -3776,6 +3610,10 @@ exception_unwind:
PUSH(val);
PUSH(exc);
JUMPTO(handler);
if (_Py_TracingPossible(ceval)) {
/* Make sure that we trace line after exception */
instr_prev = INT_MAX;
}
/* Resume normal execution */
goto main_loop;
}
@ -5477,7 +5315,7 @@ format_exc_unbound(PyThreadState *tstate, PyCodeObject *co, int oparg)
}
static void
format_awaitable_error(PyThreadState *tstate, PyTypeObject *type, int prevopcode)
format_awaitable_error(PyThreadState *tstate, PyTypeObject *type, int prevprevopcode, int prevopcode)
{
if (type->tp_as_async == NULL || type->tp_as_async->am_await == NULL) {
if (prevopcode == BEFORE_ASYNC_WITH) {
@ -5486,7 +5324,7 @@ format_awaitable_error(PyThreadState *tstate, PyTypeObject *type, int prevopcode
"that does not implement __await__: %.100s",
type->tp_name);
}
else if (prevopcode == WITH_CLEANUP_START) {
else if (prevopcode == WITH_EXCEPT_START || (prevopcode == CALL_FUNCTION && prevprevopcode == DUP_TOP)) {
_PyErr_Format(tstate, PyExc_TypeError,
"'async with' received an object from __aexit__ "
"that does not implement __await__: %.100s",

428
Python/compile.c
View File

@ -81,14 +81,16 @@ It's called a frame block to distinguish it from a basic block in the
compiler IR.
*/
enum fblocktype { WHILE_LOOP, FOR_LOOP, EXCEPT, FINALLY_TRY, FINALLY_TRY2, FINALLY_END,
WITH, ASYNC_WITH, HANDLER_CLEANUP };
enum fblocktype { WHILE_LOOP, FOR_LOOP, EXCEPT, FINALLY_TRY, FINALLY_END,
WITH, ASYNC_WITH, HANDLER_CLEANUP, POP_VALUE };
struct fblockinfo {
enum fblocktype fb_type;
basicblock *fb_block;
/* (optional) type-specific exit or cleanup block */
basicblock *fb_exit;
/* (optional) additional information required for unwinding */
void *fb_datum;
};
enum {
@ -960,12 +962,6 @@ stack_effect(int opcode, int oparg, int jump)
* Restore the stack position and push 6 values before jumping to
* the handler if an exception be raised. */
return jump ? 6 : 1;
case WITH_CLEANUP_START:
return 2; /* or 1, depending on TOS */
case WITH_CLEANUP_FINISH:
/* Pop a variable number of values pushed by WITH_CLEANUP_START
* + __exit__ or __aexit__. */
return -3;
case RETURN_VALUE:
return -1;
case IMPORT_STAR:
@ -980,10 +976,6 @@ stack_effect(int opcode, int oparg, int jump)
return 0;
case POP_EXCEPT:
return -3;
case END_FINALLY:
case POP_FINALLY:
/* Pop 6 values when an exception was raised. */
return -6;
case STORE_NAME:
return -1;
@ -1056,14 +1048,11 @@ stack_effect(int opcode, int oparg, int jump)
* Restore the stack position and push 6 values before jumping to
* the handler if an exception be raised. */
return jump ? 6 : 0;
case BEGIN_FINALLY:
/* Actually pushes 1 value, but count 6 for balancing with
* END_FINALLY and POP_FINALLY.
* This is the main reason of using this opcode instead of
* "LOAD_CONST None". */
return 6;
case CALL_FINALLY:
return jump ? 1 : 0;
case RERAISE:
return -3;
case WITH_EXCEPT_START:
return 1;
case LOAD_FAST:
return 1;
@ -1629,7 +1618,7 @@ find_ann(asdl_seq *stmts)
static int
compiler_push_fblock(struct compiler *c, enum fblocktype t, basicblock *b,
basicblock *exit)
basicblock *exit, void *datum)
{
struct fblockinfo *f;
if (c->u->u_nfblocks >= CO_MAXBLOCKS) {
@ -1641,6 +1630,7 @@ compiler_push_fblock(struct compiler *c, enum fblocktype t, basicblock *b,
f->fb_type = t;
f->fb_block = b;
f->fb_exit = exit;
f->fb_datum = datum;
return 1;
}
@ -1654,8 +1644,19 @@ compiler_pop_fblock(struct compiler *c, enum fblocktype t, basicblock *b)
assert(u->u_fblock[u->u_nfblocks].fb_block == b);
}
static int
compiler_call_exit_with_nones(struct compiler *c) {
ADDOP_O(c, LOAD_CONST, Py_None, consts);
ADDOP(c, DUP_TOP);
ADDOP(c, DUP_TOP);
ADDOP_I(c, CALL_FUNCTION, 3);
return 1;
}
/* Unwind a frame block. If preserve_tos is true, the TOS before
* popping the blocks will be restored afterwards.
* popping the blocks will be restored afterwards, unless another
* return, break or continue is found. In which case, the TOS will
* be popped.
*/
static int
compiler_unwind_fblock(struct compiler *c, struct fblockinfo *info,
@ -1665,15 +1666,6 @@ compiler_unwind_fblock(struct compiler *c, struct fblockinfo *info,
case WHILE_LOOP:
return 1;
case FINALLY_END:
info->fb_exit = NULL;
ADDOP_I(c, POP_FINALLY, preserve_tos);
if (preserve_tos) {
ADDOP(c, ROT_TWO);
}
ADDOP(c, POP_TOP);
return 1;
case FOR_LOOP:
/* Pop the iterator */
if (preserve_tos) {
@ -1688,20 +1680,28 @@ compiler_unwind_fblock(struct compiler *c, struct fblockinfo *info,
case FINALLY_TRY:
ADDOP(c, POP_BLOCK);
ADDOP_JREL(c, CALL_FINALLY, info->fb_exit);
if (preserve_tos) {
if (!compiler_push_fblock(c, POP_VALUE, NULL, NULL, NULL)) {
return 0;
}
}
VISIT_SEQ(c, stmt, info->fb_datum);
if (preserve_tos) {
compiler_pop_fblock(c, POP_VALUE, NULL);
}
return 1;
case FINALLY_TRY2:
ADDOP(c, POP_BLOCK);
case FINALLY_END:
if (preserve_tos) {
ADDOP(c, ROT_TWO);
ADDOP(c, POP_TOP);
ADDOP_JREL(c, CALL_FINALLY, info->fb_exit);
ADDOP(c, ROT_FOUR);
}
else {
ADDOP_JREL(c, CALL_FINALLY, info->fb_exit);
ADDOP(c, POP_TOP);
ADDOP(c, POP_TOP);
ADDOP(c, POP_TOP);
if (preserve_tos) {
ADDOP(c, ROT_FOUR);
}
ADDOP(c, POP_EXCEPT);
return 1;
case WITH:
@ -1710,34 +1710,66 @@ compiler_unwind_fblock(struct compiler *c, struct fblockinfo *info,
if (preserve_tos) {
ADDOP(c, ROT_TWO);
}
ADDOP(c, BEGIN_FINALLY);
ADDOP(c, WITH_CLEANUP_START);
if(!compiler_call_exit_with_nones(c)) {
return 0;
}
if (info->fb_type == ASYNC_WITH) {
ADDOP(c, GET_AWAITABLE);
ADDOP_LOAD_CONST(c, Py_None);
ADDOP(c, YIELD_FROM);
}
ADDOP(c, WITH_CLEANUP_FINISH);
ADDOP_I(c, POP_FINALLY, 0);
ADDOP(c, POP_TOP);
return 1;
case HANDLER_CLEANUP:
if (info->fb_datum) {
ADDOP(c, POP_BLOCK);
}
if (preserve_tos) {
ADDOP(c, ROT_FOUR);
}
if (info->fb_exit) {
ADDOP(c, POP_BLOCK);
ADDOP(c, POP_EXCEPT);
ADDOP_JREL(c, CALL_FINALLY, info->fb_exit);
if (info->fb_datum) {
ADDOP_LOAD_CONST(c, Py_None);
compiler_nameop(c, info->fb_datum, Store);
compiler_nameop(c, info->fb_datum, Del);
}
else {
ADDOP(c, POP_EXCEPT);
return 1;
case POP_VALUE:
if (preserve_tos) {
ADDOP(c, ROT_TWO);
}
ADDOP(c, POP_TOP);
return 1;
}
Py_UNREACHABLE();
}
/** Unwind block stack. If loop is not NULL, then stop when the first loop is encountered. */
static int
compiler_unwind_fblock_stack(struct compiler *c, int preserve_tos, struct fblockinfo **loop) {
if (c->u->u_nfblocks == 0) {
return 1;
}
struct fblockinfo *top = &c->u->u_fblock[c->u->u_nfblocks-1];
if (loop != NULL && (top->fb_type == WHILE_LOOP || top->fb_type == FOR_LOOP)) {
*loop = top;
return 1;
}
struct fblockinfo copy = *top;
c->u->u_nfblocks--;
if (!compiler_unwind_fblock(c, &copy, preserve_tos)) {
return 0;
}
if (!compiler_unwind_fblock_stack(c, preserve_tos, loop)) {
return 0;
}
c->u->u_fblock[c->u->u_nfblocks] = copy;
c->u->u_nfblocks++;
return 1;
}
/* Compile a sequence of statements, checking for a docstring
and for annotations. */
@ -2634,10 +2666,12 @@ compiler_if(struct compiler *c, stmt_ty s)
if (next == NULL)
return 0;
}
else
else {
next = end;
if (!compiler_jump_if(c, s->v.If.test, next, 0))
}
if (!compiler_jump_if(c, s->v.If.test, next, 0)) {
return 0;
}
VISIT_SEQ(c, stmt, s->v.If.body);
if (asdl_seq_LEN(s->v.If.orelse)) {
ADDOP_JREL(c, JUMP_FORWARD, end);
@ -2657,12 +2691,12 @@ compiler_for(struct compiler *c, stmt_ty s)
start = compiler_new_block(c);
cleanup = compiler_new_block(c);
end = compiler_new_block(c);
if (start == NULL || end == NULL || cleanup == NULL)
if (start == NULL || end == NULL || cleanup == NULL) {
return 0;
if (!compiler_push_fblock(c, FOR_LOOP, start, end))
}
if (!compiler_push_fblock(c, FOR_LOOP, start, end, NULL)) {
return 0;
}
VISIT(c, expr, s->v.For.iter);
ADDOP(c, GET_ITER);
compiler_use_next_block(c, start);
@ -2694,16 +2728,16 @@ compiler_async_for(struct compiler *c, stmt_ty s)
except = compiler_new_block(c);
end = compiler_new_block(c);
if (start == NULL || except == NULL || end == NULL)
if (start == NULL || except == NULL || end == NULL) {
return 0;
}
VISIT(c, expr, s->v.AsyncFor.iter);
ADDOP(c, GET_AITER);
compiler_use_next_block(c, start);
if (!compiler_push_fblock(c, FOR_LOOP, start, end))
if (!compiler_push_fblock(c, FOR_LOOP, start, end, NULL)) {
return 0;
}
/* SETUP_FINALLY to guard the __anext__ call */
ADDOP_JREL(c, SETUP_FINALLY, except);
ADDOP(c, GET_ANEXT);
@ -2741,7 +2775,7 @@ compiler_while(struct compiler *c, stmt_ty s)
// Push a dummy block so the VISIT_SEQ knows that we are
// inside a while loop so it can correctly evaluate syntax
// errors.
if (!compiler_push_fblock(c, WHILE_LOOP, NULL, NULL)) {
if (!compiler_push_fblock(c, WHILE_LOOP, NULL, NULL, NULL)) {
return 0;
}
VISIT_SEQ(c, stmt, s->v.While.body);
@ -2771,7 +2805,7 @@ compiler_while(struct compiler *c, stmt_ty s)
orelse = NULL;
compiler_use_next_block(c, loop);
if (!compiler_push_fblock(c, WHILE_LOOP, loop, end))
if (!compiler_push_fblock(c, WHILE_LOOP, loop, end, NULL))
return 0;
if (constant == -1) {
if (!compiler_jump_if(c, s->v.While.test, anchor, 0))
@ -2811,12 +2845,8 @@ compiler_return(struct compiler *c, stmt_ty s)
if (preserve_tos) {
VISIT(c, expr, s->v.Return.value);
}
for (int depth = c->u->u_nfblocks; depth--;) {
struct fblockinfo *info = &c->u->u_fblock[depth];
if (!compiler_unwind_fblock(c, info, preserve_tos))
if (!compiler_unwind_fblock_stack(c, preserve_tos, NULL))
return 0;
}
if (s->v.Return.value == NULL) {
ADDOP_LOAD_CONST(c, Py_None);
}
@ -2831,33 +2861,32 @@ compiler_return(struct compiler *c, stmt_ty s)
static int
compiler_break(struct compiler *c)
{
for (int depth = c->u->u_nfblocks; depth--;) {
struct fblockinfo *info = &c->u->u_fblock[depth];
if (!compiler_unwind_fblock(c, info, 0))
struct fblockinfo *loop = NULL;
if (!compiler_unwind_fblock_stack(c, 0, &loop)) {
return 0;
if (info->fb_type == WHILE_LOOP || info->fb_type == FOR_LOOP) {
ADDOP_JABS(c, JUMP_ABSOLUTE, info->fb_exit);
return 1;
}
}
if (loop == NULL) {
return compiler_error(c, "'break' outside loop");
}
if (!compiler_unwind_fblock(c, loop, 0)) {
return 0;
}
ADDOP_JABS(c, JUMP_ABSOLUTE, loop->fb_exit);
return 1;
}
static int
compiler_continue(struct compiler *c)
{
for (int depth = c->u->u_nfblocks; depth--;) {
struct fblockinfo *info = &c->u->u_fblock[depth];
if (info->fb_type == WHILE_LOOP || info->fb_type == FOR_LOOP) {
ADDOP_JABS(c, JUMP_ABSOLUTE, info->fb_block);
return 1;
}
if (!compiler_unwind_fblock(c, info, 0))
struct fblockinfo *loop = NULL;
if (!compiler_unwind_fblock_stack(c, 0, &loop)) {
return 0;
}
if (loop == NULL) {
return compiler_error(c, "'continue' not properly in loop");
}
ADDOP_JABS(c, JUMP_ABSOLUTE, loop->fb_block);
return 1;
}
@ -2866,10 +2895,11 @@ compiler_continue(struct compiler *c)
SETUP_FINALLY L
<code for body>
POP_BLOCK
BEGIN_FINALLY
<code for finalbody>
JUMP E
L:
<code for finalbody>
END_FINALLY
E:
The special instructions use the block stack. Each block
stack entry contains the instruction that created it (here
@ -2881,11 +2911,6 @@ compiler_continue(struct compiler *c)
onto the block stack.
POP_BLOCK:
Pops en entry from the block stack.
BEGIN_FINALLY
Pushes NULL onto the value stack.
END_FINALLY:
Pops 1 (NULL or int) or 6 entries from the *value* stack and restore
the raised and the caught exceptions they specify.
The block stack is unwound when an exception is raised:
when a SETUP_FINALLY entry is found, the raised and the caught
@ -2897,47 +2922,18 @@ compiler_continue(struct compiler *c)
static int
compiler_try_finally(struct compiler *c, stmt_ty s)
{
basicblock *start, *newcurblock, *body, *end;
int break_finally = 1;
basicblock *body, *end, *exit;
body = compiler_new_block(c);
end = compiler_new_block(c);
if (body == NULL || end == NULL)
exit = compiler_new_block(c);
if (body == NULL || end == NULL || exit == NULL)
return 0;
start = c->u->u_curblock;
/* `finally` block. Compile it first to determine if any of "break",
"continue" or "return" are used in it. */
compiler_use_next_block(c, end);
if (!compiler_push_fblock(c, FINALLY_END, end, end))
return 0;
VISIT_SEQ(c, stmt, s->v.Try.finalbody);
ADDOP(c, END_FINALLY);
break_finally = (c->u->u_fblock[c->u->u_nfblocks - 1].fb_exit == NULL);
if (break_finally) {
/* Pops a placeholder. See below */
ADDOP(c, POP_TOP);
}
compiler_pop_fblock(c, FINALLY_END, end);
newcurblock = c->u->u_curblock;
c->u->u_curblock = start;
start->b_next = NULL;
/* `try` block */
c->u->u_lineno_set = 0;
c->u->u_lineno = s->lineno;
c->u->u_col_offset = s->col_offset;
if (break_finally) {
/* Pushes a placeholder for the value of "return" in the "try" block
to balance the stack for "break", "continue" and "return" in
the "finally" block. */
ADDOP_LOAD_CONST(c, Py_None);
}
ADDOP_JREL(c, SETUP_FINALLY, end);
compiler_use_next_block(c, body);
if (!compiler_push_fblock(c, break_finally ? FINALLY_TRY2 : FINALLY_TRY, body, end))
if (!compiler_push_fblock(c, FINALLY_TRY, body, end, s->v.Try.finalbody))
return 0;
if (s->v.Try.handlers && asdl_seq_LEN(s->v.Try.handlers)) {
if (!compiler_try_except(c, s))
@ -2947,12 +2943,17 @@ compiler_try_finally(struct compiler *c, stmt_ty s)
VISIT_SEQ(c, stmt, s->v.Try.body);
}
ADDOP(c, POP_BLOCK);
ADDOP(c, BEGIN_FINALLY);
compiler_pop_fblock(c, break_finally ? FINALLY_TRY2 : FINALLY_TRY, body);
c->u->u_curblock->b_next = end;
c->u->u_curblock = newcurblock;
compiler_pop_fblock(c, FINALLY_TRY, body);
VISIT_SEQ(c, stmt, s->v.Try.finalbody);
ADDOP_JREL(c, JUMP_FORWARD, exit);
/* `finally` block */
compiler_use_next_block(c, end);
if (!compiler_push_fblock(c, FINALLY_END, end, NULL, NULL))
return 0;
VISIT_SEQ(c, stmt, s->v.Try.finalbody);
compiler_pop_fblock(c, FINALLY_END, end);
ADDOP(c, RERAISE);
compiler_use_next_block(c, exit);
return 1;
}
@ -2981,7 +2982,7 @@ compiler_try_finally(struct compiler *c, stmt_ty s)
[tb, val, exc] L2: DUP
.............................etc.......................
[tb, val, exc] Ln+1: END_FINALLY # re-raise exception
[tb, val, exc] Ln+1: RERAISE # re-raise exception
[] L0: <next statement>
@ -3001,7 +3002,7 @@ compiler_try_except(struct compiler *c, stmt_ty s)
return 0;
ADDOP_JREL(c, SETUP_FINALLY, except);
compiler_use_next_block(c, body);
if (!compiler_push_fblock(c, EXCEPT, body, NULL))
if (!compiler_push_fblock(c, EXCEPT, body, NULL, NULL))
return 0;
VISIT_SEQ(c, stmt, s->v.Try.body);
ADDOP(c, POP_BLOCK);
@ -3053,28 +3054,29 @@ compiler_try_except(struct compiler *c, stmt_ty s)
/* second try: */
ADDOP_JREL(c, SETUP_FINALLY, cleanup_end);
compiler_use_next_block(c, cleanup_body);
if (!compiler_push_fblock(c, HANDLER_CLEANUP, cleanup_body, cleanup_end))
if (!compiler_push_fblock(c, HANDLER_CLEANUP, cleanup_body, NULL, handler->v.ExceptHandler.name))
return 0;
/* second # body */
VISIT_SEQ(c, stmt, handler->v.ExceptHandler.body);
ADDOP(c, POP_BLOCK);
ADDOP(c, BEGIN_FINALLY);
compiler_pop_fblock(c, HANDLER_CLEANUP, cleanup_body);
ADDOP(c, POP_BLOCK);
ADDOP(c, POP_EXCEPT);
/* name = None; del name */
ADDOP_LOAD_CONST(c, Py_None);
compiler_nameop(c, handler->v.ExceptHandler.name, Store);
compiler_nameop(c, handler->v.ExceptHandler.name, Del);
ADDOP_JREL(c, JUMP_FORWARD, end);
/* finally: */
/* except: */
compiler_use_next_block(c, cleanup_end);
if (!compiler_push_fblock(c, FINALLY_END, cleanup_end, NULL))
return 0;
/* name = None; del name */
ADDOP_LOAD_CONST(c, Py_None);
compiler_nameop(c, handler->v.ExceptHandler.name, Store);
compiler_nameop(c, handler->v.ExceptHandler.name, Del);
ADDOP(c, END_FINALLY);
ADDOP(c, POP_EXCEPT);
compiler_pop_fblock(c, FINALLY_END, cleanup_end);
ADDOP(c, RERAISE);
}
else {
basicblock *cleanup_body;
@ -3086,16 +3088,16 @@ compiler_try_except(struct compiler *c, stmt_ty s)
ADDOP(c, POP_TOP);
ADDOP(c, POP_TOP);
compiler_use_next_block(c, cleanup_body);
if (!compiler_push_fblock(c, HANDLER_CLEANUP, cleanup_body, NULL))
if (!compiler_push_fblock(c, HANDLER_CLEANUP, cleanup_body, NULL, NULL))
return 0;
VISIT_SEQ(c, stmt, handler->v.ExceptHandler.body);
ADDOP(c, POP_EXCEPT);
compiler_pop_fblock(c, HANDLER_CLEANUP, cleanup_body);
}
ADDOP(c, POP_EXCEPT);
ADDOP_JREL(c, JUMP_FORWARD, end);
}
compiler_use_next_block(c, except);
}
ADDOP(c, END_FINALLY);
ADDOP(c, RERAISE);
compiler_use_next_block(c, orelse);
VISIT_SEQ(c, stmt, s->v.Try.orelse);
compiler_use_next_block(c, end);
@ -4630,6 +4632,22 @@ expr_constant(expr_ty e)
return -1;
}
static int
compiler_with_except_finish(struct compiler *c) {
basicblock *exit;
exit = compiler_new_block(c);
if (exit == NULL)
return 0;
ADDOP_JABS(c, POP_JUMP_IF_TRUE, exit);
ADDOP(c, RERAISE);
compiler_use_next_block(c, exit);
ADDOP(c, POP_TOP);
ADDOP(c, POP_TOP);
ADDOP(c, POP_TOP);
ADDOP(c, POP_EXCEPT);
ADDOP(c, POP_TOP);
return 1;
}
/*
Implements the async with statement.
@ -4658,7 +4676,7 @@ expr_constant(expr_ty e)
static int
compiler_async_with(struct compiler *c, stmt_ty s, int pos)
{
basicblock *block, *finally;
basicblock *block, *final, *exit;
withitem_ty item = asdl_seq_GET(s->v.AsyncWith.items, pos);
assert(s->kind == AsyncWith_kind);
@ -4669,8 +4687,9 @@ compiler_async_with(struct compiler *c, stmt_ty s, int pos)
}
block = compiler_new_block(c);
finally = compiler_new_block(c);
if (!block || !finally)
final = compiler_new_block(c);
exit = compiler_new_block(c);
if (!block || !final || !exit)
return 0;
/* Evaluate EXPR */
@ -4681,11 +4700,11 @@ compiler_async_with(struct compiler *c, stmt_ty s, int pos)
ADDOP_LOAD_CONST(c, Py_None);
ADDOP(c, YIELD_FROM);
ADDOP_JREL(c, SETUP_ASYNC_WITH, finally);
ADDOP_JREL(c, SETUP_ASYNC_WITH, final);
/* SETUP_ASYNC_WITH pushes a finally block. */
compiler_use_next_block(c, block);
if (!compiler_push_fblock(c, ASYNC_WITH, block, finally)) {
if (!compiler_push_fblock(c, ASYNC_WITH, block, final, NULL)) {
return 0;
}
@ -4704,76 +4723,80 @@ compiler_async_with(struct compiler *c, stmt_ty s, int pos)
else if (!compiler_async_with(c, s, pos))
return 0;
/* End of try block; start the finally block */
ADDOP(c, POP_BLOCK);
ADDOP(c, BEGIN_FINALLY);
compiler_pop_fblock(c, ASYNC_WITH, block);
ADDOP(c, POP_BLOCK);
/* End of body; start the cleanup */
compiler_use_next_block(c, finally);
if (!compiler_push_fblock(c, FINALLY_END, finally, NULL))
/* For successful outcome:
* call __exit__(None, None, None)
*/
if(!compiler_call_exit_with_nones(c))
return 0;
ADDOP(c, GET_AWAITABLE);
ADDOP_O(c, LOAD_CONST, Py_None, consts);
ADDOP(c, YIELD_FROM);
/* Finally block starts; context.__exit__ is on the stack under
the exception or return information. Just issue our magic
opcode. */
ADDOP(c, WITH_CLEANUP_START);
ADDOP(c, POP_TOP);
ADDOP_JABS(c, JUMP_ABSOLUTE, exit);
/* For exceptional outcome: */
compiler_use_next_block(c, final);
ADDOP(c, WITH_EXCEPT_START);
ADDOP(c, GET_AWAITABLE);
ADDOP_LOAD_CONST(c, Py_None);
ADDOP(c, YIELD_FROM);
compiler_with_except_finish(c);
ADDOP(c, WITH_CLEANUP_FINISH);
/* Finally block ends. */
ADDOP(c, END_FINALLY);
compiler_pop_fblock(c, FINALLY_END, finally);
compiler_use_next_block(c, exit);
return 1;
}
/*
Implements the with statement from PEP 343.
The semantics outlined in that PEP are as follows:
with EXPR as VAR:
BLOCK
It is implemented roughly as:
context = EXPR
exit = context.__exit__ # not calling it
value = context.__enter__()
try:
VAR = value # if VAR present in the syntax
BLOCK
finally:
if an exception was raised:
exc = copy of (exception, instance, traceback)
else:
exc = (None, None, None)
exit(*exc)
is implemented as:
<code for EXPR>
SETUP_WITH E
<code to store to VAR> or POP_TOP
<code for BLOCK>
LOAD_CONST (None, None, None)
CALL_FUNCTION_EX 0
JUMP_FORWARD EXIT
E: WITH_EXCEPT_START (calls EXPR.__exit__)
POP_JUMP_IF_TRUE T:
RERAISE
T: POP_TOP * 3 (remove exception from stack)
POP_EXCEPT
POP_TOP
EXIT:
*/
static int
compiler_with(struct compiler *c, stmt_ty s, int pos)
{
basicblock *block, *finally;
basicblock *block, *final, *exit;
withitem_ty item = asdl_seq_GET(s->v.With.items, pos);
assert(s->kind == With_kind);
block = compiler_new_block(c);
finally = compiler_new_block(c);
if (!block || !finally)
final = compiler_new_block(c);
exit = compiler_new_block(c);
if (!block || !final || !exit)
return 0;
/* Evaluate EXPR */
VISIT(c, expr, item->context_expr);
ADDOP_JREL(c, SETUP_WITH, finally);
/* Will push bound __exit__ */
ADDOP_JREL(c, SETUP_WITH, final);
/* SETUP_WITH pushes a finally block. */
compiler_use_next_block(c, block);
if (!compiler_push_fblock(c, WITH, block, finally)) {
if (!compiler_push_fblock(c, WITH, block, final, NULL)) {
return 0;
}
@ -4792,24 +4815,26 @@ compiler_with(struct compiler *c, stmt_ty s, int pos)
else if (!compiler_with(c, s, pos))
return 0;
/* End of try block; start the finally block */
ADDOP(c, POP_BLOCK);
ADDOP(c, BEGIN_FINALLY);
compiler_pop_fblock(c, WITH, block);
compiler_use_next_block(c, finally);
if (!compiler_push_fblock(c, FINALLY_END, finally, NULL))
/* End of body; start the cleanup. */
/* For successful outcome:
* call __exit__(None, None, None)
*/
if (!compiler_call_exit_with_nones(c))
return 0;
ADDOP(c, POP_TOP);
ADDOP_JREL(c, JUMP_FORWARD, exit);
/* Finally block starts; context.__exit__ is on the stack under
the exception or return information. Just issue our magic
opcode. */
ADDOP(c, WITH_CLEANUP_START);
ADDOP(c, WITH_CLEANUP_FINISH);
/* For exceptional outcome: */
compiler_use_next_block(c, final);
/* Finally block ends. */
ADDOP(c, END_FINALLY);
compiler_pop_fblock(c, FINALLY_END, finally);
ADDOP(c, WITH_EXCEPT_START);
compiler_with_except_finish(c);
compiler_use_next_block(c, exit);
return 1;
}
@ -5427,7 +5452,7 @@ Py_LOCAL_INLINE(void)
stackdepth_push(basicblock ***sp, basicblock *b, int depth)
{
assert(b->b_startdepth < 0 || b->b_startdepth == depth);
if (b->b_startdepth < depth) {
if (b->b_startdepth < depth && b->b_startdepth < 100) {
assert(b->b_startdepth < 0);
b->b_startdepth = depth;
*(*sp)++ = b;
@ -5483,19 +5508,14 @@ stackdepth(struct compiler *c)
maxdepth = target_depth;
}
assert(target_depth >= 0); /* invalid code or bug in stackdepth() */
if (instr->i_opcode == CALL_FINALLY) {
assert(instr->i_target->b_startdepth >= 0);
assert(instr->i_target->b_startdepth >= target_depth);
depth = new_depth;
continue;
}
stackdepth_push(&sp, instr->i_target, target_depth);
}
depth = new_depth;
if (instr->i_opcode == JUMP_ABSOLUTE ||
instr->i_opcode == JUMP_FORWARD ||
instr->i_opcode == RETURN_VALUE ||
instr->i_opcode == RAISE_VARARGS)
instr->i_opcode == RAISE_VARARGS ||
instr->i_opcode == RERAISE)
{
/* remaining code is dead */
next = NULL;

2165
Python/importlib.h generated
View File

File diff suppressed because it is too large Load Diff

4937
Python/importlib_external.h generated
View File

File diff suppressed because it is too large Load Diff

985
Python/importlib_zipimport.h
View File

File diff suppressed because it is too large Load Diff

16
Python/opcode_targets.h generated
View File

@ -47,12 +47,12 @@ static void *opcode_targets[256] = {
&&_unknown_opcode,
&&_unknown_opcode,
&&_unknown_opcode,
&&_unknown_opcode,
&&_unknown_opcode,
&&TARGET_RERAISE,
&&TARGET_WITH_EXCEPT_START,
&&TARGET_GET_AITER,
&&TARGET_GET_ANEXT,
&&TARGET_BEFORE_ASYNC_WITH,
&&TARGET_BEGIN_FINALLY,
&&_unknown_opcode,
&&TARGET_END_ASYNC_FOR,
&&TARGET_INPLACE_ADD,
&&TARGET_INPLACE_SUBTRACT,
@ -80,14 +80,14 @@ static void *opcode_targets[256] = {
&&TARGET_INPLACE_XOR,
&&TARGET_INPLACE_OR,
&&_unknown_opcode,
&&TARGET_WITH_CLEANUP_START,
&&TARGET_WITH_CLEANUP_FINISH,
&&_unknown_opcode,
&&_unknown_opcode,
&&TARGET_RETURN_VALUE,
&&TARGET_IMPORT_STAR,
&&TARGET_SETUP_ANNOTATIONS,
&&TARGET_YIELD_VALUE,
&&TARGET_POP_BLOCK,
&&TARGET_END_FINALLY,
&&_unknown_opcode,
&&TARGET_POP_EXCEPT,
&&TARGET_STORE_NAME,
&&TARGET_DELETE_NAME,
@ -161,8 +161,8 @@ static void *opcode_targets[256] = {
&&_unknown_opcode,
&&TARGET_LOAD_METHOD,
&&TARGET_CALL_METHOD,
&&TARGET_CALL_FINALLY,
&&TARGET_POP_FINALLY,
&&_unknown_opcode,
&&_unknown_opcode,
&&_unknown_opcode,
&&_unknown_opcode,
&&_unknown_opcode,

12
Python/peephole.c
View File

@ -198,7 +198,6 @@ markblocks(_Py_CODEUNIT *code, Py_ssize_t len)
case SETUP_FINALLY:
case SETUP_WITH:
case SETUP_ASYNC_WITH:
case CALL_FINALLY:
j = GETJUMPTGT(code, i);
assert(j < len);
blocks[j] = 1;
@ -432,14 +431,10 @@ PyCode_Optimize(PyObject *code, PyObject* consts, PyObject *names,
/* Remove unreachable ops after RETURN */
case RETURN_VALUE:
h = i + 1;
/* END_FINALLY should be kept since it denotes the end of
the 'finally' block in frame_setlineno() in frameobject.c.
SETUP_FINALLY should be kept for balancing.
*/
while (h < codelen && ISBASICBLOCK(blocks, i, h) &&
_Py_OPCODE(codestr[h]) != END_FINALLY)
while (h < codelen && ISBASICBLOCK(blocks, i, h))
{
if (_Py_OPCODE(codestr[h]) == SETUP_FINALLY) {
/* Leave SETUP_FINALLY and RERAISE in place to help find block limits. */
if (_Py_OPCODE(codestr[h]) == SETUP_FINALLY || _Py_OPCODE(codestr[h]) == RERAISE) {
while (h > i + 1 &&
_Py_OPCODE(codestr[h - 1]) == EXTENDED_ARG)
{
@ -506,7 +501,6 @@ PyCode_Optimize(PyObject *code, PyObject* consts, PyObject *names,
case SETUP_FINALLY:
case SETUP_WITH:
case SETUP_ASYNC_WITH:
case CALL_FINALLY:
j = blocks[j / sizeof(_Py_CODEUNIT) + i + 1] - blocks[i] - 1;
j *= sizeof(_Py_CODEUNIT);
break;