Mirror
/
cpython
mirror of https://github.com/python/cpython
4
1
Fork 0
Code Issues Releases Wiki Activity

bpo-40222: "Zero cost" exception handling (GH-25729) 

"Zero cost" exception handling.

* Uses a lookup table to determine how to handle exceptions.
* Removes SETUP_FINALLY and POP_TOP block instructions, eliminating (most of) the runtime overhead of try statements.
* Reduces the size of the frame object by about 60%.
This commit is contained in:
Mark Shannon 2021-05-07 15:19:19 +01:00 committed by GitHub
parent b32c8e9795
commit adcd220556
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
30 changed files with 6614 additions and 5687 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

58
Doc/library/dis.rst
View File

@ -616,13 +616,6 @@ the original TOS1.
.. versionadded:: 3.5 .. versionadded:: 3.5
.. opcode:: SETUP_ASYNC_WITH
Creates a new frame object.
.. versionadded:: 3.5
**Miscellaneous opcodes** **Miscellaneous opcodes**
@ -692,28 +685,29 @@ iterations of the loop.
opcode implements ``from module import *``. opcode implements ``from module import *``.
.. opcode:: POP_BLOCK
Removes one block from the block stack. Per frame, there is a stack of
blocks, denoting :keyword:`try` statements, and such.
.. opcode:: POP_EXCEPT .. opcode:: POP_EXCEPT
Removes one block from the block stack. The popped block must be an exception Pops three values from the stack, which are used to restore the exception state.
handler block, as implicitly created when entering an except handler. In
addition to popping extraneous values from the frame stack, the last three
popped values are used to restore the exception state.
.. opcode:: RERAISE .. opcode:: RERAISE
Re-raises the exception currently on top of the stack. If oparg is non-zero, Re-raises the exception currently on top of the stack. If oparg is non-zero,
restores ``f_lasti`` of the current frame to its value when the exception was raised. pops an additional value from the stack which is used to set ``f_lasti``
of the current frame.
.. versionadded:: 3.9 .. versionadded:: 3.9
.. opcode:: PUSH_EXC_INFO
Pops the three values from the stack. Pushes the current exception to the top of the stack.
Pushes the three values originally popped back to the stack.
Used in exception handlers.
.. versionadded:: 3.11
.. opcode:: WITH_EXCEPT_START .. opcode:: WITH_EXCEPT_START
Calls the function in position 7 on the stack with the top three Calls the function in position 7 on the stack with the top three
@ -724,6 +718,17 @@ iterations of the loop.
.. versionadded:: 3.9 .. versionadded:: 3.9
.. opcode:: POP_EXCEPT_AND_RERAISE
Pops the exception currently on top of the stack. Pops the integer value on top
of the stack and sets the ``f_lasti`` attribute of the frame with that value.
Then pops the next exception from the stack uses it to restore the current exception.
Finally it re-raises the originally popped exception.
Used in excpetion handler cleanup.
.. versionadded:: 3.11
.. opcode:: LOAD_ASSERTION_ERROR .. opcode:: LOAD_ASSERTION_ERROR
Pushes :exc:`AssertionError` onto the stack. Used by the :keyword:`assert` Pushes :exc:`AssertionError` onto the stack. Used by the :keyword:`assert`
@ -738,18 +743,15 @@ iterations of the loop.
by :opcode:`CALL_FUNCTION` to construct a class. by :opcode:`CALL_FUNCTION` to construct a class.
.. opcode:: SETUP_WITH (delta) .. opcode:: BEFORE_WITH (delta)
This opcode performs several operations before a with block starts. First, This opcode performs several operations before a with block starts. First,
it loads :meth:`~object.__exit__` from the context manager and pushes it onto it loads :meth:`~object.__exit__` from the context manager and pushes it onto
the stack for later use by :opcode:`WITH_EXCEPT_START`. Then, the stack for later use by :opcode:`WITH_EXCEPT_START`. Then,
:meth:`~object.__enter__` is called, and a finally block pointing to *delta* :meth:`~object.__enter__` is called. Finally, the result of calling the
is pushed. Finally, the result of calling the ``__enter__()`` method is pushed onto ``__enter__()`` method is pushed onto the stack.
the stack. The next opcode will either ignore it (:opcode:`POP_TOP`), or
store it in (a) variable(s) (:opcode:`STORE_FAST`, :opcode:`STORE_NAME`, or
:opcode:`UNPACK_SEQUENCE`).
.. versionadded:: 3.2 .. versionadded:: 3.11
.. opcode:: COPY_DICT_WITHOUT_KEYS .. opcode:: COPY_DICT_WITHOUT_KEYS
@ -1039,12 +1041,6 @@ All of the following opcodes use their arguments.
Loads the global named ``co_names[namei]`` onto the stack. Loads the global named ``co_names[namei]`` onto the stack.
.. opcode:: SETUP_FINALLY (delta)
Pushes a try block from a try-finally or try-except clause onto the block
stack. *delta* points to the finally block or the first except block.
.. opcode:: LOAD_FAST (var_num) .. opcode:: LOAD_FAST (var_num)
Pushes a reference to the local ``co_varnames[var_num]`` onto the stack. Pushes a reference to the local ``co_varnames[var_num]`` onto the stack.

4
Doc/whatsnew/3.11.rst
View File

@ -91,10 +91,12 @@ Optimizations
============= =============
Build and C API Changes Build and C API Changes
======================= =======================
* :c:func:`PyFrame_BlockSetup` and :c:func:`PyFrame_BlockPop` have been removed.
Deprecated Deprecated
========== ==========

5
Include/cpython/code.h
View File

@ -40,6 +40,7 @@ struct PyCodeObject {
PyObject *co_name; /* unicode (name, for reference) */ PyObject *co_name; /* unicode (name, for reference) */
PyObject *co_linetable; /* string (encoding addr<->lineno mapping) See PyObject *co_linetable; /* string (encoding addr<->lineno mapping) See
Objects/lnotab_notes.txt for details. */ Objects/lnotab_notes.txt for details. */
PyObject *co_exceptiontable; /* Byte string encoding exception handling table */
void *co_zombieframe; /* for optimization only (see frameobject.c) */ void *co_zombieframe; /* for optimization only (see frameobject.c) */
PyObject *co_weakreflist; /* to support weakrefs to code objects */ PyObject *co_weakreflist; /* to support weakrefs to code objects */
/* Scratch space for extra data relating to the code object. /* Scratch space for extra data relating to the code object.
@ -117,12 +118,12 @@ PyAPI_DATA(PyTypeObject) PyCode_Type;
PyAPI_FUNC(PyCodeObject *) PyCode_New( PyAPI_FUNC(PyCodeObject *) PyCode_New(
int, int, int, int, int, PyObject *, PyObject *, int, int, int, int, int, PyObject *, PyObject *,
PyObject *, PyObject *, PyObject *, PyObject *, PyObject *, PyObject *, PyObject *, PyObject *,
PyObject *, PyObject *, int, PyObject *); PyObject *, PyObject *, int, PyObject *, PyObject *);
PyAPI_FUNC(PyCodeObject *) PyCode_NewWithPosOnlyArgs( PyAPI_FUNC(PyCodeObject *) PyCode_NewWithPosOnlyArgs(
int, int, int, int, int, int, PyObject *, PyObject *, int, int, int, int, int, int, PyObject *, PyObject *,
PyObject *, PyObject *, PyObject *, PyObject *, PyObject *, PyObject *, PyObject *, PyObject *,
PyObject *, PyObject *, int, PyObject *); PyObject *, PyObject *, int, PyObject *, PyObject *);
/* same as struct above */ /* same as struct above */
/* Creates a new empty code object with the specified source location. */ /* Creates a new empty code object with the specified source location. */

15
Include/cpython/frameobject.h
View File

@ -34,18 +34,14 @@ struct _frame {
PyObject *f_locals; /* local symbol table (any mapping) */ PyObject *f_locals; /* local symbol table (any mapping) */
PyObject **f_valuestack; /* points after the last local */ PyObject **f_valuestack; /* points after the last local */
PyObject *f_trace; /* Trace function */ PyObject *f_trace; /* Trace function */
int f_stackdepth; /* Depth of value stack */
char f_trace_lines; /* Emit per-line trace events? */
char f_trace_opcodes; /* Emit per-opcode trace events? */
/* Borrowed reference to a generator, or NULL */ /* Borrowed reference to a generator, or NULL */
PyObject *f_gen; PyObject *f_gen;
int f_stackdepth; /* Depth of value stack */
int f_lasti; /* Last instruction if called */ int f_lasti; /* Last instruction if called */
int f_lineno; /* Current line number. Only valid if non-zero */ int f_lineno; /* Current line number. Only valid if non-zero */
int f_iblock; /* index in f_blockstack */
PyFrameState f_state; /* What state the frame is in */ PyFrameState f_state; /* What state the frame is in */
PyTryBlock f_blockstack[CO_MAXBLOCKS]; /* for try and loop blocks */ char f_trace_lines; /* Emit per-line trace events? */
char f_trace_opcodes; /* Emit per-opcode trace events? */
PyObject *f_localsplus[1]; /* locals+stack, dynamically sized */ PyObject *f_localsplus[1]; /* locals+stack, dynamically sized */
}; };
@ -77,11 +73,6 @@ _PyFrame_New_NoTrack(PyThreadState *, PyFrameConstructor *, PyObject *);
/* The rest of the interface is specific for frame objects */ /* The rest of the interface is specific for frame objects */
/* Block management functions */
PyAPI_FUNC(void) PyFrame_BlockSetup(PyFrameObject *, int, int, int);
PyAPI_FUNC(PyTryBlock *) PyFrame_BlockPop(PyFrameObject *);
/* Conversions between "fast locals" and locals in dictionary */ /* Conversions between "fast locals" and locals in dictionary */
PyAPI_FUNC(void) PyFrame_LocalsToFast(PyFrameObject *, int); PyAPI_FUNC(void) PyFrame_LocalsToFast(PyFrameObject *, int);

25
Include/opcode.h generated
View File

@ -35,10 +35,13 @@ extern "C" {
#define MATCH_SEQUENCE 32 #define MATCH_SEQUENCE 32
#define MATCH_KEYS 33 #define MATCH_KEYS 33
#define COPY_DICT_WITHOUT_KEYS 34 #define COPY_DICT_WITHOUT_KEYS 34
#define PUSH_EXC_INFO 35
#define POP_EXCEPT_AND_RERAISE 37
#define WITH_EXCEPT_START 49 #define WITH_EXCEPT_START 49
#define GET_AITER 50 #define GET_AITER 50
#define GET_ANEXT 51 #define GET_ANEXT 51
#define BEFORE_ASYNC_WITH 52 #define BEFORE_ASYNC_WITH 52
#define BEFORE_WITH 53
#define END_ASYNC_FOR 54 #define END_ASYNC_FOR 54
#define INPLACE_ADD 55 #define INPLACE_ADD 55
#define INPLACE_SUBTRACT 56 #define INPLACE_SUBTRACT 56
@ -69,7 +72,6 @@ extern "C" {
#define IMPORT_STAR 84 #define IMPORT_STAR 84
#define SETUP_ANNOTATIONS 85 #define SETUP_ANNOTATIONS 85
#define YIELD_VALUE 86 #define YIELD_VALUE 86
#define POP_BLOCK 87
#define POP_EXCEPT 89 #define POP_EXCEPT 89
#define HAVE_ARGUMENT 90 #define HAVE_ARGUMENT 90
#define STORE_NAME 90 #define STORE_NAME 90
@ -103,7 +105,6 @@ extern "C" {
#define CONTAINS_OP 118 #define CONTAINS_OP 118
#define RERAISE 119 #define RERAISE 119
#define JUMP_IF_NOT_EXC_MATCH 121 #define JUMP_IF_NOT_EXC_MATCH 121
#define SETUP_FINALLY 122
#define LOAD_FAST 124 #define LOAD_FAST 124
#define STORE_FAST 125 #define STORE_FAST 125
#define DELETE_FAST 126 #define DELETE_FAST 126
@ -118,14 +119,12 @@ extern "C" {
#define DELETE_DEREF 138 #define DELETE_DEREF 138
#define CALL_FUNCTION_KW 141 #define CALL_FUNCTION_KW 141
#define CALL_FUNCTION_EX 142 #define CALL_FUNCTION_EX 142
#define SETUP_WITH 143
#define EXTENDED_ARG 144 #define EXTENDED_ARG 144
#define LIST_APPEND 145 #define LIST_APPEND 145
#define SET_ADD 146 #define SET_ADD 146
#define MAP_ADD 147 #define MAP_ADD 147
#define LOAD_CLASSDEREF 148 #define LOAD_CLASSDEREF 148
#define MATCH_CLASS 152 #define MATCH_CLASS 152
#define SETUP_ASYNC_WITH 154
#define FORMAT_VALUE 155 #define FORMAT_VALUE 155
#define BUILD_CONST_KEY_MAP 156 #define BUILD_CONST_KEY_MAP 156
#define BUILD_STRING 157 #define BUILD_STRING 157
@ -140,8 +139,8 @@ static uint32_t _PyOpcode_RelativeJump[8] = {
0U, 0U,
0U, 0U,
536870912U, 536870912U,
67125248U, 16384U,
67141632U, 0U,
0U, 0U,
0U, 0U,
0U, 0U,
@ -150,22 +149,20 @@ static uint32_t _PyOpcode_Jump[8] = {
0U, 0U,
0U, 0U,
536870912U, 536870912U,
101695488U, 34586624U,
67141632U, 0U,
0U, 0U,
0U, 0U,
0U, 0U,
}; };
#endif /* OPCODE_TABLES */ #endif /* OPCODE_TABLES */
/* EXCEPT_HANDLER is a special, implicit block type which is created when
entering an except handler. It is not an opcode but we define it here
as we want it to be available to both frameobject.c and ceval.c, while
remaining private.*/
#define EXCEPT_HANDLER 257
#define HAS_ARG(op) ((op) >= HAVE_ARGUMENT) #define HAS_ARG(op) ((op) >= HAVE_ARGUMENT)
/* Reserve some bytecodes for internal use in the compiler.
* The value of 240 is arbitrary. */
#define IS_ARTIFICIAL(op) ((op) > 240)
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif

6
Lib/ctypes/test/test_values.py
View File

@ -80,9 +80,9 @@ class PythonValuesTestCase(unittest.TestCase):
continue continue
items.append((entry.name.decode("ascii"), entry.size)) items.append((entry.name.decode("ascii"), entry.size))
expected = [("__hello__", 137), expected = [("__hello__", 142),
("__phello__", -137), ("__phello__", -142),
("__phello__.spam", 137), ("__phello__.spam", 142),
] ]
self.assertEqual(items, expected, "PyImport_FrozenModules example " self.assertEqual(items, expected, "PyImport_FrozenModules example "
"in Doc/library/ctypes.rst may be out of date") "in Doc/library/ctypes.rst may be out of date")

56
Lib/dis.py
View File

@ -203,6 +203,9 @@ _Instruction.offset.__doc__ = "Start index of operation within bytecode sequence
_Instruction.starts_line.__doc__ = "Line started by this opcode (if any), otherwise None" _Instruction.starts_line.__doc__ = "Line started by this opcode (if any), otherwise None"
_Instruction.is_jump_target.__doc__ = "True if other code jumps to here, otherwise False" _Instruction.is_jump_target.__doc__ = "True if other code jumps to here, otherwise False"
_ExceptionTableEntry = collections.namedtuple("_ExceptionTableEntry",
"start end target depth lasti")
_OPNAME_WIDTH = 20 _OPNAME_WIDTH = 20
_OPARG_WIDTH = 5 _OPARG_WIDTH = 5
@ -308,8 +311,33 @@ def _get_name_info(name_index, name_list):
return argval, argrepr return argval, argrepr
def parse_varint(iterator):
b = next(iterator)
val = b & 63
while b&64:
val <<= 6
b = next(iterator)
val |= b&63
return val
def parse_exception_table(code):
iterator = iter(code.co_exceptiontable)
entries = []
try:
while True:
start = parse_varint(iterator)*2
length = parse_varint(iterator)*2
end = start + length
target = parse_varint(iterator)*2
dl = parse_varint(iterator)
depth = dl >> 1
lasti = bool(dl&1)
entries.append(_ExceptionTableEntry(start, end, target, depth, lasti))
except StopIteration:
return entries
def _get_instructions_bytes(code, varnames=None, names=None, constants=None, def _get_instructions_bytes(code, varnames=None, names=None, constants=None,
cells=None, linestarts=None, line_offset=0): cells=None, linestarts=None, line_offset=0, exception_entries=()):
"""Iterate over the instructions in a bytecode string. """Iterate over the instructions in a bytecode string.
Generates a sequence of Instruction namedtuples giving the details of each Generates a sequence of Instruction namedtuples giving the details of each
@ -318,7 +346,10 @@ def _get_instructions_bytes(code, varnames=None, names=None, constants=None,
arguments. arguments.
""" """
labels = findlabels(code) labels = set(findlabels(code))
for start, end, target, _, _ in exception_entries:
for i in range(start, end):
labels.add(target)
starts_line = None starts_line = None
for offset, op, arg in _unpack_opargs(code): for offset, op, arg in _unpack_opargs(code):
if linestarts is not None: if linestarts is not None:
@ -369,8 +400,10 @@ def disassemble(co, lasti=-1, *, file=None):
"""Disassemble a code object.""" """Disassemble a code object."""
cell_names = co.co_cellvars + co.co_freevars cell_names = co.co_cellvars + co.co_freevars
linestarts = dict(findlinestarts(co)) linestarts = dict(findlinestarts(co))
exception_entries = parse_exception_table(co)
_disassemble_bytes(co.co_code, lasti, co.co_varnames, co.co_names, _disassemble_bytes(co.co_code, lasti, co.co_varnames, co.co_names,
co.co_consts, cell_names, linestarts, file=file) co.co_consts, cell_names, linestarts, file=file,
exception_entries=exception_entries)
def _disassemble_recursive(co, *, file=None, depth=None): def _disassemble_recursive(co, *, file=None, depth=None):
disassemble(co, file=file) disassemble(co, file=file)
@ -385,7 +418,7 @@ def _disassemble_recursive(co, *, file=None, depth=None):
def _disassemble_bytes(code, lasti=-1, varnames=None, names=None, def _disassemble_bytes(code, lasti=-1, varnames=None, names=None,
constants=None, cells=None, linestarts=None, constants=None, cells=None, linestarts=None,
*, file=None, line_offset=0): *, file=None, line_offset=0, exception_entries=()):
# Omit the line number column entirely if we have no line number info # Omit the line number column entirely if we have no line number info
show_lineno = bool(linestarts) show_lineno = bool(linestarts)
if show_lineno: if show_lineno:
@ -403,7 +436,7 @@ def _disassemble_bytes(code, lasti=-1, varnames=None, names=None,
offset_width = 4 offset_width = 4
for instr in _get_instructions_bytes(code, varnames, names, for instr in _get_instructions_bytes(code, varnames, names,
constants, cells, linestarts, constants, cells, linestarts,
line_offset=line_offset): line_offset=line_offset, exception_entries=exception_entries):
new_source_line = (show_lineno and new_source_line = (show_lineno and
instr.starts_line is not None and instr.starts_line is not None and
instr.offset > 0) instr.offset > 0)
@ -412,6 +445,12 @@ def _disassemble_bytes(code, lasti=-1, varnames=None, names=None,
is_current_instr = instr.offset == lasti is_current_instr = instr.offset == lasti
print(instr._disassemble(lineno_width, is_current_instr, offset_width), print(instr._disassemble(lineno_width, is_current_instr, offset_width),
file=file) file=file)
if exception_entries:
print("ExceptionTable:", file=file)
for entry in exception_entries:
lasti = " lasti" if entry.lasti else ""
end = entry.end-2
print(f" {entry.start} to {end} -> {entry.target} [{entry.depth}]{lasti}", file=file)
def _disassemble_str(source, **kwargs): def _disassemble_str(source, **kwargs):
"""Compile the source string, then disassemble the code object.""" """Compile the source string, then disassemble the code object."""
@ -482,13 +521,15 @@ class Bytecode:
self._linestarts = dict(findlinestarts(co)) self._linestarts = dict(findlinestarts(co))
self._original_object = x self._original_object = x
self.current_offset = current_offset self.current_offset = current_offset
self.exception_entries = parse_exception_table(co)
def __iter__(self): def __iter__(self):
co = self.codeobj co = self.codeobj
return _get_instructions_bytes(co.co_code, co.co_varnames, co.co_names, return _get_instructions_bytes(co.co_code, co.co_varnames, co.co_names,
co.co_consts, self._cell_names, co.co_consts, self._cell_names,
self._linestarts, self._linestarts,
line_offset=self._line_offset) line_offset=self._line_offset,
exception_entries=self.exception_entries)
def __repr__(self): def __repr__(self):
return "{}({!r})".format(self.__class__.__name__, return "{}({!r})".format(self.__class__.__name__,
@ -519,7 +560,8 @@ class Bytecode:
linestarts=self._linestarts, linestarts=self._linestarts,
line_offset=self._line_offset, line_offset=self._line_offset,
file=output, file=output,
lasti=offset) lasti=offset,
exception_entries=self.exception_entries)
return output.getvalue() return output.getvalue()

3
Lib/importlib/_bootstrap_external.py
View File

@ -352,6 +352,7 @@ _code_type = type(_write_atomic.__code__)
# Python 3.10b1 3437 (Undo making 'annotations' future by default - We like to dance among core devs!) # Python 3.10b1 3437 (Undo making 'annotations' future by default - We like to dance among core devs!)
# Python 3.10b1 3438 Safer line number table handling. # Python 3.10b1 3438 Safer line number table handling.
# Python 3.10b1 3439 (Add ROT_N) # Python 3.10b1 3439 (Add ROT_N)
# Python 3.11a1 3450 Use exception table for unwinding ("zero cost" exception handling)
# #
# MAGIC must change whenever the bytecode emitted by the compiler may no # MAGIC must change whenever the bytecode emitted by the compiler may no
@ -361,7 +362,7 @@ _code_type = type(_write_atomic.__code__)
# Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array # Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array
# in PC/launcher.c must also be updated. # in PC/launcher.c must also be updated.
MAGIC_NUMBER = (3439).to_bytes(2, 'little') + b'\r\n' MAGIC_NUMBER = (3450).to_bytes(2, 'little') + b'\r\n'
_RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c _RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c
_PYCACHE = '__pycache__' _PYCACHE = '__pycache__'

9
Lib/opcode.py
View File

@ -86,11 +86,15 @@ def_op('MATCH_MAPPING', 31)
def_op('MATCH_SEQUENCE', 32) def_op('MATCH_SEQUENCE', 32)
def_op('MATCH_KEYS', 33) def_op('MATCH_KEYS', 33)
def_op('COPY_DICT_WITHOUT_KEYS', 34) def_op('COPY_DICT_WITHOUT_KEYS', 34)
def_op('PUSH_EXC_INFO', 35)
def_op('POP_EXCEPT_AND_RERAISE', 37)
def_op('WITH_EXCEPT_START', 49) def_op('WITH_EXCEPT_START', 49)
def_op('GET_AITER', 50) def_op('GET_AITER', 50)
def_op('GET_ANEXT', 51) def_op('GET_ANEXT', 51)
def_op('BEFORE_ASYNC_WITH', 52) def_op('BEFORE_ASYNC_WITH', 52)
def_op('BEFORE_WITH', 53)
def_op('END_ASYNC_FOR', 54) def_op('END_ASYNC_FOR', 54)
def_op('INPLACE_ADD', 55) def_op('INPLACE_ADD', 55)
@ -124,7 +128,6 @@ def_op('RETURN_VALUE', 83)
def_op('IMPORT_STAR', 84) def_op('IMPORT_STAR', 84)
def_op('SETUP_ANNOTATIONS', 85) def_op('SETUP_ANNOTATIONS', 85)
def_op('YIELD_VALUE', 86) def_op('YIELD_VALUE', 86)
def_op('POP_BLOCK', 87)
def_op('POP_EXCEPT', 89) def_op('POP_EXCEPT', 89)
@ -164,7 +167,6 @@ def_op('CONTAINS_OP', 118)
def_op('RERAISE', 119) def_op('RERAISE', 119)
jabs_op('JUMP_IF_NOT_EXC_MATCH', 121) jabs_op('JUMP_IF_NOT_EXC_MATCH', 121)
jrel_op('SETUP_FINALLY', 122) # Distance to target address
def_op('LOAD_FAST', 124) # Local variable number def_op('LOAD_FAST', 124) # Local variable number
haslocal.append(124) haslocal.append(124)
@ -190,7 +192,7 @@ hasfree.append(138)
def_op('CALL_FUNCTION_KW', 141) # #args + #kwargs def_op('CALL_FUNCTION_KW', 141) # #args + #kwargs
def_op('CALL_FUNCTION_EX', 142) # Flags def_op('CALL_FUNCTION_EX', 142) # Flags
jrel_op('SETUP_WITH', 143)
def_op('EXTENDED_ARG', 144) def_op('EXTENDED_ARG', 144)
EXTENDED_ARG = 144 EXTENDED_ARG = 144
def_op('LIST_APPEND', 145) def_op('LIST_APPEND', 145)
@ -201,7 +203,6 @@ hasfree.append(148)
def_op('MATCH_CLASS', 152) def_op('MATCH_CLASS', 152)
jrel_op('SETUP_ASYNC_WITH', 154)
def_op('FORMAT_VALUE', 155) def_op('FORMAT_VALUE', 155)
def_op('BUILD_CONST_KEY_MAP', 156) def_op('BUILD_CONST_KEY_MAP', 156)
def_op('BUILD_STRING', 157) def_op('BUILD_STRING', 157)

1
Lib/test/test_code.py
View File

@ -227,6 +227,7 @@ class CodeTest(unittest.TestCase):
co.co_name, co.co_name,
co.co_firstlineno, co.co_firstlineno,
co.co_lnotab, co.co_lnotab,
co.co_exceptiontable,
co.co_freevars, co.co_freevars,
co.co_cellvars) co.co_cellvars)

179
Lib/test/test_dis.py
View File

@ -282,42 +282,47 @@ dis_compound_stmt_str = """\
""" """
dis_traceback = """\ dis_traceback = """\
%3d 0 SETUP_FINALLY 7 (to 16) %3d 0 NOP
%3d 2 LOAD_CONST 1 (1) %3d 2 LOAD_CONST 1 (1)
4 LOAD_CONST 2 (0) 4 LOAD_CONST 2 (0)
--> 6 BINARY_TRUE_DIVIDE --> 6 BINARY_TRUE_DIVIDE
8 POP_TOP 8 POP_TOP
10 POP_BLOCK
%3d 12 LOAD_FAST 1 (tb) %3d 10 LOAD_FAST 1 (tb)
14 RETURN_VALUE 12 RETURN_VALUE
>> 14 PUSH_EXC_INFO
%3d >> 16 DUP_TOP %3d 16 DUP_TOP
18 LOAD_GLOBAL 0 (Exception) 18 LOAD_GLOBAL 0 (Exception)
20 JUMP_IF_NOT_EXC_MATCH 29 (to 58) 20 JUMP_IF_NOT_EXC_MATCH 28 (to 56)
22 POP_TOP 22 POP_TOP
24 STORE_FAST 0 (e) 24 STORE_FAST 0 (e)
26 POP_TOP 26 POP_TOP
28 SETUP_FINALLY 10 (to 50)
%3d 30 LOAD_FAST 0 (e) %3d 28 LOAD_FAST 0 (e)
32 LOAD_ATTR 1 (__traceback__) 30 LOAD_ATTR 1 (__traceback__)
34 STORE_FAST 1 (tb) 32 STORE_FAST 1 (tb)
36 POP_BLOCK 34 POP_EXCEPT
38 POP_EXCEPT 36 LOAD_CONST 0 (None)
40 LOAD_CONST 0 (None) 38 STORE_FAST 0 (e)
42 STORE_FAST 0 (e) 40 DELETE_FAST 0 (e)
44 DELETE_FAST 0 (e)
%3d 46 LOAD_FAST 1 (tb) %3d 42 LOAD_FAST 1 (tb)
48 RETURN_VALUE 44 RETURN_VALUE
>> 50 LOAD_CONST 0 (None) >> 46 LOAD_CONST 0 (None)
52 STORE_FAST 0 (e) 48 STORE_FAST 0 (e)
54 DELETE_FAST 0 (e) 50 DELETE_FAST 0 (e)
56 RERAISE 1 52 RERAISE 1
>> 54 POP_EXCEPT_AND_RERAISE
%3d >> 58 RERAISE 0 %3d >> 56 RERAISE 0
ExceptionTable:
2 to 8 -> 14 [0]
14 to 26 -> 54 [3] lasti
28 to 32 -> 46 [3] lasti
46 to 52 -> 54 [3] lasti
56 to 56 -> 54 [3] lasti
""" % (TRACEBACK_CODE.co_firstlineno + 1, """ % (TRACEBACK_CODE.co_firstlineno + 1,
TRACEBACK_CODE.co_firstlineno + 2, TRACEBACK_CODE.co_firstlineno + 2,
TRACEBACK_CODE.co_firstlineno + 5, TRACEBACK_CODE.co_firstlineno + 5,
@ -360,38 +365,46 @@ def _tryfinallyconst(b):
b() b()
dis_tryfinally = """\ dis_tryfinally = """\
%3d 0 SETUP_FINALLY 6 (to 14) %3d 0 NOP
%3d 2 LOAD_FAST 0 (a) %3d 2 LOAD_FAST 0 (a)
4 POP_BLOCK
%3d 6 LOAD_FAST 1 (b) %3d 4 LOAD_FAST 1 (b)
8 CALL_FUNCTION 0 6 CALL_FUNCTION 0
10 POP_TOP 8 POP_TOP
12 RETURN_VALUE 10 RETURN_VALUE
>> 14 LOAD_FAST 1 (b) >> 12 PUSH_EXC_INFO
14 LOAD_FAST 1 (b)
16 CALL_FUNCTION 0 16 CALL_FUNCTION 0
18 POP_TOP 18 POP_TOP
20 RERAISE 0 20 RERAISE 0
>> 22 POP_EXCEPT_AND_RERAISE
ExceptionTable:
2 to 2 -> 12 [0]
12 to 20 -> 22 [3] lasti
""" % (_tryfinally.__code__.co_firstlineno + 1, """ % (_tryfinally.__code__.co_firstlineno + 1,
_tryfinally.__code__.co_firstlineno + 2, _tryfinally.__code__.co_firstlineno + 2,
_tryfinally.__code__.co_firstlineno + 4, _tryfinally.__code__.co_firstlineno + 4,
) )
dis_tryfinallyconst = """\ dis_tryfinallyconst = """\
%3d 0 SETUP_FINALLY 6 (to 14) %3d 0 NOP
%3d 2 POP_BLOCK %3d 2 NOP
%3d 4 LOAD_FAST 0 (b) %3d 4 LOAD_FAST 0 (b)
6 CALL_FUNCTION 0 6 CALL_FUNCTION 0
8 POP_TOP 8 POP_TOP
10 LOAD_CONST 1 (1) 10 LOAD_CONST 1 (1)
12 RETURN_VALUE 12 RETURN_VALUE
>> 14 LOAD_FAST 0 (b) 14 PUSH_EXC_INFO
16 CALL_FUNCTION 0 16 LOAD_FAST 0 (b)
18 POP_TOP 18 CALL_FUNCTION 0
20 RERAISE 0 20 POP_TOP
22 RERAISE 0
>> 24 POP_EXCEPT_AND_RERAISE
ExceptionTable:
14 to 22 -> 24 [3] lasti
""" % (_tryfinallyconst.__code__.co_firstlineno + 1, """ % (_tryfinallyconst.__code__.co_firstlineno + 1,
_tryfinallyconst.__code__.co_firstlineno + 2, _tryfinallyconst.__code__.co_firstlineno + 2,
_tryfinallyconst.__code__.co_firstlineno + 4, _tryfinallyconst.__code__.co_firstlineno + 4,
@ -833,7 +846,7 @@ Argument count: 0
Positional-only arguments: 0 Positional-only arguments: 0
Kw-only arguments: 0 Kw-only arguments: 0
Number of locals: 2 Number of locals: 2
Stack size: 9 Stack size: 10
Flags: OPTIMIZED, NEWLOCALS, NOFREE, COROUTINE Flags: OPTIMIZED, NEWLOCALS, NOFREE, COROUTINE
Constants: Constants:
0: None 0: None
@ -1059,61 +1072,64 @@ expected_opinfo_jumpy = [
Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Who let lolcatz into this test suite?', argrepr="'Who let lolcatz into this test suite?'", offset=98, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=6, argval='Who let lolcatz into this test suite?', argrepr="'Who let lolcatz into this test suite?'", offset=98, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=100, starts_line=None, is_jump_target=False), Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=100, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=102, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=102, starts_line=None, is_jump_target=False),
Instruction(opname='SETUP_FINALLY', opcode=122, arg=48, argval=202, argrepr='to 202', offset=104, starts_line=20, is_jump_target=True), Instruction(opname='NOP', opcode=9, arg=None, argval=None, argrepr='', offset=104, starts_line=20, is_jump_target=True),
Instruction(opname='SETUP_FINALLY', opcode=122, arg=6, argval=120, argrepr='to 120', offset=106, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=106, starts_line=21, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=5, argval=1, argrepr='1', offset=108, starts_line=21, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=7, argval=0, argrepr='0', offset=108, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=7, argval=0, argrepr='0', offset=110, starts_line=None, is_jump_target=False), Instruction(opname='BINARY_TRUE_DIVIDE', opcode=27, arg=None, argval=None, argrepr='', offset=110, starts_line=None, is_jump_target=False),
Instruction(opname='BINARY_TRUE_DIVIDE', opcode=27, arg=None, argval=None, argrepr='', offset=112, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=112, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=114, starts_line=None, is_jump_target=False), Instruction(opname='JUMP_FORWARD', opcode=110, arg=14, argval=144, argrepr='to 144', offset=114, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=116, starts_line=None, is_jump_target=False), Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=116, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=12, argval=144, argrepr='to 144', offset=118, starts_line=None, is_jump_target=False), Instruction(opname='DUP_TOP', opcode=4, arg=None, argval=None, argrepr='', offset=118, starts_line=22, is_jump_target=False),
Instruction(opname='DUP_TOP', opcode=4, arg=None, argval=None, argrepr='', offset=120, starts_line=22, is_jump_target=True), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=2, argval='ZeroDivisionError', argrepr='ZeroDivisionError', offset=120, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=2, argval='ZeroDivisionError', argrepr='ZeroDivisionError', offset=122, starts_line=None, is_jump_target=False), Instruction(opname='JUMP_IF_NOT_EXC_MATCH', opcode=121, arg=109, argval=218, argrepr='to 218', offset=122, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_IF_NOT_EXC_MATCH', opcode=121, arg=106, argval=212, argrepr='to 212', offset=124, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=124, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=126, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=126, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=128, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=128, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=130, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=130, starts_line=23, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=132, starts_line=23, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=8, argval='Here we go, here we go, here we go...', argrepr="'Here we go, here we go, here we go...'", offset=132, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=8, argval='Here we go, here we go, here we go...', argrepr="'Here we go, here we go, here we go...'", offset=134, starts_line=None, is_jump_target=False), Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=134, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=136, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=136, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=138, starts_line=None, is_jump_target=False), Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=138, starts_line=None, is_jump_target=False),
Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=140, starts_line=None, is_jump_target=False), Instruction(opname='JUMP_FORWARD', opcode=110, arg=25, argval=192, argrepr='to 192', offset=140, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=22, argval=188, argrepr='to 188', offset=142, starts_line=None, is_jump_target=False), Instruction(opname='POP_EXCEPT_AND_RERAISE', opcode=37, arg=None, argval=None, argrepr='', offset=142, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=144, starts_line=25, is_jump_target=True), Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='i', argrepr='i', offset=144, starts_line=25, is_jump_target=True),
Instruction(opname='SETUP_WITH', opcode=143, arg=12, argval=172, argrepr='to 172', offset=146, starts_line=None, is_jump_target=False), Instruction(opname='BEFORE_WITH', opcode=53, arg=None, argval=None, argrepr='', offset=146, starts_line=None, is_jump_target=False),
Instruction(opname='STORE_FAST', opcode=125, arg=1, argval='dodgy', argrepr='dodgy', offset=148, starts_line=None, is_jump_target=False), Instruction(opname='STORE_FAST', opcode=125, arg=1, argval='dodgy', argrepr='dodgy', offset=148, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=150, starts_line=26, is_jump_target=False), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=150, starts_line=26, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=9, argval='Never reach this', argrepr="'Never reach this'", offset=152, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=9, argval='Never reach this', argrepr="'Never reach this'", offset=152, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=154, starts_line=None, is_jump_target=False), Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=154, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=156, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=156, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=158, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=158, starts_line=25, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=160, starts_line=25, is_jump_target=False), Instruction(opname='DUP_TOP', opcode=4, arg=None, argval=None, argrepr='', offset=160, starts_line=None, is_jump_target=False),
Instruction(opname='DUP_TOP', opcode=4, arg=None, argval=None, argrepr='', offset=162, starts_line=None, is_jump_target=False), Instruction(opname='DUP_TOP', opcode=4, arg=None, argval=None, argrepr='', offset=162, starts_line=None, is_jump_target=False),
Instruction(opname='DUP_TOP', opcode=4, arg=None, argval=None, argrepr='', offset=164, starts_line=None, is_jump_target=False), Instruction(opname='CALL_FUNCTION', opcode=131, arg=3, argval=3, argrepr='', offset=164, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=3, argval=3, argrepr='', offset=166, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=166, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=168, starts_line=None, is_jump_target=False), Instruction(opname='JUMP_FORWARD', opcode=110, arg=11, argval=192, argrepr='to 192', offset=168, starts_line=None, is_jump_target=False),
Instruction(opname='JUMP_FORWARD', opcode=110, arg=8, argval=188, argrepr='to 188', offset=170, starts_line=None, is_jump_target=False), Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=170, starts_line=None, is_jump_target=False),
Instruction(opname='WITH_EXCEPT_START', opcode=49, arg=None, argval=None, argrepr='', offset=172, starts_line=None, is_jump_target=True), Instruction(opname='WITH_EXCEPT_START', opcode=49, arg=None, argval=None, argrepr='', offset=172, starts_line=None, is_jump_target=False),
Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=89, argval=178, argrepr='to 178', offset=174, starts_line=None, is_jump_target=False), Instruction(opname='POP_JUMP_IF_TRUE', opcode=115, arg=90, argval=180, argrepr='to 180', offset=174, starts_line=None, is_jump_target=False),
Instruction(opname='RERAISE', opcode=119, arg=1, argval=1, argrepr='', offset=176, starts_line=None, is_jump_target=False), Instruction(opname='RERAISE', opcode=119, arg=4, argval=4, argrepr='', offset=176, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=178, starts_line=None, is_jump_target=True), Instruction(opname='POP_EXCEPT_AND_RERAISE', opcode=37, arg=None, argval=None, argrepr='', offset=178, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=180, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=180, starts_line=None, is_jump_target=True),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=182, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=182, starts_line=None, is_jump_target=False),
Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=184, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=184, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=186, starts_line=None, is_jump_target=False), Instruction(opname='POP_EXCEPT', opcode=89, arg=None, argval=None, argrepr='', offset=186, starts_line=None, is_jump_target=False),
Instruction(opname='POP_BLOCK', opcode=87, arg=None, argval=None, argrepr='', offset=188, starts_line=None, is_jump_target=True), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=188, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=190, starts_line=28, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=190, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=192, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=192, starts_line=28, is_jump_target=True),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=194, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=194, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=196, starts_line=None, is_jump_target=False), Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=196, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=198, starts_line=None, is_jump_target=False), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=198, starts_line=None, is_jump_target=False),
Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=200, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=None, argrepr='None', offset=200, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=202, starts_line=None, is_jump_target=True), Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, argrepr='', offset=202, starts_line=None, is_jump_target=False),
Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=204, starts_line=None, is_jump_target=False), Instruction(opname='PUSH_EXC_INFO', opcode=35, arg=None, argval=None, argrepr='', offset=204, starts_line=None, is_jump_target=False),
Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=206, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_GLOBAL', opcode=116, arg=1, argval='print', argrepr='print', offset=206, starts_line=None, is_jump_target=False),
Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=208, starts_line=None, is_jump_target=False), Instruction(opname='LOAD_CONST', opcode=100, arg=10, argval="OK, now we're done", argrepr='"OK, now we\'re done"', offset=208, starts_line=None, is_jump_target=False),
Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=210, starts_line=None, is_jump_target=False), Instruction(opname='CALL_FUNCTION', opcode=131, arg=1, argval=1, argrepr='', offset=210, starts_line=None, is_jump_target=False),
Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=212, starts_line=22, is_jump_target=True), Instruction(opname='POP_TOP', opcode=1, arg=None, argval=None, argrepr='', offset=212, starts_line=None, is_jump_target=False),
Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=214, starts_line=None, is_jump_target=False),
Instruction(opname='POP_EXCEPT_AND_RERAISE', opcode=37, arg=None, argval=None, argrepr='', offset=216, starts_line=None, is_jump_target=False),
Instruction(opname='RERAISE', opcode=119, arg=0, argval=0, argrepr='', offset=218, starts_line=22, is_jump_target=True),
] ]
# One last piece of inspect fodder to check the default line number handling # One last piece of inspect fodder to check the default line number handling
@ -1211,6 +1227,7 @@ class BytecodeTests(unittest.TestCase):
self.assertEqual(b.current_offset, tb.tb_lasti) self.assertEqual(b.current_offset, tb.tb_lasti)
def test_from_traceback_dis(self): def test_from_traceback_dis(self):
self.maxDiff = None
tb = get_tb() tb = get_tb()
b = dis.Bytecode.from_traceback(tb) b = dis.Bytecode.from_traceback(tb)
self.assertEqual(b.dis(), dis_traceback) self.assertEqual(b.dis(), dis_traceback)

2
Lib/test/test_exceptions.py
View File

@ -218,7 +218,7 @@ class ExceptionTests(unittest.TestCase):
check('class foo:return 1', 1, 11) check('class foo:return 1', 1, 11)
check('def f():\n continue', 2, 3) check('def f():\n continue', 2, 3)
check('def f():\n break', 2, 3) check('def f():\n break', 2, 3)
check('try:\n pass\nexcept:\n pass\nexcept ValueError:\n pass', 2, 3) check('try:\n pass\nexcept:\n pass\nexcept ValueError:\n pass', 3, 1)
# Errors thrown by tokenizer.c # Errors thrown by tokenizer.c
check('(0x+1)', 1, 3) check('(0x+1)', 1, 3)

7
Lib/test/test_sys.py
View File

@ -1273,13 +1273,12 @@ class SizeofTest(unittest.TestCase):
check(sys.float_info, vsize('') + self.P * len(sys.float_info)) check(sys.float_info, vsize('') + self.P * len(sys.float_info))
# frame # frame
import inspect import inspect
CO_MAXBLOCKS = 20
x = inspect.currentframe() x = inspect.currentframe()
ncells = len(x.f_code.co_cellvars) ncells = len(x.f_code.co_cellvars)
nfrees = len(x.f_code.co_freevars) nfrees = len(x.f_code.co_freevars)
extras = x.f_code.co_stacksize + x.f_code.co_nlocals +\ localsplus = x.f_code.co_stacksize + x.f_code.co_nlocals +\
ncells + nfrees - 1 ncells + nfrees
check(x, vsize('4Pi2c4P3ic' + CO_MAXBLOCKS*'3i' + 'P' + extras*'P')) check(x, vsize('8P3i3c' + localsplus*'P'))
# function # function
def func(): pass def func(): pass
check(func, size('14P')) check(func, size('14P'))

56
Lib/test/test_sys_settrace.py
View File

@ -1253,7 +1253,7 @@ class JumpTestCase(unittest.TestCase):
output.append(6) output.append(6)
output.append(7) output.append(7)
@async_jump_test(4, 5, [3, 5]) @async_jump_test(4, 5, [3], (ValueError, 'into'))
async def test_jump_out_of_async_for_block_forwards(output): async def test_jump_out_of_async_for_block_forwards(output):
for i in [1]: for i in [1]:
async for i in asynciter([1, 2]): async for i in asynciter([1, 2]):
@ -1295,7 +1295,7 @@ class JumpTestCase(unittest.TestCase):
output.append(8) output.append(8)
output.append(9) output.append(9)
@jump_test(6, 7, [2, 7], (ZeroDivisionError, '')) @jump_test(6, 7, [2], (ValueError, 'within'))
def test_jump_in_nested_finally_2(output): def test_jump_in_nested_finally_2(output):
try: try:
output.append(2) output.append(2)
@ -1306,7 +1306,7 @@ class JumpTestCase(unittest.TestCase):
output.append(7) output.append(7)
output.append(8) output.append(8)
@jump_test(6, 11, [2, 11], (ZeroDivisionError, '')) @jump_test(6, 11, [2], (ValueError, 'within'))
def test_jump_in_nested_finally_3(output): def test_jump_in_nested_finally_3(output):
try: try:
output.append(2) output.append(2)
@ -1321,7 +1321,7 @@ class JumpTestCase(unittest.TestCase):
output.append(11) output.append(11)
output.append(12) output.append(12)
@jump_test(5, 11, [2, 4], (ValueError, 'after')) @jump_test(5, 11, [2, 4], (ValueError, 'exception'))
def test_no_jump_over_return_try_finally_in_finally_block(output): def test_no_jump_over_return_try_finally_in_finally_block(output):
try: try:
output.append(2) output.append(2)
@ -1417,8 +1417,8 @@ class JumpTestCase(unittest.TestCase):
output.append(5) output.append(5)
raise raise
@jump_test(5, 7, [4, 7, 8]) @jump_test(5, 7, [4], (ValueError, 'within'))
def test_jump_between_except_blocks(output): def test_no_jump_between_except_blocks(output):
try: try:
1/0 1/0
except ZeroDivisionError: except ZeroDivisionError:
@ -1428,8 +1428,8 @@ class JumpTestCase(unittest.TestCase):
output.append(7) output.append(7)
output.append(8) output.append(8)
@jump_test(5, 6, [4, 6, 7]) @jump_test(5, 6, [4], (ValueError, 'within'))
def test_jump_within_except_block(output): def test_no_jump_within_except_block(output):
try: try:
1/0 1/0
except: except:
@ -1654,54 +1654,54 @@ class JumpTestCase(unittest.TestCase):
output.append(2) output.append(2)
output.append(3) output.append(3)
@async_jump_test(3, 2, [2, 2], (ValueError, 'into')) @async_jump_test(3, 2, [2, 2], (ValueError, 'within'))
async def test_no_jump_backwards_into_async_for_block(output): async def test_no_jump_backwards_into_async_for_block(output):
async for i in asynciter([1, 2]): async for i in asynciter([1, 2]):
output.append(2) output.append(2)
output.append(3) output.append(3)
@jump_test(1, 3, [], (ValueError, 'into')) @jump_test(1, 3, [], (ValueError, 'depth'))
def test_no_jump_forwards_into_with_block(output): def test_no_jump_forwards_into_with_block(output):
output.append(1) output.append(1)
with tracecontext(output, 2): with tracecontext(output, 2):
output.append(3) output.append(3)
@async_jump_test(1, 3, [], (ValueError, 'into')) @async_jump_test(1, 3, [], (ValueError, 'depth'))
async def test_no_jump_forwards_into_async_with_block(output): async def test_no_jump_forwards_into_async_with_block(output):
output.append(1) output.append(1)
async with asynctracecontext(output, 2): async with asynctracecontext(output, 2):
output.append(3) output.append(3)
@jump_test(3, 2, [1, 2, -1], (ValueError, 'into')) @jump_test(3, 2, [1, 2, -1], (ValueError, 'depth'))
def test_no_jump_backwards_into_with_block(output): def test_no_jump_backwards_into_with_block(output):
with tracecontext(output, 1): with tracecontext(output, 1):
output.append(2) output.append(2)
output.append(3) output.append(3)
@async_jump_test(3, 2, [1, 2, -1], (ValueError, 'into')) @async_jump_test(3, 2, [1, 2, -1], (ValueError, 'depth'))
async def test_no_jump_backwards_into_async_with_block(output): async def test_no_jump_backwards_into_async_with_block(output):
async with asynctracecontext(output, 1): async with asynctracecontext(output, 1):
output.append(2) output.append(2)
output.append(3) output.append(3)
@jump_test(1, 3, [], (ValueError, 'into')) @jump_test(1, 3, [3, 5])
def test_no_jump_forwards_into_try_finally_block(output): def test_jump_forwards_into_try_finally_block(output):
output.append(1) output.append(1)
try: try:
output.append(3) output.append(3)
finally: finally:
output.append(5) output.append(5)
@jump_test(5, 2, [2, 4], (ValueError, 'into')) @jump_test(5, 2, [2, 4, 2, 4, 5])
def test_no_jump_backwards_into_try_finally_block(output): def test_jump_backwards_into_try_finally_block(output):
try: try:
output.append(2) output.append(2)
finally: finally:
output.append(4) output.append(4)
output.append(5) output.append(5)
@jump_test(1, 3, [], (ValueError, 'into')) @jump_test(1, 3, [3])
def test_no_jump_forwards_into_try_except_block(output): def test_jump_forwards_into_try_except_block(output):
output.append(1) output.append(1)
try: try:
output.append(3) output.append(3)
@ -1709,8 +1709,8 @@ class JumpTestCase(unittest.TestCase):
output.append(5) output.append(5)
raise raise
@jump_test(6, 2, [2], (ValueError, 'into')) @jump_test(6, 2, [2, 2, 6])
def test_no_jump_backwards_into_try_except_block(output): def test_jump_backwards_into_try_except_block(output):
try: try:
output.append(2) output.append(2)
except: except:
@ -1719,7 +1719,7 @@ class JumpTestCase(unittest.TestCase):
output.append(6) output.append(6)
# 'except' with a variable creates an implicit finally block # 'except' with a variable creates an implicit finally block
@jump_test(5, 7, [4], (ValueError, 'into')) @jump_test(5, 7, [4], (ValueError, 'within'))
def test_no_jump_between_except_blocks_2(output): def test_no_jump_between_except_blocks_2(output):
try: try:
1/0 1/0
@ -1756,7 +1756,7 @@ class JumpTestCase(unittest.TestCase):
finally: finally:
output.append(5) output.append(5)
@jump_test(1, 5, [], (ValueError, "into an 'except'")) @jump_test(1, 5, [], (ValueError, "into an exception"))
def test_no_jump_into_bare_except_block(output): def test_no_jump_into_bare_except_block(output):
output.append(1) output.append(1)
try: try:
@ -1764,7 +1764,7 @@ class JumpTestCase(unittest.TestCase):
except: except:
output.append(5) output.append(5)
@jump_test(1, 5, [], (ValueError, "into an 'except'")) @jump_test(1, 5, [], (ValueError, "into an exception"))
def test_no_jump_into_qualified_except_block(output): def test_no_jump_into_qualified_except_block(output):
output.append(1) output.append(1)
try: try:
@ -1772,7 +1772,7 @@ class JumpTestCase(unittest.TestCase):
except Exception: except Exception:
output.append(5) output.append(5)
@jump_test(3, 6, [2, 5, 6], (ValueError, "into an 'except'")) @jump_test(3, 6, [2, 5, 6], (ValueError, "into an exception"))
def test_no_jump_into_bare_except_block_from_try_block(output): def test_no_jump_into_bare_except_block_from_try_block(output):
try: try:
output.append(2) output.append(2)
@ -1783,7 +1783,7 @@ class JumpTestCase(unittest.TestCase):
raise raise
output.append(8) output.append(8)
@jump_test(3, 6, [2], (ValueError, "into an 'except'")) @jump_test(3, 6, [2], (ValueError, "into an exception"))
def test_no_jump_into_qualified_except_block_from_try_block(output): def test_no_jump_into_qualified_except_block_from_try_block(output):
try: try:
output.append(2) output.append(2)
@ -1794,7 +1794,7 @@ class JumpTestCase(unittest.TestCase):
raise raise
output.append(8) output.append(8)
@jump_test(7, 1, [1, 3, 6], (ValueError, "out of an 'except'")) @jump_test(7, 1, [1, 3, 6], (ValueError, "within"))
def test_no_jump_out_of_bare_except_block(output): def test_no_jump_out_of_bare_except_block(output):
output.append(1) output.append(1)
try: try:
@ -1804,7 +1804,7 @@ class JumpTestCase(unittest.TestCase):
output.append(6) output.append(6)
output.append(7) output.append(7)
@jump_test(7, 1, [1, 3, 6], (ValueError, "out of an 'except'")) @jump_test(7, 1, [1, 3, 6], (ValueError, "within"))
def test_no_jump_out_of_qualified_except_block(output): def test_no_jump_out_of_qualified_except_block(output):
output.append(1) output.append(1)
try: try:

7
Misc/NEWS.d/next/Core and Builtins/2021-04-30-15-48-36.bpo-40222.j3VxeX.rst Normal file
View File

@ -0,0 +1,7 @@
"Zero cost" exception handling.
* Uses a lookup table to determine how to handle exceptions.
* Removes SETUP_FINALLY and POP_TOP block instructions, eliminating the runtime overhead of try statements.
* Reduces the size of the frame object by about 60%.
Patch by Mark Shannon

5
Modules/_opcode.c
View File

@ -59,7 +59,12 @@ _opcode_stack_effect_impl(PyObject *module, int opcode, PyObject *oparg,
"stack_effect: jump must be False, True or None"); "stack_effect: jump must be False, True or None");
return -1; return -1;
} }
if (IS_ARTIFICIAL(opcode)) {
effect = PY_INVALID_STACK_EFFECT;
}
else {
effect = PyCompile_OpcodeStackEffectWithJump(opcode, oparg_int, jump_int); effect = PyCompile_OpcodeStackEffectWithJump(opcode, oparg_int, jump_int);
}
if (effect == PY_INVALID_STACK_EFFECT) { if (effect == PY_INVALID_STACK_EFFECT) {
PyErr_SetString(PyExc_ValueError, PyErr_SetString(PyExc_ValueError,
"invalid opcode or oparg"); "invalid opcode or oparg");

55
Objects/clinic/codeobject.c.h generated
View File

@ -5,7 +5,8 @@ preserve
PyDoc_STRVAR(code_new__doc__, PyDoc_STRVAR(code_new__doc__,
"code(argcount, posonlyargcount, kwonlyargcount, nlocals, stacksize,\n" "code(argcount, posonlyargcount, kwonlyargcount, nlocals, stacksize,\n"
" flags, codestring, constants, names, varnames, filename, name,\n" " flags, codestring, constants, names, varnames, filename, name,\n"
" firstlineno, linetable, freevars=(), cellvars=(), /)\n" " firstlineno, linetable, exceptiontable, freevars=(), cellvars=(),\n"
" /)\n"
"--\n" "--\n"
"\n" "\n"
"Create a code object. Not for the faint of heart."); "Create a code object. Not for the faint of heart.");
@ -15,8 +16,8 @@ code_new_impl(PyTypeObject *type, int argcount, int posonlyargcount,
int kwonlyargcount, int nlocals, int stacksize, int flags, int kwonlyargcount, int nlocals, int stacksize, int flags,
PyObject *code, PyObject *consts, PyObject *names, PyObject *code, PyObject *consts, PyObject *names,
PyObject *varnames, PyObject *filename, PyObject *name, PyObject *varnames, PyObject *filename, PyObject *name,
int firstlineno, PyObject *linetable, PyObject *freevars, int firstlineno, PyObject *linetable, PyObject *exceptiontable,
PyObject *cellvars); PyObject *freevars, PyObject *cellvars);
static PyObject * static PyObject *
code_new(PyTypeObject *type, PyObject *args, PyObject *kwargs) code_new(PyTypeObject *type, PyObject *args, PyObject *kwargs)
@ -36,6 +37,7 @@ code_new(PyTypeObject *type, PyObject *args, PyObject *kwargs)
PyObject *name; PyObject *name;
int firstlineno; int firstlineno;
PyObject *linetable; PyObject *linetable;
PyObject *exceptiontable;
PyObject *freevars = NULL; PyObject *freevars = NULL;
PyObject *cellvars = NULL; PyObject *cellvars = NULL;
@ -43,7 +45,7 @@ code_new(PyTypeObject *type, PyObject *args, PyObject *kwargs)
!_PyArg_NoKeywords("code", kwargs)) { !_PyArg_NoKeywords("code", kwargs)) {
goto exit; goto exit;
} }
if (!_PyArg_CheckPositional("code", PyTuple_GET_SIZE(args), 14, 16)) { if (!_PyArg_CheckPositional("code", PyTuple_GET_SIZE(args), 15, 17)) {
goto exit; goto exit;
} }
argcount = _PyLong_AsInt(PyTuple_GET_ITEM(args, 0)); argcount = _PyLong_AsInt(PyTuple_GET_ITEM(args, 0));
@ -115,14 +117,11 @@ code_new(PyTypeObject *type, PyObject *args, PyObject *kwargs)
goto exit; goto exit;
} }
linetable = PyTuple_GET_ITEM(args, 13); linetable = PyTuple_GET_ITEM(args, 13);
if (PyTuple_GET_SIZE(args) < 15) { if (!PyBytes_Check(PyTuple_GET_ITEM(args, 14))) {
goto skip_optional; _PyArg_BadArgument("code", "argument 15", "bytes", PyTuple_GET_ITEM(args, 14));
}
if (!PyTuple_Check(PyTuple_GET_ITEM(args, 14))) {
_PyArg_BadArgument("code", "argument 15", "tuple", PyTuple_GET_ITEM(args, 14));
goto exit; goto exit;
} }
freevars = PyTuple_GET_ITEM(args, 14); exceptiontable = PyTuple_GET_ITEM(args, 14);
if (PyTuple_GET_SIZE(args) < 16) { if (PyTuple_GET_SIZE(args) < 16) {
goto skip_optional; goto skip_optional;
} }
@ -130,9 +129,17 @@ code_new(PyTypeObject *type, PyObject *args, PyObject *kwargs)
_PyArg_BadArgument("code", "argument 16", "tuple", PyTuple_GET_ITEM(args, 15)); _PyArg_BadArgument("code", "argument 16", "tuple", PyTuple_GET_ITEM(args, 15));
goto exit; goto exit;
} }
cellvars = PyTuple_GET_ITEM(args, 15); freevars = PyTuple_GET_ITEM(args, 15);
if (PyTuple_GET_SIZE(args) < 17) {
goto skip_optional;
}
if (!PyTuple_Check(PyTuple_GET_ITEM(args, 16))) {
_PyArg_BadArgument("code", "argument 17", "tuple", PyTuple_GET_ITEM(args, 16));
goto exit;
}
cellvars = PyTuple_GET_ITEM(args, 16);
skip_optional: skip_optional:
return_value = code_new_impl(type, argcount, posonlyargcount, kwonlyargcount, nlocals, stacksize, flags, code, consts, names, varnames, filename, name, firstlineno, linetable, freevars, cellvars); return_value = code_new_impl(type, argcount, posonlyargcount, kwonlyargcount, nlocals, stacksize, flags, code, consts, names, varnames, filename, name, firstlineno, linetable, exceptiontable, freevars, cellvars);
exit: exit:
return return_value; return return_value;
@ -144,7 +151,7 @@ PyDoc_STRVAR(code_replace__doc__,
" co_flags=-1, co_firstlineno=-1, co_code=None, co_consts=None,\n" " co_flags=-1, co_firstlineno=-1, co_code=None, co_consts=None,\n"
" co_names=None, co_varnames=None, co_freevars=None,\n" " co_names=None, co_varnames=None, co_freevars=None,\n"
" co_cellvars=None, co_filename=None, co_name=None,\n" " co_cellvars=None, co_filename=None, co_name=None,\n"
" co_linetable=None)\n" " co_linetable=None, co_exceptiontable=None)\n"
"--\n" "--\n"
"\n" "\n"
"Return a copy of the code object with new values for the specified fields."); "Return a copy of the code object with new values for the specified fields.");
@ -160,15 +167,16 @@ code_replace_impl(PyCodeObject *self, int co_argcount,
PyObject *co_consts, PyObject *co_names, PyObject *co_consts, PyObject *co_names,
PyObject *co_varnames, PyObject *co_freevars, PyObject *co_varnames, PyObject *co_freevars,
PyObject *co_cellvars, PyObject *co_filename, PyObject *co_cellvars, PyObject *co_filename,
PyObject *co_name, PyBytesObject *co_linetable); PyObject *co_name, PyBytesObject *co_linetable,
PyBytesObject *co_exceptiontable);
static PyObject * static PyObject *
code_replace(PyCodeObject *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames) code_replace(PyCodeObject *self, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
{ {
PyObject *return_value = NULL; PyObject *return_value = NULL;
static const char * const _keywords[] = {"co_argcount", "co_posonlyargcount", "co_kwonlyargcount", "co_nlocals", "co_stacksize", "co_flags", "co_firstlineno", "co_code", "co_consts", "co_names", "co_varnames", "co_freevars", "co_cellvars", "co_filename", "co_name", "co_linetable", NULL}; static const char * const _keywords[] = {"co_argcount", "co_posonlyargcount", "co_kwonlyargcount", "co_nlocals", "co_stacksize", "co_flags", "co_firstlineno", "co_code", "co_consts", "co_names", "co_varnames", "co_freevars", "co_cellvars", "co_filename", "co_name", "co_linetable", "co_exceptiontable", NULL};
static _PyArg_Parser _parser = {NULL, _keywords, "replace", 0}; static _PyArg_Parser _parser = {NULL, _keywords, "replace", 0};
PyObject *argsbuf[16]; PyObject *argsbuf[17];
Py_ssize_t noptargs = nargs + (kwnames ? PyTuple_GET_SIZE(kwnames) : 0) - 0; Py_ssize_t noptargs = nargs + (kwnames ? PyTuple_GET_SIZE(kwnames) : 0) - 0;
int co_argcount = self->co_argcount; int co_argcount = self->co_argcount;
int co_posonlyargcount = self->co_posonlyargcount; int co_posonlyargcount = self->co_posonlyargcount;
@ -186,6 +194,7 @@ code_replace(PyCodeObject *self, PyObject *const *args, Py_ssize_t nargs, PyObje
PyObject *co_filename = self->co_filename; PyObject *co_filename = self->co_filename;
PyObject *co_name = self->co_name; PyObject *co_name = self->co_name;
PyBytesObject *co_linetable = (PyBytesObject *)self->co_linetable; PyBytesObject *co_linetable = (PyBytesObject *)self->co_linetable;
PyBytesObject *co_exceptiontable = (PyBytesObject *)self->co_exceptiontable;
args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 0, 0, 0, argsbuf); args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser, 0, 0, 0, argsbuf);
if (!args) { if (!args) {
@ -343,15 +352,25 @@ code_replace(PyCodeObject *self, PyObject *const *args, Py_ssize_t nargs, PyObje
goto skip_optional_kwonly; goto skip_optional_kwonly;
} }
} }
if (args[15]) {
if (!PyBytes_Check(args[15])) { if (!PyBytes_Check(args[15])) {
_PyArg_BadArgument("replace", "argument 'co_linetable'", "bytes", args[15]); _PyArg_BadArgument("replace", "argument 'co_linetable'", "bytes", args[15]);
goto exit; goto exit;
} }
co_linetable = (PyBytesObject *)args[15]; co_linetable = (PyBytesObject *)args[15];
if (!--noptargs) {
goto skip_optional_kwonly;
}
}
if (!PyBytes_Check(args[16])) {
_PyArg_BadArgument("replace", "argument 'co_exceptiontable'", "bytes", args[16]);
goto exit;
}
co_exceptiontable = (PyBytesObject *)args[16];
skip_optional_kwonly: skip_optional_kwonly:
return_value = code_replace_impl(self, co_argcount, co_posonlyargcount, co_kwonlyargcount, co_nlocals, co_stacksize, co_flags, co_firstlineno, co_code, co_consts, co_names, co_varnames, co_freevars, co_cellvars, co_filename, co_name, co_linetable); return_value = code_replace_impl(self, co_argcount, co_posonlyargcount, co_kwonlyargcount, co_nlocals, co_stacksize, co_flags, co_firstlineno, co_code, co_consts, co_names, co_varnames, co_freevars, co_cellvars, co_filename, co_name, co_linetable, co_exceptiontable);
exit: exit:
return return_value; return return_value;
} }
/*[clinic end generated code: output=e3091c7baaaaa420 input=a9049054013a1b77]*/ /*[clinic end generated code: output=a272b22f63ea002e input=a9049054013a1b77]*/

35
Objects/codeobject.c
View File

@ -119,7 +119,7 @@ PyCode_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount,
PyObject *code, PyObject *consts, PyObject *names, PyObject *code, PyObject *consts, PyObject *names,
PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *varnames, PyObject *freevars, PyObject *cellvars,
PyObject *filename, PyObject *name, int firstlineno, PyObject *filename, PyObject *name, int firstlineno,
PyObject *linetable) PyObject *linetable, PyObject *exceptiontable)
{ {
PyCodeObject *co; PyCodeObject *co;
Py_ssize_t *cell2arg = NULL; Py_ssize_t *cell2arg = NULL;
@ -137,7 +137,8 @@ PyCode_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount,
cellvars == NULL || !PyTuple_Check(cellvars) || cellvars == NULL || !PyTuple_Check(cellvars) ||
name == NULL || !PyUnicode_Check(name) || name == NULL || !PyUnicode_Check(name) ||
filename == NULL || !PyUnicode_Check(filename) || filename == NULL || !PyUnicode_Check(filename) ||
linetable == NULL || !PyBytes_Check(linetable)) { linetable == NULL || !PyBytes_Check(linetable) ||
exceptiontable == NULL || !PyBytes_Check(exceptiontable)) {
PyErr_BadInternalCall(); PyErr_BadInternalCall();
return NULL; return NULL;
} }
@ -260,6 +261,8 @@ PyCode_NewWithPosOnlyArgs(int argcount, int posonlyargcount, int kwonlyargcount,
co->co_firstlineno = firstlineno; co->co_firstlineno = firstlineno;
Py_INCREF(linetable); Py_INCREF(linetable);
co->co_linetable = linetable; co->co_linetable = linetable;
Py_INCREF(exceptiontable);
co->co_exceptiontable = exceptiontable;
co->co_zombieframe = NULL; co->co_zombieframe = NULL;
co->co_weakreflist = NULL; co->co_weakreflist = NULL;
co->co_extra = NULL; co->co_extra = NULL;
@ -277,12 +280,12 @@ PyCode_New(int argcount, int kwonlyargcount,
PyObject *code, PyObject *consts, PyObject *names, PyObject *code, PyObject *consts, PyObject *names,
PyObject *varnames, PyObject *freevars, PyObject *cellvars, PyObject *varnames, PyObject *freevars, PyObject *cellvars,
PyObject *filename, PyObject *name, int firstlineno, PyObject *filename, PyObject *name, int firstlineno,
PyObject *linetable) PyObject *linetable, PyObject *exceptiontable)
{ {
return PyCode_NewWithPosOnlyArgs(argcount, 0, kwonlyargcount, nlocals, return PyCode_NewWithPosOnlyArgs(argcount, 0, kwonlyargcount, nlocals,
stacksize, flags, code, consts, names, stacksize, flags, code, consts, names,
varnames, freevars, cellvars, filename, varnames, freevars, cellvars, filename,
name, firstlineno, linetable); name, firstlineno, linetable, exceptiontable);
} }
int int
@ -369,7 +372,8 @@ PyCode_NewEmpty(const char *filename, const char *funcname, int firstlineno)
filename_ob, /* filename */ filename_ob, /* filename */
funcname_ob, /* name */ funcname_ob, /* name */
firstlineno, /* firstlineno */ firstlineno, /* firstlineno */
emptystring /* linetable */ emptystring, /* linetable */
emptystring /* exception table */
); );
failed: failed:
@ -397,6 +401,7 @@ static PyMemberDef code_memberlist[] = {
{"co_name", T_OBJECT, OFF(co_name), READONLY}, {"co_name", T_OBJECT, OFF(co_name), READONLY},
{"co_firstlineno", T_INT, OFF(co_firstlineno), READONLY}, {"co_firstlineno", T_INT, OFF(co_firstlineno), READONLY},
{"co_linetable", T_OBJECT, OFF(co_linetable), READONLY}, {"co_linetable", T_OBJECT, OFF(co_linetable), READONLY},
{"co_exceptiontable", T_OBJECT, OFF(co_exceptiontable), READONLY},
{NULL} /* Sentinel */ {NULL} /* Sentinel */
}; };
@ -538,6 +543,7 @@ code.__new__ as code_new
name: unicode name: unicode
firstlineno: int firstlineno: int
linetable: object(subclass_of="&PyBytes_Type") linetable: object(subclass_of="&PyBytes_Type")
exceptiontable: object(subclass_of="&PyBytes_Type")
freevars: object(subclass_of="&PyTuple_Type", c_default="NULL") = () freevars: object(subclass_of="&PyTuple_Type", c_default="NULL") = ()
cellvars: object(subclass_of="&PyTuple_Type", c_default="NULL") = () cellvars: object(subclass_of="&PyTuple_Type", c_default="NULL") = ()
/ /
@ -550,9 +556,9 @@ code_new_impl(PyTypeObject *type, int argcount, int posonlyargcount,
int kwonlyargcount, int nlocals, int stacksize, int flags, int kwonlyargcount, int nlocals, int stacksize, int flags,
PyObject *code, PyObject *consts, PyObject *names, PyObject *code, PyObject *consts, PyObject *names,
PyObject *varnames, PyObject *filename, PyObject *name, PyObject *varnames, PyObject *filename, PyObject *name,
int firstlineno, PyObject *linetable, PyObject *freevars, int firstlineno, PyObject *linetable, PyObject *exceptiontable,
PyObject *cellvars) PyObject *freevars, PyObject *cellvars)
/*[clinic end generated code: output=42c1839b082ba293 input=0ec80da632b99f57]*/ /*[clinic end generated code: output=a3899259c3b4cace input=f823c686da4b3a03]*/
{ {
PyObject *co = NULL; PyObject *co = NULL;
PyObject *ournames = NULL; PyObject *ournames = NULL;
@ -618,7 +624,9 @@ code_new_impl(PyTypeObject *type, int argcount, int posonlyargcount,
code, consts, ournames, code, consts, ournames,
ourvarnames, ourfreevars, ourvarnames, ourfreevars,
ourcellvars, filename, ourcellvars, filename,
name, firstlineno, linetable); name, firstlineno, linetable,
exceptiontable
);
cleanup: cleanup:
Py_XDECREF(ournames); Py_XDECREF(ournames);
Py_XDECREF(ourvarnames); Py_XDECREF(ourvarnames);
@ -663,6 +671,7 @@ code_dealloc(PyCodeObject *co)
Py_XDECREF(co->co_filename); Py_XDECREF(co->co_filename);
Py_XDECREF(co->co_name); Py_XDECREF(co->co_name);
Py_XDECREF(co->co_linetable); Py_XDECREF(co->co_linetable);
Py_XDECREF(co->co_exceptiontable);
if (co->co_cell2arg != NULL) if (co->co_cell2arg != NULL)
PyMem_Free(co->co_cell2arg); PyMem_Free(co->co_cell2arg);
if (co->co_zombieframe != NULL) if (co->co_zombieframe != NULL)
@ -715,6 +724,7 @@ code.replace
co_filename: unicode(c_default="self->co_filename") = None co_filename: unicode(c_default="self->co_filename") = None
co_name: unicode(c_default="self->co_name") = None co_name: unicode(c_default="self->co_name") = None
co_linetable: PyBytesObject(c_default="(PyBytesObject *)self->co_linetable") = None co_linetable: PyBytesObject(c_default="(PyBytesObject *)self->co_linetable") = None
co_exceptiontable: PyBytesObject(c_default="(PyBytesObject *)self->co_exceptiontable") = None
Return a copy of the code object with new values for the specified fields. Return a copy of the code object with new values for the specified fields.
[clinic start generated code]*/ [clinic start generated code]*/
@ -727,8 +737,9 @@ code_replace_impl(PyCodeObject *self, int co_argcount,
PyObject *co_consts, PyObject *co_names, PyObject *co_consts, PyObject *co_names,
PyObject *co_varnames, PyObject *co_freevars, PyObject *co_varnames, PyObject *co_freevars,
PyObject *co_cellvars, PyObject *co_filename, PyObject *co_cellvars, PyObject *co_filename,
PyObject *co_name, PyBytesObject *co_linetable) PyObject *co_name, PyBytesObject *co_linetable,
/*[clinic end generated code: output=50d77e668d3b449b input=a5f997b173d7f636]*/ PyBytesObject *co_exceptiontable)
/*[clinic end generated code: output=80957472b7f78ed6 input=38376b1193efbbae]*/
{ {
#define CHECK_INT_ARG(ARG) \ #define CHECK_INT_ARG(ARG) \
if (ARG < 0) { \ if (ARG < 0) { \
@ -758,7 +769,7 @@ code_replace_impl(PyCodeObject *self, int co_argcount,
co_argcount, co_posonlyargcount, co_kwonlyargcount, co_nlocals, co_argcount, co_posonlyargcount, co_kwonlyargcount, co_nlocals,
co_stacksize, co_flags, (PyObject*)co_code, co_consts, co_names, co_stacksize, co_flags, (PyObject*)co_code, co_consts, co_names,
co_varnames, co_freevars, co_cellvars, co_filename, co_name, co_varnames, co_freevars, co_cellvars, co_filename, co_name,
co_firstlineno, (PyObject*)co_linetable); co_firstlineno, (PyObject*)co_linetable, (PyObject*)co_exceptiontable);
} }
static PyObject * static PyObject *

179
Objects/exception_handling_notes.txt Normal file
View File

@ -0,0 +1,179 @@
Description of exception handling in Python 3.11
------------------------------------------------
Python 3.11 uses what is known as "zero-cost" exception handling.
Prior to 3.11, exceptions were handled by a runtime stack of "blocks".
In zero-cost exception handling, the cost of supporting exceptions is minimized.
In the common case (where no exception is raised) the cost is reduced
to zero (or close to zero).
The cost of raising an exception is increased, but not by much.
The following code:
def f():
try:
g(0)
except:
return "fail"
compiles as follows in 3.10:
2 0 SETUP_FINALLY 7 (to 16)
3 2 LOAD_GLOBAL 0 (g)
4 LOAD_CONST 1 (0)
6 CALL_FUNCTION 1
8 POP_TOP
10 POP_BLOCK
12 LOAD_CONST 0 (None)
14 RETURN_VALUE
4 >> 16 POP_TOP
18 POP_TOP
20 POP_TOP
5 22 POP_EXCEPT
24 LOAD_CONST 3 ('fail')
26 RETURN_VALUE
Note the explicit instructions to push and pop from the "block" stack:
SETUP_FINALLY and POP_BLOCK.
In 3.11, the SETUP_FINALLY and POP_BLOCK are eliminated, replaced with
a table to determine where to jump to when an exception is raised.
2 0 NOP
3 2 LOAD_GLOBAL 0 (g)
4 LOAD_CONST 1 (0)
6 CALL_FUNCTION 1
8 POP_TOP
10 LOAD_CONST 0 (None)
12 RETURN_VALUE
>> 14 PUSH_EXC_INFO
4 16 POP_TOP
18 POP_TOP
20 POP_TOP
5 22 POP_EXCEPT
24 LOAD_CONST 2 ('fail')
26 RETURN_VALUE
>> 28 POP_EXCEPT_AND_RERAISE
ExceptionTable:
2 to 8 -> 14 [0]
14 to 20 -> 28 [3] lasti
(Note this code is from an early 3.11 alpha, the NOP may well have be removed before release).
If an instruction raises an exception then its offset is used to find the target to jump to.
For example, the CALL_FUNCTION at offset 6, falls into the range 2 to 8.
So, if g() raises an exception, then control jumps to offset 14.
Unwinding
---------
When an exception is raised, the current instruction offset is used to find following:
target to jump to, stack depth, and 'lasti', which determines whether the instruction
offset of the raising instruction should be pushed.
This information is stored in the exception table, described below.
If there is no relevant entry, the exception bubbles up to the caller.
If there is an entry, then:
1. pop values from the stack until it matches the stack depth for the handler,
2. if 'lasti' is true, then push the offset that the exception was raised at.
3. push the exception to the stack as three values: traceback, value, type,
4. jump to the target offset and resume execution.
Format of the exception table
-----------------------------
Conceptually, the exception table consists of a sequence of 5-tuples:
1. start-offset (inclusive)
2. end-offset (exclusive)
3. target
4. stack-depth
5. push-lasti (boolean)
All offsets and lengths are in instructions, not bytes.
We want the format to be compact, but quickly searchable.
For it to be compact, it needs to have variable sized entries so that we can store common (small) offsets compactly, but handle large offsets if needed.
For it to be searchable quickly, we need to support binary search giving us log(n) performance in all cases.
Binary search typically assumes fixed size entries, but that is not necesary, as long as we can identify the start of an entry.
It is worth noting that the size (end-start) is always smaller than the end, so we encode the entries as:
start, size, target, depth, push-lasti
Also, sizes are limited to 2**30 as the code length cannot exceed 2**31 and each instruction takes 2 bytes.
It also happens that depth is generally quite small.
So, we need to encode:
start (up to 30 bits)
size (up to 30 bits)
target (up to 30 bits)
depth (up to ~8 bits)
lasti (1 bit)
We need a marker for the start of the entry, so the first byte of entry will have the most significant bit set.
Since the most significant bit is reserved for marking the start of an entry, we have 7 bits per byte to encode offsets.
Encoding uses a standard varint encoding, but with only 7 bits instead of the usual 8.
The 8 bits of a bit are (msb left) SXdddddd where S is the start bit. X is the extend bit meaning that the next byte is required to extend the offset.
In addition, we will combine depth and lasti into a single value, ((depth<<1)+lasti), before encoding.
For example, the exception entry:
start: 20
end: 28
target: 100
depth: 3
lasti: False
is encoded first by converting to the more compact four value form:
start: 20
size: 8
target: 100
depth<<1+lasti: 6
which is then encoded as:
148 (MSB + 20 for start)
8 (size)
65 (Extend bit + 1)
36 (Remainder of target, 100 == (1<<6)+36)
6
for a total of five bytes.
Script to parse the exception table
-----------------------------------
def parse_varint(iterator):
b = next(iterator)
val = b & 63
while b&64:
val <<= 6
b = next(iterator)
val |= b&63
return val
def parse_exception_table(code):
iterator = iter(code.co_exceptiontable)
try:
while True:
start = parse_varint(iterator)*2
length = parse_varint(iterator)*2
end = start + length - 2 # Present as inclusive, not exclusive
target = parse_varint(iterator)*2
dl = parse_varint(iterator)
depth = dl >> 1
lasti = bool(dl&1)
yield start, end, target, depth, lasti
except StopIteration:
return

323
Objects/frameobject.c
View File

@ -91,56 +91,71 @@ get_arg(const _Py_CODEUNIT *codestr, Py_ssize_t i)
return oparg; return oparg;
} }
/* Model the evaluation stack, to determine which jumps
* are safe and how many values needs to be popped.
* The stack is modelled by a 64 integer, treating any
* stack that can't fit into 64 bits as "overflowed".
*/
typedef enum kind { typedef enum kind {
With = 1, Iterator = 1,
Loop = 2, Except = 2,
Try = 3, Object = 3,
Except = 4,
} Kind; } Kind;
#define BITS_PER_BLOCK 3 #define BITS_PER_BLOCK 2
#define UNINITIALIZED -2
#define OVERFLOWED -1
#define MAX_STACK_ENTRIES (63/BITS_PER_BLOCK)
#define WILL_OVERFLOW (1ULL<<((MAX_STACK_ENTRIES-1)*BITS_PER_BLOCK))
static inline int64_t static inline int64_t
push_block(int64_t stack, Kind kind) push_value(int64_t stack, Kind kind)
{ {
assert(stack < ((int64_t)1)<<(BITS_PER_BLOCK*CO_MAXBLOCKS)); if (((uint64_t)stack) >= WILL_OVERFLOW) {
return OVERFLOWED;
}
else {
return (stack << BITS_PER_BLOCK) | kind; return (stack << BITS_PER_BLOCK) | kind;
}
} }
static inline int64_t static inline int64_t
pop_block(int64_t stack) pop_value(int64_t stack)
{ {
assert(stack > 0); return Py_ARITHMETIC_RIGHT_SHIFT(int64_t, stack, BITS_PER_BLOCK);
return stack >> BITS_PER_BLOCK;
} }
static inline Kind static inline Kind
top_block(int64_t stack) top_of_stack(int64_t stack)
{ {
return stack & ((1<<BITS_PER_BLOCK)-1); return stack & ((1<<BITS_PER_BLOCK)-1);
} }
static int64_t * static int64_t *
markblocks(PyCodeObject *code_obj, int len) mark_stacks(PyCodeObject *code_obj, int len)
{ {
const _Py_CODEUNIT *code = const _Py_CODEUNIT *code =
(const _Py_CODEUNIT *)PyBytes_AS_STRING(code_obj->co_code); (const _Py_CODEUNIT *)PyBytes_AS_STRING(code_obj->co_code);
int64_t *blocks = PyMem_New(int64_t, len+1); int64_t *stacks = PyMem_New(int64_t, len+1);
int i, j, opcode; int i, j, opcode;
if (blocks == NULL) { if (stacks == NULL) {
PyErr_NoMemory(); PyErr_NoMemory();
return NULL; return NULL;
} }
memset(blocks, -1, (len+1)*sizeof(int64_t)); for (int i = 1; i <= len; i++) {
blocks[0] = 0; stacks[i] = UNINITIALIZED;
}
stacks[0] = 0;
int todo = 1; int todo = 1;
while (todo) { while (todo) {
todo = 0; todo = 0;
for (i = 0; i < len; i++) { for (i = 0; i < len; i++) {
int64_t block_stack = blocks[i]; int64_t next_stack = stacks[i];
int64_t except_stack; if (next_stack == UNINITIALIZED) {
if (block_stack == -1) {
continue; continue;
} }
opcode = _Py_OPCODE(code[i]); opcode = _Py_OPCODE(code[i]);
@ -150,109 +165,153 @@ markblocks(PyCodeObject *code_obj, int len)
case POP_JUMP_IF_FALSE: case POP_JUMP_IF_FALSE:
case POP_JUMP_IF_TRUE: case POP_JUMP_IF_TRUE:
case JUMP_IF_NOT_EXC_MATCH: case JUMP_IF_NOT_EXC_MATCH:
j = get_arg(code, i); {
int64_t target_stack;
int j = get_arg(code, i);
assert(j < len); assert(j < len);
if (blocks[j] == -1 && j < i) { if (stacks[j] == UNINITIALIZED && j < i) {
todo = 1; todo = 1;
} }
assert(blocks[j] == -1 || blocks[j] == block_stack); if (opcode == JUMP_IF_NOT_EXC_MATCH) {
blocks[j] = block_stack; next_stack = pop_value(pop_value(next_stack));
blocks[i+1] = block_stack; target_stack = next_stack;
}
else if (opcode == JUMP_IF_FALSE_OR_POP ||
opcode == JUMP_IF_TRUE_OR_POP)
{
target_stack = next_stack;
next_stack = pop_value(next_stack);
}
else {
next_stack = pop_value(next_stack);
target_stack = next_stack;
}
assert(stacks[j] == UNINITIALIZED || stacks[j] == target_stack);
stacks[j] = target_stack;
stacks[i+1] = next_stack;
break; break;
}
case JUMP_ABSOLUTE: case JUMP_ABSOLUTE:
j = get_arg(code, i); j = get_arg(code, i);
assert(j < len); assert(j < len);
if (blocks[j] == -1 && j < i) { if (stacks[j] == UNINITIALIZED && j < i) {
todo = 1; todo = 1;
} }
assert(blocks[j] == -1 || blocks[j] == block_stack); assert(stacks[j] == UNINITIALIZED || stacks[j] == next_stack);
blocks[j] = block_stack; stacks[j] = next_stack;
break; break;
case SETUP_FINALLY: case POP_EXCEPT:
j = get_arg(code, i) + i + 1; next_stack = pop_value(pop_value(pop_value(next_stack)));
assert(j < len); stacks[i+1] = next_stack;
except_stack = push_block(block_stack, Except);
assert(blocks[j] == -1 || blocks[j] == except_stack);
blocks[j] = except_stack;
block_stack = push_block(block_stack, Try);
blocks[i+1] = block_stack;
break;
case SETUP_WITH:
case SETUP_ASYNC_WITH:
j = get_arg(code, i) + i + 1;
assert(j < len);
except_stack = push_block(block_stack, Except);
assert(blocks[j] == -1 || blocks[j] == except_stack);
blocks[j] = except_stack;
block_stack = push_block(block_stack, With);
blocks[i+1] = block_stack;
break; break;
case JUMP_FORWARD: case JUMP_FORWARD:
j = get_arg(code, i) + i + 1; j = get_arg(code, i) + i + 1;
assert(j < len); assert(j < len);
assert(blocks[j] == -1 || blocks[j] == block_stack); assert(stacks[j] == UNINITIALIZED || stacks[j] == next_stack);
blocks[j] = block_stack; stacks[j] = next_stack;
break; break;
case GET_ITER: case GET_ITER:
case GET_AITER: case GET_AITER:
block_stack = push_block(block_stack, Loop); next_stack = push_value(pop_value(next_stack), Iterator);
blocks[i+1] = block_stack; stacks[i+1] = next_stack;
break; break;
case FOR_ITER: case FOR_ITER:
blocks[i+1] = block_stack; {
block_stack = pop_block(block_stack); int64_t target_stack = pop_value(next_stack);
stacks[i+1] = push_value(next_stack, Object);
j = get_arg(code, i) + i + 1; j = get_arg(code, i) + i + 1;
assert(j < len); assert(j < len);
assert(blocks[j] == -1 || blocks[j] == block_stack); assert(stacks[j] == UNINITIALIZED || stacks[j] == target_stack);
blocks[j] = block_stack; stacks[j] = target_stack;
break;
case POP_BLOCK:
case POP_EXCEPT:
block_stack = pop_block(block_stack);
blocks[i+1] = block_stack;
break; break;
}
case END_ASYNC_FOR: case END_ASYNC_FOR:
block_stack = pop_block(pop_block(block_stack)); next_stack = pop_value(pop_value(pop_value(next_stack)));
blocks[i+1] = block_stack; stacks[i+1] = next_stack;
break; break;
case PUSH_EXC_INFO:
next_stack = push_value(next_stack, Except);
next_stack = push_value(next_stack, Except);
next_stack = push_value(next_stack, Except);
stacks[i+1] = next_stack;
case RETURN_VALUE: case RETURN_VALUE:
case RAISE_VARARGS: case RAISE_VARARGS:
case RERAISE: case RERAISE:
case POP_EXCEPT_AND_RERAISE:
/* End of block */ /* End of block */
break; break;
case GEN_START:
stacks[i+1] = next_stack;
break;
default: default:
blocks[i+1] = block_stack; {
int delta = PyCompile_OpcodeStackEffect(opcode, _Py_OPARG(code[i]));
while (delta < 0) {
next_stack = pop_value(next_stack);
delta++;
}
while (delta > 0) {
next_stack = push_value(next_stack, Object);
delta--;
}
stacks[i+1] = next_stack;
} }
} }
} }
return blocks; }
return stacks;
} }
static int static int
compatible_block_stack(int64_t from_stack, int64_t to_stack) compatible_kind(Kind from, Kind to) {
if (to == 0) {
return 0;
}
if (to == Object) {
return 1;
}
return from == to;
}
static int
compatible_stack(int64_t from_stack, int64_t to_stack)
{ {
if (to_stack < 0) { if (from_stack < 0 || to_stack < 0) {
return 0; return 0;
} }
while(from_stack > to_stack) { while(from_stack > to_stack) {
from_stack = pop_block(from_stack); from_stack = pop_value(from_stack);
} }
return from_stack == to_stack; while(from_stack) {
Kind from_top = top_of_stack(from_stack);
Kind to_top = top_of_stack(to_stack);
if (!compatible_kind(from_top, to_top)) {
return 0;
}
from_stack = pop_value(from_stack);
to_stack = pop_value(to_stack);
}
return to_stack == 0;
} }
static const char * static const char *
explain_incompatible_block_stack(int64_t to_stack) explain_incompatible_stack(int64_t to_stack)
{ {
Kind target_kind = top_block(to_stack); assert(to_stack != 0);
if (to_stack == OVERFLOWED) {
return "stack is too deep to analyze";
}
if (to_stack == UNINITIALIZED) {
return "can't jump into an exception handler, or code may be unreachable";
}
Kind target_kind = top_of_stack(to_stack);
switch(target_kind) { switch(target_kind) {
case Except: case Except:
return "can't jump into an 'except' block as there's no exception"; return "can't jump into an 'except' block as there's no exception";
case Try: case Object:
return "can't jump into the body of a try statement"; return "differing stack depth";
case With: case Iterator:
return "can't jump into the body of a with statement";
case Loop:
return "can't jump into the body of a for loop"; return "can't jump into the body of a for loop";
default: default:
Py_UNREACHABLE(); Py_UNREACHABLE();
@ -299,27 +358,12 @@ first_line_not_before(int *lines, int len, int line)
static void static void
frame_stack_pop(PyFrameObject *f) frame_stack_pop(PyFrameObject *f)
{ {
assert(f->f_stackdepth >= 0); assert(f->f_stackdepth > 0);
f->f_stackdepth--; f->f_stackdepth--;
PyObject *v = f->f_valuestack[f->f_stackdepth]; PyObject *v = f->f_valuestack[f->f_stackdepth];
Py_DECREF(v); Py_DECREF(v);
} }
static void
frame_block_unwind(PyFrameObject *f)
{
assert(f->f_stackdepth >= 0);
assert(f->f_iblock > 0);
f->f_iblock--;
PyTryBlock *b = &f->f_blockstack[f->f_iblock];
intptr_t delta = f->f_stackdepth - b->b_level;
while (delta > 0) {
frame_stack_pop(f);
delta--;
}
}
/* Setter for f_lineno - you can set f_lineno from within a trace function in /* Setter for f_lineno - you can set f_lineno from within a trace function in
* order to jump to a given line of code, subject to some restrictions. Most * order to jump to a given line of code, subject to some restrictions. Most
* lines are OK to jump to because they don't make any assumptions about the * lines are OK to jump to because they don't make any assumptions about the
@ -327,13 +371,7 @@ frame_block_unwind(PyFrameObject *f)
* would still work without any stack errors), but there are some constructs * would still work without any stack errors), but there are some constructs
* that limit jumping: * that limit jumping:
* *
* o Lines with an 'except' statement on them can't be jumped to, because * o Any excpetion handlers.
* they expect an exception to be on the top of the stack.
* o Lines that live in a 'finally' block can't be jumped from or to, since
* we cannot be sure which state the interpreter was in or would be in
* during execution of the finally block.
* o 'try', 'with' and 'async with' blocks can't be jumped into because
* the blockstack needs to be set up before their code runs.
* o 'for' and 'async for' loops can't be jumped into because the * o 'for' and 'async for' loops can't be jumped into because the
* iterator needs to be on the stack. * iterator needs to be on the stack.
* o Jumps cannot be made from within a trace function invoked with a * o Jumps cannot be made from within a trace function invoked with a
@ -428,67 +466,56 @@ frame_setlineno(PyFrameObject *f, PyObject* p_new_lineno, void *Py_UNUSED(ignore
return -1; return -1;
} }
int64_t *blocks = markblocks(f->f_code, len); int64_t *stacks = mark_stacks(f->f_code, len);
if (blocks == NULL) { if (stacks == NULL) {
PyMem_Free(lines); PyMem_Free(lines);
return -1; return -1;
} }
int64_t target_block_stack = -1; int64_t best_stack = OVERFLOWED;
int64_t best_block_stack = -1;
int best_addr = -1; int best_addr = -1;
int64_t start_block_stack = blocks[f->f_lasti]; int64_t start_stack = stacks[f->f_lasti];
int err = -1;
const char *msg = "cannot find bytecode for specified line"; const char *msg = "cannot find bytecode for specified line";
for (int i = 0; i < len; i++) { for (int i = 0; i < len; i++) {
if (lines[i] == new_lineno) { if (lines[i] == new_lineno) {
target_block_stack = blocks[i]; int64_t target_stack = stacks[i];
if (compatible_block_stack(start_block_stack, target_block_stack)) { if (compatible_stack(start_stack, target_stack)) {
msg = NULL; err = 0;
if (target_block_stack > best_block_stack) { if (target_stack > best_stack) {
best_block_stack = target_block_stack; best_stack = target_stack;
best_addr = i; best_addr = i;
} }
} }
else if (msg) { else if (err < 0) {
if (target_block_stack >= 0) { if (start_stack == OVERFLOWED) {
msg = explain_incompatible_block_stack(target_block_stack); msg = "stack to deep to analyze";
}
else if (start_stack == UNINITIALIZED) {
msg = "can't jump from within an exception handler";
} }
else { else {
msg = "code may be unreachable."; msg = explain_incompatible_stack(target_stack);
err = 1;
} }
} }
} }
} }
PyMem_Free(blocks); PyMem_Free(stacks);
PyMem_Free(lines); PyMem_Free(lines);
if (msg != NULL) { if (err) {
PyErr_SetString(PyExc_ValueError, msg); PyErr_SetString(PyExc_ValueError, msg);
return -1; return -1;
} }
/* Unwind block stack. */ /* Unwind block stack. */
while (start_block_stack > best_block_stack) { if (f->f_state == FRAME_SUSPENDED) {
Kind kind = top_block(start_block_stack); /* Account for value popped by yield */
switch(kind) { start_stack = pop_value(start_stack);
case Loop:
frame_stack_pop(f);
break;
case Try:
frame_block_unwind(f);
break;
case With:
frame_block_unwind(f);
// Pop the exit function
frame_stack_pop(f);
break;
case Except:
PyErr_SetString(PyExc_ValueError,
"can't jump out of an 'except' block");
return -1;
} }
start_block_stack = pop_block(start_block_stack); while (start_stack > best_stack) {
frame_stack_pop(f);
start_stack = pop_value(start_stack);
} }
/* Finally set the new f_lasti and return OK. */ /* Finally set the new f_lasti and return OK. */
f->f_lineno = 0; f->f_lineno = 0;
f->f_lasti = best_addr; f->f_lasti = best_addr;
@ -852,7 +879,6 @@ _PyFrame_New_NoTrack(PyThreadState *tstate, PyFrameConstructor *con, PyObject *l
f->f_gen = NULL; f->f_gen = NULL;
f->f_lasti = -1; f->f_lasti = -1;
f->f_lineno = 0; f->f_lineno = 0;
f->f_iblock = 0;
f->f_state = FRAME_CREATED; f->f_state = FRAME_CREATED;
// f_blockstack and f_localsplus initialized by frame_alloc() // f_blockstack and f_localsplus initialized by frame_alloc()
return f; return f;
@ -884,33 +910,6 @@ PyFrame_New(PyThreadState *tstate, PyCodeObject *code,
return f; return f;
} }
/* Block management */
void
PyFrame_BlockSetup(PyFrameObject *f, int type, int handler, int level)
{
PyTryBlock *b;
if (f->f_iblock >= CO_MAXBLOCKS) {
Py_FatalError("block stack overflow");
}
b = &f->f_blockstack[f->f_iblock++];
b->b_type = type;
b->b_level = level;
b->b_handler = handler;
}
PyTryBlock *
PyFrame_BlockPop(PyFrameObject *f)
{
PyTryBlock *b;
if (f->f_iblock <= 0) {
Py_FatalError("block stack underflow");
}
b = &f->f_blockstack[--f->f_iblock];
return b;
}
/* Convert between "fast" version of locals and dictionary version. /* Convert between "fast" version of locals and dictionary version.
map and values are input arguments. map is a tuple of strings. map and values are input arguments. map is a tuple of strings.

3
PC/launcher.c
View File

@ -1258,7 +1258,8 @@ static PYC_MAGIC magic_values[] = {
{ 3390, 3399, L"3.7" }, { 3390, 3399, L"3.7" },
{ 3400, 3419, L"3.8" }, { 3400, 3419, L"3.8" },
{ 3420, 3429, L"3.9" }, { 3420, 3429, L"3.9" },
{ 3430, 3439, L"3.10" }, { 3430, 3449, L"3.10" },
{ 3450, 3469, L"3.11" },
{ 0 } { 0 }
}; };

325
Python/ceval.c
View File

@ -95,6 +95,7 @@ static PyObject * special_lookup(PyThreadState *, PyObject *, _Py_Identifier *);
static int check_args_iterable(PyThreadState *, PyObject *func, PyObject *vararg); static int check_args_iterable(PyThreadState *, PyObject *func, PyObject *vararg);
static void format_kwargs_error(PyThreadState *, PyObject *func, PyObject *kwargs); static void format_kwargs_error(PyThreadState *, PyObject *func, PyObject *kwargs);
static void format_awaitable_error(PyThreadState *, PyTypeObject *, int, int); static void format_awaitable_error(PyThreadState *, PyTypeObject *, int, int);
static PyTryBlock get_exception_handler(PyCodeObject *, int);
#define NAME_ERROR_MSG \ #define NAME_ERROR_MSG \
"name '%.200s' is not defined" "name '%.200s' is not defined"
@ -1448,34 +1449,6 @@ eval_frame_handle_pending(PyThreadState *tstate)
GETLOCAL(i) = value; \ GETLOCAL(i) = value; \
Py_XDECREF(tmp); } while (0) Py_XDECREF(tmp); } while (0)
#define UNWIND_BLOCK(b) \
while (STACK_LEVEL() > (b)->b_level) { \
PyObject *v = POP(); \
Py_XDECREF(v); \
}
#define UNWIND_EXCEPT_HANDLER(b) \
do { \
PyObject *type, *value, *traceback; \
_PyErr_StackItem *exc_info; \
assert(STACK_LEVEL() >= (b)->b_level + 3); \
while (STACK_LEVEL() > (b)->b_level + 3) { \
value = POP(); \
Py_XDECREF(value); \
} \
exc_info = tstate->exc_info; \
type = exc_info->exc_type; \
value = exc_info->exc_value; \
traceback = exc_info->exc_traceback; \
exc_info->exc_type = POP(); \
exc_info->exc_value = POP(); \
exc_info->exc_traceback = POP(); \
Py_XDECREF(type); \
Py_XDECREF(value); \
Py_XDECREF(traceback); \
} while(0)
/* macros for opcode cache */ /* macros for opcode cache */
#define OPCACHE_CHECK() \ #define OPCACHE_CHECK() \
do { \ do { \
@ -1738,7 +1711,6 @@ _PyEval_EvalFrameDefault(PyThreadState *tstate, PyFrameObject *f, int throwflag)
assert(!_PyErr_Occurred(tstate)); assert(!_PyErr_Occurred(tstate));
#endif #endif
main_loop:
for (;;) { for (;;) {
assert(stack_pointer >= f->f_valuestack); /* else underflow */ assert(stack_pointer >= f->f_valuestack); /* else underflow */
assert(STACK_LEVEL() <= co->co_stacksize); /* else overflow */ assert(STACK_LEVEL() <= co->co_stacksize); /* else overflow */
@ -1754,9 +1726,7 @@ main_loop:
if (_Py_atomic_load_relaxed(eval_breaker)) { if (_Py_atomic_load_relaxed(eval_breaker)) {
opcode = _Py_OPCODE(*next_instr); opcode = _Py_OPCODE(*next_instr);
if (opcode != SETUP_FINALLY && if (opcode != BEFORE_ASYNC_WITH &&
opcode != SETUP_WITH &&
opcode != BEFORE_ASYNC_WITH &&
opcode != YIELD_FROM) { opcode != YIELD_FROM) {
/* Few cases where we skip running signal handlers and other /* Few cases where we skip running signal handlers and other
pending calls: pending calls:
@ -1800,14 +1770,14 @@ main_loop:
tstate->c_traceobj, tstate->c_traceobj,
tstate, f, tstate, f,
&trace_info); &trace_info);
/* Reload possibly changed frame fields */
JUMPTO(f->f_lasti);
stack_pointer = f->f_valuestack+f->f_stackdepth;
f->f_stackdepth = -1;
if (err) { if (err) {
/* trace function raised an exception */ /* trace function raised an exception */
goto error; goto error;
} }
/* Reload possibly changed frame fields */
JUMPTO(f->f_lasti);
stack_pointer = f->f_valuestack+f->f_stackdepth;
f->f_stackdepth = -1;
NEXTOPARG(); NEXTOPARG();
} }
@ -2425,7 +2395,6 @@ main_loop:
case TARGET(RETURN_VALUE): { case TARGET(RETURN_VALUE): {
retval = POP(); retval = POP();
assert(f->f_iblock == 0);
assert(EMPTY()); assert(EMPTY());
f->f_state = FRAME_RETURNED; f->f_state = FRAME_RETURNED;
f->f_stackdepth = 0; f->f_stackdepth = 0;
@ -2664,14 +2633,6 @@ main_loop:
case TARGET(POP_EXCEPT): { case TARGET(POP_EXCEPT): {
PyObject *type, *value, *traceback; PyObject *type, *value, *traceback;
_PyErr_StackItem *exc_info; _PyErr_StackItem *exc_info;
PyTryBlock *b = PyFrame_BlockPop(f);
if (b->b_type != EXCEPT_HANDLER) {
_PyErr_SetString(tstate, PyExc_SystemError,
"popped block is not an except handler");
goto error;
}
assert(STACK_LEVEL() >= (b)->b_level + 3 &&
STACK_LEVEL() <= (b)->b_level + 4);
exc_info = tstate->exc_info; exc_info = tstate->exc_info;
type = exc_info->exc_type; type = exc_info->exc_type;
value = exc_info->exc_value; value = exc_info->exc_value;
@ -2685,15 +2646,48 @@ main_loop:
DISPATCH(); DISPATCH();
} }
case TARGET(POP_BLOCK): { case TARGET(POP_EXCEPT_AND_RERAISE): {
PyFrame_BlockPop(f); PyObject *lasti = PEEK(4);
DISPATCH(); if (PyLong_Check(lasti)) {
f->f_lasti = PyLong_AsLong(lasti);
assert(!_PyErr_Occurred(tstate));
}
else {
_PyErr_SetString(tstate, PyExc_SystemError, "lasti is not an int");
goto error;
}
PyObject *type, *value, *traceback;
_PyErr_StackItem *exc_info;
type = POP();
value = POP();
traceback = POP();
Py_DECREF(POP()); /* lasti */
_PyErr_Restore(tstate, type, value, traceback);
exc_info = tstate->exc_info;
type = exc_info->exc_type;
value = exc_info->exc_value;
traceback = exc_info->exc_traceback;
exc_info->exc_type = POP();
exc_info->exc_value = POP();
exc_info->exc_traceback = POP();
Py_XDECREF(type);
Py_XDECREF(value);
Py_XDECREF(traceback);
goto exception_unwind;
} }
case TARGET(RERAISE): { case TARGET(RERAISE): {
assert(f->f_iblock > 0);
if (oparg) { if (oparg) {
f->f_lasti = f->f_blockstack[f->f_iblock-1].b_handler; PyObject *lasti = PEEK(oparg+3);
if (PyLong_Check(lasti)) {
f->f_lasti = PyLong_AsLong(lasti);
assert(!_PyErr_Occurred(tstate));
}
else {
assert(PyLong_Check(lasti));
_PyErr_SetString(tstate, PyExc_SystemError, "lasti is not an int");
goto error;
}
} }
PyObject *exc = POP(); PyObject *exc = POP();
PyObject *val = POP(); PyObject *val = POP();
@ -2705,19 +2699,17 @@ main_loop:
case TARGET(END_ASYNC_FOR): { case TARGET(END_ASYNC_FOR): {
PyObject *exc = POP(); PyObject *exc = POP();
PyObject *val = POP();
PyObject *tb = POP();
assert(PyExceptionClass_Check(exc)); assert(PyExceptionClass_Check(exc));
if (PyErr_GivenExceptionMatches(exc, PyExc_StopAsyncIteration)) { if (PyErr_GivenExceptionMatches(exc, PyExc_StopAsyncIteration)) {
PyTryBlock *b = PyFrame_BlockPop(f);
assert(b->b_type == EXCEPT_HANDLER);
Py_DECREF(exc); Py_DECREF(exc);
UNWIND_EXCEPT_HANDLER(b); Py_DECREF(val);
Py_DECREF(tb);
Py_DECREF(POP()); Py_DECREF(POP());
JUMPBY(oparg);
DISPATCH(); DISPATCH();
} }
else { else {
PyObject *val = POP();
PyObject *tb = POP();
_PyErr_Restore(tstate, exc, val, tb); _PyErr_Restore(tstate, exc, val, tb);
goto exception_unwind; goto exception_unwind;
} }
@ -4022,12 +4014,6 @@ main_loop:
DISPATCH(); DISPATCH();
} }
case TARGET(SETUP_FINALLY): {
PyFrame_BlockSetup(f, SETUP_FINALLY, INSTR_OFFSET() + oparg,
STACK_LEVEL());
DISPATCH();
}
case TARGET(BEFORE_ASYNC_WITH): { case TARGET(BEFORE_ASYNC_WITH): {
_Py_IDENTIFIER(__aenter__); _Py_IDENTIFIER(__aenter__);
_Py_IDENTIFIER(__aexit__); _Py_IDENTIFIER(__aexit__);
@ -4053,17 +4039,7 @@ main_loop:
DISPATCH(); DISPATCH();
} }
case TARGET(SETUP_ASYNC_WITH): { case TARGET(BEFORE_WITH): {
PyObject *res = POP();
/* Setup the finally block before pushing the result
of __aenter__ on the stack. */
PyFrame_BlockSetup(f, SETUP_FINALLY, INSTR_OFFSET() + oparg,
STACK_LEVEL());
PUSH(res);
DISPATCH();
}
case TARGET(SETUP_WITH): {
_Py_IDENTIFIER(__enter__); _Py_IDENTIFIER(__enter__);
_Py_IDENTIFIER(__exit__); _Py_IDENTIFIER(__exit__);
PyObject *mgr = TOP(); PyObject *mgr = TOP();
@ -4081,23 +4057,20 @@ main_loop:
Py_DECREF(mgr); Py_DECREF(mgr);
res = _PyObject_CallNoArg(enter); res = _PyObject_CallNoArg(enter);
Py_DECREF(enter); Py_DECREF(enter);
if (res == NULL) if (res == NULL) {
goto error; goto error;
/* Setup the finally block before pushing the result }
of __enter__ on the stack. */
PyFrame_BlockSetup(f, SETUP_FINALLY, INSTR_OFFSET() + oparg,
STACK_LEVEL());
PUSH(res); PUSH(res);
DISPATCH(); DISPATCH();
} }
case TARGET(WITH_EXCEPT_START): { case TARGET(WITH_EXCEPT_START): {
/* At the top of the stack are 7 values: /* At the top of the stack are 8 values:
- (TOP, SECOND, THIRD) = exc_info() - (TOP, SECOND, THIRD) = exc_info()
- (FOURTH, FIFTH, SIXTH) = previous exception for EXCEPT_HANDLER - (FOURTH, FIFTH, SIXTH) = previous exception
- SEVENTH: the context.__exit__ bound method - SEVENTH: lasti of exception in exc_info()
We call SEVENTH(TOP, SECOND, THIRD). - EIGHTH: the context.__exit__ bound method
We call EIGHTH(TOP, SECOND, THIRD).
Then we push again the TOP exception and the __exit__ Then we push again the TOP exception and the __exit__
return value. return value.
*/ */
@ -4109,7 +4082,8 @@ main_loop:
tb = THIRD(); tb = THIRD();
assert(!Py_IsNone(exc)); assert(!Py_IsNone(exc));
assert(!PyLong_Check(exc)); assert(!PyLong_Check(exc));
exit_func = PEEK(7); assert(PyLong_Check(PEEK(7)));
exit_func = PEEK(8);
PyObject *stack[4] = {NULL, exc, val, tb}; PyObject *stack[4] = {NULL, exc, val, tb};
res = PyObject_Vectorcall(exit_func, stack + 1, res = PyObject_Vectorcall(exit_func, stack + 1,
3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL); 3 | PY_VECTORCALL_ARGUMENTS_OFFSET, NULL);
@ -4120,6 +4094,37 @@ main_loop:
DISPATCH(); DISPATCH();
} }
case TARGET(PUSH_EXC_INFO): {
PyObject *type = TOP();
PyObject *value = SECOND();
PyObject *tb = THIRD();
_PyErr_StackItem *exc_info = tstate->exc_info;
SET_THIRD(exc_info->exc_traceback);
SET_SECOND(exc_info->exc_value);
if (exc_info->exc_type != NULL) {
SET_TOP(exc_info->exc_type);
}
else {
Py_INCREF(Py_None);
SET_TOP(Py_None);
}
Py_INCREF(tb);
PUSH(tb);
exc_info->exc_traceback = tb;
Py_INCREF(value);
PUSH(value);
assert(PyExceptionInstance_Check(value));
exc_info->exc_value = value;
Py_INCREF(type);
PUSH(type);
assert(PyExceptionClass_Check(type));
exc_info->exc_type = type;
DISPATCH();
}
case TARGET(LOAD_METHOD): { case TARGET(LOAD_METHOD): {
/* Designed to work in tandem with CALL_METHOD. */ /* Designed to work in tandem with CALL_METHOD. */
PyObject *name = GETITEM(names, oparg); PyObject *name = GETITEM(names, oparg);
@ -4455,30 +4460,27 @@ error:
} }
exception_unwind: exception_unwind:
f->f_state = FRAME_UNWINDING; f->f_state = FRAME_UNWINDING;
/* Unwind stacks if an exception occurred */ /* We can't use f->f_lasti here, as RERAISE may have set it */
while (f->f_iblock > 0) { int lasti = INSTR_OFFSET()-1;
/* Pop the current block. */ PyTryBlock from_table = get_exception_handler(co, lasti);
PyTryBlock *b = &f->f_blockstack[--f->f_iblock]; if (from_table.b_handler < 0) {
// No handlers, so exit.
break;
}
if (b->b_type == EXCEPT_HANDLER) { assert(STACK_LEVEL() >= from_table.b_level);
UNWIND_EXCEPT_HANDLER(b); while (STACK_LEVEL() > from_table.b_level) {
continue; PyObject *v = POP();
Py_XDECREF(v);
} }
UNWIND_BLOCK(b);
if (b->b_type == SETUP_FINALLY) {
PyObject *exc, *val, *tb; PyObject *exc, *val, *tb;
int handler = b->b_handler; int handler = from_table.b_handler;
_PyErr_StackItem *exc_info = tstate->exc_info; if (from_table.b_type) {
/* Beware, this invalidates all b->b_* fields */ PyObject *lasti = PyLong_FromLong(f->f_lasti);
PyFrame_BlockSetup(f, EXCEPT_HANDLER, f->f_lasti, STACK_LEVEL()); if (lasti == NULL) {
PUSH(exc_info->exc_traceback); goto exception_unwind;
PUSH(exc_info->exc_value);
if (exc_info->exc_type != NULL) {
PUSH(exc_info->exc_type);
} }
else { PUSH(lasti);
Py_INCREF(Py_None);
PUSH(Py_None);
} }
_PyErr_Fetch(tstate, &exc, &val, &tb); _PyErr_Fetch(tstate, &exc, &val, &tb);
/* Make the raw exception data /* Make the raw exception data
@ -4490,14 +4492,10 @@ exception_unwind:
PyException_SetTraceback(val, tb); PyException_SetTraceback(val, tb);
else else
PyException_SetTraceback(val, Py_None); PyException_SetTraceback(val, Py_None);
Py_INCREF(exc); if (tb == NULL) {
exc_info->exc_type = exc;
Py_INCREF(val);
exc_info->exc_value = val;
exc_info->exc_traceback = tb;
if (tb == NULL)
tb = Py_None; tb = Py_None;
Py_INCREF(tb); Py_INCREF(Py_None);
}
PUSH(tb); PUSH(tb);
PUSH(val); PUSH(val);
PUSH(exc); PUSH(exc);
@ -4507,12 +4505,9 @@ exception_unwind:
} }
/* Resume normal execution */ /* Resume normal execution */
f->f_state = FRAME_EXECUTING; f->f_state = FRAME_EXECUTING;
goto main_loop; f->f_lasti = handler;
} NEXTOPARG();
} /* unwind stack */ goto dispatch_opcode;
/* End the loop as we still have an error */
break;
} /* main loop */ } /* main loop */
assert(retval == NULL); assert(retval == NULL);
@ -4777,6 +4772,102 @@ fail:
} }
/* Exception table parsing code.
* See Objects/exception_table_notes.txt for details.
*/
static inline unsigned char *
parse_varint(unsigned char *p, int *result) {
int val = p[0] & 63;
while (p[0] & 64) {
p++;
val = (val << 6) | (p[0] & 63);
}
*result = val;
return p+1;
}
static inline unsigned char *
scan_back_to_entry_start(unsigned char *p) {
for (; (p[0]&128) == 0; p--);
return p;
}
static inline unsigned char *
skip_to_next_entry(unsigned char *p) {
for (; (p[0]&128) == 0; p++);
return p;
}
static inline unsigned char *
parse_range(unsigned char *p, int *start, int*end)
{
p = parse_varint(p, start);
int size;
p = parse_varint(p, &size);
*end = *start + size;
return p;
}
static inline void
parse_block(unsigned char *p, PyTryBlock *block) {
int depth_and_lasti;
p = parse_varint(p, &block->b_handler);
p = parse_varint(p, &depth_and_lasti);
block->b_level = depth_and_lasti >> 1;
block->b_type = depth_and_lasti & 1;
}
#define MAX_LINEAR_SEARCH 40
static PyTryBlock
get_exception_handler(PyCodeObject *code, int index)
{
PyTryBlock res;
unsigned char *start = (unsigned char *)PyBytes_AS_STRING(code->co_exceptiontable);
unsigned char *end = start + PyBytes_GET_SIZE(code->co_exceptiontable);
/* Invariants:
* start_table == end_table OR
* start_table points to a legal entry and end_table points
* beyond the table or to a legal entry that is after index.
*/
if (end - start > MAX_LINEAR_SEARCH) {
int offset;
parse_varint(start, &offset);
if (offset > index) {
res.b_handler = -1;
return res;
}
do {
unsigned char * mid = start + ((end-start)>>1);
mid = scan_back_to_entry_start(mid);
parse_varint(mid, &offset);
if (offset > index) {
end = mid;
}
else {
start = mid;
}
} while (end - start > MAX_LINEAR_SEARCH);
}
unsigned char *scan = start;
while (scan < end) {
int start_offset, size;
scan = parse_varint(scan, &start_offset);
if (start_offset > index) {
break;
}
scan = parse_varint(scan, &size);
if (start_offset + size > index) {
parse_block(scan, &res);
return res;
}
scan = skip_to_next_entry(scan);
}
res.b_handler = -1;
return res;
}
PyFrameObject * PyFrameObject *
_PyEval_MakeFrameVector(PyThreadState *tstate, _PyEval_MakeFrameVector(PyThreadState *tstate,

484
Python/compile.c
View File

@ -67,6 +67,14 @@
*/ */
#define MAX_ALLOWED_STACK_USE (STACK_USE_GUIDELINE * 100) #define MAX_ALLOWED_STACK_USE (STACK_USE_GUIDELINE * 100)
/* Pseudo-instructions used in the compiler,
* but turned into NOPs by the assembler. */
#define SETUP_FINALLY 255
#define SETUP_CLEANUP 254
#define SETUP_WITH 253
#define POP_BLOCK 252
#define IS_TOP_LEVEL_AWAIT(c) ( \ #define IS_TOP_LEVEL_AWAIT(c) ( \
(c->c_flags->cf_flags & PyCF_ALLOW_TOP_LEVEL_AWAIT) \ (c->c_flags->cf_flags & PyCF_ALLOW_TOP_LEVEL_AWAIT) \
&& (c->u->u_ste->ste_type == ModuleBlock)) && (c->u->u_ste->ste_type == ModuleBlock))
@ -74,10 +82,23 @@
struct instr { struct instr {
unsigned char i_opcode; unsigned char i_opcode;
int i_oparg; int i_oparg;
struct basicblock_ *i_target; /* target block (if jump instruction) */ /* target block (if jump instruction) */
struct basicblock_ *i_target;
/* target block when exception is raised, should not be set by front-end. */
struct basicblock_ *i_except;
int i_lineno; int i_lineno;
}; };
typedef struct excepthandler {
struct instr *setup;
int offset;
} ExceptHandler;
typedef struct exceptstack {
struct basicblock_ *handlers[CO_MAXBLOCKS+1];
int depth;
} ExceptStack;
#define LOG_BITS_PER_INT 5 #define LOG_BITS_PER_INT 5
#define MASK_LOW_LOG_BITS 31 #define MASK_LOW_LOG_BITS 31
@ -101,7 +122,7 @@ is_relative_jump(struct instr *i)
static inline int static inline int
is_jump(struct instr *i) is_jump(struct instr *i)
{ {
return is_bit_set_in_table(_PyOpcode_Jump, i->i_opcode); return i->i_opcode >= SETUP_WITH || is_bit_set_in_table(_PyOpcode_Jump, i->i_opcode);
} }
typedef struct basicblock_ { typedef struct basicblock_ {
@ -124,12 +145,18 @@ typedef struct basicblock_ {
int b_predecessors; int b_predecessors;
/* Basic block has no fall through (it ends with a return, raise or jump) */ /* Basic block has no fall through (it ends with a return, raise or jump) */
unsigned b_nofallthrough : 1; unsigned b_nofallthrough : 1;
/* Basic block is an exception handler that preserves lasti */
unsigned b_preserve_lasti : 1;
/* Used by compiler passes to mark whether they have visited a basic block. */
unsigned b_visited : 1;
/* Basic block exits scope (it ends with a return or raise) */ /* Basic block exits scope (it ends with a return or raise) */
unsigned b_exit : 1; unsigned b_exit : 1;
/* depth of stack upon entry of block, computed by stackdepth() */ /* depth of stack upon entry of block, computed by stackdepth() */
int b_startdepth; int b_startdepth;
/* instruction offset for block, computed by assemble_jump_offsets() */ /* instruction offset for block, computed by assemble_jump_offsets() */
int b_offset; int b_offset;
/* Exception stack at start of block, used by assembler to create the exception handling table */
ExceptStack *b_exceptstack;
} basicblock; } basicblock;
/* fblockinfo tracks the current frame block. /* fblockinfo tracks the current frame block.
@ -305,6 +332,8 @@ static int compiler_match(struct compiler *, stmt_ty);
static int compiler_pattern_subpattern(struct compiler *, pattern_ty, static int compiler_pattern_subpattern(struct compiler *, pattern_ty,
pattern_context *); pattern_context *);
static void clean_basic_block(basicblock *bb);
static PyCodeObject *assemble(struct compiler *, int addNone); static PyCodeObject *assemble(struct compiler *, int addNone);
static PyObject *__doc__, *__annotations__; static PyObject *__doc__, *__annotations__;
@ -1029,11 +1058,6 @@ stack_effect(int opcode, int oparg, int jump)
case INPLACE_OR: case INPLACE_OR:
return -1; return -1;
case SETUP_WITH:
/* 1 in the normal flow.
* Restore the stack position and push 6 values before jumping to
* the handler if an exception be raised. */
return jump ? 6 : 1;
case RETURN_VALUE: case RETURN_VALUE:
return -1; return -1;
case IMPORT_STAR: case IMPORT_STAR:
@ -1048,6 +1072,8 @@ stack_effect(int opcode, int oparg, int jump)
return 0; return 0;
case POP_EXCEPT: case POP_EXCEPT:
return -3; return -3;
case POP_EXCEPT_AND_RERAISE:
return -7;
case STORE_NAME: case STORE_NAME:
return -1; return -1;
@ -1111,14 +1137,26 @@ stack_effect(int opcode, int oparg, int jump)
case LOAD_GLOBAL: case LOAD_GLOBAL:
return 1; return 1;
/* Exception handling */ /* Exception handling pseudo-instructions */
case SETUP_FINALLY: case SETUP_FINALLY:
/* 0 in the normal flow. /* 0 in the normal flow.
* Restore the stack position and push 6 values before jumping to * Restore the stack position and push 3 values before jumping to
* the handler if an exception be raised. */ * the handler if an exception be raised. */
return jump ? 6 : 0; return jump ? 3 : 0;
case SETUP_CLEANUP:
/* As SETUP_FINALLY, but pushes lasti as well */
return jump ? 4 : 0;
case SETUP_WITH:
/* 0 in the normal flow.
* Restore the stack position to the position before the result
* of __(a)enter__ and push 4 values before jumping to the handler
* if an exception be raised. */
return jump ? -1 + 4 : 0;
case RERAISE: case RERAISE:
return -3; return -3;
case PUSH_EXC_INFO:
return 3;
case WITH_EXCEPT_START: case WITH_EXCEPT_START:
return 1; return 1;
@ -1165,13 +1203,9 @@ stack_effect(int opcode, int oparg, int jump)
/* Iterators and generators */ /* Iterators and generators */
case GET_AWAITABLE: case GET_AWAITABLE:
return 0; return 0;
case SETUP_ASYNC_WITH:
/* 0 in the normal flow.
* Restore the stack position to the position before the result
* of __aenter__ and push 6 values before jumping to the handler
* if an exception be raised. */
return jump ? -1 + 6 : 0;
case BEFORE_ASYNC_WITH: case BEFORE_ASYNC_WITH:
case BEFORE_WITH:
return 1; return 1;
case GET_AITER: case GET_AITER:
return 0; return 0;
@ -1180,7 +1214,7 @@ stack_effect(int opcode, int oparg, int jump)
case GET_YIELD_FROM_ITER: case GET_YIELD_FROM_ITER:
return 0; return 0;
case END_ASYNC_FOR: case END_ASYNC_FOR:
return -7; return -4;
case FORMAT_VALUE: case FORMAT_VALUE:
/* If there's a fmt_spec on the stack, we go from 2->1, /* If there's a fmt_spec on the stack, we go from 2->1,
else 1->1. */ else 1->1. */
@ -1238,7 +1272,7 @@ compiler_addop_line(struct compiler *c, int opcode, int line)
basicblock *b; basicblock *b;
struct instr *i; struct instr *i;
int off; int off;
assert(!HAS_ARG(opcode)); assert(!HAS_ARG(opcode) || IS_ARTIFICIAL(opcode));
off = compiler_next_instr(c->u->u_curblock); off = compiler_next_instr(c->u->u_curblock);
if (off < 0) if (off < 0)
return 0; return 0;
@ -1815,6 +1849,7 @@ compiler_unwind_fblock(struct compiler *c, struct fblockinfo *info,
if (preserve_tos) { if (preserve_tos) {
ADDOP(c, ROT_FOUR); ADDOP(c, ROT_FOUR);
} }
ADDOP(c, POP_BLOCK);
ADDOP(c, POP_EXCEPT); ADDOP(c, POP_EXCEPT);
return 1; return 1;
@ -1845,6 +1880,7 @@ compiler_unwind_fblock(struct compiler *c, struct fblockinfo *info,
if (preserve_tos) { if (preserve_tos) {
ADDOP(c, ROT_FOUR); ADDOP(c, ROT_FOUR);
} }
ADDOP(c, POP_BLOCK);
ADDOP(c, POP_EXCEPT); ADDOP(c, POP_EXCEPT);
if (info->fb_datum) { if (info->fb_datum) {
ADDOP_LOAD_CONST(c, Py_None); ADDOP_LOAD_CONST(c, Py_None);
@ -3072,14 +3108,15 @@ compiler_continue(struct compiler *c)
static int static int
compiler_try_finally(struct compiler *c, stmt_ty s) compiler_try_finally(struct compiler *c, stmt_ty s)
{ {
basicblock *body, *end, *exit; basicblock *body, *end, *exit, *cleanup;
body = compiler_new_block(c); body = compiler_new_block(c);
end = compiler_new_block(c); end = compiler_new_block(c);
exit = compiler_new_block(c); exit = compiler_new_block(c);
if (body == NULL || end == NULL || exit == NULL) cleanup = compiler_new_block(c);
if (body == NULL || end == NULL || exit == NULL || cleanup == NULL) {
return 0; return 0;
}
/* `try` block */ /* `try` block */
ADDOP_JUMP(c, SETUP_FINALLY, end); ADDOP_JUMP(c, SETUP_FINALLY, end);
compiler_use_next_block(c, body); compiler_use_next_block(c, body);
@ -3098,11 +3135,17 @@ compiler_try_finally(struct compiler *c, stmt_ty s)
ADDOP_JUMP_NOLINE(c, JUMP_FORWARD, exit); ADDOP_JUMP_NOLINE(c, JUMP_FORWARD, exit);
/* `finally` block */ /* `finally` block */
compiler_use_next_block(c, end); compiler_use_next_block(c, end);
c->u->u_lineno = -1;
ADDOP_JUMP(c, SETUP_CLEANUP, cleanup);
ADDOP(c, PUSH_EXC_INFO);
if (!compiler_push_fblock(c, FINALLY_END, end, NULL, NULL)) if (!compiler_push_fblock(c, FINALLY_END, end, NULL, NULL))
return 0; return 0;
VISIT_SEQ(c, stmt, s->v.Try.finalbody); VISIT_SEQ(c, stmt, s->v.Try.finalbody);
compiler_pop_fblock(c, FINALLY_END, end); compiler_pop_fblock(c, FINALLY_END, end);
ADDOP_I(c, RERAISE, 0); ADDOP_I(c, RERAISE, 0);
compiler_use_next_block(c, cleanup);
ADDOP(c, POP_EXCEPT_AND_RERAISE);
compiler_use_next_block(c, exit); compiler_use_next_block(c, exit);
return 1; return 1;
} }
@ -3140,14 +3183,15 @@ compiler_try_finally(struct compiler *c, stmt_ty s)
static int static int
compiler_try_except(struct compiler *c, stmt_ty s) compiler_try_except(struct compiler *c, stmt_ty s)
{ {
basicblock *body, *orelse, *except, *end; basicblock *body, *orelse, *except, *end, *cleanup;
Py_ssize_t i, n; Py_ssize_t i, n;
body = compiler_new_block(c); body = compiler_new_block(c);
except = compiler_new_block(c); except = compiler_new_block(c);
orelse = compiler_new_block(c); orelse = compiler_new_block(c);
end = compiler_new_block(c); end = compiler_new_block(c);
if (body == NULL || except == NULL || orelse == NULL || end == NULL) cleanup = compiler_new_block(c);
if (body == NULL || except == NULL || orelse == NULL || end == NULL || cleanup == NULL)
return 0; return 0;
ADDOP_JUMP(c, SETUP_FINALLY, except); ADDOP_JUMP(c, SETUP_FINALLY, except);
compiler_use_next_block(c, body); compiler_use_next_block(c, body);
@ -3159,15 +3203,20 @@ compiler_try_except(struct compiler *c, stmt_ty s)
ADDOP_JUMP_NOLINE(c, JUMP_FORWARD, orelse); ADDOP_JUMP_NOLINE(c, JUMP_FORWARD, orelse);
n = asdl_seq_LEN(s->v.Try.handlers); n = asdl_seq_LEN(s->v.Try.handlers);
compiler_use_next_block(c, except); compiler_use_next_block(c, except);
c->u->u_lineno = -1;
ADDOP_JUMP(c, SETUP_CLEANUP, cleanup);
ADDOP(c, PUSH_EXC_INFO);
/* Runtime will push a block here, so we need to account for that */ /* Runtime will push a block here, so we need to account for that */
if (!compiler_push_fblock(c, EXCEPTION_HANDLER, NULL, NULL, NULL)) if (!compiler_push_fblock(c, EXCEPTION_HANDLER, NULL, NULL, NULL))
return 0; return 0;
for (i = 0; i < n; i++) { for (i = 0; i < n; i++) {
excepthandler_ty handler = (excepthandler_ty)asdl_seq_GET( excepthandler_ty handler = (excepthandler_ty)asdl_seq_GET(
s->v.Try.handlers, i); s->v.Try.handlers, i);
if (!handler->v.ExceptHandler.type && i < n-1)
return compiler_error(c, "default 'except:' must be last");
SET_LOC(c, handler); SET_LOC(c, handler);
if (!handler->v.ExceptHandler.type && i < n-1) {
return compiler_error(c, "default 'except:' must be last");
}
except = compiler_new_block(c); except = compiler_new_block(c);
if (except == NULL) if (except == NULL)
return 0; return 0;
@ -3202,7 +3251,7 @@ compiler_try_except(struct compiler *c, stmt_ty s)
*/ */
/* second try: */ /* second try: */
ADDOP_JUMP(c, SETUP_FINALLY, cleanup_end); ADDOP_JUMP(c, SETUP_CLEANUP, cleanup_end);
compiler_use_next_block(c, cleanup_body); compiler_use_next_block(c, cleanup_body);
if (!compiler_push_fblock(c, HANDLER_CLEANUP, cleanup_body, NULL, handler->v.ExceptHandler.name)) if (!compiler_push_fblock(c, HANDLER_CLEANUP, cleanup_body, NULL, handler->v.ExceptHandler.name))
return 0; return 0;
@ -3211,6 +3260,7 @@ compiler_try_except(struct compiler *c, stmt_ty s)
VISIT_SEQ(c, stmt, handler->v.ExceptHandler.body); VISIT_SEQ(c, stmt, handler->v.ExceptHandler.body);
compiler_pop_fblock(c, HANDLER_CLEANUP, cleanup_body); compiler_pop_fblock(c, HANDLER_CLEANUP, cleanup_body);
ADDOP(c, POP_BLOCK); ADDOP(c, POP_BLOCK);
ADDOP(c, POP_BLOCK);
ADDOP(c, POP_EXCEPT); ADDOP(c, POP_EXCEPT);
/* name = None; del name; # Mark as artificial */ /* name = None; del name; # Mark as artificial */
c->u->u_lineno = -1; c->u->u_lineno = -1;
@ -3224,6 +3274,7 @@ compiler_try_except(struct compiler *c, stmt_ty s)
/* name = None; del name; # Mark as artificial */ /* name = None; del name; # Mark as artificial */
c->u->u_lineno = -1; c->u->u_lineno = -1;
ADDOP_LOAD_CONST(c, Py_None); ADDOP_LOAD_CONST(c, Py_None);
compiler_nameop(c, handler->v.ExceptHandler.name, Store); compiler_nameop(c, handler->v.ExceptHandler.name, Store);
compiler_nameop(c, handler->v.ExceptHandler.name, Del); compiler_nameop(c, handler->v.ExceptHandler.name, Del);
@ -3246,15 +3297,18 @@ compiler_try_except(struct compiler *c, stmt_ty s)
compiler_pop_fblock(c, HANDLER_CLEANUP, cleanup_body); compiler_pop_fblock(c, HANDLER_CLEANUP, cleanup_body);
/* name = None; del name; # Mark as artificial */ /* name = None; del name; # Mark as artificial */
c->u->u_lineno = -1; c->u->u_lineno = -1;
ADDOP(c, POP_BLOCK);
ADDOP(c, POP_EXCEPT); ADDOP(c, POP_EXCEPT);
ADDOP_JUMP(c, JUMP_FORWARD, end); ADDOP_JUMP(c, JUMP_FORWARD, end);
} }
compiler_use_next_block(c, except); compiler_use_next_block(c, except);
} }
compiler_pop_fblock(c, EXCEPTION_HANDLER, NULL);
/* Mark as artificial */ /* Mark as artificial */
c->u->u_lineno = -1; c->u->u_lineno = -1;
compiler_pop_fblock(c, EXCEPTION_HANDLER, NULL);
ADDOP_I(c, RERAISE, 0); ADDOP_I(c, RERAISE, 0);
compiler_use_next_block(c, cleanup);
ADDOP(c, POP_EXCEPT_AND_RERAISE);
compiler_use_next_block(c, orelse); compiler_use_next_block(c, orelse);
VISIT_SEQ(c, stmt, s->v.Try.orelse); VISIT_SEQ(c, stmt, s->v.Try.orelse);
compiler_use_next_block(c, end); compiler_use_next_block(c, end);
@ -4764,6 +4818,8 @@ compiler_async_comprehension_generator(struct compiler *c,
compiler_pop_fblock(c, ASYNC_COMPREHENSION_GENERATOR, start); compiler_pop_fblock(c, ASYNC_COMPREHENSION_GENERATOR, start);
compiler_use_next_block(c, except); compiler_use_next_block(c, except);
//c->u->u_lineno = -1;
ADDOP(c, END_ASYNC_FOR); ADDOP(c, END_ASYNC_FOR);
return 1; return 1;
@ -4944,20 +5000,24 @@ compiler_visit_keyword(struct compiler *c, keyword_ty k)
*/ */
static int static int
compiler_with_except_finish(struct compiler *c) { compiler_with_except_finish(struct compiler *c, basicblock * cleanup) {
basicblock *exit; basicblock *exit;
exit = compiler_new_block(c); exit = compiler_new_block(c);
if (exit == NULL) if (exit == NULL)
return 0; return 0;
ADDOP_JUMP(c, POP_JUMP_IF_TRUE, exit); ADDOP_JUMP(c, POP_JUMP_IF_TRUE, exit);
NEXT_BLOCK(c); NEXT_BLOCK(c);
ADDOP_I(c, RERAISE, 1); ADDOP_I(c, RERAISE, 4);
compiler_use_next_block(c, cleanup);
ADDOP(c, POP_EXCEPT_AND_RERAISE);
compiler_use_next_block(c, exit); compiler_use_next_block(c, exit);
ADDOP(c, POP_TOP); ADDOP(c, POP_TOP);
ADDOP(c, POP_TOP); ADDOP(c, POP_TOP);
ADDOP(c, POP_TOP); ADDOP(c, POP_TOP);
ADDOP(c, POP_BLOCK);
ADDOP(c, POP_EXCEPT); ADDOP(c, POP_EXCEPT);
ADDOP(c, POP_TOP); ADDOP(c, POP_TOP);
ADDOP(c, POP_TOP);
return 1; return 1;
} }
@ -4988,7 +5048,7 @@ compiler_with_except_finish(struct compiler *c) {
static int static int
compiler_async_with(struct compiler *c, stmt_ty s, int pos) compiler_async_with(struct compiler *c, stmt_ty s, int pos)
{ {
basicblock *block, *final, *exit; basicblock *block, *final, *exit, *cleanup;
withitem_ty item = asdl_seq_GET(s->v.AsyncWith.items, pos); withitem_ty item = asdl_seq_GET(s->v.AsyncWith.items, pos);
assert(s->kind == AsyncWith_kind); assert(s->kind == AsyncWith_kind);
@ -5001,7 +5061,8 @@ compiler_async_with(struct compiler *c, stmt_ty s, int pos)
block = compiler_new_block(c); block = compiler_new_block(c);
final = compiler_new_block(c); final = compiler_new_block(c);
exit = compiler_new_block(c); exit = compiler_new_block(c);
if (!block || !final || !exit) cleanup = compiler_new_block(c);
if (!block || !final || !exit || !cleanup)
return 0; return 0;
/* Evaluate EXPR */ /* Evaluate EXPR */
@ -5012,9 +5073,9 @@ compiler_async_with(struct compiler *c, stmt_ty s, int pos)
ADDOP_LOAD_CONST(c, Py_None); ADDOP_LOAD_CONST(c, Py_None);
ADDOP(c, YIELD_FROM); ADDOP(c, YIELD_FROM);
ADDOP_JUMP(c, SETUP_ASYNC_WITH, final); ADDOP_JUMP(c, SETUP_WITH, final);
/* SETUP_ASYNC_WITH pushes a finally block. */ /* SETUP_WITH pushes a finally block. */
compiler_use_next_block(c, block); compiler_use_next_block(c, block);
if (!compiler_push_fblock(c, ASYNC_WITH, block, final, s)) { if (!compiler_push_fblock(c, ASYNC_WITH, block, final, s)) {
return 0; return 0;
@ -5055,13 +5116,17 @@ compiler_async_with(struct compiler *c, stmt_ty s, int pos)
/* For exceptional outcome: */ /* For exceptional outcome: */
compiler_use_next_block(c, final); compiler_use_next_block(c, final);
c->u->u_lineno = -1;
ADDOP_JUMP(c, SETUP_CLEANUP, cleanup);
ADDOP(c, PUSH_EXC_INFO);
ADDOP(c, WITH_EXCEPT_START); ADDOP(c, WITH_EXCEPT_START);
ADDOP(c, GET_AWAITABLE); ADDOP(c, GET_AWAITABLE);
ADDOP_LOAD_CONST(c, Py_None); ADDOP_LOAD_CONST(c, Py_None);
ADDOP(c, YIELD_FROM); ADDOP(c, YIELD_FROM);
compiler_with_except_finish(c); compiler_with_except_finish(c, cleanup);
compiler_use_next_block(c, exit); compiler_use_next_block(c, exit);
return 1; return 1;
} }
@ -5090,7 +5155,7 @@ compiler_use_next_block(c, exit);
static int static int
compiler_with(struct compiler *c, stmt_ty s, int pos) compiler_with(struct compiler *c, stmt_ty s, int pos)
{ {
basicblock *block, *final, *exit; basicblock *block, *final, *exit, *cleanup;
withitem_ty item = asdl_seq_GET(s->v.With.items, pos); withitem_ty item = asdl_seq_GET(s->v.With.items, pos);
assert(s->kind == With_kind); assert(s->kind == With_kind);
@ -5098,12 +5163,14 @@ compiler_with(struct compiler *c, stmt_ty s, int pos)
block = compiler_new_block(c); block = compiler_new_block(c);
final = compiler_new_block(c); final = compiler_new_block(c);
exit = compiler_new_block(c); exit = compiler_new_block(c);
if (!block || !final || !exit) cleanup = compiler_new_block(c);
if (!block || !final || !exit || !cleanup)
return 0; return 0;
/* Evaluate EXPR */ /* Evaluate EXPR */
VISIT(c, expr, item->context_expr); VISIT(c, expr, item->context_expr);
/* Will push bound __exit__ */ /* Will push bound __exit__ */
ADDOP(c, BEFORE_WITH);
ADDOP_JUMP(c, SETUP_WITH, final); ADDOP_JUMP(c, SETUP_WITH, final);
/* SETUP_WITH pushes a finally block. */ /* SETUP_WITH pushes a finally block. */
@ -5146,8 +5213,12 @@ compiler_with(struct compiler *c, stmt_ty s, int pos)
/* For exceptional outcome: */ /* For exceptional outcome: */
compiler_use_next_block(c, final); compiler_use_next_block(c, final);
c->u->u_lineno = -1;
ADDOP_JUMP(c, SETUP_CLEANUP, cleanup);
ADDOP(c, PUSH_EXC_INFO);
ADDOP(c, WITH_EXCEPT_START); ADDOP(c, WITH_EXCEPT_START);
compiler_with_except_finish(c); compiler_with_except_finish(c, cleanup);
compiler_use_next_block(c, exit); compiler_use_next_block(c, exit);
return 1; return 1;
@ -6383,11 +6454,13 @@ compiler_match(struct compiler *c, stmt_ty s)
*/ */
struct assembler { struct assembler {
PyObject *a_bytecode; /* string containing bytecode */ PyObject *a_bytecode; /* bytes containing bytecode */
int a_offset; /* offset into bytecode */ int a_offset; /* offset into bytecode */
int a_nblocks; /* number of reachable blocks */ int a_nblocks; /* number of reachable blocks */
PyObject *a_lnotab; /* string containing lnotab */ PyObject *a_lnotab; /* bytes containing lnotab */
int a_lnotab_off; /* offset into lnotab */ int a_lnotab_off; /* offset into lnotab */
PyObject *a_except_table; /* bytes containing exception table */
int a_except_table_off; /* offset into exception table */
int a_prevlineno; /* lineno of last emitted line in line table */ int a_prevlineno; /* lineno of last emitted line in line table */
int a_lineno; /* lineno of last emitted instruction */ int a_lineno; /* lineno of last emitted instruction */
int a_lineno_start; /* bytecode start offset of current lineno */ int a_lineno_start; /* bytecode start offset of current lineno */
@ -6466,7 +6539,8 @@ stackdepth(struct compiler *c)
instr->i_opcode == JUMP_FORWARD || instr->i_opcode == JUMP_FORWARD ||
instr->i_opcode == RETURN_VALUE || instr->i_opcode == RETURN_VALUE ||
instr->i_opcode == RAISE_VARARGS || instr->i_opcode == RAISE_VARARGS ||
instr->i_opcode == RERAISE) instr->i_opcode == RERAISE ||
instr->i_opcode == POP_EXCEPT_AND_RERAISE)
{ {
/* remaining code is dead */ /* remaining code is dead */
next = NULL; next = NULL;
@ -6488,6 +6562,7 @@ assemble_init(struct assembler *a, int nblocks, int firstlineno)
memset(a, 0, sizeof(struct assembler)); memset(a, 0, sizeof(struct assembler));
a->a_prevlineno = a->a_lineno = firstlineno; a->a_prevlineno = a->a_lineno = firstlineno;
a->a_lnotab = NULL; a->a_lnotab = NULL;
a->a_except_table = NULL;
a->a_bytecode = PyBytes_FromStringAndSize(NULL, DEFAULT_CODE_SIZE); a->a_bytecode = PyBytes_FromStringAndSize(NULL, DEFAULT_CODE_SIZE);
if (a->a_bytecode == NULL) { if (a->a_bytecode == NULL) {
goto error; goto error;
@ -6496,6 +6571,10 @@ assemble_init(struct assembler *a, int nblocks, int firstlineno)
if (a->a_lnotab == NULL) { if (a->a_lnotab == NULL) {
goto error; goto error;
} }
a->a_except_table = PyBytes_FromStringAndSize(NULL, DEFAULT_LNOTAB_SIZE);
if (a->a_except_table == NULL) {
goto error;
}
if ((size_t)nblocks > SIZE_MAX / sizeof(basicblock *)) { if ((size_t)nblocks > SIZE_MAX / sizeof(basicblock *)) {
PyErr_NoMemory(); PyErr_NoMemory();
goto error; goto error;
@ -6504,6 +6583,7 @@ assemble_init(struct assembler *a, int nblocks, int firstlineno)
error: error:
Py_XDECREF(a->a_bytecode); Py_XDECREF(a->a_bytecode);
Py_XDECREF(a->a_lnotab); Py_XDECREF(a->a_lnotab);
Py_XDECREF(a->a_except_table);
return 0; return 0;
} }
@ -6512,6 +6592,7 @@ assemble_free(struct assembler *a)
{ {
Py_XDECREF(a->a_bytecode); Py_XDECREF(a->a_bytecode);
Py_XDECREF(a->a_lnotab); Py_XDECREF(a->a_lnotab);
Py_XDECREF(a->a_except_table);
} }
static int static int
@ -6541,6 +6622,253 @@ assemble_emit_linetable_pair(struct assembler *a, int bdelta, int ldelta)
return 1; return 1;
} }
static int
is_block_push(struct instr *instr)
{
int opcode = instr->i_opcode;
return opcode == SETUP_FINALLY || opcode == SETUP_WITH || opcode == SETUP_CLEANUP;
}
static basicblock *
push_except_block(ExceptStack *stack, struct instr *setup) {
assert(is_block_push(setup));
int opcode = setup->i_opcode;
basicblock * target = setup->i_target;
if (opcode == SETUP_WITH || opcode == SETUP_CLEANUP) {
target->b_preserve_lasti = 1;
}
stack->handlers[++stack->depth] = target;
return target;
}
static basicblock *
pop_except_block(ExceptStack *stack) {
assert(stack->depth > 0);
return stack->handlers[--stack->depth];
}
static basicblock *
except_stack_top(ExceptStack *stack) {
return stack->handlers[stack->depth];
}
static ExceptStack *
make_except_stack(void) {
ExceptStack *new = PyMem_Malloc(sizeof(ExceptStack));
if (new == NULL) {
PyErr_NoMemory();
return NULL;
}
new->depth = 0;
new->handlers[0] = NULL;
return new;
}
static ExceptStack *
copy_except_stack(ExceptStack *stack) {
ExceptStack *copy = PyMem_Malloc(sizeof(ExceptStack));
if (copy == NULL) {
PyErr_NoMemory();
return NULL;
}
memcpy(copy, stack, sizeof(ExceptStack));
return copy;
}
static int
label_exception_targets(basicblock *entry) {
int nblocks = 0;
for (basicblock *b = entry; b != NULL; b = b->b_next) {
b->b_visited = 0;
nblocks++;
}
basicblock **todo_stack = PyMem_Malloc(sizeof(basicblock *)*nblocks);
if (todo_stack == NULL) {
PyErr_NoMemory();
return -1;
}
ExceptStack *except_stack = make_except_stack();
if (except_stack == NULL) {
PyMem_Free(todo_stack);
PyErr_NoMemory();
return -1;
}
except_stack->depth = 0;
todo_stack[0] = entry;
entry->b_visited = 1;
entry->b_exceptstack = except_stack;
basicblock **todo = &todo_stack[1];
basicblock *handler = NULL;
while (todo > todo_stack) {
todo--;
basicblock *b = todo[0];
assert(b->b_visited == 1);
except_stack = b->b_exceptstack;
assert(except_stack != NULL);
b->b_exceptstack = NULL;
handler = except_stack_top(except_stack);
for (int i = 0; i < b->b_iused; i++) {
struct instr *instr = &b->b_instr[i];
if (is_block_push(instr)) {
if (!instr->i_target->b_visited) {
ExceptStack *copy = copy_except_stack(except_stack);
if (copy == NULL) {
goto error;
}
instr->i_target->b_exceptstack = copy;
todo[0] = instr->i_target;
instr->i_target->b_visited = 1;
todo++;
}
handler = push_except_block(except_stack, instr);
}
else if (instr->i_opcode == POP_BLOCK) {
handler = pop_except_block(except_stack);
}
else if (is_jump(instr)) {
instr->i_except = handler;
assert(i == b->b_iused -1);
if (!instr->i_target->b_visited) {
if (b->b_nofallthrough == 0) {
ExceptStack *copy = copy_except_stack(except_stack);
if (copy == NULL) {
goto error;
}
instr->i_target->b_exceptstack = copy;
}
else {
instr->i_target->b_exceptstack = except_stack;
except_stack = NULL;
}
todo[0] = instr->i_target;
instr->i_target->b_visited = 1;
todo++;
}
}
else {
instr->i_except = handler;
}
}
if (b->b_nofallthrough == 0 && !b->b_next->b_visited) {
assert(except_stack != NULL);
b->b_next->b_exceptstack = except_stack;
todo[0] = b->b_next;
b->b_next->b_visited = 1;
todo++;
}
else if (except_stack != NULL) {
PyMem_Free(except_stack);
}
}
#ifdef Py_DEBUG
for (basicblock *b = entry; b != NULL; b = b->b_next) {
assert(b->b_exceptstack == NULL);
}
#endif
PyMem_Free(todo_stack);
return 0;
error:
PyMem_Free(todo_stack);
PyMem_Free(except_stack);
return -1;
}
static void
convert_exception_handlers_to_nops(basicblock *entry) {
for (basicblock *b = entry; b != NULL; b = b->b_next) {
for (int i = 0; i < b->b_iused; i++) {
struct instr *instr = &b->b_instr[i];
if (is_block_push(instr) || instr->i_opcode == POP_BLOCK) {
instr->i_opcode = NOP;
}
}
}
}
static inline void
write_except_byte(struct assembler *a, int byte) {
unsigned char *p = (unsigned char *) PyBytes_AS_STRING(a->a_except_table);
p[a->a_except_table_off++] = byte;
}
#define CONTINUATION_BIT 64
static void
assemble_emit_exception_table_item(struct assembler *a, int value, int msb)
{
assert ((msb | 128) == 128);
assert(value >= 0 && value < (1 << 30));
if (value >= 1 << 24) {
write_except_byte(a, (value >> 24) | CONTINUATION_BIT | msb);
msb = 0;
}
if (value >= 1 << 18) {
write_except_byte(a, ((value >> 18)&0x3f) | CONTINUATION_BIT | msb);
msb = 0;
}
if (value >= 1 << 12) {
write_except_byte(a, ((value >> 12)&0x3f) | CONTINUATION_BIT | msb);
msb = 0;
}
if (value >= 1 << 6) {
write_except_byte(a, ((value >> 6)&0x3f) | CONTINUATION_BIT | msb);
msb = 0;
}
write_except_byte(a, (value&0x3f) | msb);
}
/* See Objects/exception_table_notes.txt for details of layout */
#define MAX_SIZE_OF_ENTRY 20
static int
assemble_emit_exception_table_entry(struct assembler *a, int start, int end, basicblock *handler)
{
Py_ssize_t len = PyBytes_GET_SIZE(a->a_except_table);
if (a->a_except_table_off + MAX_SIZE_OF_ENTRY >= len) {
if (_PyBytes_Resize(&a->a_except_table, len * 2) < 0)
return 0;
}
int size = end-start;
assert(end > start);
int target = handler->b_offset;
int depth = handler->b_preserve_lasti ? handler->b_startdepth-4 : handler->b_startdepth-3;
assert(depth >= 0);
int depth_lasti = (depth<<1) | handler->b_preserve_lasti;
assemble_emit_exception_table_item(a, start, (1<<7));
assemble_emit_exception_table_item(a, size, 0);
assemble_emit_exception_table_item(a, target, 0);
assemble_emit_exception_table_item(a, depth_lasti, 0);
return 1;
}
static int
assemble_exception_table(struct assembler *a)
{
basicblock *b;
int ioffset = 0;
basicblock *handler = NULL;
int start = -1;
for (b = a->a_entry; b != NULL; b = b->b_next) {
ioffset = b->b_offset;
for (int i = 0; i < b->b_iused; i++) {
struct instr *instr = &b->b_instr[i];
if (instr->i_except != handler) {
if (handler != NULL) {
RETURN_IF_FALSE(assemble_emit_exception_table_entry(a, start, ioffset, handler));
}
start = ioffset;
handler = instr->i_except;
}
ioffset += instrsize(instr->i_oparg);
}
}
if (handler != NULL) {
RETURN_IF_FALSE(assemble_emit_exception_table_entry(a, start, ioffset, handler));
}
return 1;
}
/* Appends a range to the end of the line number table. See /* Appends a range to the end of the line number table. See
* Objects/lnotab_notes.txt for the description of the line number table. */ * Objects/lnotab_notes.txt for the description of the line number table. */
@ -6793,7 +7121,7 @@ merge_const_one(struct compiler *c, PyObject **obj)
} }
static PyCodeObject * static PyCodeObject *
makecode(struct compiler *c, struct assembler *a, PyObject *consts) makecode(struct compiler *c, struct assembler *a, PyObject *consts, int maxdepth)
{ {
PyCodeObject *co = NULL; PyCodeObject *co = NULL;
PyObject *names = NULL; PyObject *names = NULL;
@ -6804,7 +7132,7 @@ makecode(struct compiler *c, struct assembler *a, PyObject *consts)
Py_ssize_t nlocals; Py_ssize_t nlocals;
int nlocals_int; int nlocals_int;
int flags; int flags;
int posorkeywordargcount, posonlyargcount, kwonlyargcount, maxdepth; int posorkeywordargcount, posonlyargcount, kwonlyargcount;
names = dict_keys_inorder(c->u->u_names, 0); names = dict_keys_inorder(c->u->u_names, 0);
varnames = dict_keys_inorder(c->u->u_varnames, 0); varnames = dict_keys_inorder(c->u->u_varnames, 0);
@ -6846,23 +7174,11 @@ makecode(struct compiler *c, struct assembler *a, PyObject *consts)
posonlyargcount = Py_SAFE_DOWNCAST(c->u->u_posonlyargcount, Py_ssize_t, int); posonlyargcount = Py_SAFE_DOWNCAST(c->u->u_posonlyargcount, Py_ssize_t, int);
posorkeywordargcount = Py_SAFE_DOWNCAST(c->u->u_argcount, Py_ssize_t, int); posorkeywordargcount = Py_SAFE_DOWNCAST(c->u->u_argcount, Py_ssize_t, int);
kwonlyargcount = Py_SAFE_DOWNCAST(c->u->u_kwonlyargcount, Py_ssize_t, int); kwonlyargcount = Py_SAFE_DOWNCAST(c->u->u_kwonlyargcount, Py_ssize_t, int);
maxdepth = stackdepth(c);
if (maxdepth < 0) {
Py_DECREF(consts);
goto error;
}
if (maxdepth > MAX_ALLOWED_STACK_USE) {
PyErr_Format(PyExc_SystemError,
"excessive stack use: stack is %d deep",
maxdepth);
Py_DECREF(consts);
goto error;
}
co = PyCode_NewWithPosOnlyArgs(posonlyargcount+posorkeywordargcount, co = PyCode_NewWithPosOnlyArgs(posonlyargcount+posorkeywordargcount,
posonlyargcount, kwonlyargcount, nlocals_int, posonlyargcount, kwonlyargcount, nlocals_int,
maxdepth, flags, a->a_bytecode, consts, names, maxdepth, flags, a->a_bytecode, consts, names,
varnames, freevars, cellvars, c->c_filename, varnames, freevars, cellvars, c->c_filename,
c->u->u_name, c->u->u_firstlineno, a->a_lnotab); c->u->u_name, c->u->u_firstlineno, a->a_lnotab, a->a_except_table);
Py_DECREF(consts); Py_DECREF(consts);
error: error:
Py_XDECREF(names); Py_XDECREF(names);
@ -7015,6 +7331,25 @@ assemble(struct compiler *c, int addNone)
goto error; goto error;
} }
int maxdepth = stackdepth(c);
if (maxdepth < 0) {
goto error;
}
if (maxdepth > MAX_ALLOWED_STACK_USE) {
PyErr_Format(PyExc_SystemError,
"excessive stack use: stack is %d deep",
maxdepth);
goto error;
}
if (label_exception_targets(entryblock)) {
goto error;
}
convert_exception_handlers_to_nops(entryblock);
for (basicblock *b = a.a_entry; b != NULL; b = b->b_next) {
clean_basic_block(b);
}
/* Can't modify the bytecode after computing jump offsets. */ /* Can't modify the bytecode after computing jump offsets. */
assemble_jump_offsets(&a, c); assemble_jump_offsets(&a, c);
@ -7024,6 +7359,10 @@ assemble(struct compiler *c, int addNone)
if (!assemble_emit(&a, &b->b_instr[j])) if (!assemble_emit(&a, &b->b_instr[j]))
goto error; goto error;
} }
if (!assemble_exception_table(&a)) {
return 0;
}
if (!assemble_line_range(&a)) { if (!assemble_line_range(&a)) {
return 0; return 0;
} }
@ -7031,6 +7370,11 @@ assemble(struct compiler *c, int addNone)
if (_PyBytes_Resize(&a.a_lnotab, a.a_lnotab_off) < 0) { if (_PyBytes_Resize(&a.a_lnotab, a.a_lnotab_off) < 0) {
goto error; goto error;
} }
if (_PyBytes_Resize(&a.a_except_table, a.a_except_table_off) < 0) {
goto error;
}
if (!merge_const_one(c, &a.a_lnotab)) { if (!merge_const_one(c, &a.a_lnotab)) {
goto error; goto error;
} }
@ -7041,7 +7385,7 @@ assemble(struct compiler *c, int addNone)
goto error; goto error;
} }
co = makecode(c, &a, consts); co = makecode(c, &a, consts, maxdepth);
error: error:
Py_XDECREF(consts); Py_XDECREF(consts);
assemble_free(&a); assemble_free(&a);
@ -7417,9 +7761,10 @@ error:
static void static void
clean_basic_block(basicblock *bb, int prev_lineno) { clean_basic_block(basicblock *bb) {
/* Remove NOPs when legal to do so. */ /* Remove NOPs when legal to do so. */
int dest = 0; int dest = 0;
int prev_lineno = -1;
for (int src = 0; src < bb->b_iused; src++) { for (int src = 0; src < bb->b_iused; src++) {
int lineno = bb->b_instr[src].i_lineno; int lineno = bb->b_instr[src].i_lineno;
if (bb->b_instr[src].i_opcode == NOP) { if (bb->b_instr[src].i_opcode == NOP) {
@ -7451,7 +7796,6 @@ clean_basic_block(basicblock *bb, int prev_lineno) {
} }
} }
} }
} }
if (dest != src) { if (dest != src) {
bb->b_instr[dest] = bb->b_instr[src]; bb->b_instr[dest] = bb->b_instr[src];
@ -7472,6 +7816,7 @@ normalize_basic_block(basicblock *bb) {
case RETURN_VALUE: case RETURN_VALUE:
case RAISE_VARARGS: case RAISE_VARARGS:
case RERAISE: case RERAISE:
case POP_EXCEPT_AND_RERAISE:
bb->b_exit = 1; bb->b_exit = 1;
bb->b_nofallthrough = 1; bb->b_nofallthrough = 1;
break; break;
@ -7588,9 +7933,9 @@ propogate_line_numbers(struct assembler *a) {
if (is_jump(&b->b_instr[b->b_iused-1])) { if (is_jump(&b->b_instr[b->b_iused-1])) {
switch (b->b_instr[b->b_iused-1].i_opcode) { switch (b->b_instr[b->b_iused-1].i_opcode) {
/* Note: Only actual jumps, not exception handlers */ /* Note: Only actual jumps, not exception handlers */
case SETUP_ASYNC_WITH:
case SETUP_WITH: case SETUP_WITH:
case SETUP_FINALLY: case SETUP_FINALLY:
case SETUP_CLEANUP:
continue; continue;
} }
basicblock *target = b->b_instr[b->b_iused-1].i_target; basicblock *target = b->b_instr[b->b_iused-1].i_target;
@ -7619,7 +7964,7 @@ optimize_cfg(struct compiler *c, struct assembler *a, PyObject *consts)
if (optimize_basic_block(c, b, consts)) { if (optimize_basic_block(c, b, consts)) {
return -1; return -1;
} }
clean_basic_block(b, -1); clean_basic_block(b);
assert(b->b_predecessors == 0); assert(b->b_predecessors == 0);
} }
if (mark_reachable(a)) { if (mark_reachable(a)) {
@ -7632,16 +7977,10 @@ optimize_cfg(struct compiler *c, struct assembler *a, PyObject *consts)
b->b_nofallthrough = 0; b->b_nofallthrough = 0;
} }
} }
basicblock *pred = NULL;
for (basicblock *b = a->a_entry; b != NULL; b = b->b_next) {
int prev_lineno = -1;
if (pred && pred->b_iused) {
prev_lineno = pred->b_instr[pred->b_iused-1].i_lineno;
}
clean_basic_block(b, prev_lineno);
pred = b->b_nofallthrough ? NULL : b;
}
eliminate_empty_basic_blocks(a->a_entry); eliminate_empty_basic_blocks(a->a_entry);
for (basicblock *b = a->a_entry; b != NULL; b = b->b_next) {
clean_basic_block(b);
}
/* Delete jump instructions made redundant by previous step. If a non-empty /* Delete jump instructions made redundant by previous step. If a non-empty
block ends with a jump instruction, check if the next non-empty block block ends with a jump instruction, check if the next non-empty block
reached through normal flow control is the target of that jump. If it reached through normal flow control is the target of that jump. If it
@ -7657,7 +7996,6 @@ optimize_cfg(struct compiler *c, struct assembler *a, PyObject *consts)
assert(b->b_next->b_iused); assert(b->b_next->b_iused);
b->b_nofallthrough = 0; b->b_nofallthrough = 0;
b_last_instr->i_opcode = NOP; b_last_instr->i_opcode = NOP;
clean_basic_block(b, -1);
maybe_empty_blocks = 1; maybe_empty_blocks = 1;
} }
} }
@ -7691,12 +8029,13 @@ ensure_exits_have_lineno(struct compiler *c)
/* Copy all exit blocks without line number that are targets of a jump. /* Copy all exit blocks without line number that are targets of a jump.
*/ */
for (basicblock *b = c->u->u_blocks; b != NULL; b = b->b_list) { for (basicblock *b = c->u->u_blocks; b != NULL; b = b->b_list) {
entry = b;
if (b->b_iused > 0 && is_jump(&b->b_instr[b->b_iused-1])) { if (b->b_iused > 0 && is_jump(&b->b_instr[b->b_iused-1])) {
switch (b->b_instr[b->b_iused-1].i_opcode) { switch (b->b_instr[b->b_iused-1].i_opcode) {
/* Note: Only actual jumps, not exception handlers */ /* Note: Only actual jumps, not exception handlers */
case SETUP_ASYNC_WITH:
case SETUP_WITH: case SETUP_WITH:
case SETUP_FINALLY: case SETUP_FINALLY:
case SETUP_CLEANUP:
continue; continue;
} }
basicblock *target = b->b_instr[b->b_iused-1].i_target; basicblock *target = b->b_instr[b->b_iused-1].i_target;
@ -7709,7 +8048,6 @@ ensure_exits_have_lineno(struct compiler *c)
b->b_instr[b->b_iused-1].i_target = new_target; b->b_instr[b->b_iused-1].i_target = new_target;
} }
} }
entry = b;
} }
assert(entry != NULL); assert(entry != NULL);
if (is_exit_without_lineno(entry)) { if (is_exit_without_lineno(entry)) {

2
Python/frozen_hello.h
View File

@ -8,5 +8,5 @@ const unsigned char _Py_M__hello[] = {
5,112,114,105,110,116,169,0,114,1,0,0,0,114,1,0, 5,112,114,105,110,116,169,0,114,1,0,0,0,114,1,0,
0,0,122,14,60,102,114,111,122,101,110,32,104,101,108,108, 0,0,122,14,60,102,114,111,122,101,110,32,104,101,108,108,
111,62,218,8,60,109,111,100,117,108,101,62,1,0,0,0, 111,62,218,8,60,109,111,100,117,108,101,62,1,0,0,0,
115,4,0,0,0,4,0,12,1, 115,4,0,0,0,4,0,12,1,243,0,0,0,0,
}; };

3507
Python/importlib.h generated
View File

File diff suppressed because it is too large Load Diff

4981
Python/importlib_external.h generated
View File

File diff suppressed because it is too large Load Diff

1878
Python/importlib_zipimport.h
View File

File diff suppressed because it is too large Load Diff

9
Python/marshal.c
View File

@ -525,6 +525,7 @@ w_complex_object(PyObject *v, char flag, WFILE *p)
w_object(co->co_name, p); w_object(co->co_name, p);
w_long(co->co_firstlineno, p); w_long(co->co_firstlineno, p);
w_object(co->co_linetable, p); w_object(co->co_linetable, p);
w_object(co->co_exceptiontable, p);
} }
else if (PyObject_CheckBuffer(v)) { else if (PyObject_CheckBuffer(v)) {
/* Write unknown bytes-like objects as a bytes object */ /* Write unknown bytes-like objects as a bytes object */
@ -1313,6 +1314,7 @@ r_object(RFILE *p)
PyObject *name = NULL; PyObject *name = NULL;
int firstlineno; int firstlineno;
PyObject *linetable = NULL; PyObject *linetable = NULL;
PyObject *exceptiontable = NULL;
idx = r_ref_reserve(flag, p); idx = r_ref_reserve(flag, p);
if (idx < 0) if (idx < 0)
@ -1370,6 +1372,10 @@ r_object(RFILE *p)
linetable = r_object(p); linetable = r_object(p);
if (linetable == NULL) if (linetable == NULL)
goto code_error; goto code_error;
exceptiontable = r_object(p);
if (exceptiontable == NULL)
goto code_error;
if (PySys_Audit("code.__new__", "OOOiiiiii", if (PySys_Audit("code.__new__", "OOOiiiiii",
code, filename, name, argcount, posonlyargcount, code, filename, name, argcount, posonlyargcount,
@ -1382,7 +1388,7 @@ r_object(RFILE *p)
nlocals, stacksize, flags, nlocals, stacksize, flags,
code, consts, names, varnames, code, consts, names, varnames,
freevars, cellvars, filename, name, freevars, cellvars, filename, name,
firstlineno, linetable); firstlineno, linetable, exceptiontable);
v = r_ref_insert(v, idx, flag, p); v = r_ref_insert(v, idx, flag, p);
code_error: code_error:
@ -1395,6 +1401,7 @@ r_object(RFILE *p)
Py_XDECREF(filename); Py_XDECREF(filename);
Py_XDECREF(name); Py_XDECREF(name);
Py_XDECREF(linetable); Py_XDECREF(linetable);
Py_XDECREF(exceptiontable);
} }
retval = v; retval = v;
break; break;

14
Python/opcode_targets.h generated
View File

@ -34,9 +34,9 @@ static void *opcode_targets[256] = {
&&TARGET_MATCH_SEQUENCE, &&TARGET_MATCH_SEQUENCE,
&&TARGET_MATCH_KEYS, &&TARGET_MATCH_KEYS,
&&TARGET_COPY_DICT_WITHOUT_KEYS, &&TARGET_COPY_DICT_WITHOUT_KEYS,
&&TARGET_PUSH_EXC_INFO,
&&_unknown_opcode, &&_unknown_opcode,
&&_unknown_opcode, &&TARGET_POP_EXCEPT_AND_RERAISE,
&&_unknown_opcode,
&&_unknown_opcode, &&_unknown_opcode,
&&_unknown_opcode, &&_unknown_opcode,
&&_unknown_opcode, &&_unknown_opcode,
@ -52,7 +52,7 @@ static void *opcode_targets[256] = {
&&TARGET_GET_AITER, &&TARGET_GET_AITER,
&&TARGET_GET_ANEXT, &&TARGET_GET_ANEXT,
&&TARGET_BEFORE_ASYNC_WITH, &&TARGET_BEFORE_ASYNC_WITH,
&&_unknown_opcode, &&TARGET_BEFORE_WITH,
&&TARGET_END_ASYNC_FOR, &&TARGET_END_ASYNC_FOR,
&&TARGET_INPLACE_ADD, &&TARGET_INPLACE_ADD,
&&TARGET_INPLACE_SUBTRACT, &&TARGET_INPLACE_SUBTRACT,
@ -86,7 +86,7 @@ static void *opcode_targets[256] = {
&&TARGET_IMPORT_STAR, &&TARGET_IMPORT_STAR,
&&TARGET_SETUP_ANNOTATIONS, &&TARGET_SETUP_ANNOTATIONS,
&&TARGET_YIELD_VALUE, &&TARGET_YIELD_VALUE,
&&TARGET_POP_BLOCK, &&_unknown_opcode,
&&_unknown_opcode, &&_unknown_opcode,
&&TARGET_POP_EXCEPT, &&TARGET_POP_EXCEPT,
&&TARGET_STORE_NAME, &&TARGET_STORE_NAME,
@ -121,7 +121,7 @@ static void *opcode_targets[256] = {
&&TARGET_RERAISE, &&TARGET_RERAISE,
&&_unknown_opcode, &&_unknown_opcode,
&&TARGET_JUMP_IF_NOT_EXC_MATCH, &&TARGET_JUMP_IF_NOT_EXC_MATCH,
&&TARGET_SETUP_FINALLY, &&_unknown_opcode,
&&_unknown_opcode, &&_unknown_opcode,
&&TARGET_LOAD_FAST, &&TARGET_LOAD_FAST,
&&TARGET_STORE_FAST, &&TARGET_STORE_FAST,
@ -142,7 +142,7 @@ static void *opcode_targets[256] = {
&&_unknown_opcode, &&_unknown_opcode,
&&TARGET_CALL_FUNCTION_KW, &&TARGET_CALL_FUNCTION_KW,
&&TARGET_CALL_FUNCTION_EX, &&TARGET_CALL_FUNCTION_EX,
&&TARGET_SETUP_WITH, &&_unknown_opcode,
&&TARGET_EXTENDED_ARG, &&TARGET_EXTENDED_ARG,
&&TARGET_LIST_APPEND, &&TARGET_LIST_APPEND,
&&TARGET_SET_ADD, &&TARGET_SET_ADD,
@ -153,7 +153,7 @@ static void *opcode_targets[256] = {
&&_unknown_opcode, &&_unknown_opcode,
&&TARGET_MATCH_CLASS, &&TARGET_MATCH_CLASS,
&&_unknown_opcode, &&_unknown_opcode,
&&TARGET_SETUP_ASYNC_WITH, &&_unknown_opcode,
&&TARGET_FORMAT_VALUE, &&TARGET_FORMAT_VALUE,
&&TARGET_BUILD_CONST_KEY_MAP, &&TARGET_BUILD_CONST_KEY_MAP,
&&TARGET_BUILD_STRING, &&TARGET_BUILD_STRING,

10
Tools/scripts/generate_opcode_h.py
View File

@ -16,14 +16,12 @@ extern "C" {
""".lstrip() """.lstrip()
footer = """ footer = """
/* EXCEPT_HANDLER is a special, implicit block type which is created when
entering an except handler. It is not an opcode but we define it here
as we want it to be available to both frameobject.c and ceval.c, while
remaining private.*/
#define EXCEPT_HANDLER 257
#define HAS_ARG(op) ((op) >= HAVE_ARGUMENT) #define HAS_ARG(op) ((op) >= HAVE_ARGUMENT)
/* Reserve some bytecodes for internal use in the compiler.
* The value of 240 is arbitrary. */
#define IS_ARTIFICIAL(op) ((op) > 240)
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif