The majority of the changes are in the compiler. The mainloop changes
primarily to implement the new opcodes and to pass a function's
closure to eval_code2(). Frames and functions got new slots to hold
the closure.
Include/compile.h
Add co_freevars and co_cellvars slots to code objects.
Update PyCode_New() to take freevars and cellvars as arguments
Include/funcobject.h
Add func_closure slot to function objects.
Add GetClosure()/SetClosure() functions (and corresponding
macros) for getting at the closure.
Include/frameobject.h
PyFrame_New() now takes a closure.
Include/opcode.h
Add four new opcodes: MAKE_CLOSURE, LOAD_CLOSURE, LOAD_DEREF,
STORE_DEREF.
Remove comment about old requirement for opcodes to fit in 7
bits.
compile.c
Implement changes to code objects for co_freevars and co_cellvars.
Modify symbol table to use st_cur_name (string object for the name
of the current scope) and st_cur_children (list of nested blocks).
Also define st_nested, which might more properly be called
st_cur_nested. Add several DEF_XXX flags to track def-use
information for free variables.
New or modified functions of note:
com_make_closure(struct compiling *, PyCodeObject *)
Emit LOAD_CLOSURE opcodes as needed to pass cells for free
variables into nested scope.
com_addop_varname(struct compiling *, int, char *)
Emits opcodes for LOAD_DEREF and STORE_DEREF.
get_ref_type(struct compiling *, char *name)
Return NAME_CLOSURE if ref type is FREE or CELL
symtable_load_symbols(struct compiling *)
Decides what variables are cell or free based on def-use info.
Can now raise SyntaxError if nested scopes are mixed with
exec or from blah import *.
make_scope_info(PyObject *, PyObject *, int, int)
Helper functions for symtable scope stack.
symtable_update_free_vars(struct symtable *)
After a code block has been analyzed, it must check each of
its children for free variables that are not defined in the
block. If a variable is free in a child and not defined in
the parent, then it is defined by block the enclosing the
current one or it is a global. This does the right logic.
symtable_add_use() is now a macro for symtable_add_def()
symtable_assign(struct symtable *, node *)
Use goto instead of for (;;)
Fixed bug in symtable where name of keyword argument in function
call was treated as assignment in the scope of the call site. Ex:
def f():
g(a=2) # a was considered a local of f
ceval.c
eval_code2() now take one more argument, a closure.
Implement LOAD_CLOSURE, LOAD_DEREF, STORE_DEREF, MAKE_CLOSURE>
Also: When name error occurs for global variable, report that the
name was global in the error mesage.
Objects/frameobject.c
Initialize f_closure to be a tuple containing space for cellvars
and freevars. f_closure is NULL if neither are present.
Objects/funcobject.c
Add support for func_closure.
Python/import.c
Change the magic number.
Python/marshal.c
Track changes to code objects.
parameters that contained both anonymous tuples and *arg or **arg. Ex:
def f(a, (b, c), *d): pass
Fix the symtable_params() to generate names in the right order for
co_varnames slot of code object. Consider *arg and **arg before the
"complex" names introduced by anonymous tuples.
module__doc__: Document the Warning subclass heirarchy.
make_class(): Added a "goto finally" so that if populate_methods()
fails, the return status will be -1 (failure) instead of 0 (success).
fini_exceptions(): When decref'ing the static pointers to the
exception classes, clear out their dictionaries too. This breaks a
cycle from class->dict->method->class and allows the classes with
unbound methods to be reclaimed. This plugs a large memory leak in a
common Py_Initialize()/dosomething/Py_Finalize() loop.
pythonrun.c: In Py_Finalize, don't reset the initialized flag until after
the exit funcs have run.
atexit.py: in _run_exitfuncs, mutate the list of pending calls in a
threadsafe way. This wasn't a contributor to bug 128475, it just burned
my eyeballs when looking at that bug.
symbol table for each top-level compilation unit. The information in
the symbol table allows the elimination of the later optimize() pass;
the bytecode generation emits the correct opcodes.
The current version passes the complete regression test, but may still
contain some bugs. It's a fairly substantial revision. The current
code adds an assert() and a test that may lead to a Py_FatalError().
I expect to remove these before 2.1 beta 1.
The symbol table (struct symtable) is described in comments in the
code.
The changes affects the several com_XXX() functions that were used to
emit LOAD_NAME and its ilk. The primary interface for this bytecode
is now com_addop_varname() which takes a kind and a name, where kind
is one of VAR_LOAD, VAR_STORE, or VAR_DELETE.
There are many other smaller changes:
- The name mangling code is no longer contained in ifdefs. There are
two functions that expose the mangling logical: com_mangle() and
symtable_mangle().
- The com_error() function can accept NULL for its first argument;
this is useful with is_constant_false() is called during symbol
table generation.
- The loop index names used by list comprehensions have been changed
from __1__ to [1], so that they can not be accessed by Python code.
- in com_funcdef(), com_argdefs() is now called before the body of the
function is compiled. This provides consistency with com_lambdef()
and symtable_funcdef().
- Helpers do_pad(), dump(), and DUMP() are added to aid in debugging
the compiler.
except that it always returns Unicode objects.
A new C API PyObject_Unicode() is also provided.
This closes patch #101664.
Written by Marc-Andre Lemburg. Copyright assigned to Guido van Rossum.
- Use PyObject_RichCompare*() where possible: when comparing
keyword arguments, in _PyEval_SliceIndex(), and of course in
cmp_outcome().
Unrelated stuff:
- Removed all trailing whitespace.
- Folded some long lines.
message, and tries to make the messages more consistent and helpful when
the wrong number of arguments or duplicate keyword arguments are supplied.
Comes with more tests for test_extcall.py and and an update to an error
message in test/output/test_pyexpat.
re-initializing Python (Py_Finalize() followed by Py_Initialize()) to
blow up quickly. With the DECREF removed I can't get it to fail any
more. (Except it still leaks, but that's probably a separate issue.)
1) "from M import X" now works even if M is not a real module; it's
basically a getattr() operation with AttributeError exceptions
changed into ImportError.
2) "from M import *" now looks for M.__all__ to decide which names to
import; if M.__all__ doesn't exist, it uses M.__dict__.keys() but
filters out names starting with '_' as before. Whether or not
__all__ exists, there's no restriction on the type of M.
- Make error messages from issubclass() and isinstance() a bit more
descriptive (Ping, modified by Guido)
- Couple of tiny fixes to other docstrings (Ping)
- Get rid of trailing whitespace (Guido)
Cygwin Python DLL and Shared Extension Patch). Add module.dll as a
valid extension.
jlt63 writes: Note that his change essentially backs out the fix for
bug #115973. Should ".pyd" be retained instead for posterity?
an empty keywords dictionary (via apply() or the extended call syntax),
the keywords dict should be ignored. If the keywords dict is not empty,
TypeError should be raised. (Between the restructuring of the call
machinery and this patch, an empty dict in this situation would trigger
a SystemError via PyErr_BadInternalCall().)
Added regression tests to detect errors for this.
More revision still needed.
Much of the code that was in the mainloop was moved to a series of
helper functions. PyEval_CallObjectWithKeywords was split into two
parts. The first part now only does argument handling. The second
part is now named call_object and delegates the call to a
call_(function,method,etc.) helper.
XXX The call_XXX helper functions should be replaced with tp_call
functions for the respective types.
The CALL_FUNCTION implementation contains three kinds of optimization:
1. fast_cfunction and fast_function are called when the arguments on
the stack can be passed directly to eval_code2() without copying
them into a tuple.
2. PyCFunction objects are dispatched immediately, because they are
presumed to occur more often than anything else.
3. Bound methods are dispatched inline. The method object contains a
pointer to the function object that will be called. The function
is called from within the mainloop, which may allow optimization #1
to be used, too.
The extened call implementation -- f(*args) and f(**kw) -- are
implemented as a separate case in the mainloop. This allows the
common case of normal function calls to execute without wasting time
on checks for extended calls, although it does introduce a small
amount of code duplication.
Also, the unused final argument of eval_code2() was removed. This is
probably the last trace of the access statement :-).
Jeremy Hylton
Barry Warsaw
Guido van Rossum
Tim Peters
Jack Jansen
Marc-André Lemburg
Ka-Ping Yee
Fred Drake
Moshe Zadka
Greg Stein
Charles G. Waldman
Martin v. Löwis
Neil Schemenauer
Andrew M. Kuchling