Commit Graph

197 Commits

Author SHA1 Message Date
Jeremy Hylton 897b82123d Make it illegal to assign to __debug__ as per Guido's request. 2001-03-23 14:08:38 +00:00
Jeremy Hylton 2e2cded1b5 Set the line number correctly for a nested function with an exec or
import *.  Mark the offending stmt rather than the function def line.
2001-03-22 03:57:58 +00:00
Jeremy Hylton 280e6bd742 Make error messages clearer for illegal combinations of nested
functions and import */exec.
2001-03-22 03:51:05 +00:00
Jeremy Hylton 061d106a0f If a code object is compiled with nested scopes, define the CO_NESTED flag.
Add PyEval_GetNestedScopes() which returns a non-zero value if the
code for the current interpreter frame has CO_NESTED defined.
2001-03-22 02:32:48 +00:00
Jeremy Hylton ded4bd776f Update PyNode_CompileSymtable() to understand future statements 2001-03-21 19:01:33 +00:00
Jeremy Hylton 220ae7c0bf Fix PyFrame_FastToLocals() and counterpart to deal with cells and
frees.  Note there doesn't seem to be any way to test LocalsToFast(),
because the instructions that trigger it are illegal in nested scopes
with free variables.

Fix allocation strategy for cells that are also formal parameters.
Instead of emitting LOAD_FAST / STORE_DEREF pairs for each parameter,
have the argument handling code in eval_code2() do the right thing.

A side-effect of this change is that cell variables that are also
arguments are listed at the front of co_cellvars in the order they
appear in the argument list.
2001-03-21 16:43:47 +00:00
Jeremy Hylton ce7ef599d2 Fixup handling of free variables in methods when the class scope also
has a binding for the name.  The fix is in two places:

  - in symtable_update_free_vars, ignore a global stmt in a class scope
  - in symtable_load_symbols, add extra handling for names that are
    defined at class scope and free in a method

Closes SF bug 407800
2001-03-20 00:25:43 +00:00
Jeremy Hylton 23b4227ec8 Fix crashes in nested list comprehensions
SF bugs 409230 and 407800

Also remove bogus list comp code from symtable_assign().
2001-03-19 20:38:06 +00:00
Guido van Rossum 207fda61a5 Refactored the warning-issuing code more.
Made sure that the warnings issued by symtable_check_unoptimized()
(about import * and exec) contain the proper filename and line number,
and are transformed into SyntaxError exceptions with -Werror.
2001-03-02 03:30:41 +00:00
Jeremy Hylton 9f324e964e Useful future statement support for the interactive interpreter
(Also remove warning about module-level global decl, because we can't
distinguish from code passed to exec.)

Define PyCompilerFlags type contains a single element,
cf_nested_scopes, that is true if a nested scopes future statement has
been entered at the interactive prompt.

New API functions:
    PyNode_CompileFlags()
    PyRun_InteractiveOneFlags()
    -- same as their non Flags counterparts except that the take an
       optional PyCompilerFlags pointer

compile.c: In jcompile() use PyCompilerFlags argument.  If
    cf_nested_scopes is true, compile code with nested scopes.  If it
    is false, but the code has a valid future nested scopes statement,
    set it to true.

pythonrun.c: Create a new PyCompilerFlags object in
    PyRun_InteractiveLoop() and thread it through to
    PyRun_InteractiveOneFlags().
2001-03-01 22:59:14 +00:00
Jeremy Hylton 7889107be7 Fix core dump in example from Samuele Pedroni:
from __future__ import nested_scopes
x=7
def f():
    x=1
    def g():
        global x
        def i():
            def h():
                return x
            return h()
        return i()
    return g()

print f()
print x

This kind of code didn't work correctly because x was treated as free
in i, leading to an attempt to load x in g to make a closure for i.

Solution is to make global decl apply to nested scopes unless their is
an assignment.  Thus, x in h is global.
2001-03-01 06:09:34 +00:00
Jeremy Hylton 5125773ff1 Don't add global names to st->st_global if we're already iterating
over the elements of st->st_global!
2001-03-01 00:42:55 +00:00
Jeremy Hylton 3dd5ad3b4f undo introduction of st_global_star 2001-02-28 23:47:55 +00:00
Jeremy Hylton c176132d63 Warn about global statement at the module level.
Do better accounting for global variables.
2001-02-28 23:44:45 +00:00
Jeremy Hylton 4419ac1a97 Add warning/error handlin for problematic nested scopes cases as
described in PEP 227.

symtable_check_unoptimized() warns about import * and exec with "in"
when it is used in a function that contains a nested function with
free variables.  Warnings are issued unless nested scopes are in
effect, in which case these are SyntaxErrors.

symtable_check_shadow() warns about assignments in a function scope
that shadow free variables defined in a nested scope.  This will
always generate a warning -- and will behave differently with nested
scopes than without.

Restore full checking for free vars in children, even when nested
scopes are not enabled.  This is needed to support warnings for
shadowing.

Change symtable_warn() to return an int-- the return value of
PyErr_WarnExplicit.

Sundry cleanup: Remove commented out code.  Break long lines.
2001-02-28 22:54:51 +00:00
Guido van Rossum ee34ac124a Let's have some sanity. Introduce a helper to issue a symbol table
warning.
2001-02-28 22:08:12 +00:00
Guido van Rossum 0bba7f83f2 Use the new PyErr_WarnExplicit() API to issue better warnings for
global after assign / use.

Note: I'm not updating the PyErr_Warn() call for import * / exec
combined with a function, because I can't trigger it with an example.
Jeremy, just follow the example of the call to PyErr_WarnExplicit()
that I *did* include.
2001-02-28 21:55:38 +00:00
Jeremy Hylton ad3d3f2f3f Improve SyntaxErrors for bad future statements. Set file and location
for errors raised in future.c.

Move some helper functions from compile.c to errors.c and make them
API functions: PyErr_SyntaxLocation() and PyErr_ProgramText().
2001-02-28 17:47:12 +00:00
Jeremy Hylton 9f1b9932b8 Print the offending line of code in the traceback for SyntaxErrors
raised by the compiler.

XXX For now, text entered into the interactive intepreter is not
printed in the traceback.

Inspired by a patch from Roman Sulzhyk

compile.c:

Add helper fetch_program_text() that opens a file and reads until it
finds the specified line number.  The code is a near duplicate of
similar code in traceback.c.

Modify com_error() to pass two arguments to SyntaxError constructor,
where the second argument contains the offending text when possible.

Modify set_error_location(), now used only by the symtable pass, to
set the text attribute on existing exceptions.

pythonrun.c:

Change parse_syntax_error() to continue of the offset attribute of a
SyntaxError is None.  In this case, it sets offset to -1.

Move code from PyErr_PrintEx() into helper function
print_error_text().  In the helper, only print the caret for a
SyntaxError if offset > 0.
2001-02-28 07:07:43 +00:00
Jeremy Hylton 39e2f3f824 Presumed correct compiler pass for future statements
XXX still need to integrate into symtable API

compile.h: Remove ff_n_simple_stmt; obsolete.

           Add ff_found_docstring used internally to skip one and only
           one string at the beginning of a module.

compile.c: Add check for from __future__ imports to far into the file.

 	   In symtable_global() check for -1 returned from
	   symtable_lookup(), which signifies name not defined.

	   Add missing DECERF in symtable_add_def.

           Free c->c_future.

future.c:  Add special handling for multiple statements joined on a
	   single line using one or more semicolons; this form can
           include an illegal future statement that would otherwise be
           hard to detect.

	   Add support for detecting and skipping doc strings.
2001-02-28 01:58:08 +00:00
Jeremy Hylton 4db62b1e14 Improved __future__ parser; still more to do
Makefile.pre.in: add target future.o

Include/compile.h: define PyFutureFeaters and PyNode_Future()
                   add c_future slot to struct compiling

Include/symtable.h: add st_future slot to struct symtable

Python/future.c: implementation of PyNode_Future()

Python/compile.c: use PyNode_Future() for nested_scopes support

Python/symtable.c: include compile.h to pick up PyFutureFeatures decl
2001-02-27 19:07:02 +00:00
Jeremy Hylton be77cf7d57 Add warnings about undefined "global"
SF bug #233532

XXX Can't figure out how to write test cases that work with warnings
2001-02-27 05:15:57 +00:00
Jeremy Hylton 29906eef3a Preliminary support for future nested scopes
compile.h: #define NESTED_SCOPES_DEFAULT 0 for Python 2.1
           __future__ feature name: "nested_scopes"

symtable.h: Add st_nested_scopes slot.  Define flags to track exec and
    import star.

Lib/test/test_scope.py: requires nested scopes

compile.c: Fiddle with error messages.

    Reverse the sense of ste_optimized flag on
    PySymtableEntryObjects.  If it is true, there is an optimization
    conflict.

    Modify get_ref_type to respect st_nested_scopes flags.

    Refactor symtable_load_symbols() into several smaller functions,
    which use struct symbol_info to share variables.  In new function
    symtable_update_flags(), raise an error or warning for import * or
    bare exec that conflicts with nested scopes.  Also, modify handle
    for free variables to respect st_nested_scopes flag.

    In symtable_init() assign st_nested_scopes flag to
    NESTED_SCOPES_DEFAULT (defined in compile.h).

    Add preliminary and often incorrect implementation of
    symtable_check_future().

    Add symtable_lookup() helper for future use.
2001-02-27 04:23:34 +00:00
Tim Peters 1e542110f9 Shuffle premature decref; nuke unreachable code block.
Fixes the "debug-build -O test_builtin.py and no test_b2.pyo" crash just
discussed on Python-Dev.
2001-02-23 22:23:53 +00:00
Barry Warsaw 0372af754e symtable_update_free_vars(), symtable_undo_free(),
symtable_enter_scope(): Removed some unnecessary backslashes at the
end of lines.  C != Python. :)
2001-02-23 18:22:59 +00:00
Jeremy Hylton 74b3bc47df Fix for bug 133489: compiler leaks memory
Two different but related problems:

1. PySymtable_Free() must explicitly DECREF(st->st_cur), which should
always point to the global symtable entry.  This entry is setup by the
first enter_scope() call, but there is never a corresponding
exit_scope() call.

Since each entry has a reference to scopes defined within it, the
missing DECREF caused all symtable entries to be leaked.

2. The leak here masked a separate problem with
PySymtableEntry_New().  When the requested entry was found in
st->st_symbols, the entry was returned without doing an INCREF.

And problem c) The ste_children slot was getting two copies of each
child entry, because it was populating the slot on the first and
second passes.  Now only populate on the first pass.
2001-02-23 17:55:27 +00:00
Jeremy Hylton 2b3f0ca2ac Fix for implicit tuple + default arguments, courtesy of Michael Hudson.
SF patch #103749
2001-02-19 23:52:49 +00:00
Jeremy Hylton 384639f80e When running python -O, do not include blocks defined in asserts in
the symbol table pass.  These blocks were already ignored by the code
gen pass.  Both passes must visit the same set of blocks in the same
order.

Fixes SF buf 132820
2001-02-19 15:50:51 +00:00
Jeremy Hylton 17820c4f1b Tolerate ill-formed trees in symtable_assign(). Fixes SF bug 132510. 2001-02-19 15:33:10 +00:00
Tim Peters 4e30378e80 Bug #132313 error message confusing for assignment in lambda.
They're actually complaining about something more specific, an assignment
in a lambda as an actual argument, so that Python parses the
lambda as if it were a keyword argument.  Like f(lambda x: x[0]=42).
The "lambda x: x[0]" part gets parsed as if it were a keyword, being
bound to 42, and the resulting error msg didn't make much sense.
2001-02-18 04:45:10 +00:00
Tim Peters 3081421d9e Change temp names created by listcomps from [%d] to _[%d], so the one-liner
[k for k in dir() if k[0] != "_"]
can be used to get the non-private names (used to contain "[1]").
2001-02-17 05:30:26 +00:00
Jeremy Hylton d7f393edae In symtable_update_free_vars() do not modify the dictionary while
iterating over it using PyDict_Next().

This bug fix brought to you by the letters b, c, d, g, h, ... and the
reporter Ping.
2001-02-12 16:01:03 +00:00
Jeremy Hylton 6492bf71da SF patch 103589: Fix handling of cell vars that are either * or ** parameters.
(Nick Mathewson)

Remove to XXX comments
2001-02-09 22:55:26 +00:00
Jeremy Hylton cb17ae8b19 Relax the rules for using 'from ... import *' and exec in the presence
of nested functions.  Either is allowed in a function if it contains
no defs or lambdas or the defs and lambdas it contains have no free
variables.  If a function is itself nested and has free variables,
either is illegal.

Revise the symtable to use a PySymtableEntryObject, which holds all
the revelent information for a scope, rather than using a bunch of
st_cur_XXX pointers in the symtable struct.  The changes simplify the
internal management of the current symtable scope and of the stack.

Added new C source file: Python/symtable.c.  (Does the Windows build
process need to be updated?)

As part of these changes, the initial _symtable module interface
introduced in 2.1a2 is replaced.  A dictionary of
PySymtableEntryObjects are returned.
2001-02-09 22:22:18 +00:00
Guido van Rossum cd90c20b62 Reindent a function that was somehow indented by 7 spaces. Also did a
spaces->tab conversion for fields added to struct compiling.
2001-02-09 15:06:42 +00:00
Jeremy Hylton 5acc0c0cfc Fix symbol table pass to generation SyntaxError exceptions that
include the filename and line number.
2001-02-02 20:01:10 +00:00
Jeremy Hylton 4b38da664c Move a bunch of definitions that were internal to compile.c to
symtable.h, so that they can be used by external module.

Improve error handling in symtable_enter_scope(), which return an
error code that went unchecked by most callers. XXX The error handling
in symtable code is sloppy in general.

Modify symtable to record the line number that begins each scope.
This can help to identify which code block is being referred to when
multiple blocks are bound to the same name.

Add st_scopes dict that is used to preserve scope info when
PyNode_CompileSymtable() is called.  Otherwise, this information is
tossed as soon as it is no longer needed.

Add Py_SymtableString() to pythonrun; analogous to Py_CompileString().
2001-02-02 18:19:15 +00:00
Jeremy Hylton bbf10b59d1 add missing DECREF (thanks, Barry) 2001-02-02 02:58:48 +00:00
Jeremy Hylton 3faa52ecc4 Allow 'continue' inside 'try' clause
SF patch 102989 by Thomas Wouters
2001-02-01 22:48:12 +00:00
Jeremy Hylton 483638c9a8 Undo recent change that banned using import to bind a global, as per
discussion on python-dev.  'from mod import *' is still banned except
at the module level.

Fix value for special NOOPT entry in symtable.  Initialze to 0 instead
of None, so that later uses of PyInt_AS_LONG() are valid.  (Bug
reported by Donn Cave.)

replace local REPR macros with PyObject_REPR in object.h
2001-02-01 20:20:45 +00:00
Jeremy Hylton eab156f8eb Enforce two illegal import statements that were outlawed in the
reference manual but not checked: Names bound by import statemants may
not occur in global statements in the same scope. The from ... import *
form may only occur in a module scope.

I guess these changes could break code, but the reference manual
warned about them.

Several other small changes

If a variable is declared global in the nearest enclosing scope of a
free variable, then treat it is a global in the nested scope too.

Get rid of com_mangle and symtable_mangle functions and call mangle
directly.

If errors occur during symtable table creation, return -1 from
symtable_build().

Do not increment st_errors in assignment to lambda, because exception
is not set.

Add extra argument to symtable_assign(); the argument, flag, is ORed
with DEF_LOCAL for each symtable_add_def() call.
2001-01-30 01:24:43 +00:00
Jeremy Hylton 2fdfadf6dd plug leak detected by Barry 2001-01-29 22:42:28 +00:00
Jeremy Hylton 64949cb753 PEP 227 implementation
The majority of the changes are in the compiler.  The mainloop changes
primarily to implement the new opcodes and to pass a function's
closure to eval_code2().  Frames and functions got new slots to hold
the closure.

Include/compile.h
    Add co_freevars and co_cellvars slots to code objects.
    Update PyCode_New() to take freevars and cellvars as arguments
Include/funcobject.h
    Add func_closure slot to function objects.
    Add GetClosure()/SetClosure() functions (and corresponding
    macros) for getting at the closure.
Include/frameobject.h
    PyFrame_New() now takes a closure.
Include/opcode.h
    Add four new opcodes: MAKE_CLOSURE, LOAD_CLOSURE, LOAD_DEREF,
    STORE_DEREF.
    Remove comment about old requirement for opcodes to fit in 7
    bits.
compile.c
    Implement changes to code objects for co_freevars and co_cellvars.

    Modify symbol table to use st_cur_name (string object for the name
    of the current scope) and st_cur_children (list of nested blocks).
    Also define st_nested, which might more properly be called
    st_cur_nested.  Add several DEF_XXX flags to track def-use
    information for free variables.

    New or modified functions of note:
    com_make_closure(struct compiling *, PyCodeObject *)
        Emit LOAD_CLOSURE opcodes as needed to pass cells for free
        variables into nested scope.
    com_addop_varname(struct compiling *, int, char *)
        Emits opcodes for LOAD_DEREF and STORE_DEREF.
    get_ref_type(struct compiling *, char *name)
        Return NAME_CLOSURE if ref type is FREE or CELL
    symtable_load_symbols(struct compiling *)
        Decides what variables are cell or free based on def-use info.
        Can now raise SyntaxError if nested scopes are mixed with
        exec or from blah import *.
    make_scope_info(PyObject *, PyObject *, int, int)
        Helper functions for symtable scope stack.
    symtable_update_free_vars(struct symtable *)
        After a code block has been analyzed, it must check each of
        its children for free variables that are not defined in the
        block.  If a variable is free in a child and not defined in
        the parent, then it is defined by block the enclosing the
        current one or it is a global.  This does the right logic.
    symtable_add_use() is now a macro for symtable_add_def()
    symtable_assign(struct symtable *, node *)
        Use goto instead of for (;;)

    Fixed bug in symtable where name of keyword argument in function
    call was treated as assignment in the scope of the call site. Ex:
        def f():
            g(a=2) # a was considered a local of f

ceval.c
    eval_code2() now take one more argument, a closure.
    Implement LOAD_CLOSURE, LOAD_DEREF, STORE_DEREF, MAKE_CLOSURE>

    Also: When name error occurs for global variable, report that the
    name was global in the error mesage.

Objects/frameobject.c
    Initialize f_closure to be a tuple containing space for cellvars
    and freevars.  f_closure is NULL if neither are present.
Objects/funcobject.c
    Add support for func_closure.
Python/import.c
    Change the magic number.
Python/marshal.c
    Track changes to code objects.
2001-01-25 20:06:59 +00:00
Jeremy Hylton a6ebc4841d Fix bug reported by Ka-Ping Yee: The compiler botched parsing function
parameters that contained both anonymous tuples and *arg or **arg. Ex:
def f(a, (b, c), *d): pass

Fix the symtable_params() to generate names in the right order for
co_varnames slot of code object.  Consider *arg and **arg before the
"complex" names introduced by anonymous tuples.
2001-01-25 17:01:49 +00:00
Jeremy Hylton 5f827f4e9b Visit the initial test element of the listmaker for a list
comprehension.  Fixes bug reported by Tim Peters.
2001-01-23 01:26:20 +00:00
Jeremy Hylton 1113cfc767 prevent symtable_params() from dereferencing off the end of the
varagslist node. based on fix from Thomas Wouters.
2001-01-23 00:50:52 +00:00
Barry Warsaw 174e8018c4 com_init(): My entry into the smallest patch possible category.
(cosmetic whitespace change).
2001-01-22 04:35:57 +00:00
Tim Peters 7f3e4adf60 SF patch #103336: Missing cast. 2001-01-20 05:15:26 +00:00
Jeremy Hylton e36f77814e This patch introduces an extra pass to the compiler that generates a
symbol table for each top-level compilation unit.  The information in
the symbol table allows the elimination of the later optimize() pass;
the bytecode generation emits the correct opcodes.

The current version passes the complete regression test, but may still
contain some bugs.  It's a fairly substantial revision.  The current
code adds an assert() and a test that may lead to a Py_FatalError().
I expect to remove these before 2.1 beta 1.

The symbol table (struct symtable) is described in comments in the
code.

The changes affects the several com_XXX() functions that were used to
emit LOAD_NAME and its ilk.  The primary interface for this bytecode
is now com_addop_varname() which takes a kind and a name, where kind
is one of VAR_LOAD, VAR_STORE, or VAR_DELETE.

There are many other smaller changes:

- The name mangling code is no longer contained in ifdefs.  There are
  two functions that expose the mangling logical: com_mangle() and
  symtable_mangle().

- The com_error() function can accept NULL for its first argument;
  this is useful with is_constant_false() is called during symbol
  table generation.

- The loop index names used by list comprehensions have been changed
  from __1__ to [1], so that they can not be accessed by Python code.

- in com_funcdef(), com_argdefs() is now called before the body of the
  function is compiled.  This provides consistency with com_lambdef()
  and symtable_funcdef().

- Helpers do_pad(), dump(), and DUMP() are added to aid in debugging
  the compiler.
2001-01-19 03:21:30 +00:00
Guido van Rossum fc5ce61abd SF Patch #103250, by pj99: Optimize a strspn() out of startup.
Minor startup speedup: avoid a call to strspn().
2001-01-19 00:24:06 +00:00