Commit Graph

166 Commits

Author SHA1 Message Date
Jeremy Hylton d7f393edae In symtable_update_free_vars() do not modify the dictionary while
iterating over it using PyDict_Next().

This bug fix brought to you by the letters b, c, d, g, h, ... and the
reporter Ping.
2001-02-12 16:01:03 +00:00
Jeremy Hylton 6492bf71da SF patch 103589: Fix handling of cell vars that are either * or ** parameters.
(Nick Mathewson)

Remove to XXX comments
2001-02-09 22:55:26 +00:00
Jeremy Hylton cb17ae8b19 Relax the rules for using 'from ... import *' and exec in the presence
of nested functions.  Either is allowed in a function if it contains
no defs or lambdas or the defs and lambdas it contains have no free
variables.  If a function is itself nested and has free variables,
either is illegal.

Revise the symtable to use a PySymtableEntryObject, which holds all
the revelent information for a scope, rather than using a bunch of
st_cur_XXX pointers in the symtable struct.  The changes simplify the
internal management of the current symtable scope and of the stack.

Added new C source file: Python/symtable.c.  (Does the Windows build
process need to be updated?)

As part of these changes, the initial _symtable module interface
introduced in 2.1a2 is replaced.  A dictionary of
PySymtableEntryObjects are returned.
2001-02-09 22:22:18 +00:00
Guido van Rossum cd90c20b62 Reindent a function that was somehow indented by 7 spaces. Also did a
spaces->tab conversion for fields added to struct compiling.
2001-02-09 15:06:42 +00:00
Jeremy Hylton 5acc0c0cfc Fix symbol table pass to generation SyntaxError exceptions that
include the filename and line number.
2001-02-02 20:01:10 +00:00
Jeremy Hylton 4b38da664c Move a bunch of definitions that were internal to compile.c to
symtable.h, so that they can be used by external module.

Improve error handling in symtable_enter_scope(), which return an
error code that went unchecked by most callers. XXX The error handling
in symtable code is sloppy in general.

Modify symtable to record the line number that begins each scope.
This can help to identify which code block is being referred to when
multiple blocks are bound to the same name.

Add st_scopes dict that is used to preserve scope info when
PyNode_CompileSymtable() is called.  Otherwise, this information is
tossed as soon as it is no longer needed.

Add Py_SymtableString() to pythonrun; analogous to Py_CompileString().
2001-02-02 18:19:15 +00:00
Jeremy Hylton bbf10b59d1 add missing DECREF (thanks, Barry) 2001-02-02 02:58:48 +00:00
Jeremy Hylton 3faa52ecc4 Allow 'continue' inside 'try' clause
SF patch 102989 by Thomas Wouters
2001-02-01 22:48:12 +00:00
Jeremy Hylton 483638c9a8 Undo recent change that banned using import to bind a global, as per
discussion on python-dev.  'from mod import *' is still banned except
at the module level.

Fix value for special NOOPT entry in symtable.  Initialze to 0 instead
of None, so that later uses of PyInt_AS_LONG() are valid.  (Bug
reported by Donn Cave.)

replace local REPR macros with PyObject_REPR in object.h
2001-02-01 20:20:45 +00:00
Jeremy Hylton eab156f8eb Enforce two illegal import statements that were outlawed in the
reference manual but not checked: Names bound by import statemants may
not occur in global statements in the same scope. The from ... import *
form may only occur in a module scope.

I guess these changes could break code, but the reference manual
warned about them.

Several other small changes

If a variable is declared global in the nearest enclosing scope of a
free variable, then treat it is a global in the nested scope too.

Get rid of com_mangle and symtable_mangle functions and call mangle
directly.

If errors occur during symtable table creation, return -1 from
symtable_build().

Do not increment st_errors in assignment to lambda, because exception
is not set.

Add extra argument to symtable_assign(); the argument, flag, is ORed
with DEF_LOCAL for each symtable_add_def() call.
2001-01-30 01:24:43 +00:00
Jeremy Hylton 2fdfadf6dd plug leak detected by Barry 2001-01-29 22:42:28 +00:00
Jeremy Hylton 64949cb753 PEP 227 implementation
The majority of the changes are in the compiler.  The mainloop changes
primarily to implement the new opcodes and to pass a function's
closure to eval_code2().  Frames and functions got new slots to hold
the closure.

Include/compile.h
    Add co_freevars and co_cellvars slots to code objects.
    Update PyCode_New() to take freevars and cellvars as arguments
Include/funcobject.h
    Add func_closure slot to function objects.
    Add GetClosure()/SetClosure() functions (and corresponding
    macros) for getting at the closure.
Include/frameobject.h
    PyFrame_New() now takes a closure.
Include/opcode.h
    Add four new opcodes: MAKE_CLOSURE, LOAD_CLOSURE, LOAD_DEREF,
    STORE_DEREF.
    Remove comment about old requirement for opcodes to fit in 7
    bits.
compile.c
    Implement changes to code objects for co_freevars and co_cellvars.

    Modify symbol table to use st_cur_name (string object for the name
    of the current scope) and st_cur_children (list of nested blocks).
    Also define st_nested, which might more properly be called
    st_cur_nested.  Add several DEF_XXX flags to track def-use
    information for free variables.

    New or modified functions of note:
    com_make_closure(struct compiling *, PyCodeObject *)
        Emit LOAD_CLOSURE opcodes as needed to pass cells for free
        variables into nested scope.
    com_addop_varname(struct compiling *, int, char *)
        Emits opcodes for LOAD_DEREF and STORE_DEREF.
    get_ref_type(struct compiling *, char *name)
        Return NAME_CLOSURE if ref type is FREE or CELL
    symtable_load_symbols(struct compiling *)
        Decides what variables are cell or free based on def-use info.
        Can now raise SyntaxError if nested scopes are mixed with
        exec or from blah import *.
    make_scope_info(PyObject *, PyObject *, int, int)
        Helper functions for symtable scope stack.
    symtable_update_free_vars(struct symtable *)
        After a code block has been analyzed, it must check each of
        its children for free variables that are not defined in the
        block.  If a variable is free in a child and not defined in
        the parent, then it is defined by block the enclosing the
        current one or it is a global.  This does the right logic.
    symtable_add_use() is now a macro for symtable_add_def()
    symtable_assign(struct symtable *, node *)
        Use goto instead of for (;;)

    Fixed bug in symtable where name of keyword argument in function
    call was treated as assignment in the scope of the call site. Ex:
        def f():
            g(a=2) # a was considered a local of f

ceval.c
    eval_code2() now take one more argument, a closure.
    Implement LOAD_CLOSURE, LOAD_DEREF, STORE_DEREF, MAKE_CLOSURE>

    Also: When name error occurs for global variable, report that the
    name was global in the error mesage.

Objects/frameobject.c
    Initialize f_closure to be a tuple containing space for cellvars
    and freevars.  f_closure is NULL if neither are present.
Objects/funcobject.c
    Add support for func_closure.
Python/import.c
    Change the magic number.
Python/marshal.c
    Track changes to code objects.
2001-01-25 20:06:59 +00:00
Jeremy Hylton a6ebc4841d Fix bug reported by Ka-Ping Yee: The compiler botched parsing function
parameters that contained both anonymous tuples and *arg or **arg. Ex:
def f(a, (b, c), *d): pass

Fix the symtable_params() to generate names in the right order for
co_varnames slot of code object.  Consider *arg and **arg before the
"complex" names introduced by anonymous tuples.
2001-01-25 17:01:49 +00:00
Jeremy Hylton 5f827f4e9b Visit the initial test element of the listmaker for a list
comprehension.  Fixes bug reported by Tim Peters.
2001-01-23 01:26:20 +00:00
Jeremy Hylton 1113cfc767 prevent symtable_params() from dereferencing off the end of the
varagslist node. based on fix from Thomas Wouters.
2001-01-23 00:50:52 +00:00
Barry Warsaw 174e8018c4 com_init(): My entry into the smallest patch possible category.
(cosmetic whitespace change).
2001-01-22 04:35:57 +00:00
Tim Peters 7f3e4adf60 SF patch #103336: Missing cast. 2001-01-20 05:15:26 +00:00
Jeremy Hylton e36f77814e This patch introduces an extra pass to the compiler that generates a
symbol table for each top-level compilation unit.  The information in
the symbol table allows the elimination of the later optimize() pass;
the bytecode generation emits the correct opcodes.

The current version passes the complete regression test, but may still
contain some bugs.  It's a fairly substantial revision.  The current
code adds an assert() and a test that may lead to a Py_FatalError().
I expect to remove these before 2.1 beta 1.

The symbol table (struct symtable) is described in comments in the
code.

The changes affects the several com_XXX() functions that were used to
emit LOAD_NAME and its ilk.  The primary interface for this bytecode
is now com_addop_varname() which takes a kind and a name, where kind
is one of VAR_LOAD, VAR_STORE, or VAR_DELETE.

There are many other smaller changes:

- The name mangling code is no longer contained in ifdefs.  There are
  two functions that expose the mangling logical: com_mangle() and
  symtable_mangle().

- The com_error() function can accept NULL for its first argument;
  this is useful with is_constant_false() is called during symbol
  table generation.

- The loop index names used by list comprehensions have been changed
  from __1__ to [1], so that they can not be accessed by Python code.

- in com_funcdef(), com_argdefs() is now called before the body of the
  function is compiled.  This provides consistency with com_lambdef()
  and symtable_funcdef().

- Helpers do_pad(), dump(), and DUMP() are added to aid in debugging
  the compiler.
2001-01-19 03:21:30 +00:00
Guido van Rossum fc5ce61abd SF Patch #103250, by pj99: Optimize a strspn() out of startup.
Minor startup speedup: avoid a call to strspn().
2001-01-19 00:24:06 +00:00
Guido van Rossum 83fb073a03 Plug a memory leak in com_import_stmt(): the tuple created to hold the
"..." in "from M import ..." was never DECREFed.  Leak reported by
James Slaughter and nailed by Barry, who also provided an earlier
version of this patch.
2000-11-27 22:22:36 +00:00
Tim Peters 102e457a01 SF bug 119622: compile errors due to redundant atof decls. I don't understand
the bug report (for details, look at it), but agree there's no need for Python
to declare atof itself:  we #include stdlib.h, and ANSI C sez atof is declared
there already.
2000-11-14 20:44:53 +00:00
Fred Drake d5fadf75e4 Rationalize use of limits.h, moving the inclusion to Python.h.
Add definitions of INT_MAX and LONG_MAX to pyport.h.
Remove includes of limits.h and conditional definitions of INT_MAX
and LONG_MAX elsewhere.

This closes SourceForge patch #101659 and bug #115323.
2000-09-26 05:46:01 +00:00
Fred Drake fd1f1be98d com_continue_stmt(): Improve error message when continue is found
in a try statement in a loop.

This is related to SourceForge bug #110830.
2000-09-08 16:31:24 +00:00
Fredrik Lundh 1fa0b895ec changed \x to consume exactly two hex digits. implements PEP-223
for 8-bit strings.
2000-09-02 20:11:27 +00:00
Guido van Rossum 8586991099 REMOVED all CWI, CNRI and BeOpen copyright markings.
This should match the situation in the 1.6b1 tree.
2000-09-01 23:29:29 +00:00
Thomas Wouters dd13e4f91f Replace the run-time 'future-bytecode-stream-inspection' hack to find out
how 'import' was called with a compiletime mechanism: create either a tuple
of the import arguments, or None (in the case of a normal import), add it to
the code-block constants, and load it onto the stack before calling
IMPORT_NAME.
2000-08-27 20:31:27 +00:00
Thomas Wouters e753ef8d1b Re-allow 'import mod.submod as s', and change its meaning to what it should
mean; the same as 'from mod import submod as s'.
2000-08-27 20:16:32 +00:00
Thomas Wouters 0ae722e0a2 Oops, one pop too many. 2000-08-27 19:01:33 +00:00
Thomas Wouters b59290f5b2 Fix allowable node-types for assignment, need to add 'listmaker'.
(This fix is a bit broken, just as the test already was: the test for
testlist and listmaker are done always, whereas the test for exprlist and
the actual abort() are only done if Py_DEBUG is defined. Suggestions
welcome, I guess ;)
2000-08-25 05:41:11 +00:00
Thomas Wouters 434d0828d8 Support for three-token characters (**=, >>=, <<=) which was written by
Michael Hudson, and support in general for the augmented assignment syntax.
The graminit.c patch is large!
2000-08-24 20:11:32 +00:00
Fred Drake ef8ace3a6f Charles G. Waldman <cgw@fnal.gov>:
Add the EXTENDED_ARG opcode to the virtual machine, allowing 32-bit
arguments to opcodes instead of being forced to stick to the 16-bit
limit.  This is especially useful for machine-generated code, which
can be too long for the SET_LINENO parameter to fit into 16 bits.

This closes the implementation portion of SourceForge patch #100893.
2000-08-24 00:32:09 +00:00
Skip Montanaro 46dfa5f4ed require list comprehensions to start with a for clause 2000-08-22 02:43:07 +00:00
Barry Warsaw 24703a02e7 com_print_stmt(): Guido rightly points out that the stream expression
in extended prints should only be evaluated once.  This patch plays
stack games (documented!) to fix this.
2000-08-21 17:07:20 +00:00
Barry Warsaw 29c574e30c PEP 214, Extended print Statement, has been accepted by the BDFL.
com_print_stmt(): Implement recognition of, and byte compilation for,
extended print using new byte codes PRINT_ITEM_TO and
PRINT_NEWLINE_TO.
2000-08-21 15:38:56 +00:00
Thomas Wouters 8bad612881 Disallow "import mod.submod as m", because the result is ambiguous. Does it
load mod.submod as m, or mod as m ? Both can be achieved differently, and
unambiguously. Also attempt to document this restriction (editor
appreciated!)

Note that this is an artificial check during compile, because incorporating
this in the grammar is hard, and then adjusting the compiler to do the right
thing with the right nodes is harder.
2000-08-19 20:55:02 +00:00
Barry Warsaw 73f77b4495 com_error(): Quiet gcc -Wall warning. 2000-08-18 19:59:20 +00:00
Thomas Wouters 5215225ea1 Apply SF patch #101135, adding 'import module as m' and 'from module import
name as n'. By doing some twists and turns, "as" is not a reserved word.

There is a slight change in semantics for 'from module import name' (it will
now honour the 'global' keyword) but only in cases that are explicitly
undocumented.
2000-08-17 22:55:00 +00:00
Tim Peters b59ab42487 Fix new compiler warnings. Unused var in compile.c. Argsize mismatches
in binascii.c (only on platforms with signed chars -- although Py_CHARMASK
is documented as returning an int, it only does so on platforms with
signed chars).
2000-08-15 16:41:26 +00:00
Fred Drake a811a5b13a Remove the osdefs.h #include; it was not needed in the final version of
my last set of changes.
2000-08-15 16:13:37 +00:00
Fred Drake dcf08e0dfe When raising a SyntaxError, make a best-effort attempt to set the
filename and lineno attributes, but do not mask the SyntaxError if we
fail.

This is part of what is needed to close SoruceForge bug #110628
(Jitterbug PR#278).

Wrap a long line to fit in under 80 columns.
2000-08-15 15:49:44 +00:00
Thomas Wouters 87df80d542 The list comp patch checked for the second child node of the 'listmaker'
node, without checking if the node actually had more than one child. It can
have only one node, though: '[' test ']'. This fixes it.
2000-08-13 17:05:17 +00:00
Thomas Wouters 361852f80e The list comprehensions patch partly reversed the removal of UNPACK_LIST,
re-introducing com_assign_list, now unused. Removed it.
2000-08-12 22:03:16 +00:00
Skip Montanaro 803d6e5451 list comprehensions. see
http://sourceforge.net/patch/?func=detailpatch&patch_id=100654&group_id=5470

for details.
2000-08-12 18:09:51 +00:00
Thomas Wouters 0be5aab04d Merge UNPACK_LIST and UNPACK_TUPLE into a single UNPACK_SEQUENCE, since they
did the same anyway.

I'm not sure what to do with Tools/compiler/compiler/* -- that isn't part of
distutils, is it ? Should it try to be compatible with old bytecode version ?
2000-08-11 22:15:52 +00:00
Guido van Rossum 6c2f0c73a1 When returning an error from jcompile() (which is passed through by
PyNode_Compile()), make sure that an exception is actually set --
otherwise someone stomped on our error.  [2.0 checkin of this fix.]
2000-08-07 19:22:43 +00:00
Thomas Wouters e8643c465d Fix some strange indentation and grammar that have been bugging me for
weeks.
2000-08-05 21:37:50 +00:00
Thomas Wouters f70ef4f860 Mass ANSIfication of function definitions. Doesn't cover all 'extern'
declarations yet, those come later.
2000-07-22 18:47:25 +00:00
Peter Schneider-Kamp 9a5086c598 just fixing the indentation 2000-07-13 06:24:29 +00:00
Peter Schneider-Kamp 11384c60f6 raise error on duplicate function arguments
example:

>>> def f(a,a):print a
...
SyntaxError: duplicate argument in function definition
2000-07-13 06:15:04 +00:00
Tim Peters dbd9ba6a6c Nuke all remaining occurrences of Py_PROTO and Py_FPROTO. 2000-07-09 03:09:57 +00:00