Commit Graph

93 Commits

Author SHA1 Message Date
Jeremy Hylton f354575328 Generate FOR_ITER-based loops instead of old FOR_LOOP-based loops 2001-08-28 16:35:18 +00:00
Jeremy Hylton c59e220000 Handle private names
(Hard to believe these were never handled before)

Add misc.mangle() that mangles based on the rules in compile.c.
XXX Need to test the corner cases

Update CodeGenerator with a class_name attribute bound to None.  If a
particular instance is created within a class scope, the instance's
class_name is bound to that class's name.

Add mangle() method to CodeGenerator that mangles if the class_name
has a class_name in it.

Modify the FunctionCodeGenerator family to handle an extra argument--
the class_name.

Wrap all name ops and attrnames in calls to self.mangle()
2001-08-27 22:56:16 +00:00
Jeremy Hylton 80ea40d858 emit SET_LINENO for augmented assignments 2001-08-27 21:58:09 +00:00
Jeremy Hylton 2afff324ea Many changes -- bug fixes and sundry improvements
Make nested scopes enabled by default

Add is_constant_false() helper so that compiled code and symbols are
consistent with builtin compiler's handling of "if 0:"

Fix doc string handling to be consistent with recent change that
eliminates the doc string from the Module's node attribute.

Add fix to print handling from Evan & Shane.

Track change to visitor api by making "verbose" explicit.

Comment out setting CO_NESTED flag (it's unnecessary in 2.2).
2001-08-27 21:51:52 +00:00
Jeremy Hylton bb0bae6da2 Pop loop off the loop stack before handling the loop's else clause.
Otherwise, continue/break will attempt to affect the wrong loop.

A few more fiddles to get the SET_LINENOs consistent across compilers.
2001-04-12 21:54:41 +00:00
Jeremy Hylton 13d70944cb Use new _implicitNameOp() to generate name op code for list comprehensions.
Always emit a SET_LINENO 0 at the beginning of the module.  The
builtin compiler does this, and it's much easier to compare bytecode
generated by the two compilers if they both do.

Move the SET_LINENO inside the FOR_LOOP block for list
comprehensions.  Also for compat. with builtin compiler.
2001-04-12 21:04:43 +00:00
Jeremy Hylton 614e87f286 Add support for visitAssAttr to findOp(). 2001-04-12 20:24:26 +00:00
Jeremy Hylton 542b11acfd pyassem.py:
Fix annoying bugs in flow graph layout code.  In some cases the
    implicit control transfers weren't honored.  In other cases,
    JUMP_FORWARD instructions jumped backwards.

    Remove unused arg from nextBlock().

pycodegen.py

    Add optional force kwarg to set_lineno() that will emit a
    SET_LINENO even if it is the same as the previous lineno.

    Use explicit LOAD_FAST and STORE_FAST to access list comp implicit
    variables.  (The symbol table doesn't know about them.)
2001-04-12 20:21:39 +00:00
Jeremy Hylton bfb0cf822b Revise handling of tuple arguments so that the variables names match
those used by compile.c.  (test_grammar now depends on the names)
2001-04-12 17:33:34 +00:00
Jeremy Hylton 3f76b7e6e4 Fix unpackSequence() to use _nameOp() rather than LOAD_FAST 2001-04-12 06:52:27 +00:00
Jeremy Hylton 660cc77abb Inside a class scope always use LOAD_NAME, STORE_NAME, DEL_NAME 2001-04-12 06:49:00 +00:00
Jeremy Hylton 364f9b9e2f Preliminary support for nested scopes
XXX Still doesn't work right for classes
XXX Still doesn't do sufficient error checking
2001-04-12 06:40:42 +00:00
Jeremy Hylton 84ec1f9159 typo 2001-04-11 16:43:13 +00:00
Jeremy Hylton 01d12937df [finishing fix from earlier checkins]
Call set_lineno() in visitDiscard(), which will generate linenos for
discard statements, e.g. the statement "1/0"

Fixes SF bug #409587
2001-04-11 16:36:25 +00:00
Jeremy Hylton 9ab019bee7 Generate docstrings.
Fixes SF buf #217004

Add method fixDocstring() to CodeGenerator.  It converts the Discard
node containing the docstring into an assignment to __doc__.
2001-04-11 16:24:30 +00:00
Jeremy Hylton 80e29bd139 Add support for future statements 2001-04-09 04:28:48 +00:00
Jeremy Hylton 314e3fb215 Change the graph structure to contain the code generator object for
embedded code objects (e.g. functions) rather than the generated code
object.  This change means that the compiler generates code for
everything at the end, rather then generating code for each function
as it finds it.  Implementation note: _convert_LOAD_CONST in
pyassem.py must be change to call getCode().

Other changes follow.  Several changes creates extra edges between
basic blocks to reflect control flow for loops and exceptions.  These
missing edges had gone unnoticed because they do not affect the
current compilation process.

pyassem.py:
    Add _enable_debug() and _disable_debug() methods that print
    instructions and blocks to stdout as they are generated.

    Add edges between blocks for instructions like SETUP_LOOP,
    FOR_LOOP, etc.

    Add pruneNext to get rid of bogus edges remaining after
    unconditional transfer ops (e.g. JUMP_FORWARD)

    Change repr of Block to omit block length.

pycodegen.py:
    Make sure a new block is started after FOR_LOOP, etc.

    Change assert implementation to use RAISE_VARARGS 1 when there is
    no user-specified failure output.

misc.py:
    Implement __contains__ and copy for Set.
2000-11-06 03:43:11 +00:00
Jeremy Hylton 9c048f9f65 Now supports entire Python 2.0 language and still supports Python
1.5.2.  The compiler generates code for the version of the interpreter
it is run under.

ast.py:
    Print and Printnl add dest attr for extended print
    new node AugAssign for augmented assignments
    new nodes ListComp, ListCompFor, and ListCompIf for list
        comprehensions

pyassem.py:
    add work around for string-Unicode comparison raising UnicodeError
        on comparison of two objects in code object's const table

pycodegen.py:
    define VERSION, the Python major version number
    get magic number using imp.get_magic() instead of hard coding
    implement list comprehensions, extended print, and augmented
        assignment; augmented assignment uses Delegator classes (see
        doc string)
    fix import and tuple unpacking for 1.5.2

transformer.py:
    various changes to support new 2.0 grammar and old 1.5 grammar
    add debug_tree helper than converts and symbol and token numbers
    to their names
2000-10-13 21:58:13 +00:00
Jeremy Hylton 4e1be72e6b Fix SF bug #116263: support for from .. import *
transformer.py: return '*', None from com_import_as_name
pycodegen.py: special case for name == '*'
pyassem.py: fix stack counting for IMPORT_ opcodes
2000-10-12 20:23:23 +00:00
Jeremy Hylton 92f3972090 patch by Neil Schemenauer to improve (fix?) line number generation 2000-09-01 20:47:37 +00:00
Jeremy Hylton 2051608616 Update magic number.
Fix import support to work with import as variant of Python 2.0.  The
grammar for import changed, requiring changes in transformer and code
generator, even to handle compilation of imports with as.
2000-09-01 20:33:26 +00:00
Thomas Wouters 46cc7c0f7b Bring Tools/compiler almost up to date. Specifically:
- fix tab space issues (SF patch #101167 by Neil Schemenauer)
- fix co_flags for classes to include CO_NEWLOCALS (SF patch #101145 by Neil)
- fix for merger of UNPACK_LIST and UNPACK_TUPLE into UNPACK_SEQUENCE,
  (SF patch #101168 by, well, Neil :)
- Adjust bytecode MAGIC to current bytecode.

TODO: teach compile.py about list comprehensions.
2000-08-12 20:32:46 +00:00
Jeremy Hylton 7daf04d9e0 replace most calls to emit 'SET_LINENO' will call to method set_lineno
based on bug report by Neil Schemenauer
2000-08-04 16:56:51 +00:00
Jeremy Hylton be317e615e patches from Mark Hammond
Attached is a set of diffs for the .py compiler that adds support
for the new extended call syntax.

compiler/ast.py:
CallFunc node gets 2 new children to support extended call syntax -
"star_args" (for "*args") and "dstar_args" (for "**args")

compiler/pyassem.py
It appear that self.lnotab is supposed to be responsible for
tracking line numbers, but self.firstlineno was still hanging
around.  Removed self.firstlineno completely.  NOTE - I didnt
actually test that the generated code has the correct line numbers!!

Stack depth tracking appeared a little broken - the checks never
made it beyond the "self.patterns" check - thus, the custom methods
were never called!  Fixed this.

(XXX Jeremy notes: I think this code is still broken because it
doesn't track stack effects across block bounaries.)

Added support for the new extended call syntax opcodes for depth
calculations.

compiler/pycodegen.py

Added support for the new extended call syntax opcodes.

compiler/transformer.py

Added support for the new extended call syntax.
2000-05-02 22:32:59 +00:00
Jeremy Hylton 36cc6a2197 complete rewrite
code generator uses flowgraph as intermediate representation.  the old
    rep uses a list with explicit "StackRefs" to indicate the target
    of jumps.

pyassem converts flowgraph to bytecode, breaks up individual steps of
    generating bytecode
2000-03-16 20:06:59 +00:00
Jeremy Hylton 7fab23e9cb rename compile.py to pycodegen.py
fix imports
remove parse functions and visitor code
track name change: Classdef to Class

add some comments and tweak order of visitXXX methods

get rid of if __name__ == "__main__ section
2000-03-06 19:10:54 +00:00
Jeremy Hylton 3ec7e2c4be the previous quick hack to fix def foo((x,y)) failed on some cases
(big surprise).  new solution is a little less hackish.

Code gen adds a TupleArg instance in the argument slot. The tuple arg
includes a copy of the names that it is responsble for binding.  The
PyAssembler uses this information to calculate the correct argcount.

all fix this wacky case: del (a, ((b,), c)), d
which is the same as: del a, b, c, d
(Can't wait for Guido to tell me why.)

solution uses findOp which walks a tree to find out whether it
contains OP_ASSIGN or OP_DELETE or ...
2000-02-17 22:09:35 +00:00
Jeremy Hylton 873bdc18e4 satisfy the tabnanny
fix broken references to filename var in generateXXX methods
2000-02-17 17:56:29 +00:00
Jeremy Hylton ad9a86fb1c support for arglists with implicit tuple unpacks
- added a number of support methods to generate code just before the
  body
- hack protocol for communicating number of args to PyAssembler

fix TryExcept generation for case where exception handler has no body
fix visitAssAttr
add comment about incomplete visitAssName

stop using the ExampleASTVisitor

change script invocation to accept a list of .py files (e.g. Lib/*.py)
2000-02-16 00:55:44 +00:00
Jeremy Hylton 4f6bcd80fc finish first impl of code generator
add support for nodes TryExcept, TryFinally, Sliceobj
fix visitSubscript to properly handle x[a,b,c]
2000-02-15 23:45:26 +00:00
Jeremy Hylton 9e1cd688b5 edit a doc string
(real intent is to test out rsync install)
2000-02-14 23:57:56 +00:00
Jeremy Hylton a46fb3841b get rid of spurious print 2000-02-14 21:54:57 +00:00
Jeremy Hylton 126960b744 looks like everything is working except for try/except (pystone
compiles correctly)
2000-02-14 21:33:10 +00:00
Jeremy Hylton a50581228e split compile.py into two files
add StackDepthFinder (and remove push/pop from CodeGen)
add several nodes, including Ellipsis, Bit&|^, Exec
2000-02-14 14:14:29 +00:00
Jeremy Hylton 3050d51571 change MODULE_NAMESPACE/FUNCTION_NAMESPACE stuff to have a single flag
named OPTIMIZED, which matches compile.c and makes more sense for
classes

revamp call signature for PythonVMCode.__init__; doesn't really matter
'cuz this code is going to be refactored out of existence

add generateClassCode and modify Func & Lambda generation

add support for nodes Classdef, Keyword,

fix CallFunc to generate right op arg when calling w/ keywords

add ugly hack to properly compute offsets when the same stack ref is
used multiple times
2000-02-12 00:12:38 +00:00
Jeremy Hylton 76d01b820c add support for Lambda nodes
change resolution of local name ops (LOAD_FAST).  i think it makes
sense now.  if it is an argument or a local var name that it used, it
must be in varnames.  if it is a local var name that is used, it must
also be in names
2000-02-11 20:27:07 +00:00
Jeremy Hylton 4eb504c2af add loop handling via
Loop object to handle StackRegs
loops stack attr on CodeGenreeator to hold the current loop object

add support for nodes While, Break, Continue
2000-02-10 20:55:50 +00:00
Jeremy Hylton 3e0910c10c add namespace attr to CodeGenerator, can be either MODULE_NAMESPACE or
FUNCTION_NAMESPACE.  initialize in __init__ and reset in
generateFunctionCode.

replace direct issue of STORE_FAST, STORE_GLOBAL, etc. with call to
storeName; same for loadName and deleteName

the new {store,load,delete}Name methods use the namespace attr and the
local variable stack to determine the correct bytecode to issue
2000-02-10 17:20:39 +00:00
Jeremy Hylton 5e0ce53e0e add ExampleASTVisitor:
* prints out examples of nodes that are handled by visitor.  simply a
  development convenience

remove NestedCodeGenerator -- it was bogus after all
replace with generateFunctionCode, a method to call to generate code
  for a function instead of a top-level module

fix impl of visitDiscard (most pop stack)
emit lineno for pass

handle the following new node types: Import, From, Getattr, Subscript,
Slice, AssAttr, AssTuple, Mod, Not, And, Or, List

LocalNameFinder: remove names declared as globals for locals

PythonVMCode: pass arg names to constructor, force varnames to contain
them all (even if they aren't referenced)

add -q option on command line to disable stdout
2000-02-10 00:47:08 +00:00
Jeremy Hylton 402456020b add optional verbose arg to walk function. it overrides the global
VERBOSE setting for the ASTVisitor

add getopt handling for one or more -v args

rename ForwardRef to StackRef, because it isn't necessarily directional

CodeGenerator:
* add assertStackEmpty method.  prints warning if stack is not empty
  when it should be
* define methods for AssName, UNARY_*, For

PythonVMCode:
* fix mix up between hasjrel and hasjabs for address calculation
2000-02-08 21:15:48 +00:00
Jeremy Hylton 53187f32eb now produces valid pyc files for a least a trivial subset of the
language.

CodeGenerator:
* modify to track stack depth
* add emit method that call's PythonVMCode's makeCodeObject
* thread filenames through in hackish way
* set flags for code objects for modules and functions
  XXX the docs for the flags seem out of date and/or incomplete

PythonVMCode:
* add doc string describing the elements of a real code object

LineAddrTable:
* creates an lnotab (no quite correctly though)
2000-02-08 19:01:29 +00:00
Jeremy Hylton 0fdffcf916 rapid evolution towards producing real .pyc files (even though I don't
handle most of the language syntax yet)

create NestedCodeGenerator used to generator the separate code object
that needs to be passed as an argument to MAKE_FUNCTION when a def
stmt is found (probably useful for class too)

change CodeGenerator.visitFunction to use the NestedCG

add CompiledModule class to handle creation of .pyc (pretty minimal
for now)

add makeCodeObject method to PythonVMCode that replaces symbolic names
with indexes into slots of the code code.  the design of this
class will probably need to be revised.
2000-02-04 19:37:35 +00:00
Jeremy Hylton 8b6323d3ef checking in initial weekend's work
compile.py: ASTVisitor framework plus bits of a code generator that
    should be bug-for-buf compatible with compile.c

misc.py: Set and Stack helpers

test.py: a bit of simple sample code that compile.py will work on
2000-02-04 00:28:21 +00:00