Commit Graph

74 Commits

Author SHA1 Message Date
Jeremy Hylton 4ebf3be407 a few small optimizations that seem to give a 5-10% speedup; the
further optimization of com_node makes the most difference.
2000-11-06 16:03:52 +00:00
Jeremy Hylton eefaeb78b3 move pruneNext method to correct object (doh!) 2000-11-06 03:47:39 +00:00
Jeremy Hylton 314e3fb215 Change the graph structure to contain the code generator object for
embedded code objects (e.g. functions) rather than the generated code
object.  This change means that the compiler generates code for
everything at the end, rather then generating code for each function
as it finds it.  Implementation note: _convert_LOAD_CONST in
pyassem.py must be change to call getCode().

Other changes follow.  Several changes creates extra edges between
basic blocks to reflect control flow for loops and exceptions.  These
missing edges had gone unnoticed because they do not affect the
current compilation process.

pyassem.py:
    Add _enable_debug() and _disable_debug() methods that print
    instructions and blocks to stdout as they are generated.

    Add edges between blocks for instructions like SETUP_LOOP,
    FOR_LOOP, etc.

    Add pruneNext to get rid of bogus edges remaining after
    unconditional transfer ops (e.g. JUMP_FORWARD)

    Change repr of Block to omit block length.

pycodegen.py:
    Make sure a new block is started after FOR_LOOP, etc.

    Change assert implementation to use RAISE_VARARGS 1 when there is
    no user-specified failure output.

misc.py:
    Implement __contains__ and copy for Set.
2000-11-06 03:43:11 +00:00
Jeremy Hylton a59ac0a7df If a function contains a doc string, remove the doc string node from
the function's body.

If assert is used without an error message, make the AST node None
rather than Name('None').
2000-11-06 03:33:52 +00:00
Jeremy Hylton 7cff7fe21f Many changes.
Reformatting -- long lines, "[ ]" -> "[]", a few indentation nits.

Replace calls to Node function (which constructed ast nodes) with
calls to actual constructors imported from ast module.

Optimize com_node (most frequently used method) for the common case --
the appropriate method is found in _dispatch.

Fix com_augassign to use class object's rather than node names
(rendered invalid by recent changes to ast)

Remove expensive tests for sequence-ness in com_stmt and
com_append_stmt. These tests should never fail; if they do, something
is really broken and exception will be raised elsewhere.

Fix com_stmt and com_append_stmt to use isinstance rather than
testing's type slot of ast node (this slot disappeared with recent
changes to ast).
2000-10-25 18:10:32 +00:00
Jeremy Hylton 628d289d12 Generated from rev 1.1 of ast.txt 2000-10-25 18:02:59 +00:00
Jeremy Hylton 66d2c1f7e5 Small optimizations in dispatch method: 1) lookup node's __class__ once
and store in local; 2) define _preorder to be dispatch (rather than
method that called dispatch).
2000-10-25 18:02:02 +00:00
Jeremy Hylton 821eee3321 Support for generation of ast.py from simple description of node
structure (ast.txt).  Usage is python astgen.py > ast.py.
2000-10-25 17:59:17 +00:00
Jeremy Hylton ac148b5fcd run the std regression test suite using bytecode produced by the compiler 2000-10-13 22:00:13 +00:00
Jeremy Hylton e7f710cc06 add -d option that dumps entire AST before compiling 2000-10-13 21:59:32 +00:00
Jeremy Hylton 9c048f9f65 Now supports entire Python 2.0 language and still supports Python
1.5.2.  The compiler generates code for the version of the interpreter
it is run under.

ast.py:
    Print and Printnl add dest attr for extended print
    new node AugAssign for augmented assignments
    new nodes ListComp, ListCompFor, and ListCompIf for list
        comprehensions

pyassem.py:
    add work around for string-Unicode comparison raising UnicodeError
        on comparison of two objects in code object's const table

pycodegen.py:
    define VERSION, the Python major version number
    get magic number using imp.get_magic() instead of hard coding
    implement list comprehensions, extended print, and augmented
        assignment; augmented assignment uses Delegator classes (see
        doc string)
    fix import and tuple unpacking for 1.5.2

transformer.py:
    various changes to support new 2.0 grammar and old 1.5 grammar
    add debug_tree helper than converts and symbol and token numbers
    to their names
2000-10-13 21:58:13 +00:00
Jeremy Hylton 4e1be72e6b Fix SF bug #116263: support for from .. import *
transformer.py: return '*', None from com_import_as_name
pycodegen.py: special case for name == '*'
pyassem.py: fix stack counting for IMPORT_ opcodes
2000-10-12 20:23:23 +00:00
Jeremy Hylton b769e80056 read in a .pyc file and disassemble the code objects 2000-10-09 14:35:24 +00:00
Jeremy Hylton 572bdce4b3 change 2-space indent to 4-space indent 2000-09-20 02:47:28 +00:00
Jeremy Hylton 9c36a41694 distutils setup script 2000-09-20 02:35:19 +00:00
Jeremy Hylton 92f3972090 patch by Neil Schemenauer to improve (fix?) line number generation 2000-09-01 20:47:37 +00:00
Jeremy Hylton 2051608616 Update magic number.
Fix import support to work with import as variant of Python 2.0.  The
grammar for import changed, requiring changes in transformer and code
generator, even to handle compilation of imports with as.
2000-09-01 20:33:26 +00:00
Thomas Wouters 46cc7c0f7b Bring Tools/compiler almost up to date. Specifically:
- fix tab space issues (SF patch #101167 by Neil Schemenauer)
- fix co_flags for classes to include CO_NEWLOCALS (SF patch #101145 by Neil)
- fix for merger of UNPACK_LIST and UNPACK_TUPLE into UNPACK_SEQUENCE,
  (SF patch #101168 by, well, Neil :)
- Adjust bytecode MAGIC to current bytecode.

TODO: teach compile.py about list comprehensions.
2000-08-12 20:32:46 +00:00
Jeremy Hylton 7daf04d9e0 replace most calls to emit 'SET_LINENO' will call to method set_lineno
based on bug report by Neil Schemenauer
2000-08-04 16:56:51 +00:00
Jeremy Hylton 8612f1c152 update my email address
fix com_call_function to cope with trailing comma in "f(a, b,)"
2000-08-04 16:54:54 +00:00
Greg Stein fd342bf453 add a bit more legal junk
(too lazy to paste in the whole BSD license tho; included by ref)
2000-08-03 17:39:13 +00:00
Jeremy Hylton ffe968b8fe This script demonstrates use of the visitor interface of the compiler
package.
2000-07-25 16:43:23 +00:00
Jeremy Hylton be317e615e patches from Mark Hammond
Attached is a set of diffs for the .py compiler that adds support
for the new extended call syntax.

compiler/ast.py:
CallFunc node gets 2 new children to support extended call syntax -
"star_args" (for "*args") and "dstar_args" (for "**args")

compiler/pyassem.py
It appear that self.lnotab is supposed to be responsible for
tracking line numbers, but self.firstlineno was still hanging
around.  Removed self.firstlineno completely.  NOTE - I didnt
actually test that the generated code has the correct line numbers!!

Stack depth tracking appeared a little broken - the checks never
made it beyond the "self.patterns" check - thus, the custom methods
were never called!  Fixed this.

(XXX Jeremy notes: I think this code is still broken because it
doesn't track stack effects across block bounaries.)

Added support for the new extended call syntax opcodes for depth
calculations.

compiler/pycodegen.py

Added support for the new extended call syntax opcodes.

compiler/transformer.py

Added support for the new extended call syntax.
2000-05-02 22:32:59 +00:00
Jeremy Hylton 0a4f1ff64e patches from Mark Hammond
compile.py:
On Windows, use 'nul' instead of '/dev/null'.

test.py:
Use double-quotes for the command-line, as Windows doesnt recognise
singles.
2000-05-02 22:29:46 +00:00
Jeremy Hylton 36cc6a2197 complete rewrite
code generator uses flowgraph as intermediate representation.  the old
    rep uses a list with explicit "StackRefs" to indicate the target
    of jumps.

pyassem converts flowgraph to bytecode, breaks up individual steps of
    generating bytecode
2000-03-16 20:06:59 +00:00
Jeremy Hylton f635abee3a simplify visitor walker class
- remove postorder
- remove protocol for automatically walking children based on visitor
  method return value; now only walks if there is no method
2000-03-16 20:04:16 +00:00
Jeremy Hylton b631b8ede5 fix list.append problems 2000-03-16 20:03:04 +00:00
Jeremy Hylton e4d6293383 change name of Set method: items -> elements (avoids confusion with
dict)
2000-03-16 20:02:38 +00:00
Jeremy Hylton 0c3208aa62 compiler command-line interface moved here from compiler.pycodegen 2000-03-06 19:13:21 +00:00
Jeremy Hylton f728f9a13e import compile function form pycodegen 2000-03-06 19:12:33 +00:00
Jeremy Hylton 7fab23e9cb rename compile.py to pycodegen.py
fix imports
remove parse functions and visitor code
track name change: Classdef to Class

add some comments and tweak order of visitXXX methods

get rid of if __name__ == "__main__ section
2000-03-06 19:10:54 +00:00
Jeremy Hylton 8c78341f45 add a doc string
import some useful functions from contained modules
2000-03-06 19:04:14 +00:00
Jeremy Hylton 9812e7bc6a fix import to refer to compiler package 2000-03-06 18:54:30 +00:00
Jeremy Hylton abd7ebf70b revise arguments for addCode method on lnotab. take several numbers
that are internally converted to chars, rather than taking a string.
2000-03-06 18:53:14 +00:00
Jeremy Hylton fa974a9d06 change node Classdef to Class
add doc string to transformer module
add two helper functions:
    parse(buf) -> AST
    parseFile(path) -> AST
2000-03-06 18:50:48 +00:00
Jeremy Hylton ed9586174d factor out the tree walking/visitor code that was in compile.py 2000-03-06 18:49:31 +00:00
Jeremy Hylton 772dd417f7 satisfy the tabnanny (thanks to MH for noticing the problem) 2000-02-21 22:46:00 +00:00
Jeremy Hylton efd0694a2d changes to _lookupName
- removed now (happily) unused second arg
- need to verify results of [].index are correct; for building consts,
  need to have same value and same type, e.g. 2 not the same as 2L
2000-02-17 22:58:54 +00:00
Jeremy Hylton 3ec7e2c4be the previous quick hack to fix def foo((x,y)) failed on some cases
(big surprise).  new solution is a little less hackish.

Code gen adds a TupleArg instance in the argument slot. The tuple arg
includes a copy of the names that it is responsble for binding.  The
PyAssembler uses this information to calculate the correct argcount.

all fix this wacky case: del (a, ((b,), c)), d
which is the same as: del a, b, c, d
(Can't wait for Guido to tell me why.)

solution uses findOp which walks a tree to find out whether it
contains OP_ASSIGN or OP_DELETE or ...
2000-02-17 22:09:35 +00:00
Jeremy Hylton 7708d697ee add varargs and kwargs flags to Lambda nodes 2000-02-17 22:06:20 +00:00
Jeremy Hylton 873bdc18e4 satisfy the tabnanny
fix broken references to filename var in generateXXX methods
2000-02-17 17:56:29 +00:00
Jeremy Hylton ad9a86fb1c support for arglists with implicit tuple unpacks
- added a number of support methods to generate code just before the
  body
- hack protocol for communicating number of args to PyAssembler

fix TryExcept generation for case where exception handler has no body
fix visitAssAttr
add comment about incomplete visitAssName

stop using the ExampleASTVisitor

change script invocation to accept a list of .py files (e.g. Lib/*.py)
2000-02-16 00:55:44 +00:00
Jeremy Hylton 3d9f5e4de2 more robust assignment of lineno for keyword args
get the lineno from the name of the keyword arg

example of case that didn't work--
def foo(x, y, a = None,
	b = None):
2000-02-16 00:51:37 +00:00
Jeremy Hylton 2ce27b223b fix argcount generation for arg lists containing tuple unpacks
this is sort of a hack
2000-02-16 00:50:29 +00:00
Jeremy Hylton 65d4ea05d2 add flatten helper function 2000-02-16 00:49:47 +00:00
Jeremy Hylton 4f6bcd80fc finish first impl of code generator
add support for nodes TryExcept, TryFinally, Sliceobj
fix visitSubscript to properly handle x[a,b,c]
2000-02-15 23:45:26 +00:00
Jeremy Hylton 1ebba96871 fix creation of Ellipsis node 2000-02-15 23:43:19 +00:00
Jeremy Hylton 410e840c85 add a little debugging support when new.code raises SystemError 2000-02-15 21:59:50 +00:00
Jeremy Hylton d603dee4a3 add line numbers to nodes in the except clause (when possible) 2000-02-15 21:30:48 +00:00
Jeremy Hylton ca1ad182f2 tidy up tryexcept and tryfinally nodes 2000-02-15 21:30:00 +00:00