Commit Graph

41 Commits

Author SHA1 Message Date
Jeffrey Yasskin 68d6852006 Backport r69961 to trunk, replacing JUMP_IF_{TRUE,FALSE} with
POP_JUMP_IF_{TRUE,FALSE} and JUMP_IF_{TRUE,FALSE}_OR_POP. This avoids executing
a POP_TOP on each conditional and sometimes allows the peephole optimizer to
skip a JUMP_ABSOLUTE entirely. It speeds up list comprehensions significantly.
2009-02-28 19:03:21 +00:00
Neil Schemenauer 4db626f95d Overhaul Lib/compiler block ordering. The previous code was filled with
hacks.  The new code is based on issue #2472 posted by Antoine Pitrou.  I
did some further cleanups of the pyassem code and optimized the block
ordering pass.
2009-02-06 21:08:52 +00:00
Georg Brandl 2d2fe572a4 #4578: fix has_key() usage in compiler package. 2008-12-15 08:58:59 +00:00
Christian Heimes c756d00cf2 Replaced import of the 'new' module with 'types' module and added a deprecation warning to the 'new' module. 2007-11-27 21:34:01 +00:00
Guido van Rossum f669436189 Um, I thought I'd already checked this in.
Anyway, this is the changes to the with-statement
so that __exit__ must return a true value in order
for a pending exception to be ignored.
The PEP (343) is already updated.
2006-03-10 02:28:35 +00:00
Neal Norwitz 10be2ea85d SF bug 1442442: LIST_APPEND optimization got lost in the AST merge.
Add it back.
2006-03-03 20:29:11 +00:00
Thomas Wouters fa0cf4f3ae Add support for absolute/relative imports and if/else expressions:
- regenerate ast.py
 - add future flags for absolute-import and with-statement so they
   (hopefully) properly get set in code-object flags
 - try out if/else expressions in actual code for the hell of it.

Seems to generate the same kind of bytecode as the normal compiler.
2006-03-03 18:16:20 +00:00
Guido van Rossum baf3eb2410 Forgot to explain the effect of the new opcode.
Sigh -- we really should have one place for all opcode metadata.
2006-02-28 00:40:16 +00:00
Neal Norwitz d752f7d8e8 No need for types, use isinstance 2005-11-25 03:17:59 +00:00
Jeremy Hylton 3e0055f8c6 Merge ast-branch to head
This change implements a new bytecode compiler, based on a
transformation of the parse tree to an abstract syntax defined in
Parser/Python.asdl.

The compiler implementation is not complete, but it is in stable
enough shape to run the entire test suite excepting two disabled
tests.
2005-10-20 19:59:25 +00:00
Jeremy Hylton accb62b28e SF patch [ 597919 ] compiler package and SET_LINENO
A variety of changes from Michael Hudson to get the compiler working
with 2.3.  The primary change is the handling of SET_LINENO:

# The set_lineno() function and the explicit emit() calls for
# SET_LINENO below are only used to generate the line number table.
# As of Python 2.3, the interpreter does not have a SET_LINENO
# instruction.  pyassem treats SET_LINENO opcodes as a special case.

A few other small changes:
 - Remove unused code from pycodegen and pyassem.
 - Fix error handling in parsermodule.  When PyParser_SimplerParseString()
   fails, it sets an exception with detailed info.  The parsermodule
   was clobbering that exception and replacing it was a generic
   "could not parse string" exception.  Keep the original exception.
2002-12-31 18:17:44 +00:00
Neal Norwitz a312c3ade7 Remove uses of string module and stat.ST_MODE 2002-06-06 18:30:10 +00:00
Tim Peters e0c446bb4a Whitespace normalization. 2001-10-18 21:57:37 +00:00
Jeremy Hylton 138d90eb73 Vastly improved stacksize calculation.
There are now no known cases where the compiler package computes a
stack depth lower than the one computed by the builtin compiler.  (To
achieve this state, we had to fix bugs in both compilers :-).

The chief change is to do the depth calculations with respect to basic
blocks.  The stack effect of block is calculated.  Then the flow graph
is traversed using breadth-first search to find the max weight path
through the graph.

Had to fix the StackDepthTracker to calculate the right info for
several opcodes: LOAD_ATTR, CALL_FUNCTION (and friends), MAKE_CLOSURE,
and DUP_TOPX.

XXX Still need to handle free variables in MAKE_CLOSURE.

XXX There are still a lot of places where the computed stack depth is
larger than for the builtin compiler.  These won't cause the
interpreter to overflow the frame, but they waste space.
2001-10-17 13:37:29 +00:00
Jeremy Hylton 1e99a77120 Various sundry changes for 2.2 compatibility
Remove the option to have nested scopes or old LGB scopes.  This has a
large impact on the code base, by removing the need for two variants
of each CodeGenerator.

Add a get_module() method to CodeGenerator objects, used to get the
future features for the current module.

Set CO_GENERATOR, CO_GENERATOR_ALLOWED, and CO_FUTURE_DIVISION flags
as appropriate.

Attempt to fix the value of nlocals in newCodeObject(), assuming that
nlocals is 0 if CO_NEWLOCALS is not defined.
2001-09-14 22:49:08 +00:00
Jeremy Hylton 71ebc3359b Fix _convert_NAME() so that it doesn't store locals for class bodies.
Fix list comp code generation -- emit GET_ITER instead of Const(0)
after the list.

Add CO_GENERATOR flag to generators.

Get CO_xxx flags from the new module
2001-08-30 20:25:55 +00:00
Jeremy Hylton 9263848fa1 Improve stack depth computation for try/except and try/finally
Add CONTINUE_LOOP to the list of unconditional transfers
2001-08-29 22:27:14 +00:00
Jeremy Hylton 4ba9001f5c Fix off-by-one errors in code to find depth of stack.
XXX The code is still widely inaccurate, but most (all?) of the time
it's an overestimate.
2001-08-29 20:55:17 +00:00
Jeremy Hylton bf77c465bd Undo change from list to dict for handling varnames, consts, etc.
As the doc string for _lookupName() explains:

    This routine uses a list instead of a dictionary, because a
    dictionary can't store two different keys if the keys have the
    same value but different types, e.g. 2 and 2L.  The compiler
    must treat these two separately, so it does an explicit type
    comparison before comparing the values.
2001-08-29 19:45:33 +00:00
Jeremy Hylton e4e9cd4c01 Modify name conversion to be (hopefully) a bit more efficient.
Use a dictionary instead of a list to map objects to their offsets in
a const/name tuple of a code object.

XXX The conversion is perhaps incomplete, in that we shouldn't have to
do the list2dict to start.
2001-08-29 18:09:50 +00:00
Jeremy Hylton 63db7b9ca1 XXX_NAME ops should affect varnames
varnames should list all the local variables (with arguments first).
The XXX_NAME ops typically occur at the module level and assignment
ops should create locals.
2001-08-28 16:36:12 +00:00
Tim Peters 2a7f384122 SF bug 430991: wrong co_lnotab
Armin Rigo pointed out that the way the line-# table got built didn't work
for lines generating more than 255 bytes of bytecode.  Fixed as he
suggested, plus corresponding changes to pyassem.py, plus added some
long overdue docs about this subtle table to compile.c.

Bugfix candidate (line numbers may be off in tracebacks under -O).
2001-06-09 09:26:21 +00:00
Jeremy Hylton 542b11acfd pyassem.py:
Fix annoying bugs in flow graph layout code.  In some cases the
    implicit control transfers weren't honored.  In other cases,
    JUMP_FORWARD instructions jumped backwards.

    Remove unused arg from nextBlock().

pycodegen.py

    Add optional force kwarg to set_lineno() that will emit a
    SET_LINENO even if it is the same as the previous lineno.

    Use explicit LOAD_FAST and STORE_FAST to access list comp implicit
    variables.  (The symbol table doesn't know about them.)
2001-04-12 20:21:39 +00:00
Jeremy Hylton bfb0cf822b Revise handling of tuple arguments so that the variables names match
those used by compile.c.  (test_grammar now depends on the names)
2001-04-12 17:33:34 +00:00
Jeremy Hylton 364f9b9e2f Preliminary support for nested scopes
XXX Still doesn't work right for classes
XXX Still doesn't do sufficient error checking
2001-04-12 06:40:42 +00:00
Jeremy Hylton 5af105eec9 Make sure the docstring is always entered as the first element in the
consts, even if it is None.

Simplify _lookupName() by removing lots of redundant tests.
2001-04-11 16:21:51 +00:00
Jeremy Hylton eefaeb78b3 move pruneNext method to correct object (doh!) 2000-11-06 03:47:39 +00:00
Jeremy Hylton 314e3fb215 Change the graph structure to contain the code generator object for
embedded code objects (e.g. functions) rather than the generated code
object.  This change means that the compiler generates code for
everything at the end, rather then generating code for each function
as it finds it.  Implementation note: _convert_LOAD_CONST in
pyassem.py must be change to call getCode().

Other changes follow.  Several changes creates extra edges between
basic blocks to reflect control flow for loops and exceptions.  These
missing edges had gone unnoticed because they do not affect the
current compilation process.

pyassem.py:
    Add _enable_debug() and _disable_debug() methods that print
    instructions and blocks to stdout as they are generated.

    Add edges between blocks for instructions like SETUP_LOOP,
    FOR_LOOP, etc.

    Add pruneNext to get rid of bogus edges remaining after
    unconditional transfer ops (e.g. JUMP_FORWARD)

    Change repr of Block to omit block length.

pycodegen.py:
    Make sure a new block is started after FOR_LOOP, etc.

    Change assert implementation to use RAISE_VARARGS 1 when there is
    no user-specified failure output.

misc.py:
    Implement __contains__ and copy for Set.
2000-11-06 03:43:11 +00:00
Jeremy Hylton 9c048f9f65 Now supports entire Python 2.0 language and still supports Python
1.5.2.  The compiler generates code for the version of the interpreter
it is run under.

ast.py:
    Print and Printnl add dest attr for extended print
    new node AugAssign for augmented assignments
    new nodes ListComp, ListCompFor, and ListCompIf for list
        comprehensions

pyassem.py:
    add work around for string-Unicode comparison raising UnicodeError
        on comparison of two objects in code object's const table

pycodegen.py:
    define VERSION, the Python major version number
    get magic number using imp.get_magic() instead of hard coding
    implement list comprehensions, extended print, and augmented
        assignment; augmented assignment uses Delegator classes (see
        doc string)
    fix import and tuple unpacking for 1.5.2

transformer.py:
    various changes to support new 2.0 grammar and old 1.5 grammar
    add debug_tree helper than converts and symbol and token numbers
    to their names
2000-10-13 21:58:13 +00:00
Jeremy Hylton 4e1be72e6b Fix SF bug #116263: support for from .. import *
transformer.py: return '*', None from com_import_as_name
pycodegen.py: special case for name == '*'
pyassem.py: fix stack counting for IMPORT_ opcodes
2000-10-12 20:23:23 +00:00
Jeremy Hylton 92f3972090 patch by Neil Schemenauer to improve (fix?) line number generation 2000-09-01 20:47:37 +00:00
Thomas Wouters 46cc7c0f7b Bring Tools/compiler almost up to date. Specifically:
- fix tab space issues (SF patch #101167 by Neil Schemenauer)
- fix co_flags for classes to include CO_NEWLOCALS (SF patch #101145 by Neil)
- fix for merger of UNPACK_LIST and UNPACK_TUPLE into UNPACK_SEQUENCE,
  (SF patch #101168 by, well, Neil :)
- Adjust bytecode MAGIC to current bytecode.

TODO: teach compile.py about list comprehensions.
2000-08-12 20:32:46 +00:00
Jeremy Hylton be317e615e patches from Mark Hammond
Attached is a set of diffs for the .py compiler that adds support
for the new extended call syntax.

compiler/ast.py:
CallFunc node gets 2 new children to support extended call syntax -
"star_args" (for "*args") and "dstar_args" (for "**args")

compiler/pyassem.py
It appear that self.lnotab is supposed to be responsible for
tracking line numbers, but self.firstlineno was still hanging
around.  Removed self.firstlineno completely.  NOTE - I didnt
actually test that the generated code has the correct line numbers!!

Stack depth tracking appeared a little broken - the checks never
made it beyond the "self.patterns" check - thus, the custom methods
were never called!  Fixed this.

(XXX Jeremy notes: I think this code is still broken because it
doesn't track stack effects across block bounaries.)

Added support for the new extended call syntax opcodes for depth
calculations.

compiler/pycodegen.py

Added support for the new extended call syntax opcodes.

compiler/transformer.py

Added support for the new extended call syntax.
2000-05-02 22:32:59 +00:00
Jeremy Hylton 36cc6a2197 complete rewrite
code generator uses flowgraph as intermediate representation.  the old
    rep uses a list with explicit "StackRefs" to indicate the target
    of jumps.

pyassem converts flowgraph to bytecode, breaks up individual steps of
    generating bytecode
2000-03-16 20:06:59 +00:00
Jeremy Hylton abd7ebf70b revise arguments for addCode method on lnotab. take several numbers
that are internally converted to chars, rather than taking a string.
2000-03-06 18:53:14 +00:00
Jeremy Hylton 772dd417f7 satisfy the tabnanny (thanks to MH for noticing the problem) 2000-02-21 22:46:00 +00:00
Jeremy Hylton efd0694a2d changes to _lookupName
- removed now (happily) unused second arg
- need to verify results of [].index are correct; for building consts,
  need to have same value and same type, e.g. 2 not the same as 2L
2000-02-17 22:58:54 +00:00
Jeremy Hylton 3ec7e2c4be the previous quick hack to fix def foo((x,y)) failed on some cases
(big surprise).  new solution is a little less hackish.

Code gen adds a TupleArg instance in the argument slot. The tuple arg
includes a copy of the names that it is responsble for binding.  The
PyAssembler uses this information to calculate the correct argcount.

all fix this wacky case: del (a, ((b,), c)), d
which is the same as: del a, b, c, d
(Can't wait for Guido to tell me why.)

solution uses findOp which walks a tree to find out whether it
contains OP_ASSIGN or OP_DELETE or ...
2000-02-17 22:09:35 +00:00
Jeremy Hylton 2ce27b223b fix argcount generation for arg lists containing tuple unpacks
this is sort of a hack
2000-02-16 00:50:29 +00:00
Jeremy Hylton 410e840c85 add a little debugging support when new.code raises SystemError 2000-02-15 21:59:50 +00:00
Jeremy Hylton a50581228e split compile.py into two files
add StackDepthFinder (and remove push/pop from CodeGen)
add several nodes, including Ellipsis, Bit&|^, Exec
2000-02-14 14:14:29 +00:00