(With slight cosmetic improvements to shorten lines and a grammar fix
to a docstring.)
This addes -X and -E options to freeze. From the docstring:
-X module Like -x, except the module can never be imported by
the frozen binary.
-E: Freeze will fail if any modules can't be found (that
were not excluded using -x or -X).
The strerror attribute contained only partial information about the
exception and produced some very confusing error messages. By passing
err (the exception object itself) and letting it convert itself to a
string, the error messages are better.
compile() becomes replacement for builtin compile()
compileFile() generates a .pyc from a .py
both are exported in __init__
compiler.parse() gets optional second argument to specify compilation
mode, e.g. single, eval, exec
Add AbstractCompileMode as parent class and Module, Expression, and
Interactive as concrete subclasses. Each corresponds to a compilation
mode.
THe AbstractCompileMode instances in turn delegate to CodeGeneration
subclasses specialized for their particular functions --
ModuleCodeGenerator, ExpressionCodeGeneration,
InteractiveCodeGenerator.
The argument properties are ordered from easiest to hardest. The
harder the arg, the more complicated that code that must be generated
to return it from getChildren() and/or getChildNodes(). The old
calculation routine was bogus, because it always set hardest_arg to
the hardness of the last argument. Now use max() to always set it to
the hardness of the hardest argument.
Remove the only test in the syntax module. It ends up that the
transformer must handle this error case.
In the transformer, check for a list compression in com_assign_list()
by looking for a list_for node where a comma is expected.
In pycodegen.compile() re-raise the SyntaxError rather than catching
it and exiting
Invoke compiler.syntax.check() after building AST. If a SyntaxError
occurs, print the error and exit without generating a .pyc file.
Refactor code to use compiler.misc.set_filename() rather than passing
filename argument around to each CodeGenerator instance.
introspection incompatibility, but in fact it's just that calltips
always gave up on a docstring that started with a newline (but
didn't realize they were giving up <wink>).
Remove the option to have nested scopes or old LGB scopes. This has a
large impact on the code base, by removing the need for two variants
of each CodeGenerator.
Add a get_module() method to CodeGenerator objects, used to get the
future features for the current module.
Set CO_GENERATOR, CO_GENERATOR_ALLOWED, and CO_FUTURE_DIVISION flags
as appropriate.
Attempt to fix the value of nlocals in newCodeObject(), assuming that
nlocals is 0 if CO_NEWLOCALS is not defined.
operators per line or statement are now on by default, and -m turns
these warnings off.
- Change the way multiple / operators are reported; a regular
recommendation is always emitted after the warning.
- Report ambiguous warnings (both int|long and float|complex used for
the same operator).
- Update the doc string again to clarify all this and describe the
possible messages more precisely.
percolated out, and some general cleanup. The output is still the
same, except it now prints "Index: <file>" instead of "Processing:
<file>", so that the output can be used as input for patch (but only
the diff-style parts of it).
Fix list comp code generation -- emit GET_ITER instead of Const(0)
after the list.
Add CO_GENERATOR flag to generators.
Get CO_xxx flags from the new module
try/except or try/finally.
Previous versions had only track SETUP_LOOP blocks and ignored the
exception part. This meant that it allowed continue inside a
try/except but generated buggy code. Now it does the right thing.
As the doc string for _lookupName() explains:
This routine uses a list instead of a dictionary, because a
dictionary can't store two different keys if the keys have the
same value but different types, e.g. 2 and 2L. The compiler
must treat these two separately, so it does an explicit type
comparison before comparing the values.
Avoid if/elif/elif/else tests where the final else is supposed to
handle exactly one case instead of all other cases. When the list of
operators is extended, the catchall else treats all new operators as
the last operator in the set of tests. Instead, raise an exception if
an unexpected operator occurs.
Use a dictionary instead of a list to map objects to their offsets in
a const/name tuple of a code object.
XXX The conversion is perhaps incomplete, in that we shouldn't have to
do the list2dict to start.
Add support for floor division (// and //=)
The implementation of getChildren() and getChildNodes() is intended to
be faster, because it avoids calling flatten() on every return value.
But it's not clear that it is a lot faster, because constructing a
tuple with just the right values ends up being slow. (Too many
attribute lookups probably.)
The ast.txt file is much more complicated, with funny characters at
the ends of names (*, &, !) to indicate the types of each child node.
The astgen script is also much more complex, making me wonder if it's
still useful.
varnames should list all the local variables (with arguments first).
The XXX_NAME ops typically occur at the module level and assignment
ops should create locals.
(Hard to believe these were never handled before)
Add misc.mangle() that mangles based on the rules in compile.c.
XXX Need to test the corner cases
Update CodeGenerator with a class_name attribute bound to None. If a
particular instance is created within a class scope, the instance's
class_name is bound to that class's name.
Add mangle() method to CodeGenerator that mangles if the class_name
has a class_name in it.
Modify the FunctionCodeGenerator family to handle an extra argument--
the class_name.
Wrap all name ops and attrnames in calls to self.mangle()
Make nested scopes enabled by default
Add is_constant_false() helper so that compiled code and symbols are
consistent with builtin compiler's handling of "if 0:"
Fix doc string handling to be consistent with recent change that
eliminates the doc string from the Module's node attribute.
Add fix to print handling from Evan & Shane.
Track change to visitor api by making "verbose" explicit.
Comment out setting CO_NESTED flag (it's unnecessary in 2.2).
Evan Simpson's fix. And his explanation:
If you defined two nested functions in a row that refer to the
same non-global variable, the second one will be generated as
though the variable were global.
The use of com_node() introduces a lot of extra stack frames, enough
to cause a stack overflow compiling test.test_parser with the standard
interpreter recursionlimit. The com_node() is a convenience function
that hides the dispatch details, but comes at a very high cost. It is
more efficient to dispatch directly in the callers. In these cases,
use lookup_node() and call the dispatched node directly.
Also handle yield_stmt in a way that will work with Python 2.1
(suggested by Shane Hathaway)
Remove _preorder as alias for dispatch and call dispatch directly.
Add an extra optional argument to walk()
XXX Also comment out some code that does debugging prints.
The tests are run from a copy of the library directory, where
everything has been compiled by the compiler package.
Add a raw_input() call at the end of the script, so that I can check
the output before the temp directory with the compiled code is
removed.
- make the selftests work again (they were apparently not used since
very early in bgen's development), with some minor cleanup by me
- make emacs python mode happier
ChipWidget.__init__(): Added a message area just below the color
name. Both the message and name widgets are now FLAT, DISABLED
Entry widgets instead of Labels. This allows users to
copy-n-paste the color names or color specs. Also, the contents
of both widgets are now driven by StringVars.
set_color(): This only sets the chip color; it does not set the name
widgets.
set_name(): New method which only sets the name widget contents.
set_message(): New method which only sets the message widget contents.
ChipViewer.update_yourself(): Set the color, name, and message for
each chip as follows: the first line always contains the color
spec in #rrggbb format. The second line will contain the color
name, but slightly differently for each widget. For the Selected
widget, if the color exactly matches the Nearest color, the name
is shown, otherwise the message field will be empty. The name
field of the Nearest widget will always contain the color name.
Change several sections to subsections (part of the manual -> howto
transformation).
Flesh out discussion of assignment nodes (and delete statements).
Add an example of manipulating AST objects at a >>> prompt
Remove the module index; there aren't enough modules documented yet
for this to make sense.
Add a couple more index entries, fixed a few typos, and adjusted a few
more things for consistency.
Cater to that.
+ Major speed boost via not reading more of files than necessary. This
was no slouch before; now it screams.
+ Improve msg when giving up on a goofy future statement.
If multiple header files are processed simultaneously which include each
other, the corresponding modules mport each other. Specifically, if h2py
is invoked with sys/types.h first, later header files won't contain the
complete contents of TYPES.py.
list of files to not extract docstrings from when the -D option is
given. This isn't optimal, but I didn't want to change the semantics
of -D, and it's bad form to allow optional switch arguments.
Bumping __version__ to 1.4.
TokenEater.__init__(): Initialize __curfile to None.
__waiting(): In order to extract docstrings from the module, both the
-D flag should be set, and the __curfile should not be named in
the -X filename (i.e. it isn't in opts.nodocstrings).
set_filename(): Fixed a bug where once the first module docstring is
extracted, no subsequent module docstrings will be extracted. The
bug was that the first extraction set __freshmodule to 0, but that
flag was never reset back to 1. set_filename() is always called
when the next file is being processed, so use it to reset the
__freshmodule flag.
main(): Add support for -X/--no-docstring.
fragile. Now the leading "0x" on hex numbers are displayed as labels
and the type-in entry fields just accept the hex digits. Be sure to
strip off the "0x" string when displaying hex values too.
Also, de-string-module-ification, and other Python 2.x improvements.
found a bug here. Here's the deal:
Class PyShell derives from class OutputWindow. Method PyShell.close()
wants to invoke its parent method, but because PyShell long ago was
inherited from class PyShellEditorWindow, it invokes
PyShelEditorWindow.close(self). Now, class PyShellEditorWindow itself
derives from class OutputWindow, and inherits the close() method from
there without overriding it. Under the old rules,
PyShellEditorWindow.close would return an unbound method restricted to
the class that defined the implementation of close(), which was
OutputWindow.close. Under the new rules, the unbound method is
restricted to the class whose method was requested, that is
PyShellEditorWindow, and this was correctly trapped as an error.
This allows system libs to be weak-linked, thereby allowing us to generate functions that are only available on some OS versions without getting a NULL dereference if the function isn't available.
I published it on the web as http://www.python.org/2.1/md5sum.py
so I thought I might as well check it in.
Works with Python 1.5.2 and later.
Works like the Linux tool ``mdfsum file ...'' except it doesn't take
any options or read stdin.
Armin Rigo pointed out that the way the line-# table got built didn't work
for lines generating more than 255 bytes of bytecode. Fixed as he
suggested, plus corresponding changes to pyassem.py, plus added some
long overdue docs about this subtle table to compile.c.
Bugfix candidate (line numbers may be off in tracebacks under -O).
indicating whether the entry was extracted from a docstring or not.
write(): If any of the locations of a string appearance came from a
docstring, add a comment such as
#. docstring
before the references (after a suggestion by Martin von Loewis).
codec files to codecs.py and added logic so that multi mappings
in the decoding maps now result in mappings to None (undefined mapping)
in the encoding maps.
rather than the idle.py script. This has advantages and
disadvantages; the biggest advantage being that we can more easily
have an alternative main program.
Assertion error message had typos in arguments to string format.
.cover files for modules in packages are now put in the right place.
The code that generate .cover files seemed to prepend a "./" to many
absolute paths, causing them to fail. The code now checks explicitly
for absolute paths and leaves them alone.
In trace/coverage code, recover from case where module has no __name__
attribute, when e.g. it is executed by PyRun_String(). In this case,
assign modulename to None and hope for the best. There isn't anywhere
to write out coverage data for this code anyway.
Also, replace several sys.stderr.writes with print >> sys.stderr.
New features:
-C/--coverdir dir: Generate .cover files in specified directory
instead of in the directory where the .py file is.
-s: Print a short summary of files coverred (# lines, % coverage,
name)
variables reported by PyChecker.
__togglegentype(): PyChecker accurately reported that the variable
__gentypevar was unused -- actually this whole method is currently
unused so comment it out.
Always emit a SET_LINENO 0 at the beginning of the module. The
builtin compiler does this, and it's much easier to compare bytecode
generated by the two compilers if they both do.
Move the SET_LINENO inside the FOR_LOOP block for list
comprehensions. Also for compat. with builtin compiler.
Fix annoying bugs in flow graph layout code. In some cases the
implicit control transfers weren't honored. In other cases,
JUMP_FORWARD instructions jumped backwards.
Remove unused arg from nextBlock().
pycodegen.py
Add optional force kwarg to set_lineno() that will emit a
SET_LINENO even if it is the same as the previous lineno.
Use explicit LOAD_FAST and STORE_FAST to access list comp implicit
variables. (The symbol table doesn't know about them.)
(Yes, this is a new feature right before the 2.1 release. No, I can't
imagine this would seriously break anybody's code. In fact, most
users of this script are probably *happy* to see this addition.)
Add mangling support
Add get_children() and add_child() methods to Scope
Skip nodes when If test is a false constant
Add test code that checks results against symtable module
Fix com_NEWLINE() so that is accepts arguments, which occurs for lines like:
stmt; # note trailing semicolon
Add XXX about checking for assignment to list comps
<christopher.mccafferty@csg.ch>:
Add javascript: and telnet: to the types of URLs we ignore.
Add support for several additional URL-valued attributes on the BODY,
FRAME, IFRAME, LINK, OBJECT, and SCRIPT elements.
Was getting mystery
ModifiedInterpreter instance has no attribute 'save_warnings_filters'
errors at odd times (for sure in a fresh IDLE, fresh file, then Run Script).
This patch was developed primarily to reduce the size of the
frozen binary. It is particularly useful when freezing for 'small'
platforms, such as Palm OS, where you really want to save that
last miserable byte.
A limitation of this patch is that it does not provide any feedback
about the replacements being made. As the path matching
is case-sensitive this may lead to unexpected behaviour for DOS
and Windows people, eg
> freeze.py -r C:\Python\Lib\=py\ goats.py
should probably be:
> freeze.py -r c:\python\lib\=py\ goats.py
this just copies the __name__=='__main__' logic from pydoc.py.
?!ng can decide whether he wants to create a main() in pydoc, or rip
it out of pydoc.py completely.
Guido told me to do this <wink>.
Greatly expanded docstrings, and fleshed out with examples.
New std test.
Added new get_close_matches() function for ESR.
Needs docs, but LaTeXification of the module docstring is all it needs.
\CVS: ----------------------------------------------------------------------
fields. You can now backspace out the 0 in 0x0, and you can clear the
field when in decimal mode. There are still some oddities about
typing into these fields, but it should be much less annoying. The
real solution is to ditch the update-while-typing "feature".
\t\t\t\t\treal code
##\t\t\t\t\tunused code
\t\t\t\t\treal code
via untabifying and shifting the real code left. Semantically the
same but made the intent of the commented-out-in-column-0 unused code
unclear. The exact same unused code appears to have gotten copied from
file to file over the years.
codec to not apply Latin-1 mappings for keys which are not found
in the mapping dictionaries, but instead treat them as undefined
mappings.
The patch was originally written by Martin v. Loewis with some
additional (cosmetic) changes and an updated test script
by Marc-Andre Lemburg.
The standard codecs were recreated from the most current files
available at the Unicode.org site using the Tools/scripts/gencodec.py
tool.
This patch closes the bugs #116285 and #119960.
embedded code objects (e.g. functions) rather than the generated code
object. This change means that the compiler generates code for
everything at the end, rather then generating code for each function
as it finds it. Implementation note: _convert_LOAD_CONST in
pyassem.py must be change to call getCode().
Other changes follow. Several changes creates extra edges between
basic blocks to reflect control flow for loops and exceptions. These
missing edges had gone unnoticed because they do not affect the
current compilation process.
pyassem.py:
Add _enable_debug() and _disable_debug() methods that print
instructions and blocks to stdout as they are generated.
Add edges between blocks for instructions like SETUP_LOOP,
FOR_LOOP, etc.
Add pruneNext to get rid of bogus edges remaining after
unconditional transfer ops (e.g. JUMP_FORWARD)
Change repr of Block to omit block length.
pycodegen.py:
Make sure a new block is started after FOR_LOOP, etc.
Change assert implementation to use RAISE_VARARGS 1 when there is
no user-specified failure output.
misc.py:
Implement __contains__ and copy for Set.
Reformatting -- long lines, "[ ]" -> "[]", a few indentation nits.
Replace calls to Node function (which constructed ast nodes) with
calls to actual constructors imported from ast module.
Optimize com_node (most frequently used method) for the common case --
the appropriate method is found in _dispatch.
Fix com_augassign to use class object's rather than node names
(rendered invalid by recent changes to ast)
Remove expensive tests for sequence-ness in com_stmt and
com_append_stmt. These tests should never fail; if they do, something
is really broken and exception will be raised elsewhere.
Fix com_stmt and com_append_stmt to use isinstance rather than
testing's type slot of ast node (this slot disappeared with recent
changes to ast).
1.5.2. The compiler generates code for the version of the interpreter
it is run under.
ast.py:
Print and Printnl add dest attr for extended print
new node AugAssign for augmented assignments
new nodes ListComp, ListCompFor, and ListCompIf for list
comprehensions
pyassem.py:
add work around for string-Unicode comparison raising UnicodeError
on comparison of two objects in code object's const table
pycodegen.py:
define VERSION, the Python major version number
get magic number using imp.get_magic() instead of hard coding
implement list comprehensions, extended print, and augmented
assignment; augmented assignment uses Delegator classes (see
doc string)
fix import and tuple unpacking for 1.5.2
transformer.py:
various changes to support new 2.0 grammar and old 1.5 grammar
add debug_tree helper than converts and symbol and token numbers
to their names
parsing functions in support of auto-indent weren't expecting Unicode
strings, but text.get() can now return them (although it remains muddy as
to exactly when or why that can happen). Fixed that with a Big Hammer.
- use unidb compression for the unicodectype module. smaller,
faster, and slightly more portable...
- also mention the unicode directory in Tools/README
- fixed attributions
- moved decomposition data to a separate table, in preparation
for step 3 (which won't happen before 2.0 final, promise!)
- use relative paths in the generator script
I have a lot more stuff in the works for 2.1, but let's leave
that for another day...
a bit, sped it a lot primarily by removing the unused assumption that None was
a legit bin entry (the function doesn't really need to assume that there's
anything special about 0), added an optional "trace" argument, and in __debug__
mode added exhaustive verification that the decomposition is both correct and
doesn't overstep any array bounds (which wasn't obvious to me from staring at the
generated C code -- now I feel safe!). Did not commit a new unicodedata_db.h, as
the one produced by this version is identical to the one already checked in.
Guido van Rossum
Jeremy Hylton
Tim Peters
Fred Drake
Mark Hammond
Jack Jansen
Barry Warsaw
Andrew M. Kuchling
Martin v. Löwis
Neil Schemenauer
Marc-André Lemburg
Ka-Ping Yee
Eric S. Raymond
Fredrik Lundh