compile() becomes replacement for builtin compile()
compileFile() generates a .pyc from a .py
both are exported in __init__
compiler.parse() gets optional second argument to specify compilation
mode, e.g. single, eval, exec
Add AbstractCompileMode as parent class and Module, Expression, and
Interactive as concrete subclasses. Each corresponds to a compilation
mode.
THe AbstractCompileMode instances in turn delegate to CodeGeneration
subclasses specialized for their particular functions --
ModuleCodeGenerator, ExpressionCodeGeneration,
InteractiveCodeGenerator.
The argument properties are ordered from easiest to hardest. The
harder the arg, the more complicated that code that must be generated
to return it from getChildren() and/or getChildNodes(). The old
calculation routine was bogus, because it always set hardest_arg to
the hardness of the last argument. Now use max() to always set it to
the hardness of the hardest argument.
Remove the only test in the syntax module. It ends up that the
transformer must handle this error case.
In the transformer, check for a list compression in com_assign_list()
by looking for a list_for node where a comma is expected.
In pycodegen.compile() re-raise the SyntaxError rather than catching
it and exiting
Invoke compiler.syntax.check() after building AST. If a SyntaxError
occurs, print the error and exit without generating a .pyc file.
Refactor code to use compiler.misc.set_filename() rather than passing
filename argument around to each CodeGenerator instance.
introspection incompatibility, but in fact it's just that calltips
always gave up on a docstring that started with a newline (but
didn't realize they were giving up <wink>).
Remove the option to have nested scopes or old LGB scopes. This has a
large impact on the code base, by removing the need for two variants
of each CodeGenerator.
Add a get_module() method to CodeGenerator objects, used to get the
future features for the current module.
Set CO_GENERATOR, CO_GENERATOR_ALLOWED, and CO_FUTURE_DIVISION flags
as appropriate.
Attempt to fix the value of nlocals in newCodeObject(), assuming that
nlocals is 0 if CO_NEWLOCALS is not defined.
operators per line or statement are now on by default, and -m turns
these warnings off.
- Change the way multiple / operators are reported; a regular
recommendation is always emitted after the warning.
- Report ambiguous warnings (both int|long and float|complex used for
the same operator).
- Update the doc string again to clarify all this and describe the
possible messages more precisely.
percolated out, and some general cleanup. The output is still the
same, except it now prints "Index: <file>" instead of "Processing:
<file>", so that the output can be used as input for patch (but only
the diff-style parts of it).
Fix list comp code generation -- emit GET_ITER instead of Const(0)
after the list.
Add CO_GENERATOR flag to generators.
Get CO_xxx flags from the new module
try/except or try/finally.
Previous versions had only track SETUP_LOOP blocks and ignored the
exception part. This meant that it allowed continue inside a
try/except but generated buggy code. Now it does the right thing.
As the doc string for _lookupName() explains:
This routine uses a list instead of a dictionary, because a
dictionary can't store two different keys if the keys have the
same value but different types, e.g. 2 and 2L. The compiler
must treat these two separately, so it does an explicit type
comparison before comparing the values.
Avoid if/elif/elif/else tests where the final else is supposed to
handle exactly one case instead of all other cases. When the list of
operators is extended, the catchall else treats all new operators as
the last operator in the set of tests. Instead, raise an exception if
an unexpected operator occurs.
Use a dictionary instead of a list to map objects to their offsets in
a const/name tuple of a code object.
XXX The conversion is perhaps incomplete, in that we shouldn't have to
do the list2dict to start.
Add support for floor division (// and //=)
The implementation of getChildren() and getChildNodes() is intended to
be faster, because it avoids calling flatten() on every return value.
But it's not clear that it is a lot faster, because constructing a
tuple with just the right values ends up being slow. (Too many
attribute lookups probably.)
The ast.txt file is much more complicated, with funny characters at
the ends of names (*, &, !) to indicate the types of each child node.
The astgen script is also much more complex, making me wonder if it's
still useful.
varnames should list all the local variables (with arguments first).
The XXX_NAME ops typically occur at the module level and assignment
ops should create locals.
(Hard to believe these were never handled before)
Add misc.mangle() that mangles based on the rules in compile.c.
XXX Need to test the corner cases
Update CodeGenerator with a class_name attribute bound to None. If a
particular instance is created within a class scope, the instance's
class_name is bound to that class's name.
Add mangle() method to CodeGenerator that mangles if the class_name
has a class_name in it.
Modify the FunctionCodeGenerator family to handle an extra argument--
the class_name.
Wrap all name ops and attrnames in calls to self.mangle()
Make nested scopes enabled by default
Add is_constant_false() helper so that compiled code and symbols are
consistent with builtin compiler's handling of "if 0:"
Fix doc string handling to be consistent with recent change that
eliminates the doc string from the Module's node attribute.
Add fix to print handling from Evan & Shane.
Track change to visitor api by making "verbose" explicit.
Comment out setting CO_NESTED flag (it's unnecessary in 2.2).
Evan Simpson's fix. And his explanation:
If you defined two nested functions in a row that refer to the
same non-global variable, the second one will be generated as
though the variable were global.
The use of com_node() introduces a lot of extra stack frames, enough
to cause a stack overflow compiling test.test_parser with the standard
interpreter recursionlimit. The com_node() is a convenience function
that hides the dispatch details, but comes at a very high cost. It is
more efficient to dispatch directly in the callers. In these cases,
use lookup_node() and call the dispatched node directly.
Also handle yield_stmt in a way that will work with Python 2.1
(suggested by Shane Hathaway)
Remove _preorder as alias for dispatch and call dispatch directly.
Add an extra optional argument to walk()
XXX Also comment out some code that does debugging prints.
The tests are run from a copy of the library directory, where
everything has been compiled by the compiler package.
Add a raw_input() call at the end of the script, so that I can check
the output before the temp directory with the compiled code is
removed.
- make the selftests work again (they were apparently not used since
very early in bgen's development), with some minor cleanup by me
- make emacs python mode happier
ChipWidget.__init__(): Added a message area just below the color
name. Both the message and name widgets are now FLAT, DISABLED
Entry widgets instead of Labels. This allows users to
copy-n-paste the color names or color specs. Also, the contents
of both widgets are now driven by StringVars.
set_color(): This only sets the chip color; it does not set the name
widgets.
set_name(): New method which only sets the name widget contents.
set_message(): New method which only sets the message widget contents.
ChipViewer.update_yourself(): Set the color, name, and message for
each chip as follows: the first line always contains the color
spec in #rrggbb format. The second line will contain the color
name, but slightly differently for each widget. For the Selected
widget, if the color exactly matches the Nearest color, the name
is shown, otherwise the message field will be empty. The name
field of the Nearest widget will always contain the color name.
Change several sections to subsections (part of the manual -> howto
transformation).
Flesh out discussion of assignment nodes (and delete statements).
Add an example of manipulating AST objects at a >>> prompt
Remove the module index; there aren't enough modules documented yet
for this to make sense.
Add a couple more index entries, fixed a few typos, and adjusted a few
more things for consistency.
Cater to that.
+ Major speed boost via not reading more of files than necessary. This
was no slouch before; now it screams.
+ Improve msg when giving up on a goofy future statement.
If multiple header files are processed simultaneously which include each
other, the corresponding modules mport each other. Specifically, if h2py
is invoked with sys/types.h first, later header files won't contain the
complete contents of TYPES.py.
list of files to not extract docstrings from when the -D option is
given. This isn't optimal, but I didn't want to change the semantics
of -D, and it's bad form to allow optional switch arguments.
Bumping __version__ to 1.4.
TokenEater.__init__(): Initialize __curfile to None.
__waiting(): In order to extract docstrings from the module, both the
-D flag should be set, and the __curfile should not be named in
the -X filename (i.e. it isn't in opts.nodocstrings).
set_filename(): Fixed a bug where once the first module docstring is
extracted, no subsequent module docstrings will be extracted. The
bug was that the first extraction set __freshmodule to 0, but that
flag was never reset back to 1. set_filename() is always called
when the next file is being processed, so use it to reset the
__freshmodule flag.
main(): Add support for -X/--no-docstring.
fragile. Now the leading "0x" on hex numbers are displayed as labels
and the type-in entry fields just accept the hex digits. Be sure to
strip off the "0x" string when displaying hex values too.
Also, de-string-module-ification, and other Python 2.x improvements.
found a bug here. Here's the deal:
Class PyShell derives from class OutputWindow. Method PyShell.close()
wants to invoke its parent method, but because PyShell long ago was
inherited from class PyShellEditorWindow, it invokes
PyShelEditorWindow.close(self). Now, class PyShellEditorWindow itself
derives from class OutputWindow, and inherits the close() method from
there without overriding it. Under the old rules,
PyShellEditorWindow.close would return an unbound method restricted to
the class that defined the implementation of close(), which was
OutputWindow.close. Under the new rules, the unbound method is
restricted to the class whose method was requested, that is
PyShellEditorWindow, and this was correctly trapped as an error.
This allows system libs to be weak-linked, thereby allowing us to generate functions that are only available on some OS versions without getting a NULL dereference if the function isn't available.
I published it on the web as http://www.python.org/2.1/md5sum.py
so I thought I might as well check it in.
Works with Python 1.5.2 and later.
Works like the Linux tool ``mdfsum file ...'' except it doesn't take
any options or read stdin.
Armin Rigo pointed out that the way the line-# table got built didn't work
for lines generating more than 255 bytes of bytecode. Fixed as he
suggested, plus corresponding changes to pyassem.py, plus added some
long overdue docs about this subtle table to compile.c.
Bugfix candidate (line numbers may be off in tracebacks under -O).
indicating whether the entry was extracted from a docstring or not.
write(): If any of the locations of a string appearance came from a
docstring, add a comment such as
#. docstring
before the references (after a suggestion by Martin von Loewis).
codec files to codecs.py and added logic so that multi mappings
in the decoding maps now result in mappings to None (undefined mapping)
in the encoding maps.
rather than the idle.py script. This has advantages and
disadvantages; the biggest advantage being that we can more easily
have an alternative main program.
Assertion error message had typos in arguments to string format.
.cover files for modules in packages are now put in the right place.
The code that generate .cover files seemed to prepend a "./" to many
absolute paths, causing them to fail. The code now checks explicitly
for absolute paths and leaves them alone.
In trace/coverage code, recover from case where module has no __name__
attribute, when e.g. it is executed by PyRun_String(). In this case,
assign modulename to None and hope for the best. There isn't anywhere
to write out coverage data for this code anyway.
Also, replace several sys.stderr.writes with print >> sys.stderr.
New features:
-C/--coverdir dir: Generate .cover files in specified directory
instead of in the directory where the .py file is.
-s: Print a short summary of files coverred (# lines, % coverage,
name)
variables reported by PyChecker.
__togglegentype(): PyChecker accurately reported that the variable
__gentypevar was unused -- actually this whole method is currently
unused so comment it out.
Always emit a SET_LINENO 0 at the beginning of the module. The
builtin compiler does this, and it's much easier to compare bytecode
generated by the two compilers if they both do.
Move the SET_LINENO inside the FOR_LOOP block for list
comprehensions. Also for compat. with builtin compiler.
Fix annoying bugs in flow graph layout code. In some cases the
implicit control transfers weren't honored. In other cases,
JUMP_FORWARD instructions jumped backwards.
Remove unused arg from nextBlock().
pycodegen.py
Add optional force kwarg to set_lineno() that will emit a
SET_LINENO even if it is the same as the previous lineno.
Use explicit LOAD_FAST and STORE_FAST to access list comp implicit
variables. (The symbol table doesn't know about them.)
(Yes, this is a new feature right before the 2.1 release. No, I can't
imagine this would seriously break anybody's code. In fact, most
users of this script are probably *happy* to see this addition.)
Add mangling support
Add get_children() and add_child() methods to Scope
Skip nodes when If test is a false constant
Add test code that checks results against symtable module
Fix com_NEWLINE() so that is accepts arguments, which occurs for lines like:
stmt; # note trailing semicolon
Add XXX about checking for assignment to list comps
<christopher.mccafferty@csg.ch>:
Add javascript: and telnet: to the types of URLs we ignore.
Add support for several additional URL-valued attributes on the BODY,
FRAME, IFRAME, LINK, OBJECT, and SCRIPT elements.
Was getting mystery
ModifiedInterpreter instance has no attribute 'save_warnings_filters'
errors at odd times (for sure in a fresh IDLE, fresh file, then Run Script).
This patch was developed primarily to reduce the size of the
frozen binary. It is particularly useful when freezing for 'small'
platforms, such as Palm OS, where you really want to save that
last miserable byte.
A limitation of this patch is that it does not provide any feedback
about the replacements being made. As the path matching
is case-sensitive this may lead to unexpected behaviour for DOS
and Windows people, eg
> freeze.py -r C:\Python\Lib\=py\ goats.py
should probably be:
> freeze.py -r c:\python\lib\=py\ goats.py
this just copies the __name__=='__main__' logic from pydoc.py.
?!ng can decide whether he wants to create a main() in pydoc, or rip
it out of pydoc.py completely.
Guido told me to do this <wink>.
Greatly expanded docstrings, and fleshed out with examples.
New std test.
Added new get_close_matches() function for ESR.
Needs docs, but LaTeXification of the module docstring is all it needs.
\CVS: ----------------------------------------------------------------------
fields. You can now backspace out the 0 in 0x0, and you can clear the
field when in decimal mode. There are still some oddities about
typing into these fields, but it should be much less annoying. The
real solution is to ditch the update-while-typing "feature".
\t\t\t\t\treal code
##\t\t\t\t\tunused code
\t\t\t\t\treal code
via untabifying and shifting the real code left. Semantically the
same but made the intent of the commented-out-in-column-0 unused code
unclear. The exact same unused code appears to have gotten copied from
file to file over the years.
codec to not apply Latin-1 mappings for keys which are not found
in the mapping dictionaries, but instead treat them as undefined
mappings.
The patch was originally written by Martin v. Loewis with some
additional (cosmetic) changes and an updated test script
by Marc-Andre Lemburg.
The standard codecs were recreated from the most current files
available at the Unicode.org site using the Tools/scripts/gencodec.py
tool.
This patch closes the bugs #116285 and #119960.
embedded code objects (e.g. functions) rather than the generated code
object. This change means that the compiler generates code for
everything at the end, rather then generating code for each function
as it finds it. Implementation note: _convert_LOAD_CONST in
pyassem.py must be change to call getCode().
Other changes follow. Several changes creates extra edges between
basic blocks to reflect control flow for loops and exceptions. These
missing edges had gone unnoticed because they do not affect the
current compilation process.
pyassem.py:
Add _enable_debug() and _disable_debug() methods that print
instructions and blocks to stdout as they are generated.
Add edges between blocks for instructions like SETUP_LOOP,
FOR_LOOP, etc.
Add pruneNext to get rid of bogus edges remaining after
unconditional transfer ops (e.g. JUMP_FORWARD)
Change repr of Block to omit block length.
pycodegen.py:
Make sure a new block is started after FOR_LOOP, etc.
Change assert implementation to use RAISE_VARARGS 1 when there is
no user-specified failure output.
misc.py:
Implement __contains__ and copy for Set.
Reformatting -- long lines, "[ ]" -> "[]", a few indentation nits.
Replace calls to Node function (which constructed ast nodes) with
calls to actual constructors imported from ast module.
Optimize com_node (most frequently used method) for the common case --
the appropriate method is found in _dispatch.
Fix com_augassign to use class object's rather than node names
(rendered invalid by recent changes to ast)
Remove expensive tests for sequence-ness in com_stmt and
com_append_stmt. These tests should never fail; if they do, something
is really broken and exception will be raised elsewhere.
Fix com_stmt and com_append_stmt to use isinstance rather than
testing's type slot of ast node (this slot disappeared with recent
changes to ast).
1.5.2. The compiler generates code for the version of the interpreter
it is run under.
ast.py:
Print and Printnl add dest attr for extended print
new node AugAssign for augmented assignments
new nodes ListComp, ListCompFor, and ListCompIf for list
comprehensions
pyassem.py:
add work around for string-Unicode comparison raising UnicodeError
on comparison of two objects in code object's const table
pycodegen.py:
define VERSION, the Python major version number
get magic number using imp.get_magic() instead of hard coding
implement list comprehensions, extended print, and augmented
assignment; augmented assignment uses Delegator classes (see
doc string)
fix import and tuple unpacking for 1.5.2
transformer.py:
various changes to support new 2.0 grammar and old 1.5 grammar
add debug_tree helper than converts and symbol and token numbers
to their names
parsing functions in support of auto-indent weren't expecting Unicode
strings, but text.get() can now return them (although it remains muddy as
to exactly when or why that can happen). Fixed that with a Big Hammer.
- use unidb compression for the unicodectype module. smaller,
faster, and slightly more portable...
- also mention the unicode directory in Tools/README
- fixed attributions
- moved decomposition data to a separate table, in preparation
for step 3 (which won't happen before 2.0 final, promise!)
- use relative paths in the generator script
I have a lot more stuff in the works for 2.1, but let's leave
that for another day...
a bit, sped it a lot primarily by removing the unused assumption that None was
a legit bin entry (the function doesn't really need to assume that there's
anything special about 0), added an optional "trace" argument, and in __debug__
mode added exhaustive verification that the decomposition is both correct and
doesn't overstep any array bounds (which wasn't obvious to me from staring at the
generated C code -- now I feel safe!). Did not commit a new unicodedata_db.h, as
the one produced by this version is identical to the one already checked in.
subset of Win32 ShellExecute's functionality. Guido wants this because
IDLE's Help -> Docs function currently crashes his machine because of a
conflict between his version of Norton AntiVirus (6.10.20) and MS's
_popen. Docs for startfile are being mailed to Fred (or just read the
docstring -- it tells the whole story).
Changed webbrowser.py to use os.startfile instead of os.popen on Windows.
Changed IDLE's EditorWindow.py to pass an absolute path for the docs
(hardcoding ShellExecute's "directory" arg to "." as used to be done let
IDLE work, but made the startfile command exceedingly obscure for other
uses -- the MS docs are terrible, of course, & still not sure I
understand it).
Note that Windows Python must link with shell32.lib now! That's where
ShellExecute lives.
The cause was that the replace code necessarily used a PCRE internal
function to to template expansion.
The fix changes the code to use an SRE internal if SRE is used, and a
PCRE internal if SRE is used; in a way that should work with 1.5.2.
The solution can be sped up tremendously under the assumption that the
choice between sre and pre is not changed during the execution of the
program; especially replace-all will be slow.
But I'll leave that to someone else.
created. This allows the application-specific Tkinter
initialization to be executed if present. Also pass an explicit
className parameter to the Tk() constructor.
This closes SourceForge bug #110618.
Fix import support to work with import as variant of Python 2.0. The
grammar for import changed, requiring changes in transformer and code
generator, even to handle compilation of imports with as.
by Martin v. Loewis, proofed by Barry Warsaw for coding standards,
typos, and to make command line options compatible with GNU msgfmt
where they overlap.
Closes patch #101295.
- fix tab space issues (SF patch #101167 by Neil Schemenauer)
- fix co_flags for classes to include CO_NEWLOCALS (SF patch #101145 by Neil)
- fix for merger of UNPACK_LIST and UNPACK_TUPLE into UNPACK_SEQUENCE,
(SF patch #101168 by, well, Neil :)
- Adjust bytecode MAGIC to current bytecode.
TODO: teach compile.py about list comprehensions.
originally submitted by Bill Tutt
Note: This code is actually going to be replaced in 2.0 by /F's new
database. Until then, this patch keeps the test suite working.
comments, docstrings or error messages. I fixed two minor things in
test_winreg.py ("didn't" -> "Didn't" and "Didnt" -> "Didn't").
There is a minor style issue involved: Guido seems to have preferred English
grammar (behaviour, honour) in a couple places. This patch changes that to
American, which is the more prominent style in the source. I prefer English
myself, so if English is preferred, I'd be happy to supply a patch myself ;)
mislabeled.
(Using -c and then -e rearranges some comments, so I won't check that
in -- but it's a good test anyway.
Note that pindent is not perfect -- e.g. it doesn't know about
triple-quoted strings!)
Problem:
A Python program can be completed and reformatted using
Tools/scripts/pindent.py. Unfortunately there is no option for removal
of the generated "# end"-tags. Although a few Python commands or a
"grep -v '# end '" can do wonders here, there are two drawbacks:
- not everyone has grep/time to write a Python script
- it is not checked whether the "# end"-tags were used validly
Solution:
add extra option "-e" (eliminate) to pindent.py
Perfect hash table generator. Outputs a Python extension module
which provides access to the hash table (which is stored in static
C data) using custom code.
This module can currently only generates code for the ucnhash
module, but can easily be adapted to produce perfect hash tables
for other tasks where fast lookup in large tables is needed.
By Bill Tutt.
Instead of using Netscape through os.system(), use the new
browser.open() function to load the documentation on the
default browser. On Windows, this will use the installed
documentation if it exists, instead of hitting python.org.
Attached is a set of diffs for the .py compiler that adds support
for the new extended call syntax.
compiler/ast.py:
CallFunc node gets 2 new children to support extended call syntax -
"star_args" (for "*args") and "dstar_args" (for "**args")
compiler/pyassem.py
It appear that self.lnotab is supposed to be responsible for
tracking line numbers, but self.firstlineno was still hanging
around. Removed self.firstlineno completely. NOTE - I didnt
actually test that the generated code has the correct line numbers!!
Stack depth tracking appeared a little broken - the checks never
made it beyond the "self.patterns" check - thus, the custom methods
were never called! Fixed this.
(XXX Jeremy notes: I think this code is still broken because it
doesn't track stack effects across block bounaries.)
Added support for the new extended call syntax opcodes for depth
calculations.
compiler/pycodegen.py
Added support for the new extended call syntax opcodes.
compiler/transformer.py
Added support for the new extended call syntax.
can't remember who. :-) Changes:
- Owner name+email made generic instead of GvR
- Support for Windows NT (running from a .bat file)
- DOcument <HTML>...</HTML> tags
The robotparser.py module currently lives in Tools/webchecker. In
preparation for its migration to Lib, I made the following changes:
* renamed the test() function _test
* corrected the URLs in _test() so they refer to actual documents
* added an "if __name__ == '__main__'" catcher to invoke _test()
when run as a main program
* added doc strings for the two main methods, parse and can_fetch
* replaced usage of regsub and regex with corresponding re code
code generator uses flowgraph as intermediate representation. the old
rep uses a list with explicit "StackRefs" to indicate the target
of jumps.
pyassem converts flowgraph to bytecode, breaks up individual steps of
generating bytecode
Fix bad auto-indent I recently introduced when replacing the regexp that
could cause re to blow up:
if or_any_other_block_opener:
# one indenting comment line
^ cursor ended up at the caret (the bug)
^ but belongs here (the post-patch behavior)
(inspired by Detlef Lannert). Specifically,
-k/--keyword no longer takes an optional argument to clear the
default keywords. Instead, use -K/--no-default-keywords to clear
them.
-n/--add-location also no longer takes an optional argument to set
the comment style. Instead, use -S/--style to set the comment
style to GNU or Solaris.
-o/--output can take `-' as the filename, meaning write to
standard output.
The inputfile name can also be `-' meaning read from standard in.
A few other changes include
Kludge to mark the file docstring as translatable. Since the
marking is to place _() around the docstring, and because we
actually have to define the _() function before we use it, this
means that we have to manually assign to __doc__ the output of
_(). This doesn't seem too bad because you'll only use this idiom
when translating a script's docstring (you really don't need to
translate most module docstrings).
Convert everything to string methods and do not import the string
module.
Bump the version number to 1.1
and output windows) so that it if it doesn't succeed with the line
at the cursor, it tries the line before that. This is handy with
tracebacks, where my natural tendency is to click in the displayed
source line rather than in the file/line indicator just above it.
Now I can indulge this tendency.
I factored out a helper and changed the error handling so that a
non-existing file is treated as if the line didn't match -- this is
handy because some function calls (e.g. "foo.bar(1)") match the grep
pattern.
cursor, erase that whitespace first. This avoids a particularly
confusing case where hitting Return at the end of the command didn't
do what it was expected to do -- because it wasn't considered to be at
the end of the command. Now it is.
fix imports
remove parse functions and visitor code
track name change: Classdef to Class
add some comments and tweak order of visitXXX methods
get rid of if __name__ == "__main__ section
idle.py:
Load the config files before anything else happens
XXX Need to define standard way to get files relative to the
IDLE install dir
PyShell.py:
ColorDelegator.py:
Get color defns out of IdleConf instead of IdlePrefs
EditorWindow.py:
Replace hard-coded font & window size with config options
Get extension names via IdleConf.getextensions
extend.py:
Obsolete. Extensions defined in config file.
ParenMatch.py:
Use config file for extension options.
Revise comment about parser requirements.
Simplify logic on find returning None.
Changes the one regexp in PyParse capable of making the re module blow the C
stack when passed unreasonable <0.9 wink> program text. Jeremy Hylton
provoked this with a program of the form:
x = (1,
2,
... # 9997 lines deleted here
10000,
)
Programs "like this" will no longer (no matter how many lines they contain)
trigger re death. OTOH, you can now make another class of unreasonable
program that will take much longer to parse.
This will fold all ISO 8859 chars from the upper half of the
charset into the lower half, which is ...ummm.... unintened.
The second is a typo in the reference to options.escape in main().
make pygettext more compatible with GNU xgettext, specifically:
Added -E/--escape for allowing pass-thru of iso8859-1 characters above
7 bits.
Added -o/--output option for renaming the output file from
messages.pot (there's overlap with -d/--default-domain, but GNU
xgettext has them both).
Added -p/--output-dir for specifying the output directory for
messages.pot.
Added -V/--version for printing the version number.
Added -w/--width for specifying the output page width (this is because
now pygettext, like GNU xgettext will put several locations on the
same line to cut down on vertical space).
Added -x/--exclude-file for specifying a list of strings that are not
to be extracted from the input files.
Bumped version number to 1.0
Try to import fintl and use fintl.gettext as _ if available. Fall
back is to use identity definition of _().
Moved the escape creation to a function make_escapes() so that its
behavior can be controlled by the -E option.
__openseen(): Support the -x option.
write(): Support -w option and vertical space preserving feature.
main(): Support new options.
standard library. Added some comments:
# XXX Note: this is now a standard library module.
# XXX The API needs to undergo changes however; the current code is too
# XXX script-like. This will be addressed later.
of the current module. It also runs the tabnanny to catch any
inconsistent tabs.
Also did a little bit of refactoring: added an errorbox() method to
simplify the display of error dialogs.
- removed now (happily) unused second arg
- need to verify results of [].index are correct; for building consts,
need to have same value and same type, e.g. 2 not the same as 2L
(big surprise). new solution is a little less hackish.
Code gen adds a TupleArg instance in the argument slot. The tuple arg
includes a copy of the names that it is responsble for binding. The
PyAssembler uses this information to calculate the correct argcount.
all fix this wacky case: del (a, ((b,), c)), d
which is the same as: del a, b, c, d
(Can't wait for Guido to tell me why.)
solution uses findOp which walks a tree to find out whether it
contains OP_ASSIGN or OP_DELETE or ...
- added a number of support methods to generate code just before the
body
- hack protocol for communicating number of args to PyAssembler
fix TryExcept generation for case where exception handler has no body
fix visitAssAttr
add comment about incomplete visitAssName
stop using the ExampleASTVisitor
change script invocation to accept a list of .py files (e.g. Lib/*.py)
named OPTIMIZED, which matches compile.c and makes more sense for
classes
revamp call signature for PythonVMCode.__init__; doesn't really matter
'cuz this code is going to be refactored out of existence
add generateClassCode and modify Func & Lambda generation
add support for nodes Classdef, Keyword,
fix CallFunc to generate right op arg when calling w/ keywords
add ugly hack to properly compute offsets when the same stack ref is
used multiple times
change resolution of local name ops (LOAD_FAST). i think it makes
sense now. if it is an argument or a local var name that it used, it
must be in varnames. if it is a local var name that is used, it must
also be in names
FUNCTION_NAMESPACE. initialize in __init__ and reset in
generateFunctionCode.
replace direct issue of STORE_FAST, STORE_GLOBAL, etc. with call to
storeName; same for loadName and deleteName
the new {store,load,delete}Name methods use the namespace attr and the
local variable stack to determine the correct bytecode to issue
* prints out examples of nodes that are handled by visitor. simply a
development convenience
remove NestedCodeGenerator -- it was bogus after all
replace with generateFunctionCode, a method to call to generate code
for a function instead of a top-level module
fix impl of visitDiscard (most pop stack)
emit lineno for pass
handle the following new node types: Import, From, Getattr, Subscript,
Slice, AssAttr, AssTuple, Mod, Not, And, Or, List
LocalNameFinder: remove names declared as globals for locals
PythonVMCode: pass arg names to constructor, force varnames to contain
them all (even if they aren't referenced)
add -q option on command line to disable stdout
VERBOSE setting for the ASTVisitor
add getopt handling for one or more -v args
rename ForwardRef to StackRef, because it isn't necessarily directional
CodeGenerator:
* add assertStackEmpty method. prints warning if stack is not empty
when it should be
* define methods for AssName, UNARY_*, For
PythonVMCode:
* fix mix up between hasjrel and hasjabs for address calculation
language.
CodeGenerator:
* modify to track stack depth
* add emit method that call's PythonVMCode's makeCodeObject
* thread filenames through in hackish way
* set flags for code objects for modules and functions
XXX the docs for the flags seem out of date and/or incomplete
PythonVMCode:
* add doc string describing the elements of a real code object
LineAddrTable:
* creates an lnotab (no quite correctly though)
handle most of the language syntax yet)
create NestedCodeGenerator used to generator the separate code object
that needs to be passed as an argument to MAKE_FUNCTION when a def
stmt is found (probably useful for class too)
change CodeGenerator.visitFunction to use the NestedCG
add CompiledModule class to handle creation of .pyc (pretty minimal
for now)
add makeCodeObject method to PythonVMCode that replaces symbolic names
with indexes into slots of the code code. the design of this
class will probably need to be revised.
compile.py: ASTVisitor framework plus bits of a code generator that
should be bug-for-buf compatible with compile.c
misc.py: Set and Stack helpers
test.py: a bit of simple sample code that compile.py will work on
Three bgen mods:
- support for FSSpecs passed-by-value and points-passed-by-reference added.
- strip single-line comments when parsing header files
- if a definition is blacklisted _do_ output it, but in comment
- same trick with "import wcnew; webchecker = wcnew" as above
- updated readhtml() method to handle pair representation; used
new name suppression infrastructure from wcnew.py to suppress
processing name anchors
[And untabified --GvR]
- added -t and -a arguments
- added "import wcnew; webchecker = wcnew" in place of "import
webchecker" (I assume that if you're happy with the changes, you'll
just replace webchecker.py with wcnew.py, but if I were to do that,
the diffs would be incomprehensible)
- fixed buggy -v argument (I think you got out of sync with the
way verbosity was handled in webchecker vs. wcgui between 1.5 and
1.5.2)
- made -v actually do something by adding a call to c.setflags()
(probably the same problem as above)
- updated references to URLs to accommodate wcnew.py's pair
representation; added appropriate calls to format_url() to handle
display; added argument to ListPanel() initialization to provide
access to format_url()
[And untabified --GvR]
- same fixes from webchecker.py
- incorporated small diff between current webchecker.py and 1.5.2
- fixed bug where "extra roots" added with the -t argument were being
checked as real roots, not just as possible continuations
- added -a argument to suppress checking of name anchors
[And untabified --GvR]
- forced new done origins to set errors if they're in self.bad (fixes
bug where only the first of a number of errorful references to a
link is reported under some circumstances)
- suppressed adding duplicates to self.todo list (cleans up printout
in wcgui details)
Herzog <herzog@online.de>. Specifically,
--verbose/-v flag added
pot_header added to make msgmerge and Emacs po-mode work better
normalize(), escape(), safe_eval(): Improved normalization of strings
for more .po file compatibility (e.g. C style). Handles emmbedded
newlines better.
Also added an identity function called _() and use it in the file
where messages are printed. This allows us to selftest pygettext.py
with itself as input.
(Suggested by Moshe Zadka, but implemented differently.)
Add <<python-docs>> event which, on Unix, brings up Netscape pointing
to http://www.python.doc/current/ (a local copy would be nice but its
location can't be predicted). Windows solution TBD.
(interrupted system call) when getting the device information. I've
never seen it, but this patch should take care of the problem.
If we get that exception and we're polling, just return since we'll
wake up again soon and get the right information. If we're not
polling, try 4 times and then give up.
"""
If the filename being complained about contains a space, enclose the
file-name in quotes.
The reason is simply that when I try and parse tabnanny's output, filenames
with spaces make it very difficult to determine where the filename stops
and the linenumber begins!
"""
Tim approves.
I slightly changed the patch (use 'in' instead of string.find()) and
arbitrarily bumped the __version__ variable up to 6.
dynamically imported when Pynche is running via askcolor out of a
package. If the ImportError occurs, try again, prepending the package
name and digging out the module.
user what they want first if there's unsaved stuff, and may cancel).
It closes more than before.
Add unload_extensions() method to unload all extensions; called from
_close(). It calls an extension's close() method if it has one.
"""
When there are additional Setup files, specified by -e option of freeze,
checkextenstions.py assumes that *.o, *.a, -Lpath, and -Rpath are all
relative to where the Setup file is. select() inserts the path to the
Setup file to make them absolute. However, the assumption is not true.
There are cases that absolute paths are specified for them. The inserted
prefix, by select(), results in error.
The following fix check for absolute paths. The assumption is: an
absolute path begins with either '/' or '$'. In the latter case, it is
from the environmental variable. (Variables defined locally in the Setup
file have already been handled by expandvars())
"""
My version of the patch has been verified by Charles Waldman (a
colleague of Chih-Hao).
Toby writes:
winmakemakefile.py tries to allow for spaces in the python install
path, by adding quotes around the appropriate filenames. It doesn't
quite get this correct; sometimes the quotes end up in the middle of
the path.
Microsoft's NMAKE version 6.0 is happy with this (!!!!) unless there
is also a space in the name. I guess most users of freeze on windows
do not use the same path as the binary distribution.
I've tested the following changes on systems with and without a space
in the path.
EditorWindow.py:
+ Added get_tabwidth & set_tabwidth "virtual text" methods, that get/set the
widget's view of what a tab means.
+ Moved TK_TABWIDTH_DEFAULT here from AutoIndent.
+ Renamed Mark's get_selection_index to get_selection_indices (sorry, Mark,
but the name was plain wrong <wink>).
FormatParagraph.py: renamed use of get_selection_index.
AutoIndent.py:
+ Moved TK_TABWIDTH_DEFAULT to EditorWindow.
+ Rewrote set_indentation_params to use new VTW get/set_tabwidth methods.
+ Changed smart_backspace_event to delete whitespace back to closest
preceding virtual tab stop or real character (note that this may require
inserting characters if backspacing over a tab!).
+ Nuked almost references to the selection tag, in favor of using
get_selection_indices. The sole exception is in set_region, for which no
"set_selection" abstraction has yet been agreed upon.
+ Had too much fun using the spiffy new features of the format-paragraph
cmd.
this functionality is not present (e.g. when used with a vintage
Python 1.5.2 installation) top-level functions are not listed.
(Hmm... Any distribution of IDLE 0.5 should probably include a copy
of the new pyclbr.py!)
+ Set usetabs=1. Editing pyclbr.py was driving me nuts <0.6 wink>.
usetabs=1 is the Emacs pymode default too, and thanks to indentwidth !=
tabwidth magical usetabs disabling, new files are still created with tabs
turned off. The only implication is that if you open a file whose first
indent is a single tab, IDLE will now magically use tabs for that file (and
set indentwidth to 8). Note that the whole scheme doesn't work right for
PythonWin, though, since Windows users typically set tabwidth to 4; Mark
probably has to hide the IDLE algorithm from them (which he already knows).
+ Changed comment_region_event to stick "##" in front of every line. The
"holes" previously left on blank lines were visually confusing (made it
needlessly hard to figure out what to uncomment later).
Smarter logic for finding a parse synch point.
Does a half to a fifth the work in normal cases; don't notice the speedup,
but makes more breathing room for other extensions.
Speeds terrible cases by at least a factor of 10. "Terrible" == e.g. you put
""" at the start of Tkinter.py, undo it, zoom to the bottom, and start
typing in code. Used to take about 8 seconds for ENTER to respond, now some
large fraction of a second. The new code gets indented correctly, despite
that it all remains "string colored" until the colorizer catches up (after
which, ENTER appears instantaneous again).
IDLE is now the first Python editor in the Universe not confused by my
doctest.py <wink>.
As threatened, this defines IDLE's is_char_in_string function as a
method of EditorWindow. You just need to define one similarly in
whatever it is you pass as editwin to AutoIndent; looking at the
EditorWindow.py part of the patch should make this clear.
stuff for getting the tip. Had to fix the Key-( and Key-) events
for Unix. Will have to re-apply my patch for catching KeyRelease and
ButtonRelease events.
Unix bindings for <<toggle-tabs>> and <<change-indentwidth>> were
missing, and somehow that meant the events were never generated,
even though they were in the menu. The new Unix bindings are now
the same as the Windows bindings (M-t and M-u).
The new version (attached) is fast enough all the time in every real module
I have <whew!>. You can make it slow by, e.g., creating an open list with
5,000 90-character identifiers (+ trailing comma) each on its own line, then
adding an item to the end -- but that still consumes less than a second on
my P5-166. Response time in real code appears instantaneous.
Fixed some bugs.
New feature: when hitting ENTER and the cursor is beyond the line's leading
indentation, whitespace is removed on both sides of the cursor; before
whitespace was removed only on the left; e.g., assuming the cursor is
between the comma and the space:
def something(arg1, arg2):
^ cursor to the left of here, and hit ENTER
arg2): # new line used to end up here
arg2): # but now lines up the way you expect
New hack: AutoIndent has grown a context_use_ps1 Boolean config option,
defaulting to 0 (false) and set to 1 (only) by PyShell. Reason: handling
the fancy stuff requires looking backward for a parsing synch point; ps1
lines are the only sensible thing to look for in a shell window, but are a
bad thing to look for in a file window (ps1 lines show up in my module
docstrings often). PythonWin's shell should set this true too.
Persistent problem: strings containing def/class can still screw things up
completely. No improvement. Simplest workaround is on the user's head, and
consists of inserting e.g.
def _(): pass
(or any other def/class) after the end of the multiline string that's
screwing them up. This is especially irksome because IDLE's syntax coloring
is *not* confused, so when this happens the colors don't match the
indentation behavior they see.
[Tim, after adding some bracket smarts to AutoIndent.py]
> ...
> What it can't possibly do without reparsing large gobs of text is
> suggest a reasonable indent level after you've *closed* a bracket
> left open on some previous line.
> ...
The attached can, and actually fast enough to use -- most of the time. The
code is tricky beyond belief to achieve that, but it works so far; e.g.,
return len(string.expandtabs(str[self.stmt_start :
^ indents to caret
i],
^ indents to caret
self.tabwidth)) + 1
^ indents to caret
It's about as smart as pymode now, wrt both bracket and backslash
continuation rules. It does require reparsing large gobs of text, and if it
happens to find something that looks like a "def" or "class" or sys.ps1
buried in a multiline string, but didn't suck up enough preceding text to
see the start of the string, it's completely hosed. I can't repair that --
it's just too slow to reparse from the start of the file all the time.
AutoIndent has grown a new num_context_lines tuple attribute that controls
how far to look back, and-- like other params --this could/should be made
user-overridable at startup and per-file on the fly.
One new file in the attached, PyParse.py. The LineStudier (whatever it was
called <wink>) class was removed from AutoIndent; PyParse subsumes its
functionality.
Removed "New tabwidth" menu binding.
Added "a tab means how many spaces?" dialog to block tabify and untabify. I
think prompting for this is good now: they're usually at-most-once-per-file
commands, and IDLE can't let them change tabwidth from the Tk default
anymore, so IDLE can no longer presume to have any idea what a tab means.
Irony: for the purpose of keeping comments aligned via tabs, Tk's
non-default approach is much nicer than the Emacs/Notepad/Codewright/vi/etc
approach.
2. No longer need to reset pyclbr cache and show watch cursor when calling
ClassBrowser -- the ClassBrowser takes care of pyclbr and the TreeWidget
takes care of the watch cursor.
3. Reset the focus to the current window after error message about class
browser on buffer without filename.
I'm still unsure, but couldn't stand the virtual event trickery so tried a
different sin (adding undo_block_start/stop methods to the Text instance in
EditorWindow.py). Like it or not, it's efficient and works <wink>. Better
idea?
Give the attached a whirl. Even if you hate the implementation, I think
you'll like the results. Think I caught all the "block edit" cmds,
including Format Paragraph, plus subtler ones involving smart indents and
backspacing.
[W]hile trying to dope out how redirection works, stumbled into two
possible glitches. In the first, it doesn't appear to make sense to try to
rename a command that's already been destroyed; in the second, the name
"previous" doesn't really bring to mind "ignore the previous value" <wink>.
You can switch database by just loading the new one; the list window
and nearest colors adapt to the new database.
Some reorganizing of code. Also, the name of the database file is
stored in the ~/.pynche pickle. If it can't be loaded, fallbacks are
used.
Ho ho ho -- that's trickier than it sounded! The colorizer is working with
"line.col" strings instead of Text marks, and the absolute coordinates of
the point of interest can change across the self.update call (voice of
baffled experience, when two quick backspaces no longer fooled it, but a
backspace followed by a quick ENTER did <wink>).
Anyway, the attached appears to do the trick. CPU usage goes way up when
typing quickly into a long triple-quoted string, but the latency is fine for
me (a relatively fast typist on a relatively slow machine). Most of the
changes here are left over from reducing the # of vrbl names to help me
reason about the logic better; I hope the code is a *little* easier to
there are two new commands:
Import module (F5) imports or reloads the module and also adds its
name to the __main__ namespace. This gets executed in the PyShell
window under control of its debug settings.
Run script (Control-F5) is similar but executes the contents of the
file directly in the __main__ namespace.
idle.py -e file ... -- to edit files
idle.py script arg ... -- to run a script
idle.py -c cmd arg ... -- to run a command
Other options, see also the usage message (also new!) for more details:
-d -- enable debugger
-s -- run $IDLESTARTUP or $PYTHONSTARTUP
-t title -- set Python Shell window's title
sys.argv is set accordingly, unless -e is used.
sys.path is absolutized, and all relevant paths are inserted into it.
Other changes:
- the environment in which commands are executed is now the __main__ module
- explicitly save sys.stdout etc., don't restore from sys.__stdout__
- new interpreter methods execsource(), execfile(), stuffsource()
- a few small nits
dramatically (up to 15 times he claims). Works by reading more than
one line at a time, up to 100-line chunks (starting with one line and
then doubling up to the limit). On a typical machine (e.g. Tim's
P5-166) this doesn't reduce interactive responsiveness in a noticeable
way.
unsuccessful search wraps around and re-searches that part of the file
between the start of the search and the end of the file - only really
an issue for very large files, but... (also removes a redundant
m.span() call).
o Makes the tab key intelligently insert spaces when appropriate (see Help
list banter twixt David Ascher and me; idea stolen from every other editor
on earth <wink>).
o newline_and_indent_event trims trailing whitespace on the old line (pymode
and Codewright).
o newline_and_indent_event no longer fooled by trailing whitespace or
comment after ":" (pymode, PTUI).
o newline_and_indent_event now reduces the new line's indentation after
return, break, continue, raise and pass stmts (pymode).
The last two are easy to fool in the presence of strings & continuations,
but pymode requires Emacs's high-powered C parsing functions to avoid that
in finite time.
I should have waited overnight <wink/sigh>. Nothing wrong with the one I
sent, but I couldn't resist going on to add new -r1 / -r2 cmdline options
for recreating the original files from ndiff's output. That's attached, if
you're game! Us Windows guys don't usually have a sed sitting around
<wink>.
Attached is a cleaned-up version of ndiff (added useful module
docstring, now echo'ed in case of cmd line mistake); added -q option
to suppress initial file identification lines; + other minor cleanups,
& a slightly faster match engine.
Unfortunately his code breaks wcgui.py in a way that's not easy
to fix. I expect that this is a temporary situation --
eventually Sam's changes will be merged back in.
(The changes add a -t option to specify exceptions to the -x
option, and explicit checking for #foo style fragment ids.)
the ob_itself pointer. This allows (when using the mixin)
different Python objects pointing to the same C object and
behaving well as dictionary keys.
Or so sez Jack Jansen...
Under Windows, python freeze.py -o hello hello.py
creates all the correct files in the hello subdirectory, but the
Makefile has the directory prefix in it for frozen_extensions.c
nmake fails because it tries to locate hello/frozen_extensions.c
(His fix adds a call to os.path.basename() in the appropriate place.)
pyclbr.Class object; this can happen when the superclass is
unrecognizable (to pyclbr), e.g. when module renaming is used.
- Show a watch cursor when calling pyclbr (since it may take a while
recursively parsing imported modules!).
directories on sys.path
modules in selected directory
classes in selected module
methods of selected class
Sinlge clicking in a directory, module or class item updates the next
column with info about the selected item. Double clicking in a
module, class or method item opens the file (and selects the clicked
item if it is a class or method).
I guess eventually I should be using a tree widget for this, but the
ones I've seen don't work well enough, so for now I use the old
Smalltalk or NeXT style multi-column hierarchical browser.
with changed APIs -- it makes much more sense there.
Also add a new feature: if the first character of a menu label is
a '!', it gets a checkbox. Checkboxes are bound to Boolean Tcl variables
that can be accessed through the new getvar/setvar/getrawvar API;
the variable is named after the event to which the menu is bound.
The separator dances too much and seems to jump by arbitrary amounts
in arbitrary directions when I try to move it for resizing the frames.
This patch makes it more quiet.
Much has changed -- too much, in fact, to write down.
The big news is that there's a standard way to write IDLE extensions;
see extend.txt. Some sample extensions have been provided, and
some existing code has been converted to extensions. Probably the
biggest new user feature is a new search dialog with more options,
search and replace, and even search in files (grep).
This is exactly as downloaded from my laptop after returning
from the holidays -- it hasn't even been tested on Unix yet.
to color constants (i.e. red constant, green constant, blue
constant). But I haven't hooked this up yet because the UI gets more
crowded and the arrows don't reflect the correct values.
Added "Go to Black" and "Go to White" buttons.
show(): added color keyword here so that the selected color can be
chosen on each invocation of askcolor().
Also fixed this class, and askcolor() so that the same Chooser
instance can be re-used instead of creating a new one on each
invocation of askcolor().
Added a module function save() which can be used to explicitly save
the option database in ~/.pynche. This does not happen automatically
when used as a modal.
run either as a standalone application (by running pynche or
pynche.pyw), or as a modal dialog inside another application. This
can be done by importing pyColorChooser and running askcolor(). The
API for this is the same as the tkColorChooser.askcolor() API, namely:
When `Okay' is hit, askcolor() returns ((r, g, b), "name"). When
`Cancel' is hit, askcolor() returns (None, None).
Note the following differences:
1. pyColorChooser.askcolor() takes an optional keyword `master'
which if set tells Pynche to run as a modal dialog. `master'
is a Tkinter parent window. Without the `master' keyword
Pynche runs standalone.
2. in pyColorChooser.askcolor() will return a Tk/X11 color name as
"name" if there is an exact match, otherwise it will return a
color spec, e.g. "#rrggbb". tkColorChooser can't return a
color name.
There are also some UI differences when running standalone vs. modal.
When modal, there is no "File" menu, but instead there are "Okay" and
"Cancel" buttons.
The implementation of all this is a bit of a hack, but it seems to
work moderately well. I'm not guaranteeing the pyColorChooser.Chooser
class has the same semantics as the tkColorChooser.Chooser class.
"""
the NEWS file of Python 1.5.2a2 inspired me to look at
Tools/scripts/untabify.py. I wonder why it accepts a -t argument
but ignores it. The following patch tries to make it somewhat useful
(i.e., to override the tabsize=8 setting). Is that agreeable?
"""
priority over text colorization (which on Windows is almost the
same color as the selection background).
Define a tag and color for breakpoints ("BREAK").
Jeremy Hylton
Tim Peters
Mark Hammond
Jack Jansen
Guido van Rossum
Barry Warsaw
Fred Drake
Andrew M. Kuchling
Martin v. Löwis
Neil Schemenauer
Marc-André Lemburg
Ka-Ping Yee
Eric S. Raymond
Fredrik Lundh
Thomas Wouters
Greg Stein
Peter Schneider-Kamp