Changes the one regexp in PyParse capable of making the re module blow the C
stack when passed unreasonable <0.9 wink> program text. Jeremy Hylton
provoked this with a program of the form:
x = (1,
2,
... # 9997 lines deleted here
10000,
)
Programs "like this" will no longer (no matter how many lines they contain)
trigger re death. OTOH, you can now make another class of unreasonable
program that will take much longer to parse.
This will fold all ISO 8859 chars from the upper half of the
charset into the lower half, which is ...ummm.... unintened.
The second is a typo in the reference to options.escape in main().
make pygettext more compatible with GNU xgettext, specifically:
Added -E/--escape for allowing pass-thru of iso8859-1 characters above
7 bits.
Added -o/--output option for renaming the output file from
messages.pot (there's overlap with -d/--default-domain, but GNU
xgettext has them both).
Added -p/--output-dir for specifying the output directory for
messages.pot.
Added -V/--version for printing the version number.
Added -w/--width for specifying the output page width (this is because
now pygettext, like GNU xgettext will put several locations on the
same line to cut down on vertical space).
Added -x/--exclude-file for specifying a list of strings that are not
to be extracted from the input files.
Bumped version number to 1.0
Try to import fintl and use fintl.gettext as _ if available. Fall
back is to use identity definition of _().
Moved the escape creation to a function make_escapes() so that its
behavior can be controlled by the -E option.
__openseen(): Support the -x option.
write(): Support -w option and vertical space preserving feature.
main(): Support new options.
standard library. Added some comments:
# XXX Note: this is now a standard library module.
# XXX The API needs to undergo changes however; the current code is too
# XXX script-like. This will be addressed later.
of the current module. It also runs the tabnanny to catch any
inconsistent tabs.
Also did a little bit of refactoring: added an errorbox() method to
simplify the display of error dialogs.
- removed now (happily) unused second arg
- need to verify results of [].index are correct; for building consts,
need to have same value and same type, e.g. 2 not the same as 2L
(big surprise). new solution is a little less hackish.
Code gen adds a TupleArg instance in the argument slot. The tuple arg
includes a copy of the names that it is responsble for binding. The
PyAssembler uses this information to calculate the correct argcount.
all fix this wacky case: del (a, ((b,), c)), d
which is the same as: del a, b, c, d
(Can't wait for Guido to tell me why.)
solution uses findOp which walks a tree to find out whether it
contains OP_ASSIGN or OP_DELETE or ...
- added a number of support methods to generate code just before the
body
- hack protocol for communicating number of args to PyAssembler
fix TryExcept generation for case where exception handler has no body
fix visitAssAttr
add comment about incomplete visitAssName
stop using the ExampleASTVisitor
change script invocation to accept a list of .py files (e.g. Lib/*.py)
named OPTIMIZED, which matches compile.c and makes more sense for
classes
revamp call signature for PythonVMCode.__init__; doesn't really matter
'cuz this code is going to be refactored out of existence
add generateClassCode and modify Func & Lambda generation
add support for nodes Classdef, Keyword,
fix CallFunc to generate right op arg when calling w/ keywords
add ugly hack to properly compute offsets when the same stack ref is
used multiple times
change resolution of local name ops (LOAD_FAST). i think it makes
sense now. if it is an argument or a local var name that it used, it
must be in varnames. if it is a local var name that is used, it must
also be in names
FUNCTION_NAMESPACE. initialize in __init__ and reset in
generateFunctionCode.
replace direct issue of STORE_FAST, STORE_GLOBAL, etc. with call to
storeName; same for loadName and deleteName
the new {store,load,delete}Name methods use the namespace attr and the
local variable stack to determine the correct bytecode to issue
* prints out examples of nodes that are handled by visitor. simply a
development convenience
remove NestedCodeGenerator -- it was bogus after all
replace with generateFunctionCode, a method to call to generate code
for a function instead of a top-level module
fix impl of visitDiscard (most pop stack)
emit lineno for pass
handle the following new node types: Import, From, Getattr, Subscript,
Slice, AssAttr, AssTuple, Mod, Not, And, Or, List
LocalNameFinder: remove names declared as globals for locals
PythonVMCode: pass arg names to constructor, force varnames to contain
them all (even if they aren't referenced)
add -q option on command line to disable stdout
VERBOSE setting for the ASTVisitor
add getopt handling for one or more -v args
rename ForwardRef to StackRef, because it isn't necessarily directional
CodeGenerator:
* add assertStackEmpty method. prints warning if stack is not empty
when it should be
* define methods for AssName, UNARY_*, For
PythonVMCode:
* fix mix up between hasjrel and hasjabs for address calculation
language.
CodeGenerator:
* modify to track stack depth
* add emit method that call's PythonVMCode's makeCodeObject
* thread filenames through in hackish way
* set flags for code objects for modules and functions
XXX the docs for the flags seem out of date and/or incomplete
PythonVMCode:
* add doc string describing the elements of a real code object
LineAddrTable:
* creates an lnotab (no quite correctly though)
handle most of the language syntax yet)
create NestedCodeGenerator used to generator the separate code object
that needs to be passed as an argument to MAKE_FUNCTION when a def
stmt is found (probably useful for class too)
change CodeGenerator.visitFunction to use the NestedCG
add CompiledModule class to handle creation of .pyc (pretty minimal
for now)
add makeCodeObject method to PythonVMCode that replaces symbolic names
with indexes into slots of the code code. the design of this
class will probably need to be revised.
compile.py: ASTVisitor framework plus bits of a code generator that
should be bug-for-buf compatible with compile.c
misc.py: Set and Stack helpers
test.py: a bit of simple sample code that compile.py will work on
Three bgen mods:
- support for FSSpecs passed-by-value and points-passed-by-reference added.
- strip single-line comments when parsing header files
- if a definition is blacklisted _do_ output it, but in comment
- same trick with "import wcnew; webchecker = wcnew" as above
- updated readhtml() method to handle pair representation; used
new name suppression infrastructure from wcnew.py to suppress
processing name anchors
[And untabified --GvR]
- added -t and -a arguments
- added "import wcnew; webchecker = wcnew" in place of "import
webchecker" (I assume that if you're happy with the changes, you'll
just replace webchecker.py with wcnew.py, but if I were to do that,
the diffs would be incomprehensible)
- fixed buggy -v argument (I think you got out of sync with the
way verbosity was handled in webchecker vs. wcgui between 1.5 and
1.5.2)
- made -v actually do something by adding a call to c.setflags()
(probably the same problem as above)
- updated references to URLs to accommodate wcnew.py's pair
representation; added appropriate calls to format_url() to handle
display; added argument to ListPanel() initialization to provide
access to format_url()
[And untabified --GvR]
- same fixes from webchecker.py
- incorporated small diff between current webchecker.py and 1.5.2
- fixed bug where "extra roots" added with the -t argument were being
checked as real roots, not just as possible continuations
- added -a argument to suppress checking of name anchors
[And untabified --GvR]
- forced new done origins to set errors if they're in self.bad (fixes
bug where only the first of a number of errorful references to a
link is reported under some circumstances)
- suppressed adding duplicates to self.todo list (cleans up printout
in wcgui details)
Herzog <herzog@online.de>. Specifically,
--verbose/-v flag added
pot_header added to make msgmerge and Emacs po-mode work better
normalize(), escape(), safe_eval(): Improved normalization of strings
for more .po file compatibility (e.g. C style). Handles emmbedded
newlines better.
Also added an identity function called _() and use it in the file
where messages are printed. This allows us to selftest pygettext.py
with itself as input.
(Suggested by Moshe Zadka, but implemented differently.)
Add <<python-docs>> event which, on Unix, brings up Netscape pointing
to http://www.python.doc/current/ (a local copy would be nice but its
location can't be predicted). Windows solution TBD.
(interrupted system call) when getting the device information. I've
never seen it, but this patch should take care of the problem.
If we get that exception and we're polling, just return since we'll
wake up again soon and get the right information. If we're not
polling, try 4 times and then give up.
"""
If the filename being complained about contains a space, enclose the
file-name in quotes.
The reason is simply that when I try and parse tabnanny's output, filenames
with spaces make it very difficult to determine where the filename stops
and the linenumber begins!
"""
Tim approves.
I slightly changed the patch (use 'in' instead of string.find()) and
arbitrarily bumped the __version__ variable up to 6.
dynamically imported when Pynche is running via askcolor out of a
package. If the ImportError occurs, try again, prepending the package
name and digging out the module.
user what they want first if there's unsaved stuff, and may cancel).
It closes more than before.
Add unload_extensions() method to unload all extensions; called from
_close(). It calls an extension's close() method if it has one.
"""
When there are additional Setup files, specified by -e option of freeze,
checkextenstions.py assumes that *.o, *.a, -Lpath, and -Rpath are all
relative to where the Setup file is. select() inserts the path to the
Setup file to make them absolute. However, the assumption is not true.
There are cases that absolute paths are specified for them. The inserted
prefix, by select(), results in error.
The following fix check for absolute paths. The assumption is: an
absolute path begins with either '/' or '$'. In the latter case, it is
from the environmental variable. (Variables defined locally in the Setup
file have already been handled by expandvars())
"""
My version of the patch has been verified by Charles Waldman (a
colleague of Chih-Hao).
Toby writes:
winmakemakefile.py tries to allow for spaces in the python install
path, by adding quotes around the appropriate filenames. It doesn't
quite get this correct; sometimes the quotes end up in the middle of
the path.
Microsoft's NMAKE version 6.0 is happy with this (!!!!) unless there
is also a space in the name. I guess most users of freeze on windows
do not use the same path as the binary distribution.
I've tested the following changes on systems with and without a space
in the path.
EditorWindow.py:
+ Added get_tabwidth & set_tabwidth "virtual text" methods, that get/set the
widget's view of what a tab means.
+ Moved TK_TABWIDTH_DEFAULT here from AutoIndent.
+ Renamed Mark's get_selection_index to get_selection_indices (sorry, Mark,
but the name was plain wrong <wink>).
FormatParagraph.py: renamed use of get_selection_index.
AutoIndent.py:
+ Moved TK_TABWIDTH_DEFAULT to EditorWindow.
+ Rewrote set_indentation_params to use new VTW get/set_tabwidth methods.
+ Changed smart_backspace_event to delete whitespace back to closest
preceding virtual tab stop or real character (note that this may require
inserting characters if backspacing over a tab!).
+ Nuked almost references to the selection tag, in favor of using
get_selection_indices. The sole exception is in set_region, for which no
"set_selection" abstraction has yet been agreed upon.
+ Had too much fun using the spiffy new features of the format-paragraph
cmd.
this functionality is not present (e.g. when used with a vintage
Python 1.5.2 installation) top-level functions are not listed.
(Hmm... Any distribution of IDLE 0.5 should probably include a copy
of the new pyclbr.py!)
+ Set usetabs=1. Editing pyclbr.py was driving me nuts <0.6 wink>.
usetabs=1 is the Emacs pymode default too, and thanks to indentwidth !=
tabwidth magical usetabs disabling, new files are still created with tabs
turned off. The only implication is that if you open a file whose first
indent is a single tab, IDLE will now magically use tabs for that file (and
set indentwidth to 8). Note that the whole scheme doesn't work right for
PythonWin, though, since Windows users typically set tabwidth to 4; Mark
probably has to hide the IDLE algorithm from them (which he already knows).
+ Changed comment_region_event to stick "##" in front of every line. The
"holes" previously left on blank lines were visually confusing (made it
needlessly hard to figure out what to uncomment later).
Smarter logic for finding a parse synch point.
Does a half to a fifth the work in normal cases; don't notice the speedup,
but makes more breathing room for other extensions.
Speeds terrible cases by at least a factor of 10. "Terrible" == e.g. you put
""" at the start of Tkinter.py, undo it, zoom to the bottom, and start
typing in code. Used to take about 8 seconds for ENTER to respond, now some
large fraction of a second. The new code gets indented correctly, despite
that it all remains "string colored" until the colorizer catches up (after
which, ENTER appears instantaneous again).
IDLE is now the first Python editor in the Universe not confused by my
doctest.py <wink>.
As threatened, this defines IDLE's is_char_in_string function as a
method of EditorWindow. You just need to define one similarly in
whatever it is you pass as editwin to AutoIndent; looking at the
EditorWindow.py part of the patch should make this clear.
stuff for getting the tip. Had to fix the Key-( and Key-) events
for Unix. Will have to re-apply my patch for catching KeyRelease and
ButtonRelease events.
Unix bindings for <<toggle-tabs>> and <<change-indentwidth>> were
missing, and somehow that meant the events were never generated,
even though they were in the menu. The new Unix bindings are now
the same as the Windows bindings (M-t and M-u).
The new version (attached) is fast enough all the time in every real module
I have <whew!>. You can make it slow by, e.g., creating an open list with
5,000 90-character identifiers (+ trailing comma) each on its own line, then
adding an item to the end -- but that still consumes less than a second on
my P5-166. Response time in real code appears instantaneous.
Fixed some bugs.
New feature: when hitting ENTER and the cursor is beyond the line's leading
indentation, whitespace is removed on both sides of the cursor; before
whitespace was removed only on the left; e.g., assuming the cursor is
between the comma and the space:
def something(arg1, arg2):
^ cursor to the left of here, and hit ENTER
arg2): # new line used to end up here
arg2): # but now lines up the way you expect
New hack: AutoIndent has grown a context_use_ps1 Boolean config option,
defaulting to 0 (false) and set to 1 (only) by PyShell. Reason: handling
the fancy stuff requires looking backward for a parsing synch point; ps1
lines are the only sensible thing to look for in a shell window, but are a
bad thing to look for in a file window (ps1 lines show up in my module
docstrings often). PythonWin's shell should set this true too.
Persistent problem: strings containing def/class can still screw things up
completely. No improvement. Simplest workaround is on the user's head, and
consists of inserting e.g.
def _(): pass
(or any other def/class) after the end of the multiline string that's
screwing them up. This is especially irksome because IDLE's syntax coloring
is *not* confused, so when this happens the colors don't match the
indentation behavior they see.
[Tim, after adding some bracket smarts to AutoIndent.py]
> ...
> What it can't possibly do without reparsing large gobs of text is
> suggest a reasonable indent level after you've *closed* a bracket
> left open on some previous line.
> ...
The attached can, and actually fast enough to use -- most of the time. The
code is tricky beyond belief to achieve that, but it works so far; e.g.,
return len(string.expandtabs(str[self.stmt_start :
^ indents to caret
i],
^ indents to caret
self.tabwidth)) + 1
^ indents to caret
It's about as smart as pymode now, wrt both bracket and backslash
continuation rules. It does require reparsing large gobs of text, and if it
happens to find something that looks like a "def" or "class" or sys.ps1
buried in a multiline string, but didn't suck up enough preceding text to
see the start of the string, it's completely hosed. I can't repair that --
it's just too slow to reparse from the start of the file all the time.
AutoIndent has grown a new num_context_lines tuple attribute that controls
how far to look back, and-- like other params --this could/should be made
user-overridable at startup and per-file on the fly.
One new file in the attached, PyParse.py. The LineStudier (whatever it was
called <wink>) class was removed from AutoIndent; PyParse subsumes its
functionality.
Removed "New tabwidth" menu binding.
Added "a tab means how many spaces?" dialog to block tabify and untabify. I
think prompting for this is good now: they're usually at-most-once-per-file
commands, and IDLE can't let them change tabwidth from the Tk default
anymore, so IDLE can no longer presume to have any idea what a tab means.
Irony: for the purpose of keeping comments aligned via tabs, Tk's
non-default approach is much nicer than the Emacs/Notepad/Codewright/vi/etc
approach.
2. No longer need to reset pyclbr cache and show watch cursor when calling
ClassBrowser -- the ClassBrowser takes care of pyclbr and the TreeWidget
takes care of the watch cursor.
3. Reset the focus to the current window after error message about class
browser on buffer without filename.
I'm still unsure, but couldn't stand the virtual event trickery so tried a
different sin (adding undo_block_start/stop methods to the Text instance in
EditorWindow.py). Like it or not, it's efficient and works <wink>. Better
idea?
Give the attached a whirl. Even if you hate the implementation, I think
you'll like the results. Think I caught all the "block edit" cmds,
including Format Paragraph, plus subtler ones involving smart indents and
backspacing.
[W]hile trying to dope out how redirection works, stumbled into two
possible glitches. In the first, it doesn't appear to make sense to try to
rename a command that's already been destroyed; in the second, the name
"previous" doesn't really bring to mind "ignore the previous value" <wink>.
You can switch database by just loading the new one; the list window
and nearest colors adapt to the new database.
Some reorganizing of code. Also, the name of the database file is
stored in the ~/.pynche pickle. If it can't be loaded, fallbacks are
used.
Ho ho ho -- that's trickier than it sounded! The colorizer is working with
"line.col" strings instead of Text marks, and the absolute coordinates of
the point of interest can change across the self.update call (voice of
baffled experience, when two quick backspaces no longer fooled it, but a
backspace followed by a quick ENTER did <wink>).
Anyway, the attached appears to do the trick. CPU usage goes way up when
typing quickly into a long triple-quoted string, but the latency is fine for
me (a relatively fast typist on a relatively slow machine). Most of the
changes here are left over from reducing the # of vrbl names to help me
reason about the logic better; I hope the code is a *little* easier to
there are two new commands:
Import module (F5) imports or reloads the module and also adds its
name to the __main__ namespace. This gets executed in the PyShell
window under control of its debug settings.
Run script (Control-F5) is similar but executes the contents of the
file directly in the __main__ namespace.
idle.py -e file ... -- to edit files
idle.py script arg ... -- to run a script
idle.py -c cmd arg ... -- to run a command
Other options, see also the usage message (also new!) for more details:
-d -- enable debugger
-s -- run $IDLESTARTUP or $PYTHONSTARTUP
-t title -- set Python Shell window's title
sys.argv is set accordingly, unless -e is used.
sys.path is absolutized, and all relevant paths are inserted into it.
Other changes:
- the environment in which commands are executed is now the __main__ module
- explicitly save sys.stdout etc., don't restore from sys.__stdout__
- new interpreter methods execsource(), execfile(), stuffsource()
- a few small nits
dramatically (up to 15 times he claims). Works by reading more than
one line at a time, up to 100-line chunks (starting with one line and
then doubling up to the limit). On a typical machine (e.g. Tim's
P5-166) this doesn't reduce interactive responsiveness in a noticeable
way.
unsuccessful search wraps around and re-searches that part of the file
between the start of the search and the end of the file - only really
an issue for very large files, but... (also removes a redundant
m.span() call).
Guido van Rossum
Jeremy Hylton
Barry Warsaw