the Python Unicode implementation.
The internal buffer used for implementing the buffer protocol
is renamed to defenc to make this change visible. It now holds the
default encoded version of the Unicode object and is calculated
on demand (NULL otherwise).
Since the default encoding defaults to ASCII, this will mean that
Unicode objects which hold non-ASCII characters will no longer
work on C APIs using the "s" or "t" parser markers. C APIs must now
explicitly provide Unicode support via the "u", "U" or "es"/"es#"
parser markers in order to work with non-ASCII Unicode strings.
(Note: this patch will also have to be applied to the 1.6 branch
of the CVS tree.)
Montanaro, handle execution of indented regions by inserting an "if
1:" in front of the block. This better preserves things like triple
quoted strings and commented regions. This patch resolves PR#264.
1.5.2 was released, except those who contributed only to Doc files --
Fred has his own way of doing this.
This doesn't mean that I've got everyone who contributed *before*
1.5.2 was released in here... :-(
executive summary:
Instead of typing 'apply(f, args, kwargs)' you can type 'f(*arg, **kwargs)'.
Some file-by-file details follow.
Grammar/Grammar:
simplify varargslist, replacing '*' '*' with '**'
add * & ** options to arglist
Include/opcode.h & Lib/dis.py:
define three new opcodes
CALL_FUNCTION_VAR
CALL_FUNCTION_KW
CALL_FUNCTION_VAR_KW
Python/ceval.c:
extend TypeError "keyword parameter redefined" message to include
the name of the offending keyword
reindent CALL_FUNCTION using four spaces
add handling of sequences and dictionaries using extend calls
fix function import_from to use PyErr_Format
The attached patch set includes a workaround to get Python with
Unicode compile on BSDI 4.x (courtesy Thomas Wouters; the cause
is a bug in the BSDI wchar.h header file) and Python interfaces
for the MBCS codec donated by Mark Hammond.
Also included are some minor corrections w/r to the docs of
the new "es" and "es#" parser markers (use PyMem_Free() instead
of free(); thanks to Mark Hammond for finding these).
The unicodedata tests are now in a separate file
(test_unicodedata.py) to avoid problems if the module cannot
be found.
Attached you find the latest update of the Unicode implementation.
The patch is against the current CVS version.
It includes the fix I posted yesterday for the core dump problem
in codecs.c (was introduced by my previous patch set -- sorry),
adds more tests for the codecs and two new parser markers
"es" and "es#".
Attached you find an update of the Unicode implementation.
The patch is against the current CVS version. I would appreciate
if someone with CVS checkin permissions could check the changes
in.
The patch contains all bugs and patches sent this week and also
fixes a leak in the codecs code and a bug in the free list code
for Unicode objects (which only shows up when compiling Python
with Py_DEBUG; thanks to MarkH for spotting this one).
(python): Set defgroup :prefix to "py-" to make variable names cleaner.
(py-jpython-command, py-jpython-command-args): Set :tag for proper
capitalization of JPython in variable name display.
first time a py buffer is visited during the Emacs session. This
ensures that py-which-shells is initialized and also guarantees that
the mode lines reflect the correct shell. First bug found by GvR,
second one has long bugged :) me.
(py-toggle-shells): Programmatically, arg can also take the symbols
`cpython' or `jpython', which makes it easy to call with the value of
py-default-interpreter.
(py-shell): Don't need to initialize py-which-* variables since these
will guarantee to be initialized by python-mode when the first py
buffer is visited.
(py-default-interpreter): Update docstring.
casing when py-honor-comment-indentation is nil, but this could be a
religious issue with some. Seems to me we should still be dedenting
such comment lines one level.
buffer-syntactic-context -- just short circuit the TQS test by jumping
to point-min and doing the test from there. For long files, this will
be faster than looping with a re-search-backwards.
I don't know what its origins are but I think I've seen it
once in a NeXT dictionary application -- not sure whether
anyone owns copyright but I don't see why we should risk it.
py-newline-and-indent. These ought to get picked up by the mapcar
that follows; any existing binding to newline-and-indent gets shadowed
to py-newline-and-indent.
This will break some people who, e.g. bind C-m or C-j to newline but
still want these bound to py-newline-and-indent in Python mode. On
the other hand, the forced binding pisses off Emacs diehards. So
consider this experimental and see if any tall Dutch guys complain :-)
standard narrow-to-defun but works with Python classes and methods.
With no arg, narrows to most enclosing def/method. With C-u arg,
narrows to most enclosing class.
string we find ourselves in, based on the passed in delimiter.
(py-compute-indentation): Fixes for indentation errors when we land
inside a triple quoted string. For example:
def foo():
if os.path.isfile(o_pri_mbox_file) and os.path.isfile(o_pub_mbox_file):
print """\
I found both a private and a public mbox archive file
private: %s
public : %s
I won't move either file, but you should choose one and move it to
%s
You may want to merge them manually, but be careful about exposing private
correspondences to the public.""" % (
o_pri_mbox_file, o_pub_mbox_file, mbox_file)
*----indentation would be wrong on this line.
#simple things. First step: rename the Imenu supportive variables and
#functions in this file to py-imenu-* so I can grok what is part of
#python-mode and what is part of Imenu.
(py-imenu-create-index-engine): Fixed problem with two classes in a
single file, caused by new semantics of py-beginning-of-def-or-class
when called programmatically.
#Note, there are still some problems with Imenu when arguments to
#functions are funky, but it should be much better now.
string in the argument to execfile() so a Windows temp directory
named, e.g. c:\\tmp doesn't get interpreted as a file name with an
embedded tab! (given by C. Waldman).
this string should not end with whitespace.
(py-compute-indentation): Append whitespace regexp to
py-block-comment-prefix so that any combination of intervening
whitespace will be recognized.
change error messages to be a little more straightforward
change definition of FULL_PATH so that an error is raised if the
setuid wrapper is used un-edited
shell buffers.
(py-shell): Moved the require of comint to the top level. Also
use-local-map py-shell-map instead of hacking on the comint-mode-map.
This eliminates breakage of other comint-mode buffers (e.g. shell).
interactions with newer Emacsen, I've rewritten the way all the
process filters work in the *Python* buffer. We use more of the
comint infrastructure, specifically the default process filter. This
means that scrolling is now handled by the default comint variables
including comint-scroll-to-bottom-on-output. Note that this is
somewhat experimental change!
(py-comint-output-filter-function): Moved to here from the obsolete
py-process-filter function, the logic to pop and exec the next queued
file waiting to be executed.
(py-execute-file): Don't bind comint-scroll-to-bottom-on-output to t,
and save the excursion when inserting the "working on" message. This
lets the standard comint scrolling variables as set by the user,
continue to work.
(python-mode, py-shell, py-describe-mode): Remove description of
py-scroll-process-buffer. Also in py-shell, make
comint-output-filter-functions buffer-local, and add
py-comint-output-filter-function to this hook (instead of setting the
process filter).
(py-scroll-process-buffer): Deleted this variable. See comint
variables including comint-scroll-to-bottom-on-output.
(py-execute-region): When exec files are being queued, push the next
temp file on the end of the list.
(py-submit-bug-report): Removed reporting of py-scroll-process-buffer.
"3.67 fixes Imenu as far as classes are concerned, but some default
values for function arguments are still not supported."
This ought to fix that problem.
indetnation of normal statements: The regular expression that searches
for indenting comment lines has been changed to not require a
space/tab after the first `#'. We then explicitly look for
py-block-comment-prefix depending on the value of
py-honor-comment-indentation.
I think this more accurately reflects the documentation for
py-honor-comment-indentation.
conformations, etc., etc. inspired and given by Michael Ernst. These
include error string fixes, moving of comments to docstrings, some
other non-related typos, terminology standardizing (b/w TP and myself,
and b/w myself and myself :-) although more can still be done.
E.g. "outdenting" => "dedenting".
serial number isn't enough to uniquify the temp file name -- what if
two users are on the same machine? Add in the (emacs-pid) to help
further. Should never be tickled on Emacs 20, XEmacs 20, 21.
py-mark-def-or-class): Integrated Michael Ernst latest patches.
Primarily, it allows functions that search or mark defs/classes based
on programmatic specification, to take an 'either flag value which
allows searching for both classes and defs (stopping at the nearest
construct).
Also clean up some docstrings.
comment-indent-function's default lambda value (in simple.el), this
version finally kills this nit: auto-filling a comment that starts in
column zero with filladapt turned off would cascade the #'s to the
right.
Now auto-filling seems to work with or without filladapt, and with the
comment starting in any column.
(python-mode): Set comment-indent-function.
(py-execute-import-or-reload): Cool new command that imports or
reloads the current file as a module, so as not to clutter the global
namespace. Bound to C-c C-m.
(py-execute-def-or-class): New command that sends the current def or
class to the interpreter. Bound to C-M-x.
(py-execute-string): New command that sends arbitrary string to the
interpreter. Not bound by default.
(py-describe-mode): Doco updates.
py-beginning-of-def-or-class, and defaliased for backwards
compatibility. ME patch to add optional second argument, count.
(end-of-python-def-or-class): Renamed to py-end-of-def-or-class, and
defaliased for backwards compatibility. ME patch to add optional
second argument, count.
(py-shell): Recognize the Python debugger prompt
(py-jump-to-exception): Force into python-mode any buffer that gets
jumped to on exception. Cope with py-exception-buffer possibly a
cons.
JPython interpreters. This implementation may suck.
(py-jpython-command, py-jpython-command-args): New variables.
(py-mode-map): py-toggle-shells bound to C-c C-t
(py-toggle-shells): Command to toggle between using CPython (the
default) and JPython. This is buffer local, and notice the mode-name
change.
(py-shell): Use either CPython or JPython. Note that py-execute-*
still needs to be modified.
an open paren, do a better job of reindenting the line. For example:
def foo():
print 'hello %s, %d' % (
a, b)
Hit TAB on the line starting with `a'. Without this patch this line
will never be reindented.
(py-compute-indentation): int-to-char isn't defined in Emacs, but we
don't really need it anyway, so just remove this conversion. XEmacs
is happy either way.
(py-parse-state): The Emacs branch (i.e. w/o buffer-syntactic-context)
wasn't adjusting point correctly.
otherwise return nil.
(py-execute-region): When executing the buffer asynchronously in a
subprocess, if an exception occurred, show both the output buffer and
the file containing the exception, leaving point on the source line
containing bottom-most error in the traceback. If no exception
occurred, jump to the output buffer (no change).
from CC Mode.
(py-guess-indent-offset): Teach it about colons in `literals'
(e.g. comments and strings). Don't false hit colons in literals; keep
searching for a real block introducing line.
of a line in py-tab-face to aid in seeing mixed tab/space indentation.
This face defaults to the `default' face so it is unobtrusive until
you `M-x customize-face' py-tab-face to something obnoxious like
"Yellow".
setting of py-indent-offset and indent-tabs-mode.
(python-mode): After python-mode-hook is run, do the automagic
calculation if py-smart-indentation is non-nil.
(py-parse-state): Get rid of unused variable to quiet the
byte-compiler.
over and around triple-quoted strings:
- move the beginning-of-line to above the p-p-s call
- in the `t' clause of the big cond, where we skip over
triple-quoted strings, first find out if we're looking at a
single or TQS, then skip over it in one fell swoop, instead of
trying to loop over skipage of SQS's.
(py-parse-state): Implement XEmacs only hack to more accurately figure
out whether we're in a string or not. Can't do this in Emacs because
it lacks the necessary primitive, so we just do it the old (and mostly
accurate, but foolable) way for Emacs.
continuation lines. This fixes this bug report, reported by Frank
Stajano.
# But if I split the "raise" line and reindent, the else WRONGLY goes up a
# level (?!?)
while condition1:
if condition2:
raise error3, \
moreInfo4
else: # meant to close "if condition2"
action5()
a religious issue: RMS decrees that the Enter (RET) key should just do
a newline and a LFD (C-j) should do a newline and indent (i.e. the
python-mode version of this). Almost everyone I know disagrees and
finds that RET should do newline and indent. Almost everyone hacks
their modes to do this, if they know how. Because it's hard for
newbies to figure out how to do this, and because most DOS keyboards
lack a LFD (leaving users to the more obscure C-j), I think it makes
better sense to add this default binding.
is based on the line above, watch out for landing inside a triple
quoted string. In this case, use iterative search +
parse-partial-sexp backwards to find the beginning of the string.
Note this does affect performance, but very little in the common cases
(I hope). It could be made *much* faster by adding Emacs and XEmacs
dependent code -- different code naturally. :-(
Fixes the following reported bug:
if len(sys.argv) >= 6:
# More lines here
fptr = open('/etc/hosts', 'w')
fptr.write("""# /etc/hosts -- autocreated by /etc/ppp/ip-up
#
# Address from pppd
%-15s %s
# For loopbacking
127.0.0.1 localhost
255.255.255.255 broadcast
""" % (ipaddr, ipname) )
os.chmod('/etc/hosts', 0644)
file local variable section of a file. When set, and the user hits
C-c C-c, this file gets executed instead of the buffer's file. Idea
given by Roy Dragseth <royd@math.uit.no>, but implemented differently.
(py-execute-buffer): Support py-master-file variable. If this names a
relative path, default-directory is prepended via expand-file-name.
(python-mode): Conditionalize imenu initializations to when we can
safely require imenu. Under Emacs this should prevent python-mode
from hosing the global value of imenu-create-index-function and
messing things up for all other modes. Problem identified by
Christian Egli.
(py-describe-mode): py-delete-char => py-electric-backspace. Given by
Christian Egli.
(imenu-example--create-python-index-engine): Use
buffer-substring-no-properties. Also, don't use
imenu-create-submenu-name. Apparently it is obsolete.
These Imenu patches were given by Christian Egli
<christian.egli@stest.ch>
py-keep-region-active so that the marked def/class gets the
zmacs-region or transient-mark region highlighted. Also point should
be left at the end of the marked region.
(py-mode-map): Moved py-mark-def-or-class to M-C-h to conform to Emacs
major mode standards.
to the Web site.
(py-defun-start-re, py-class-start-re): Changed to defconst.
(py-traceback-line-re): Regular expression describing what traceback
lines look like.
(py-point): New defsubst copied from CC Mode.
(py-highlight-line): Function which does the work of making a
traceback line mouseable. This only works on XEmacs. Someone familar
with Emacs text properties and such will have to do that port.
(py-mode-map): Added C-c- bound to py-up-exception and C-c= bound to
py-down-exception. Also, more concise form for mapcar.
(py-mode-output-map): New keymap for the *Python Output* buffer which
only has keybindings for py-mouseto-exception and py-goto-exception.
All other self-insert-command's are bound to beep. This is actually
bogus because the buffer should really be made read-only and the
functions that insert in that buffer should bind inhibit-read-only.
Also, this map should be bound to highlighted extents in a *Python*
shell buffer, but this stuff hasn't been migrated into there.
(py-postprocess-output-buffer): New function which extentifies the
*Python Output* buffer. The bogosities are that this only runs when
the synchronous process in the buffer is finished (so it doesn't work
for async procs), and it should also be merged into py-process-filter
so the *Python* shell gets mouseable too.
(py-shell): Added C-c- and C-c= to the comint buffer's keymap. The
bogosity is that py-goto-exception should also be bound, but it cannot
be bound to C-cC-c (since that interferes with
comint-interrupt-subjob's typical binding). Also, traceback lines
aren't mouseable in this buffer.
(py-execute-region): Support for traceback jumping. This really is
quite a kludge, but necessary based on the way all this stuff works.
There's bound to be broken interactions here.
(py-jump-to-exception, py-mouseto-exception, py-goto-exception,
py-find-next-exception, py-down-exception, py-up-exception): All new
commands and functions to implement traceback jumping.
(py-compute-indentation): Hope this change doesn't get lost in all the
noise above!!!! This fixes broken non-indentation of a line when TAB
is hit inside a string that isn't a multi-line string.
immediately following colons. Sjoerd noticed this one too. Here's a
nonsense.py file that flexes all the font-lock keyword combinations.
class A:
class B(A):
pass
def __init__(self):
if i == 2 and j == 3 or k == 4:
import stuff
from otherstuff import cool
for i in range(cool.count):
if i == j:
break
elif j == 1:
continue
print i
else:
return not i
elif q is not i:
return lambda x: x + 1
else:
try:
try:
raise stuff.error
except stuff.error, v:
print v
except:
global q
finally:
while q > 0:
q = q - 1
assert q == 0
def make():
a = A()
exec "nonsense"
del a
on NTEmacs 19.34.6.
(py-serial-number): New variable.
(py-execute-region): If make-temp-name is broken, simply append a
serial number to the string "python-" to get a temporary file name.
It's possible concurrent NTEmacs can step on each others toes, but it
makes no sense to further coddle a busted NTEmacs.
(py-mode-map): Moved py-mark-def-or-class from M-C-h to C-c C-m since
the old binding conflicts with the standard global backward-kill-word
binding, and this new binding is more conformant with other language
modes. Moved py-mark-block to C-c C-k.
(py-electric-backspace, py-electric-delete): Support the XEmacs 20 Way
for backspace and delete mappings. In XEmacs 19, Emacs 19, and Emacs
20, both backspace and delete keysyms are bound to
py-electric-backspace. In XEmacs 20, backspace and delete keysyms are
bound separately, allowing the user to specify forward or backward
deletion of the delete keysym through the variable
delete-key-deletes-forward. All this is the Right Way To Do It and
this implementation was largely ripped from CC Mode.
(py-execute-file): Better interaction with comint. Set
comint-scroll-to-bottom-on-output to t. Wrapper buffer change in
unwind-protect in case process filter fails.
(py-shell): Start Python with -i flag to fix tty problem on Windows;
presumably -- not yet tested.
(py-clear-queue): New function to clear the pending exec file queue.
Not currently keybound.
(py-execute-region, py-execute-buffer): Added optional async flag (use
via C-u prefix) to execute the region in a new asynchrous buffer, even
if the Python shell is running.
(py-append-to-process-buffer): Removed as obsolete. Comint provides
this functionality.
Removed fbound test defun of match-string. All modern X/Emacsen have
this function.
his *own* RCS file for python-mode.el, and I've agreed that it would
be better if his version was in the Python source tree. However I
don't want to totally get rid of the old RCS file (which has
interesting info such as which version was in which Python release).
So I've moved the old one to python-mode-old.el behind the scenes,
and this checkin message indicates that I'm now deleting it.
If you do an update, you will actually get Barry's *new* version!
Introductory comment updates.
(python-font-lock-keywords): Added "assert"
(py-block-closing-keywords-re): New variable.
(py-no-outdent-re): Rewrite to use py-block-closing-keywords-re.
(py-shell): py-process-filter should no longer be necessary. Comint
should do all the work. Note that more fixes to the py-shell process
mechanism need to be done.
(py-execute-region): Check for empty region. Some questionable
changes to set-buffer after shell-command-on-region. Again, this all
needs to be closely examined for X/Emacs 19/20 compatibility.
(py-goto-beyond-final-line): py-parse-partial-sexp-works-p should no
longer be necessary.
(py-statement-closes-block-p): Use py-block-closing-keywords-re.
Currently fairly minimal, but I'll be adding to this as needed. I
think it's pretty darn close.
To use this, just load the file and in a C buffer type:
M-x c-set-style RET python RET
[there are ways to automate much of this!]
defeat extra outdentation for block closing statements (return, raise,
break, continue, pass).
(py-compute-indentation): extra argument to honor block closing
statements.
(py-electric-colon, py-indent-region): use py-compute-indentation's
extra argument
(py-statement-closes-block-p): `pass' treated as a block closing
statement.
add-change-log-entry-other-window is so bad about guessing the proper
name of Python functions, methods and variables, so finally I wrote
the following (unidiff patch against python-mode.el 2.73):
Per Cederqvist <ceder@signum.se>
right thing.
(py-comment-region): let-bind comment-start to "## " so commented
regions get transformed into non-indenting comment lines.
(py-compute-region): Implement modification to rule for recognizing
"indenting comment lines".
py-outdent-left, py-mode-map): Folded all functionality into
py-shift-region-* commands. Bound C-c C-l to py-shift-region-left and
C-c C-r to py-shift-region-right. Removed py-indent-right and
py-indent-left.
py-submit-bug-report placed on C-c C-b, and py-version placed on C-c
C-v.
(py-version, py-submit-bug-report): new functions
(py-version, py-help-address): new variables
Guido van Rossum
Jeremy Hylton
Barry Warsaw
Marc-André Lemburg
Peter Schneider-Kamp
Vladimir Marangozov
Fred Drake
Greg Ward
Jack Jansen