Expand set of errors caught in set_context(). Some new errors, some
old error messages changed for consistency.
Fixed error checking in generator expression code. The first set of
tests were impossible condition given the grammar. In general, the
ast code uses REQ() for those sanity checks.
Fix some error handling for augmented assignments. As comments in the
code explain, set_context() ought to work here, but I got unexpected
crashes when I tried it. Should come back to this.
Add note to Grammar that yield expression is a special case.
Add doctest cases for SyntaxErrors raised by ast.c.
Squash new warnings from VC 7.1 about mixing signed and
unsigned types in comparisons. I can see why `len` was
changed to size_t here, but don't see why `i` was also
changed. Change `i` back to int.
This code generated a C assertion:
assert 1, ([s for s in x] +
[s for s in x])
pass
assert was completely broken, it needed to use the proper block.
compiler_use_block() is now no longer used, so remove it.
Add C API function Py_GetBuildNumber(), add it to the interactive prompt
banner (i.e. Py_GetBuildInfo()), and add it as the sys.build_number
attribute. The build number is a string instead of an int because it may
contain a trailing 'M' if there are local modifications.
Strip off leading dots and slash so the generated files are the same regardless
of whether you configure in the checkout directory or build.
If anyone configures in a different directory, we might want a cleaner
approach using os.path.*(). Hopefully this is good enough.
If a line had multiple semi-colons and ended with a semi-colon, we would
loop too many times and access a NULL node. Exit the loop early if
there are no more children.
In C++, it's an error to pass a string literal to a char* function
without a const_cast(). Rather than require every C++ extension
module to put a cast around string literals, fix the API to state the
const-ness.
I focused on parts of the API where people usually pass literals:
PyArg_ParseTuple() and friends, Py_BuildValue(), PyMethodDef, the type
slots, etc. Predictably, there were a large set of functions that
needed to be fixed as a result of these changes. The most pervasive
change was to make the keyword args list passed to
PyArg_ParseTupleAndKewords() to be a const char *kwlist[].
One cast was required as a result of the changes: A type object
mallocs the memory for its tp_doc slot and later frees it.
PyTypeObject says that tp_doc is const char *; but if the type was
created by type_new(), we know it is safe to cast to char *.
Incorrect code was generated for:
foo(a = i for i in range(10))
This should have generated a SyntaxError. Fix the Grammar so
it raises a SyntaxError and test it.
I'm uncertain whether this should be backported. It makes
something that was Syntactically valid invalid. However,
the code would either be completely broken or do the wrong thing.
This change implements a new bytecode compiler, based on a
transformation of the parse tree to an abstract syntax defined in
Parser/Python.asdl.
The compiler implementation is not complete, but it is in stable
enough shape to run the entire test suite excepting two disabled
tests.
- SF Bug #772896, unknown encoding results in MemoryError, which is not helpful
I will only backport the segfault fix. I'll let Anthony decide if he wants
the other changes backported. I will do the backport if asked.
Fix over-aggressive PyErr_Clear(). The same code fragment appears in
various guises in list.extend(), map(), filter(), zip(), and internally
in PySequence_Tuple().
- Handle both frozenset() and frozenset([]).
- Do not use singleton for frozenset subclasses.
- Finalize the singleton.
- Add test cases.
* Factor-out set_update_internal() from set_update(). Simplifies the
code for several internal callers.
* Factor constant expressions out of loop in set_merge_internal().
* Minor comment touch-ups.
[ 1180995 ] binary formats for marshalling floats
Adds 2 new type codes for marshal (binary floats and binary complexes), a
new marshal version (2), updates MAGIC and fiddles the de-serializing of
code objects to be less likely to clobber the real reason for failing if
it fails.
[ 1181301 ] make float packing copy bytes when they can
which hasn't been reviewed, despite numerous threats to check it in
anyway if noone reviews it. Please read the diff on the checkin list,
at least!
The basic idea is to examine the bytes of some 'probe values' to see if
the current platform is a IEEE 754-ish platform, and if so
_PyFloat_{Pack,Unpack}{4,8} just copy bytes around.
The rest is hair for testing, and tests.
A problem regarding importing symlinked modules was recently reported on the
Cygwin mailing list:
http://cygwin.com/ml/cygwin/2005-04/msg00257.html
The following test case demonstrates the problem:
$ ls -l
total 1
lrwxrwxrwx 1 jt None 6 Apr 23 13:32 bar.py -> foo.py
-rw-r--r-- 1 jt None 24 Apr 18 20:13 foo.py
$ python -c 'import bar'
Traceback (most recent call last):
File "<string>", line 1, in ?
ImportError: No module named bar
Since Cygwin's case_ok() uses a modified version of the Windows's version, the
symlinked bar module actually resolves to file foo.py instead of bar.py. This
obviously causes the matching code to fail (regardless of case).
The patch fixes this problem by making Cygwin use the Mac OS X case_ok()
instead of a modified Window's version.
If we exit via the break here, we need to set ff_last_lineno or
FUTURE_POSSIBLE() will remain true. The bug affected statements
containing a variety of expressions, but not all expressions. It has
been present since Python 2.2.
Improve signal handling, especially when using threads, by forcing an early
re-execution of PyEval_EvalFrame() "periodic" code when things_to_do is not
cleared by Py_MakePendingCalls().
M Misc/NEWS
M Python/ceval.c
PyGILState_Ensure(): The fix in 2.4a3 for bug 1010677 reintroduced thread
shutdown race bug 225673. Repaired by (once again) ensuring the GIL is
held whenever deleting a thread state.
Alas, there's no useful test case for this shy bug. Four years ago, only
Guido could provoke it, on his box, and today only Armin can provoke it
on his box. I've never been able to provoke it (but not for lack of
trying!).
This is a critical fix for 2.3.5 too, since the fix for 1010677 got
backported there already and so also reintroduced 225673. I don't intend to
backport this fix. For whoever (if anyone) does, there are other thread
fixes in 2.4 that need backporting too, and I bet they need to happen first
for this patch to apply cleanly.
(Contributed by Bob Ippolito.)
This patch trims down the Python core on Darwin by making it
independent of CoreFoundation and CoreServices. It does this by:
Changed linker flags in configure/configure.in
Removed the unused PyMac_GetAppletScriptFile
Moved the implementation of PyMac_StrError to the MacOS module
Moved the implementation of PyMac_GetFullPathname to the
Carbon.File module
* Use simpler, faster two pass algorithm for markblocks().
* Free the blocks variable if not NULL and exiting without change.
* Verify that the rest of the compiler has not set an exception.
* Make the test for tuple of constants less restrictive.
* Embellish the comment for chained conditional jumps.
Peepholer could be fooled into misidentifying a tuple_of_constants.
Added code to count consecutive occurrences of LOAD_CONST.
Use the count to weed out the misidentified cases.
Added a unittest.
thread's id can't get duplicated, because (of course!) the current thread
is still running. The code should work either way, but reverting the
gratuitous change should make backporting easier, and gets the bad
reasoning out of 2.35's new comments.
can fail, check its return value, and die if it does fail.
_PyGILState_Init(): Assert that the thread doesn't already have an
association for autoTLSkey. If it does, PyThread_set_key_value() will
ignore the attempt to (re)set the association, which the code clearly
doesn't want.
code.
PyThread_set_key_value(): It's clear that this code assumes the passed-in
value isn't NULL, so document that it must not be, and assert that it
isn't. It remains unclear whether existing callers want the odd semantics
actually implemented by this function.
High level error message was stomping useful detailed messages from lower
level routines.
The new approach is to augment string error messages returned by the low
level routines. The provides both high and low level information. If
the exception value is not a string, no changes are made.
To see the improved messages in action, type:
import random
class R(random): pass
class B(bool): pass
Allows the lineno fixup code to remain simple and not have to deal with
multibyte codings.
* Add an assertion to that effect.
* Remove the XXX comment on the subject.
happen in 2.3, but nobody noticed it still was getting generated (the
warning was disabled by default). OverflowWarning and
PyExc_OverflowWarning should be removed for 2.5, and left notes all over
saying so.
* Perform the code length check earlier.
* Eliminate the extra PyMem_Free() upon hitting an EXTENDED_ARG.
* Assert that the NOP count used in jump retargeting matches the NOPs
eliminated in the final step.
* Add an XXX note to indicate that more work is being to done to
handle linenotab with intervals > 255.
* Make a pass to eliminate NOPs. Produce code that is more readable,
more compact, and a tiny bit faster. Makes the peepholer more flexible
in the scope of allowable transformations.
* With Guido's okay, bumped up the magic number so that this patch gets
widely exercised before the alpha goes out.
files and not re-optimized upon import. Saves a bit of startup time while
still remaining decoupled from the rest of the compiler.
As a side benefit, handcoded bytecode is not run through the optimizer
when new code objects are created. Hopefully, a handcoder has already
created exactly what they want to have run.
(Idea suggested by Armin Rigo and Michael Hudson. Initially avoided
because of worries about compiler coupling; however, only the nexus
point needed to be moved so there won't be a conflict when the AST
branch is loaded.)
[ 1009560 ] Fix @decorator evaluation order
From the description:
Changes in this patch:
- Change Grammar/Grammar to require
newlines between adjacent decorators.
- Fix order of evaluation of decorators
in the C (compile.c) and python
(Lib/compiler/pycodegen.py) compilers
- Add better order of evaluation check
to test_decorators.py (test_eval_order)
- Update the decorator documentation in
the reference manual (improve description
of evaluation order and update syntax
description)
and the comment:
Used Brett's evaluation order (see
http://mail.python.org/pipermail/python-dev/2004-August/047835.html)
(I'm checking this in for Anthony who was having problems getting SF to
talk to him)
because GNU/k*BSD uses gnu pth to provide pthreads, but will also happen on any
system that does the same.
python fails to build because it doesn't detect gnu pth in pthread
emulation. See C comments in patch for details.
patch taken from http://bugs.debian.org/264315
[ 1005248 ] new.code() not cleanly checking its arguments
using the result of new.code() can still destroy the sun, but merely
calling the function shouldn't any more.
I also rewrote the existing tests of new.code() to use vastly less
un-bogus arguments, and added tests for the previous insane behaviours.
hack: it would resize *interned* strings in-place! This occurred because
their reference counts do not have their expected value -- stringobject.c
hacks them. Mea culpa.
interning were not clear here -- a subclass could be mutable, for
example -- and had bugs. Explicitly interning a subclass of string
via intern() will raise a TypeError. Internal operations that attempt
to intern a string subclass will have no effect.
Added a few tests to test_builtin that includes the old buggy code and
verifies that calls like PyObject_SetAttr() don't fail. Perhaps these
tests should have gone in test_string.
[ 991812 ] PyArg_ParseTuple can miss errors with warnings as exceptions
as suggested in the report.
This is definitely a 2.3 candidate (as are most of the checkins I've
made in the last month...)
have differing refcount semantics. If anyone sees a prettier way to
acheive the same ends, then please go for it.
I think this is the first time I've ever used Py_XINCREF.
* Fixes an incorrect variable in a PyDict_CheckExact.
* Allow general mapping locals arguments for the execfile() function
and exec statement.
* Add tests.
PyImport_ReloadModule(): restore the module to sys.modules in error cases.
load_package(): semantic-neutral refactoring from an earlier stab at
this patch; giving it a common error exit made the code
easier to follow, so retaining that part.
_RemoveModule(): new little utility to delete a key from sys.modules.
__oct__, and __hex__. Raise TypeError if an invalid type is
returned. Note that PyNumber_Int and PyNumber_Long can still
return ints or longs. Fixes SF bug #966618.
The preceding case statement was missing a terminating "break" stmt,
so fell into the new code by mistake. This caused uncaught out-of-bounds
accesses to the "names" tuple, leading to a variety of insane behaviors.
expected entrypoint. The unlinking will crash the application if the
module contained ObjC code. The price of this is small: a little wasted
memory, and only in a case than isn't expected to occur often.
[ 984722 ] Py_BuildValue loses reference counts on error
I'm ever-so-slightly uneasy at the amount of work this can do with an
exception pending, but I don't think that this can result in anything
more serious than a strange error message.
[ 960406 ] unblock signals in threads
although the changes do not correspond exactly to any patch attached to
that report.
Non-main threads no longer have all signals masked.
A different interface to readline is used.
The handling of signals inside calls to PyOS_Readline is now rather
different.
These changes are all a bit scary! Review and cross-platform testing
much appreciated.
table' of the dll, to make sure that the dll really was build for the
correct Python version. It does this by looking for an entry
'pythonXY.dll' (X.Y is the Python version number).
The code now checks the size of the dll's import table before reading
entries from it. Before this patch, the code crashed trying to read
the import table when the size was zero (as in Win2k's wmi.dll, for
example).
Look for imports of 'pythonXY_d.dll' in a debug build instead of
'pythonXY.dll'.
Fixes SF 951851: Crash when reading "import table" of certain windows dlls.
Already backported to the 2.3 branch.
The builtin eval() function now accepts any mapping for the locals argument.
Time sensitive steps guarded by PyDict_CheckExact() to keep from slowing
down the normal case. My timings so no measurable impact.
Add a more informative message for the common user mistake of subclassing
from a module name rather than another class (i.e. random instead of
random.random).
(Code contributed by Jiwon Seo.)
The documentation portion of the patch is being re-worked and will be
checked-in soon. Likewise, PEP 289 will be updated to reflect Guido's
rationale for the design decisions on binding behavior (as described in
in his patch comments and in discussions on python-dev).
The test file, test_genexps.py, is written in doctest format and is
meant to exercise all aspects of the the patch. Further additions are
welcome from everyone. Please stress test this new feature as much as
possible before the alpha release.
pre-increment forms to post-increment forms. Post-incrementing
also eliminates the need for negative array indices for oparg fetches.
* In exception handling code, check for class based exceptions before
the older string based exceptions.
BINARY_SUBSCR:
* invert test for normal case fall through
* eliminate err handling code by jumping to slow_case
LOAD_LOCALS:
* invert test for normal case fall through
* continue instead of break for the non-error case
STORE_NAME and DELETE_NAME:
* invert test for normal case fall through
LOAD_NAME:
* continue instead of break for the non-error case
DELETE_FAST:
* invert test for normal case fall through
LOAD_DEREF:
* invert test for normal case fall through
* continue instead of break for the non-error case
tests of "why" against WHY_YIELD became useless. This patch removes them,
but assert()s that why != WHY_YIELD everywhere such a test was removed.
The test suite ran fine under a debug build (i.e., the asserts never
triggered).
Martin v. Löwis
Neal Norwitz
Georg Brandl
Armin Rigo
Thomas Wouters
Brett Cannon
Jeremy Hylton
Tim Peters
Hye-Shik Chang
Fredrik Lundh
Barry Warsaw
Nick Coghlan
Neil Schemenauer
Michael W. Hudson
Raymond Hettinger
Guido van Rossum
Phillip J. Eby
Jason Tishler
Anthony Baxter
Andrew MacIntyre
Fred Drake
Skip Montanaro
Kurt B. Kaiser
Matthias Klose
Jack Jansen
Thomas Heller
Nicholas Bastin