This patch enhances Python/import.c/find_module() so
that unicode objects found in sys.path will be treated
as legal directory names (The current code ignores
anything that is not a str). The unicode name is
converted to str using Py_FileSystemDefaultEncoding.
These built-in functions are replaced by their (now callable) type:
slice()
buffer()
and these types can also be called (but have no built-in named
function named after them)
classobj (type name used to be "class")
code
function
instance
instancemethod (type name used to be "instance method")
The module "new" has been replaced with a small backward compatibility
placeholder in Python.
A large portion of the patch simply removes the new module from
various platform-specific build recipes. The following binary Mac
project files still have references to it:
Mac/Build/PythonCore.mcp
Mac/Build/PythonStandSmall.mcp
Mac/Build/PythonStandalone.mcp
[I've tweaked the code layout and the doc strings here and there, and
added a comment to types.py about StringTypes vs. basestring. --Guido]
Write 4 bytes for co_stacksize, etc. to prevent writing out
bad .pyc files which can cause a crash when read back in.
(I forgot that frozen needs to be updated too for the test.)
This was a simple typo. Strange that the compiler didn't catch it!
Instead of WHY_CONTINUE, two tests used CONTINUE_LOOP, which isn't a
why_code at all, but an opcode; but even though 'why' is declared as
an enum, comparing it to an int is apparently not even worth a
warning -- not in gcc, and not in VC++. :-(
Will fix in 2.2 too.
get_file() must convert 'U' to "r" PY_STDIOTEXTMODE before calling
fopen().
imp_load_module() must accept 'r' or 'U' or something with '+'.
Also reflow some long lines.
for 'str' and 'unicode', and can be used instead of
types.StringTypes, e.g. to test whether something is "a string":
isinstance(x, string) is True for Unicode and 8-bit strings. This
is an abstract base class and cannot be instantiated directly.
The old syntax suggested that a trailing comma was OK inside backticks,
but in fact (due to ideosyncrasies of pgen) it was not. Fix the grammar
to avoid the ambiguity. Fred: you may want to update the refman.
[ 558249 ] softspace vs --disable-unicode
And #endif was in the wrong place.
Bugfix candidate, almost surely.
I think I will embark on squashing test failures in --disable-unicode builds --
a Real Bug was hiding under them.
don't understand how this function works, also beefed up the docs. The
most common usage error is of this form (often spread out across gotos):
if (_PyString_Resize(&s, n) < 0) {
Py_DECREF(s);
s = NULL;
goto outtahere;
}
The error is that if _PyString_Resize runs out of memory, it automatically
decrefs the input string object s (which also deallocates it, since its
refcount must be 1 upon entry), and sets s to NULL. So if the "if"
branch ever triggers, it's an error to call Py_DECREF(s): s is already
NULL! A correct way to write the above is the simpler (and intended)
if (_PyString_Resize(&s, n) < 0)
goto outtahere;
Bugfix candidate.
The SIGXFSZ signal is sent when the maximum file size limit is
exceeded (RLIMIT_FSIZE). Apparently, it is also sent when the 2GB
file limit is reached on platforms without large file support.
The default action for SIGXFSZ is to terminate the process and dump
core. When it is ignored, the system call that caused the limit to be
exceeded returns an error and sets errno to EFBIG. Python
always checks errno on I/O syscalls, so there is nothing to do with
the signal.
+ Redirect PyMem_{Del, DEL} to the object allocator's free() when
pymalloc is enabled. Needed so old extensions can continue to
mix PyObject_New with PyMem_DEL.
+ This implies that pgen needs to be able to see the PyObject_XYZ
declarations too. pgenheaders.h now includes Python.h. An
implication is that I expect obmalloc.o needs to get linked into
pgen on non-Windows boxes.
+ When PYMALLOC_DEBUG is defined, *all* Py memory API functions
now funnel through the debug allocator wrapper around pymalloc.
This is the default in a debug build.
+ That caused compile.c to fail: it indirectly mixed PyMem_Malloc
with raw platform free() in one place. This is verbotten.
Highlights: import and friends will understand any of \r, \n and \r\n
as end of line. Python file input will do the same if you use mode 'U'.
Everything can be disabled by configuring with --without-universal-newlines.
See PEP278 for details.
Added code to call this when PYMALLOC_DEBUG is enabled, and envar
PYTHONMALLOCSTATS is set, whenever a new arena is obtained and once
late in the Python shutdown process.
PEP 285. Everything described in the PEP is here, and there is even
some documentation. I had to fix 12 unit tests; all but one of these
were printing Boolean outcomes that changed from 0/1 to False/True.
(The exception is test_unicode.py, which did a type(x) == type(y)
style comparison. I could've fixed that with a single line using
issubtype(x, type(y)), but instead chose to be explicit about those
places where a bool is expected.
Still to do: perhaps more documentation; change standard library
modules to return False/True from predicates.
SF bug 535905 (Evil Trashcan and GC interaction).
The SETLOCAL() macro should not DECREF the local variable in-place and
then store the new value; it should copy the old value to a temporary
value, then store the new value, and then DECREF the temporary value.
This is because it is possible that during the DECREF the frame is
accessed by other code (e.g. a __del__ method or gc.collect()) and the
variable would be pointing to already-freed memory.
BUGFIX CANDIDATE!
descriptor, as used for the tp_methods slot of a type. These new flag
bits are both optional, and mutually exclusive. Most methods will not
use either. These flags are used to create special method types which
exist in the same namespace as normal methods without having to use
tedious construction code to insert the new special method objects in
the type's tp_dict after PyType_Ready() has been called.
If METH_CLASS is specified, the method will represent a class method
like that returned by the classmethod() built-in.
If METH_STATIC is specified, the method will represent a static method
like that returned by the staticmethod() built-in.
These flags may not be used in the PyMethodDef table for modules since
these special method types are not meaningful in that case; a
ValueError will be raised if these flags are found in that context.
This fixes the symptom, but PRINT_ITEM has no way to know what (if
anything) PyFile_WriteObject() writes unless the object being printed
is a string. When the object isn't a string, this fix retains the
guess that softspace should be set after PyFile_WriteObject().
We might want to say that it's the job of filelike-object write methods
to leave the file's softspace in the correct state. That would probably
be better -- but everyone relies on PRINT_ITEM to guess for them now.
A file-static "threads" dict mapped thread IDs to Windows handles, but
was never referenced, and entries never got removed. This gets rid of
the YAGNI-dict entirely.
Bugfix candidate.
Python/
dynload_shlib.c // EMX port emulates dlopen() etc. for DL extensions
import.c // changes to support 8.3 DLL name limit (VACPP+EMX)
// and case sensitive import semantics
importdl.h
thread_os2.h
Fix for the UTF-8 decoder: it will now accept isolated surrogates
(previously it raised an exception which causes round-trips to
fail).
Added new tests for UTF-8 round-trip safety (we rely on UTF-8 for
marshalling Unicode objects, so we better make sure it works for
all Unicode code points, including isolated surrogates).
Bumped the PYC magic in a non-standard way -- please review. This
was needed because the old PYC format used illegal UTF-8 sequences
for isolated high surrogates which now raise an exception.
By default every module is imported in its own namespace, but this can
be changed by defining USE_DYLD_GLOBAL_NAMESPACE. In a future version this
define will be replaced by a runtime setting, but that needs a bit more
thought.
This code is largely based on code and feedback from Steven Majewski,
Marcel Prastawa, Manoj Plakal and other on pythonmac-sig.
type.__module__ problems (again?)
This simply initializes the __module__ local in a class statement from
the __name__ global. I'm not 100% sure that this is the correct fix,
although it usually does the right thing. The problem is that if the
class statement executes in a custom namespace, the __name__ global
may be taken from __builtins__, in which case it would have the value
__builtin__, or it may not exist at all (if the custom namespace also
has a custom __builtins__), in which case the class statement will
fail.
Nevertheless, unless someone finds a better solution, this is a 2.2.1
bugfix too.
eval_frame(): Under -Qnew, INPLACE_DIVIDE wasn't getting handed off to
INPLACE_TRUE_DIVIDE (like BINARY_DIVIDE was getting handed off to
BINARY_TRUE_DIVIDE).
Bugfix candidate.
(ditto for PyMem_Free() -> PyMem_FREE()) to fix and close SF bug
#495875 on systems that HAVE_SNPRINTF=0.
Check in on both release-22 branch and trunk.
Based on the patch from Danny Yoo. The fix is in exec_statement() in
ceval.c.
There are also changes to introduce use of PyCode_GetNumFree() in
several places.
was obviously leaking an int object when whatever the heck it's looking for
was found. Repaired that. This accounts for why entering function and
class definitions at an interactive prompt leaked a reference to the
integer 1 each time.
Bugfix candidate.
still fail on importing modules that link with libraries that fail
their initialization code (such as windowing libraries when we don't have
access to the window server) and that is what I really wanted to fix.
Had nothing to do with rich comparisons -- some stack cleanup code was
lost as a result of merging in Neil Schemenauer's generators patch.
Reinserted the stack cleanup code, skipping it when yielding.
leak when a class defined a __metaclass__. This fixes the problem
reported on python-dev by Ping; I dunno if it's the same as SF bug
#489669 (since that mentions Unicode).
Big Hammer to implement -Qnew as PEP 238 says it should work (a global
option affecting all instances of "/").
pydebug.h, main.c, pythonrun.c: define a private _Py_QnewFlag flag, true
iff -Qnew is passed on the command line. This should go away (as the
comments say) when true division becomes The Rule. This is
deliberately not exposed to runtime inspection or modification: it's
a one-way one-shot switch to pretend you're using Python 3.
ceval.c: when _Py_QnewFlag is set, treat BINARY_DIVIDE as
BINARY_TRUE_DIVIDE.
test_{descr, generators, zipfile}.py: fiddle so these pass under
-Qnew too. This was just a matter of s!/!//! in test_generators and
test_zipfile. test_descr was trickier, as testbinop() is passed
assumptions that "/" is the same as calling a "__div__" method; put
a temporary hack there to call "__truediv__" instead when the method
name is "__div__" and 1/2 evaluates to 0.5.
Three standard tests still fail under -Qnew (on Windows; somebody
please try the Linux tests with -Qnew too! Linux runs a whole bunch
of tests Windows doesn't):
test_augassign
test_class
test_coercion
I can't stay awake longer to stare at this (be my guest). Offhand
cures weren't obvious, nor was it even obvious that cures are possible
without major hackery.
Question: when -Qnew is in effect, should calls to __div__ magically
change into calls to __truediv__? See "major hackery" at tail end of
last paragraph <wink>.
There's now a new structmember code, T_OBJECT_EX, which is used for
all __slot__ variables (except __weakref__, which has special behavior
anyway). This new code raises AttributeError when the variable is
NULL rather than converting NULL to None.
The error for assignment to __debug__ used ste->ste_opt_lineno instead
of n->n_lineno. The latter was at best incorrect; often the slot was
uninitialized. Two fixes here: Use the correct lineno for the error.
Initialize ste_opt_lineno in PySymtable_New(); while there are no
current cases where it is referenced unless it has already been
assigned to, there is no harm in initializing it.
use wrappers on all platforms, to make this as consistent as possible x-
platform (in particular, make sure there's at least one \0 byte in
the output buffer). Also document more of the truth about what these do.
getargs.c, seterror(): Three computations of remaining buffer size were
backwards, thus telling PyOS_snprintf the buffer is larger than it
actually is. This matters a lot now that PyOS_snprintf ensures there's a
trailing \0 byte (because it didn't get the truth about the buffer size,
it was storing \0 beyond the true end of the buffer).
sysmodule.c, mywrite(): Simplify, now that PyOS_vsnprintf guarantees to
produce a \0 byte.
than the argument string size, copy as many bytes as will fit
(including a terminating '\0'), rather than not copying anything.
This to make it satisfy the C99 spec.
vgetargskeywords(): Now that this routine is checking for bad input
(rather than dump core in some cases), some bad calls are raising errors
that previously "worked". This patch makes the error strings more
revealing, and changes the exceptions from SystemError to RuntimeError
(under the theory that SystemError is more of a "can't happen!" assert-
like thing, and so inappropriate for bad arguments to a public C API
function).
seterror() uses a char array and a pointer to the current position in
that array. Use snprintf() and compute the amount of space left in
the buffer based on the current pointer position.
If it returns -1 (which indicates overflow on old Linux platforms and
perhaps on Windows) or size greater than buffer, write a message
indicating that the previous message was truncated.
com_arglist(), symtable_check_unoptimized(), symtable_params(),
symtable_global(), symtable_list_comprehension():
Conversion of sprintf() to PyOS_snprintf() for buffer overrun
avoidance.
PyEval_EvalCodeEx(): increment tstate->recursion_depth around the
decref of the frame, because the C stack for this call is still in
use and the decref can lead to __del__ methods getting called.
While this gives tstate->recursion_depth a value proportional to the
depth of the C stack (instead of a small constant no matter how
deeply __del__s recurse), it's not enough to stop the reported crash
when using the default recursion limit on Windows.
Bugfix candidate.
Bugfix candidate.
tb_displayline(): the sprintf format was choking off the file name, but
used plain %s for the function name (which can be arbitrarily long).
Limit both to 500 chars max.
uninitialized memory reads reported in bug #478001.
Note that this doesn't address the following larger issues:
- Error conditions are not documented for PyOS_*sig() in the C API.
- Nothing that actually calls PyOS_*sig() in the core interpreter and
extension modules actually /checks/ the return value of the call.
Fixing those is left as an exercise for a later day.
This patch boosts performance for comparing identical string object
by some 20% on my machine while not causing any noticable slow-down
for other operations (according to tests done with pybench).
routines. As of 10.1 using Carbon will crash Python if no window server is
available (ssh connection, console mode, MacOSX Server). This fixes bug
#466907.
A result of this mod is that the default 8bit encoding on OSX is now ASCII,
for the time being. Also, the extension modules that need the Carbon
framework now explicitly include it in setup.py.
+ Squash another potential buffer overrun.
+ Simplify the keyword-arg loop by decrementing the count of keywords
remaining instead of incrementing Yet Another Variable; also break
out early if the number of keyword args remaining hits 0.
Since I hit the function's closing curly brace with this patch, that's
enough of this for now <wink>.
The "need" for this was probably removed by an earlier patch that stopped
the loop right before it from passing NULL to a dict lookup routine.
I still haven't convinced myself that the next loop is correct, so am
leaving the next mysterious PyErr_Clear() call in for now.
+ Generally test nkeywords against 0 instead of keywords against NULL
(saves a little work if an empty keywords dict is passed, and is
conceptually more on-target regardless).
+ When a call erroneously specifies a keyword argument both by position
and by keyword name:
- It was easy to provoke this routine into an internal buffer overrun
by using a long argument name. Now uses PyErr_format instead (which
computes a safe buffer size).
- Improved the error msg.
+ Got rid of now-redundant dict typecheck.
+ Renamed nkwds to nkwlist. Now all the "counting" vrbls have names
related to the things they're counting in an obvious way.
+ Renamed argslen to nargs.
+ Renamed kwlen to nkeywords. This one was especially confusing because
kwlen wasn't the length of the kwlist argument, but of the keywords
argument.
+ Removed now-redundant tuple typecheck.
+ Renamed "tplen" local to "argslen" (it's the length of the "args"
argument; I suppose "tp" was for "Tim Peters should rename me
someday <wink>).
introduced this bug just a little while ago, when *adding* internal error
checks).
vgetargskeywords: Rewrote the section that crawls over the format string.
+ Added block comment so it won't take the next person 15 minutes to
reverse-engineer what it's doing.
+ Lined up the "else" clauses.
+ Rearranged the ifs in decreasing order of likelihood (for speed).
and raise an error if they're insane.
vgetargskeywords: the same, except that since this is an internal routine,
just assert that the arguments are sane.
the kwlist vector whenever there was a mix of positional and keyword
arguments, and the number of positional arguments exceeded the length
of the kwlist vector. If there was just one more positional arg than
keyword, the kwlist-terminating NULL got passed to PyMapping_HasKeyString,
which set an internal error that vgetargskeywords() then squashed (but
it's impossible to say whether it knew it was masking an error). If
more than one more positional argument, it went on to pass random trash
to PyMapping_HasKeyString, which is why the example at the start
happened to kill the process.
Pure bugfix candidate.
Fred Drake
Walter Dörwald
Guido van Rossum
Michael W. Hudson
Neal Norwitz
Martin v. Löwis
Raymond Hettinger
Jeremy Hylton
Jack Jansen
Tim Peters
Neil Schemenauer
Marc-André Lemburg
Skip Montanaro
Andrew MacIntyre
Barry Warsaw