As promised in my response to the bug report, I'm not really fixing
it; in fact, one could argule over what the proper fix should do.
Instead, I'm adding a little magic that raises TypeError if you try to
pickle an instance of a class that has __slots__ but doesn't define or
override __getstate__. This is done by adding a bozo __getstate__
that always raises TypeError.
There were several places that assumed the md_dict field was always
set, but it needn't be. Fixed these to be more careful.
I changed PyModule_GetDict() to initialize md_dict to a new dictionary
if it's NULL.
Bugfix candidate.
and (b) stop trying to prevent file growth.
Beef up the file.truncate() docs.
Change test_largefile.py to stop assuming that f.truncate() moves the
file pointer to the truncation point, and to verify instead that it leaves
the file position alone. Remove the test for what happens when a
specified size exceeds the original file size (it's ill-defined, according
to the Single Unix Spec).
dropping MS's inadequate _chsize() function. This was inspired by
SF patch 498109 ("fileobject truncate support for win32"), which I
rejected.
libstdtypes.tex: Someone who knows should update the availability
blurb. For example, if it's available on Linux, it would be good to
say so.
test_largefile: Uncommented the file.truncate() tests, and reworked to
do more. The old comment about "permission errors" in the truncation
tests under Windows was almost certainly due to that the file wasn't open
for *write* access at this point, so of course MS wouldn't let you
truncate it. I'd be appalled if a Unixish system did.
CAUTION: Someone should run this test on Linux (etc) too. The
truncation part was commented out before. Note that test_largefile isn't
run by default.
Adapter from SF patch 528038; fixes SF bug 527816.
The wrapper for __nonzero__ should be wrap_inquiry rather than
wrap_unaryfunc, since the slot returns an int, not a PyObject *.
Another year in the quest to out-guess random C behavior.
Added macros Py_ADJUST_ERANGE1(X) and Py_ADJUST_ERANGE2(X, Y). The latter
is useful for functions with complex results. Two corrections to errno-
after-libm-call are attempted:
1. If the platform set errno to ERANGE due to underflow, clear errno.
Some unknown subset of libm versions and link options do this. It's
allowed by C89, but I never figured anyone would do it.
2. If the platform did not set errno but overflow occurred, force
errno to ERANGE. C89 required setting errno to ERANGE, but C99
doesn't. Some unknown subset of libm versions and link options do
it the C99 way now.
Bugfix candidate, but hold off until some Linux people actually try it,
with and without -lieee. I'll send a help plea to Python-Dev.
PyNumber_Add() tries the nb_add slot first, then falls back to
sq_concat. However, tt didn't check the return value of sq_concat.
If sq_concat returns NotImplemented, raise the standard TypeError.
[ 526072 ] pickling os.stat results round II
structseq's constructors can now take "invisible" fields in a dict.
Gave the constructors better error messages.
their __reduce__ method puts these fields in a dict.
(this is all in aid of getting os.stat_result's to pickle portably)
Also fixes
[ 526039 ] devious code can crash structseqs
Thought needed about how much of this counts as a bugfix. Certainly
#526039 needs to be fixed.
platform realloc(p, 0) returns NULL, so MALLOC_ZERO_RETURNS_NULL can
be correctly undefined yet realloc(p, 0) can return NULL anyway.
Prevent realloc(p, 0) doing free(p) and returning NULL via a different
hack. Would probably be better to get rid of MALLOC_ZERO_RETURNS_NULL
entirely.
Bugfix candidate.
Due to the bizarre definition of _PyLong_Copy(), creating an instance
of a subclass of long with a negative value could cause core dumps
later on. Unfortunately it looks like the behavior of _PyLong_Copy()
is quite intentional, so the fix is more work than feels comfortable.
This fix is almost, but not quite, the code that Naofumi Honda added;
in addition, I added a test case.
Objects/
fileobject.c
stringobject.c
unicodeobject.c
This commit doesn't include the cleanup patches for stringobject.c and
unicodeobject.c which are shown separately in the patch manager. Those
patches will be regenerated and applied in a subsequent commit, so as
to preserve a fallback position (this commit to those files).
Fix for the UTF-8 decoder: it will now accept isolated surrogates
(previously it raised an exception which causes round-trips to
fail).
Added new tests for UTF-8 round-trip safety (we rely on UTF-8 for
marshalling Unicode objects, so we better make sure it works for
all Unicode code points, including isolated surrogates).
Bumped the PYC magic in a non-standard way -- please review. This
was needed because the old PYC format used illegal UTF-8 sequences
for isolated high surrogates which now raise an exception.
NULL, so that you can call PyType_Ready() to initialize a type that
is to be separately compiled with C on Windows.
inherit_special(): Add a long comment explaining that you have to set
tp_new if your base class is PyBaseObject_Type.
Fix for SF bug #492345. (I could've sworn I checked this in, but
apparently I didn't!)
This code:
class Classic:
pass
class New(Classic):
__metaclass__ = type
attempts to create a new-style class with only classic bases -- but it
doesn't work right. Attempts to fix it so it works caused problems
elsewhere, so I'm now raising a TypeError in this case.
delivered bizarre results. Check float_divmod for a Py_NotImplemented
return and pass it along (instead of treating Py_NotImplemented as a
2-tuple).
CONVERT_TO_DOUBLE: Added comments; this macro is obscure.
PyDict_UpdateFromSeq2(): removed it.
PyDict_MergeFromSeq2(): made it public and documented it.
PyDict_Merge() docs: updated to reveal <wink> that the second
argument can be any mapping object.
no get function was defined, the property's doc string was
inaccessible. This was because the test for prop_get was made
*before* the test for a NULL/None object argument.
Also changed the property class defined in Python in a comment to test
for NULL to decide between get and delete; this makes it less Python
but then, assigning None to a property doesn't delete it!
PyString_FromString():
Since the length of the string is already being stored in size,
changed the strcpy() to a memcpy() for a small speed improvement.
out the for loop at the end intended to zero out new items wasn't
doing anything, because sv->ob_size was already equal to newsize. The
fix slightly refactors the function, introducing a variable oldsize
and doing away with sizediff (which was used only once), and using
oldsize and newsize consistently. I also added comments explaining
what the two for loops do. (Looking at the CVS annotation of this
function, it's no miracle a bug crept in -- this has been patched by
many different folks! :-)
This is best reproduced by
while 1:
class U(unicode):
pass
U(u"xxxxxx")
The unicode_dealloc() code wasn't properly freeing the str and defenc
fields of the Unicode object when freeing a subtype instance. Fixed
this by a subtle refactoring that actually reduces the amount of code
slightly.
PyCell_Set() incremenets the reference count, so the earlier XINCREF
causes a leak.
Also make a number of small performance improvements to the code on
the assumption that most of the time variables are not rebound across
a FastToLocals() / LocalsToFast() pair.
Replace uses of PyCell_Set() and PyCell_Get() with PyCell_SET() and
PyCell_GET(), since the frame is guaranteed to contain cells.
Add a missing DECREF in an obscure corner. If the str() or repr() of
an object passed to a string interpolation -- e.g. "%s" % obj --
returns a non-string, the returned object was leaked.
Repair an indentation glitch.
Replace a bunch of PyString_AsString() calls (and their ilk) with
macros.
It was easier than I thought, assuming that no other things contribute
to the instance size besides slots -- a pretty good bet. With a test
suite, no less!
happy if one could delete the __dict__ attribute of an instance. I
love to make Jim happy, so here goes...
- New-style objects now support deleting their __dict__. This is for
all intents and purposes equivalent to assigning a brand new empty
dictionary, but saves space if the object is not used further.
int_mul(): new and vastly simpler overflow checking. Whether it's
faster or slower will likely vary across platforms, favoring boxes
with fast floating point. OTOH, we no longer have to worry about
people shipping broken LONG_BIT definitions <0.9 wink>.
There's now a new structmember code, T_OBJECT_EX, which is used for
all __slot__ variables (except __weakref__, which has special behavior
anyway). This new code raises AttributeError when the variable is
NULL rather than converting NULL to None.
string object (or a Unicode that's trivially converted to ASCII).
PyObject_GetAttr(): add an 'else' to the Unicode test like
PyObject_SetAttr() already has.
Rather than tweaking the inheritance of type object slots (which turns
out to be too messy to try), this fix adds a __hash__ to the list and
dict types (the only mutable types I'm aware of) that explicitly
raises an error. This has the advantage that list.__hash__([]) also
raises an error (previously, this would invoke object.__hash__([]),
returning the argument's address); ditto for dict.__hash__.
The disadvantage for this fix is that 3rd party mutable types aren't
automatically fixed. This should be added to the rules for creating
subclassable extension types: if you don't want your object to be
hashable, add a tp_hash function that raises an exception.
Also, it's possible that I've forgotten about other mutable types for
which this should be done.
SF patch #480716 by Greg Chapman fixes the problem that super's
__get__ method always returns an instance of super, even when the
instance whose __get__ method is called is an instance of a subclass
of super.
Other issues fixed:
- super(C, C()).__class__ would return the __class__ attribute of C()
rather than the __class__ attribute of the super object. This is
confusing. To fix this, I decided to change the semantics of super
so that it only applies to code attributes, not to data attributes.
After all, overriding data attributes is not supported anyway.
- While super(C, x) carefully checked that x is an instance of C,
super(C).__get__(x) made no such check, allowing for a loophole.
This is now fixed.
slot_tp_descr_set(): When deleting an attribute described by a
descriptor implemented in Python, the descriptor's __del__ method is
called by the slot_tp_descr_set dispatch function. This is bogus --
__del__ already has a different meaning. Renaming this use of __del__
is renamed to __delete__.
Bugfix candidate.
int_repr(): we've never had a buffer big enough to hold the largest
possible result on a 64-bit box. Now that we're using snprintf instead
of sprintf, this can lead to nonsense results instead of random stack
corruption.
pass the buffer length. Stop using it. It should be deprecated, but too
late in the release cycle to do that now.
New static format_float() does the same thing but requires passing the
buffer length too. Use it instead.
const char* instead of char*. The change is conceptually correct, and
indirectly fixes a compiler wng introduced when somebody else innocently
passed a const char* to this function.
sprintf() to PyOS_snprintf() for buffer overrun avoidance.
complex_print(), complex_repr(), complex_str(): Call complex_to_buf()
passing in sizeof(buf).
confusing error messages. If a new-style class has no sequence or
mapping behavior, attempting to use the indexing notation with a
non-integer key would complain that the sequence index must be an
integer, rather than complaining that the operation is not supported.
of multiple inheritance from a mix of new- and classic-style classes.
This is his patch, plus a start at some test cases from me. Will check
in more, plus a NEWS blurb, later tonight.
object, so the "Metroworks only" section should not decref it in case
of error (the caller is responsible for decref'ing in case of error --
and does).
helping for types that defined tp_richcmp but not tp_compare, although
that's when it's most valuable, and strings moved into that category
since the fast path was first introduced. Now it helps for same-type
non-Instance objects that define rich or 3-way compares.
For all the edits here, the rest just amounts to moving the fast path from
do_richcmp into PyObject_RichCompare, saving a layer of function call
(measurable on my box!). This loses when NESTING_LIMIT is exceeded, but I
don't care about that (fast-paths are for normal cases, not pathologies).
Also added a tasteful <wink> label to get out of PyObject_RichCompare, as
the if/else nesting in this routine was getting incomprehensible.
Try to ensure that divmod(-0.0, 1.0) -> (-0.0, +0.0) across platforms.
It always did on Windows, and still does. It didn't on Linux. Alas,
there's no platform-independent way to write a test case for this.
Bugfix candidate.
presence of NaNs. So pass the issue on to the platform libm fabs();
after all, fabs() is a std C function because you can't implement it
correctly in portable C89.
should just avoid calling it in the first place to avoid waiting for a repr
of a large object like a dict or list. The result of PyObject_Repr() was
being leaked as well.
Bugfix candidate!
XXX Remaining problems:
- The GC module doesn't know about these; I think it has its reasons
to disallow calling __del__, but for now, __del__ on new-style
objects is called when the GC module discards an object, for better
or for worse.
- The code to call a __del__ handler is really ridiculously
complicated, due to all the different debug #ifdefs. I've copied
this from the similar code in classobject.c, so I'm pretty sure I
did it right, but it's not pretty. :-(
- No tests yet.
Martin v. Löwis
Guido van Rossum
Tim Peters
Jeremy Hylton
Michael W. Hudson
Andrew MacIntyre
Marc-André Lemburg
Neal Norwitz
Fred Drake
Sjoerd Mullender
Jack Jansen
Barry Warsaw