are actually getting generated. Add helpered method
ensure_opcode_in_pickle to do a correct job checking for that. Changed
test_long1(), test_long4(), and test_short_tuples() to use it.
huge. On older Linux systems, the C library's strtod() apparently
gives up before seeing the end of the string when it sees so many
digits that it thinks the result must be Infinity. (It is wrong, BTW
-- there could be an "e-10000" hiding behind 10,000 digits.) The
shorter shuge still tests what it's testing, without relying on
strtod() doing a super job.
exception, ResourceDenied. This is used to distinguish between tests that
are skipped for other reasons (platform support, missing data, etc.) from
those that are skipped because a "resource" has not been enabled. This
prevents those tests from being reported as unexpected skips for the
platform; those should only be considered unexpected skips if the resource
were enabled.
[ 676342 ] after using pdb readline does not work correctly
using Michael Stone's patch so the completer functionality of
cmd is only setup between preloop and postloop.
loops. Renamed DATA and BINDATA to DATA0 and DATA1. Included
disassemblies, but noted why we can't test them. Added XXX comment to
cPickle about a mysterious comment, where pickle and cPickle diverge
in how they number PUT indices.
Assorted code cleanups; e.g., sizeof(char) is 1 by definition, so there's
no need to do things like multiply by sizeof(char) in hairy malloc
arguments. Fixed an undetected-overflow bug in readline_file().
longobject.c: Fixed a really stupid bug in the new _PyLong_NumBits.
pickle.py: Fixed stupid bug in save_long(): When proto is 2, it
wrote LONG1 or LONG4, but forgot to return then -- it went on to
append the proto 1 LONG opcode too.
Fixed equally stupid cancelling bugs in load_long1() and
load_long4(): they *returned* the unpickled long instead of pushing
it on the stack. The return values were ignored. Tests passed
before only because save_long() pickled the long twice.
Fixed bugs in encode_long().
Noted that decode_long() is quadratic-time despite our hopes,
because long(string, 16) is still quadratic-time in len(string).
It's hex() that's linear-time. I don't know a way to make decode_long()
linear-time in Python, short of maybe transforming the 256's-complement
bytes into marshal's funky internal format, and letting marshal decode
that. It would be more valuable to make long(string, 16) linear time.
pickletester.py: Added a global "protocols" vector so tests can try
all the protocols in a sane way. Changed test_ints() and test_unicode()
to do so. Added a new test_long(), but the tail end of it is disabled
because it "takes forever" under pickle.py (but runs very quickly under
cPickle: cPickle proto 2 for longs is linear-time).
anymore either, so don't. This also allows to get rid of obscure code
making __getnewargs__ identical to __getstate__ (hmm ... hope there
wasn't more to this than I realize!).
longer needs to be public, and shoudn't be public because all datetime
objects are immutable. The Python implementation has changed
accordingly, but still need to change the C implementation.
The 4th item can be None or an iterator yielding list items, which are
used to append() or extend() the object. The 5th item can be None or
an iterator yielding a dict's (key, value) pairs, which are stuffed
into the object using __setitem__.
Also (as a separate, though related, feature) add "batching" for list
and dict items. If you pickled a dict or list with a million items in
the past, it would push a million items onto the stack. It now pushes
only 1000 items at a time on the stack, using repeated APPENDS or
SETITEMS opcodes. (For lists, I hope that using many short extend()
calls doesn't exhibit quadratic behavior.)
__module__ is the string name of the module the function was defined
in, just like __module__ of classes. In some cases, particularly for
C functions, the __module__ may be None.
Change PyCFunction_New() from a function to a macro, but keep an
unused copy of the function around so that we don't change the binary
API.
Change pickle's save_global() to use whichmodule() if __module__ is
None, but add the __module__ logic to whichmodule() since it might be
used outside of pickle.
error handers in the Unicode codecs: Negative
positions are treated as being relative to the end of
the input and out of bounds positions result in an
IndexError.
Also update the PEP and include an explanation of
this in the documentation for codecs.register_error.
Fixes a small bug in iconv_codecs: if the position
from the callback is negative *add* it to the size
instead of substracting it.
From SF patch #677429.
M rpc.py
SF Bug 676398 Doesn't handle non-built-in exceptions
1. Move exception formatting to the subprocess; allows subclassing of
exceptions, including subclasses created in the shell without
introducing excessive complexity in the RPC mechanism.
2. Provide access to linecache from subprocess to support this.
classes have a __reduce__ that returns (self.__class__,
self.__getstate__()). tzinfo.__reduce__() is a bit smarter, calling
__getinitargs__ and __getstate__ if they exist, and falling back to
__dict__ if it exists and isn't empty.
for this iconv() implementation in the init function.
For encoding: use a byteswapped version of the input if
neccessary.
For decoding: byteswap every piece returned by iconv()
if neccessary (but not those pieces returned from the
callback)
Comment out test_sane() in the test script, because
whether this works depends on whether byte swapping
is neccessary or not (an on Py_UNICODE_SIZE)
on the type instead of self.save(t). This defeated the purpose of
NEWOBJ, because it didn't generate a BINGET opcode when t was already
memoized; but moreover, it would generate multiple BINPUT opcodes for
the same type! pickletools.dis() doesn't like this.
How I found this? I was playing with picklesize.py in the datetime
sandbox, and noticed that protocol 2 pickles for multiple objects were
in fact larger than protocol 1 pickles! That was suspicious, so I
decided to disassemble one of the pickles.
This really needs a unit test, but I'm exhausted. I'll be late for
work as it is. :-(
the same function, don't save the state or write a BUILD opcode. This
is so that a type (e.g. datetime :-) can support protocol 2 using
__getnewargs__ while also supporting protocol 0 and 1 using
__getstate__. (Without this, the state would be pickled twice with
protocol 2, unless __getstate__ is defined to return None, which
breaks protocol 0 and 1.)
popped a MARK, but without stack emulation the disassembler couldn't
know that, and subsequent indentation got hosed.
Now the disassembler does do enough stack emulation to catch this. While
I was at it, also added lots of sanity checks for other stack operations,
and correct use of the memo. This goes (I think) a long way toward being
a "pickle verifier" now too.
types. The special handling for these can now be removed from save_newobj().
Add some testing for this.
Also add support for setting the 'fast' flag on the Python Pickler class,
which suppresses use of the memo.
only get run by test_pickle.py now (& not by test_cpickle.py). This
should be undone when protocol 2 is implemented in cPickle too.
test_cpickle should pass again.
object.__reduce__, do a getattr() on the class so we can explicitly
test for it. The reduce()-calling code becomes a bit more regular as
a result.
Also add support slots: if an object has slots, the default state is
(dict, slots) where dict is the __dict__ or None, and slots is a dict
mapping slot names to slot values. We do a best-effort approach to
find slot names, assuming the __slots__ fields of classes aren't
modified after class definition time to misrepresent the actual list
of slots defined by a class.
of the opcode character instead (but stripping the quotes).
Added a proto 2 test section for the canonical recursive-tuple case.
Note that since pickle's save_tuple() takes different paths depending on
tuple length now, beefier tests are really needed (but not in pickletools);
the "short tuple" case tried here was actually broken yesterday, and it's
subtle stuff so needs to be tested.
be one of 0, 1 or 2).
I should note that the previous checkin also added NEWOBJ support to
the unpickler -- but there's nothing yet that generates this.
Tim Peters
Guido van Rossum
Andrew M. Kuchling
Fred Drake
Greg Ward
Walter Dörwald
Just van Rossum
Michael W. Hudson
Gregory P. Smith
Jack Jansen
Raymond Hettinger
Thomas Heller
Jeremy Hylton
Kurt B. Kaiser
Neal Norwitz
Barry Warsaw
Neil Schemenauer
Skip Montanaro