or dumping pickles with a 2.x-compatible protocol, in order to make data
sharing and migration easier. This behaviour can be disabled using the
new `fix_imports` optional argument.
svn+ssh://pythondev@svn.python.org/python/trunk
........
r72223 | antoine.pitrou | 2009-05-02 23:13:23 +0200 (sam., 02 mai 2009) | 5 lines
Isue #5084: unpickling now interns the attribute names of pickled objects,
saving memory and avoiding growth in size of subsequent pickles. Proposal
and original patch by Jake McGuire.
........
'L' opcode always appends an 'L' on output, just as 2.x does. When
unpickling, remove the trailing 'L' (if present) before passing the
result to PyLong_FromString.
Updated documentation.
Merged revisions 63042 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r63042 | alexandre.vassalotti | 2008-05-11 04:25:28 -0400 (Sun, 11 May 2008) | 5 lines
Added module stub for copy_reg renaming in 3.0.
Renamed copy_reg to copyreg in the standard library, to avoid
spurious warnings and ease later merging to py3k branch. Public
documentation remains intact.
........
memoryview objects have a different API (such as for indexing) than
bytes and bytesarray objects, so memoryview objects shouldn't be
treated blindly as "bytes" objects.
for bytes. This is the default protocol. It intentionally cannot
be unpickled by Python 2.x.
- When a pickle written by Python 2.x contains an (8-bit) str
instance, this is now decoded to a (Unicode) str instance. The
encoding used to do this defaults to ASCII, but can be overridden
via two new keyword arguments to the Unpickler class. Previously
this would create bytes instances, which is usually wrong: str
instances are often used to pickle attribute names etc., and text is
more common than binary data anyway.
No detailed change log; just check out the change log for the py3k-pep3137
branch. The most obvious changes:
- str8 renamed to bytes (PyString at the C level);
- bytes renamed to buffer (PyBytes at the C level);
- PyString and PyUnicode are no longer compatible.
I.e. we now have an immutable bytes type and a mutable bytes type.
The behavior of PyString was modified quite a bit, to make it more
bytes-like. Some changes are still on the to-do list.
strings, in Latin-1. Bytes are once more pickled through bytes.__reduce__,
but now it returns "latin-1" as the second parameter.
Unfortunately this breaks datetime pickling. I'll have to investigate
further; reverting Martin's changes doesn't seem to help.
's' and 'c' codes.
Change pickle to dump bytes objects using the 'S'
code, and to load the 'S' code as byte objects.
Change datetime and array to generate and expect
bytes objects in reduce/unreduce.
base64, uu, zlib, rot_13, hex, quopri, bz2, string_escape.
However codecs.escape_encode() and codecs.escape_decode()
still exist, as they are used for pickling str8 objects
(so those two functions can go, when the str8 type is removed).
svn+ssh://pythondev@svn.python.org/python/branches/p3yk
........
r55797 | neal.norwitz | 2007-06-07 00:00:57 -0700 (Thu, 07 Jun 2007) | 3 lines
Get rid of some remnants of classic classes. types.ClassType == type.
Also get rid of almost all uses of the types module and use the builtin name.
........
r55798 | neal.norwitz | 2007-06-07 00:12:36 -0700 (Thu, 07 Jun 2007) | 1 line
Remove a use of types, verify commit hook works
........
r55809 | guido.van.rossum | 2007-06-07 11:11:29 -0700 (Thu, 07 Jun 2007) | 2 lines
Fix syntax error introduced by Neal in last checkin.
........
svn+ssh://pythondev@svn.python.org/python/branches/p3yk
................
r55413 | fred.drake | 2007-05-17 12:30:10 -0700 (Thu, 17 May 2007) | 1 line
fix argument name in documentation; match the implementation
................
r55430 | jack.diederich | 2007-05-18 06:39:59 -0700 (Fri, 18 May 2007) | 1 line
Implements class decorators, PEP 3129.
................
r55432 | guido.van.rossum | 2007-05-18 08:09:41 -0700 (Fri, 18 May 2007) | 2 lines
obsubmit.
................
r55434 | guido.van.rossum | 2007-05-18 09:39:10 -0700 (Fri, 18 May 2007) | 3 lines
Fix bug in test_inspect. (I presume this is how it should be fixed;
Jack Diedrich, please verify.)
................
r55460 | brett.cannon | 2007-05-20 00:31:57 -0700 (Sun, 20 May 2007) | 4 lines
Remove the imageop module. With imgfile already removed in Python 3.0 and
rgbimg gone in Python 2.6 the unit tests themselves were made worthless. Plus
third-party libraries perform the same function much better.
................
r55469 | neal.norwitz | 2007-05-20 11:28:20 -0700 (Sun, 20 May 2007) | 118 lines
Merged revisions 55324-55467 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r55348 | georg.brandl | 2007-05-15 13:19:34 -0700 (Tue, 15 May 2007) | 4 lines
HTML-escape the plain traceback in cgitb's HTML output, to prevent
the traceback inadvertently or maliciously closing the comment and
injecting HTML into the error page.
........
r55372 | neal.norwitz | 2007-05-15 21:33:50 -0700 (Tue, 15 May 2007) | 6 lines
Port rev 55353 from Guido:
Add what looks like a necessary call to PyErr_NoMemory() when PyMem_MALLOC()
fails.
Will backport.
........
r55377 | neal.norwitz | 2007-05-15 22:06:33 -0700 (Tue, 15 May 2007) | 1 line
Mention removal of some directories for obsolete platforms
........
r55380 | brett.cannon | 2007-05-15 22:50:03 -0700 (Tue, 15 May 2007) | 2 lines
Change the maintainer of the BeOS port.
........
r55383 | georg.brandl | 2007-05-16 06:44:18 -0700 (Wed, 16 May 2007) | 2 lines
Bug #1719995: don't use deprecated method in sets example.
........
r55386 | neal.norwitz | 2007-05-16 13:05:11 -0700 (Wed, 16 May 2007) | 5 lines
Fix bug in marshal where bad data would cause a segfault due to
lack of an infinite recursion check.
Contributed by Damien Miller at Google.
........
r55389 | brett.cannon | 2007-05-16 15:42:29 -0700 (Wed, 16 May 2007) | 6 lines
Remove the gopherlib module. It has been raising a DeprecationWarning since
Python 2.5.
Also remove gopher support from urllib/urllib2. As both imported gopherlib the
usage of the support would have raised a DeprecationWarning.
........
r55394 | raymond.hettinger | 2007-05-16 18:08:04 -0700 (Wed, 16 May 2007) | 1 line
calendar.py gets no benefit from xrange() instead of range()
........
r55395 | brett.cannon | 2007-05-16 19:02:56 -0700 (Wed, 16 May 2007) | 3 lines
Complete deprecation of BaseException.message. Some subclasses were directly
accessing the message attribute instead of using the descriptor.
........
r55396 | neal.norwitz | 2007-05-16 23:11:36 -0700 (Wed, 16 May 2007) | 4 lines
Reduce the max stack depth to see if this fixes the segfaults on
Windows and some other boxes. If this is successful, this rev should
be backported. I'm not sure how close to the limit we should push this.
........
r55397 | neal.norwitz | 2007-05-16 23:23:50 -0700 (Wed, 16 May 2007) | 4 lines
Set the depth to something very small to try to determine if the
crashes on Windows are really due to the stack size or possibly
some other problem.
........
r55398 | neal.norwitz | 2007-05-17 00:04:46 -0700 (Thu, 17 May 2007) | 4 lines
Last try for tweaking the max stack depth. 5000 was the original value,
4000 didn't work either. 1000 does work on Windows. If 2000 works,
that will hopefully be a reasonable balance.
........
r55412 | fred.drake | 2007-05-17 12:29:58 -0700 (Thu, 17 May 2007) | 1 line
fix argument name in documentation; match the implementation
........
r55427 | neal.norwitz | 2007-05-17 22:47:16 -0700 (Thu, 17 May 2007) | 1 line
Verify neither dumps or loads overflow the stack and segfault.
........
r55446 | collin.winter | 2007-05-18 16:11:24 -0700 (Fri, 18 May 2007) | 1 line
Backport PEP 3110's new 'except' syntax to 2.6.
........
r55448 | raymond.hettinger | 2007-05-18 18:11:16 -0700 (Fri, 18 May 2007) | 1 line
Improvements to NamedTuple's implementation, tests, and documentation
........
r55449 | raymond.hettinger | 2007-05-18 18:50:11 -0700 (Fri, 18 May 2007) | 1 line
Fix beginner mistake -- don't mix spaces and tabs.
........
r55450 | neal.norwitz | 2007-05-18 20:48:47 -0700 (Fri, 18 May 2007) | 1 line
Clear data so random memory does not get freed. Will backport.
........
r55452 | neal.norwitz | 2007-05-18 21:34:55 -0700 (Fri, 18 May 2007) | 3 lines
Whoops, need to pay attention to those test failures.
Move the clear to *before* the first use, not after.
........
r55453 | neal.norwitz | 2007-05-18 21:35:52 -0700 (Fri, 18 May 2007) | 1 line
Give some clue as to what happened if the test fails.
........
r55455 | georg.brandl | 2007-05-19 11:09:26 -0700 (Sat, 19 May 2007) | 2 lines
Fix docstring for add_package in site.py.
........
r55458 | brett.cannon | 2007-05-20 00:09:50 -0700 (Sun, 20 May 2007) | 2 lines
Remove the rgbimg module. It has been deprecated since Python 2.5.
........
r55465 | nick.coghlan | 2007-05-20 04:12:49 -0700 (Sun, 20 May 2007) | 1 line
Fix typo in example (should be backported, but my maintenance branch is woefully out of date)
........
................
r55472 | brett.cannon | 2007-05-20 12:06:18 -0700 (Sun, 20 May 2007) | 2 lines
Remove imageop from the Windows build process.
................
r55486 | neal.norwitz | 2007-05-20 23:59:52 -0700 (Sun, 20 May 2007) | 1 line
Remove callable() builtin
................
r55506 | neal.norwitz | 2007-05-22 00:43:29 -0700 (Tue, 22 May 2007) | 78 lines
Merged revisions 55468-55505 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk
........
r55468 | neal.norwitz | 2007-05-20 11:06:27 -0700 (Sun, 20 May 2007) | 1 line
rotor is long gone.
........
r55470 | neal.norwitz | 2007-05-20 11:43:00 -0700 (Sun, 20 May 2007) | 1 line
Update directories/files at the top-level.
........
r55471 | brett.cannon | 2007-05-20 12:05:06 -0700 (Sun, 20 May 2007) | 2 lines
Try to remove rgbimg from Windows builds.
........
r55474 | brett.cannon | 2007-05-20 16:17:38 -0700 (Sun, 20 May 2007) | 4 lines
Remove the macfs module. This led to the deprecation of macostools.touched();
it completely relied on macfs and is a no-op on OS X according to code
comments.
........
r55476 | brett.cannon | 2007-05-20 16:56:18 -0700 (Sun, 20 May 2007) | 3 lines
Move imgfile import to the global namespace to trigger an import error ASAP to
prevent creation of a test file.
........
r55477 | brett.cannon | 2007-05-20 16:57:38 -0700 (Sun, 20 May 2007) | 3 lines
Cause posixfile to raise a DeprecationWarning. Documented as deprecated since
Ptyhon 1.5.
........
r55479 | andrew.kuchling | 2007-05-20 17:03:15 -0700 (Sun, 20 May 2007) | 1 line
Note removed modules
........
r55481 | martin.v.loewis | 2007-05-20 21:35:47 -0700 (Sun, 20 May 2007) | 2 lines
Add Alexandre Vassalotti.
........
r55482 | george.yoshida | 2007-05-20 21:41:21 -0700 (Sun, 20 May 2007) | 4 lines
fix against r55474 [Remove the macfs module]
Remove "libmacfs.tex" from Makefile.deps and mac/mac.tex.
........
r55487 | raymond.hettinger | 2007-05-21 01:13:35 -0700 (Mon, 21 May 2007) | 1 line
Replace assertion with straight error-checking.
........
r55489 | raymond.hettinger | 2007-05-21 09:40:10 -0700 (Mon, 21 May 2007) | 1 line
Allow all alphanumeric and underscores in type and field names.
........
r55490 | facundo.batista | 2007-05-21 10:32:32 -0700 (Mon, 21 May 2007) | 5 lines
Added timeout support to HTTPSConnection, through the
socket.create_connection function. Also added a small
test for this, and updated NEWS file.
........
r55495 | georg.brandl | 2007-05-21 13:34:16 -0700 (Mon, 21 May 2007) | 2 lines
Patch #1686487: you can now pass any mapping after '**' in function calls.
........
r55502 | neal.norwitz | 2007-05-21 23:03:36 -0700 (Mon, 21 May 2007) | 1 line
Document new params to HTTPSConnection
........
r55504 | neal.norwitz | 2007-05-22 00:16:10 -0700 (Tue, 22 May 2007) | 1 line
Stop using METH_OLDARGS
........
r55505 | neal.norwitz | 2007-05-22 00:16:44 -0700 (Tue, 22 May 2007) | 1 line
Stop using METH_OLDARGS implicitly
........
................
svn+ssh://pythondev@svn.python.org/python/branches/p3yk
........
r55271 | fred.drake | 2007-05-11 10:14:47 -0700 (Fri, 11 May 2007) | 3 lines
remove jpeg, panel libraries for SGI; there is more IRIX stuff left over,
I guess that should be removed too, but will leave for someone who is sure
........
r55280 | fred.drake | 2007-05-11 19:11:37 -0700 (Fri, 11 May 2007) | 1 line
remove mention of file that has been removed
........
r55301 | brett.cannon | 2007-05-13 17:38:05 -0700 (Sun, 13 May 2007) | 4 lines
Remove rexec and Bastion from the stdlib. This also eliminates the need for
f_restricted on frames. This in turn negates the need for
PyEval_GetRestricted() and PyFrame_IsRestricted().
........
r55303 | brett.cannon | 2007-05-13 19:22:22 -0700 (Sun, 13 May 2007) | 2 lines
Remove the md5 and sha modules.
........
r55305 | george.yoshida | 2007-05-13 19:45:55 -0700 (Sun, 13 May 2007) | 2 lines
fix markup
........
r55306 | neal.norwitz | 2007-05-13 19:47:57 -0700 (Sun, 13 May 2007) | 1 line
Get the doc building again after some removals.
........
r55307 | neal.norwitz | 2007-05-13 19:50:45 -0700 (Sun, 13 May 2007) | 1 line
Get test_pyclbr passing again after getstatus was removed from commands. This "test case" was weird since it was just importing a seemingly random module. Remove the import
........
r55322 | brett.cannon | 2007-05-14 14:09:20 -0700 (Mon, 14 May 2007) | 3 lines
Remove the compiler package. Will eventually need a mechanism to byte compile
an AST.
........
svn+ssh://pythondev@svn.python.org/python/branches/p3yk
........
r55077 | guido.van.rossum | 2007-05-02 11:54:37 -0700 (Wed, 02 May 2007) | 2 lines
Use the new print syntax, at least.
........
r55142 | fred.drake | 2007-05-04 21:27:30 -0700 (Fri, 04 May 2007) | 1 line
remove old cruftiness
........
r55143 | fred.drake | 2007-05-04 21:52:16 -0700 (Fri, 04 May 2007) | 1 line
make this work with the new Python
........
r55162 | neal.norwitz | 2007-05-06 22:29:18 -0700 (Sun, 06 May 2007) | 1 line
Get asdl code gen working with Python 2.3. Should continue to work with 3.0
........
r55164 | neal.norwitz | 2007-05-07 00:00:38 -0700 (Mon, 07 May 2007) | 1 line
Verify checkins to p3yk (sic) branch go to 3000 list.
........
r55166 | neal.norwitz | 2007-05-07 00:12:35 -0700 (Mon, 07 May 2007) | 1 line
Fix this test so it runs again by importing warnings_test properly.
........
r55167 | neal.norwitz | 2007-05-07 01:03:22 -0700 (Mon, 07 May 2007) | 8 lines
So long xrange. range() now supports values that are outside
-sys.maxint to sys.maxint. floats raise a TypeError.
This has been sitting for a long time. It probably has some problems and
needs cleanup. Objects/rangeobject.c now uses 4-space indents since
it is almost completely new.
........
r55171 | guido.van.rossum | 2007-05-07 10:21:26 -0700 (Mon, 07 May 2007) | 4 lines
Fix two tests that were previously depending on significant spaces
at the end of a line (and before that on Python 2.x print behavior
that has no exact equivalent in 3.0).
........
and .keys(), .items(), .values() return dict views.
The dict views aren't fully functional yet; in particular, they can't
be compared to sets yet. but they are useful as "iterator wells".
There are still 27 failing unit tests; I expect that many of these
have fairly trivial fixes, but there are so many, I could use help.
I banged on the code (beyond what's in that branch) to make fewer tests fail;
the only tests that fail now are:
test_descr -- can't pickle ints?!
test_pickletools -- ???
test_socket -- See python.org/sf/1619659
test_sqlite -- ???
I'll deal with those later.
Not all code has been fixed yet; this is just a checkpoint...
The C API still has PyDict_HasKey() and _HasKeyString(); not sure
if I want to change those just yet.
the optional proto 2 slot state.
pickle.py, load_build(): CAUTION: Noted that cPickle's
load_build and pickle's load_build really don't do the same
things with the state, and didn't before this patch either.
cPickle never tries to do .update(), and has no backoff if
instance.__dict__ can't be retrieved. There are no tests
that can tell the difference, and part of what cPickle's
load_build() did looked accidental to me, so I don't know
what the true intent is here.
pickletester.py, test_pickle.py: Got rid of the hack for
exempting cPickle from running some of the proto 2 tests.
dictobject.c, PyDict_Next(): documented intended use.
this clarifies that they are part of an internal API (albeit shared
between pickle.py, copy_reg.py and cPickle.c).
I'd like to do the same for copy_reg.dispatch_table, but worry that it
might be used by existing code. This risk doesn't exist for the
extension registry.
outcome as __slotnames__ on the class. (Like __slots__, it's not safe
to ask for this as an attribute -- you must look for it in the
specific class's __dict__. But it must be set using attribute
notation, because __dict__ is a read-only proxy.)
Assorted code cleanups; e.g., sizeof(char) is 1 by definition, so there's
no need to do things like multiply by sizeof(char) in hairy malloc
arguments. Fixed an undetected-overflow bug in readline_file().
longobject.c: Fixed a really stupid bug in the new _PyLong_NumBits.
pickle.py: Fixed stupid bug in save_long(): When proto is 2, it
wrote LONG1 or LONG4, but forgot to return then -- it went on to
append the proto 1 LONG opcode too.
Fixed equally stupid cancelling bugs in load_long1() and
load_long4(): they *returned* the unpickled long instead of pushing
it on the stack. The return values were ignored. Tests passed
before only because save_long() pickled the long twice.
Fixed bugs in encode_long().
Noted that decode_long() is quadratic-time despite our hopes,
because long(string, 16) is still quadratic-time in len(string).
It's hex() that's linear-time. I don't know a way to make decode_long()
linear-time in Python, short of maybe transforming the 256's-complement
bytes into marshal's funky internal format, and letting marshal decode
that. It would be more valuable to make long(string, 16) linear time.
pickletester.py: Added a global "protocols" vector so tests can try
all the protocols in a sane way. Changed test_ints() and test_unicode()
to do so. Added a new test_long(), but the tail end of it is disabled
because it "takes forever" under pickle.py (but runs very quickly under
cPickle: cPickle proto 2 for longs is linear-time).
The 4th item can be None or an iterator yielding list items, which are
used to append() or extend() the object. The 5th item can be None or
an iterator yielding a dict's (key, value) pairs, which are stuffed
into the object using __setitem__.
Also (as a separate, though related, feature) add "batching" for list
and dict items. If you pickled a dict or list with a million items in
the past, it would push a million items onto the stack. It now pushes
only 1000 items at a time on the stack, using repeated APPENDS or
SETITEMS opcodes. (For lists, I hope that using many short extend()
calls doesn't exhibit quadratic behavior.)
__module__ is the string name of the module the function was defined
in, just like __module__ of classes. In some cases, particularly for
C functions, the __module__ may be None.
Change PyCFunction_New() from a function to a macro, but keep an
unused copy of the function around so that we don't change the binary
API.
Change pickle's save_global() to use whichmodule() if __module__ is
None, but add the __module__ logic to whichmodule() since it might be
used outside of pickle.
on the type instead of self.save(t). This defeated the purpose of
NEWOBJ, because it didn't generate a BINGET opcode when t was already
memoized; but moreover, it would generate multiple BINPUT opcodes for
the same type! pickletools.dis() doesn't like this.
How I found this? I was playing with picklesize.py in the datetime
sandbox, and noticed that protocol 2 pickles for multiple objects were
in fact larger than protocol 1 pickles! That was suspicious, so I
decided to disassemble one of the pickles.
This really needs a unit test, but I'm exhausted. I'll be late for
work as it is. :-(
the same function, don't save the state or write a BUILD opcode. This
is so that a type (e.g. datetime :-) can support protocol 2 using
__getnewargs__ while also supporting protocol 0 and 1 using
__getstate__. (Without this, the state would be pickled twice with
protocol 2, unless __getstate__ is defined to return None, which
breaks protocol 0 and 1.)
types. The special handling for these can now be removed from save_newobj().
Add some testing for this.
Also add support for setting the 'fast' flag on the Python Pickler class,
which suppresses use of the memo.
object.__reduce__, do a getattr() on the class so we can explicitly
test for it. The reduce()-calling code becomes a bit more regular as
a result.
Also add support slots: if an object has slots, the default state is
(dict, slots) where dict is the __dict__ or None, and slots is a dict
mapping slot names to slot values. We do a best-effort approach to
find slot names, assuming the __slots__ fields of classes aren't
modified after class definition time to misrepresent the actual list
of slots defined by a class.
Antoine Pitrou
Benjamin Peterson
Alexander Belopolsky
Victor Stinner
Alexandre Vassalotti
Mark Dickinson
Hirokazu Yamamoto
Guido van Rossum
Martin v. Löwis
Walter Dörwald
Georg Brandl
Nick Coghlan
Raymond Hettinger
Andrew M. Kuchling
Jeremy Hylton
Tim Peters
Neal Norwitz