Commit Graph

285 Commits

Author SHA1 Message Date
Guido van Rossum 99603b0c1e Getting rid of cPickle. Mmm, feels good! 2007-07-20 00:22:32 +00:00
Guido van Rossum f93254d299 Fix test_pickle, by reverting the string opcodes (S, T, U) to returning
strings, in Latin-1.  Bytes are once more pickled through bytes.__reduce__,
but now it returns "latin-1" as the second parameter.

Unfortunately this breaks datetime pickling.  I'll have to investigate
further; reverting Martin's changes doesn't seem to help.
2007-07-19 22:19:35 +00:00
Martin v. Löwis 10a60b3ec0 Change Py_BuildValue to generate Unicode objects for
's' and 'c' codes.
Change pickle to dump bytes objects using the 'S'
code, and to load the 'S' code as byte objects.
Change datetime and array to generate and expect
bytes objects in reduce/unreduce.
2007-07-18 02:28:27 +00:00
Guido van Rossum aa588c4699 Fix some problems introduced by the str8 repr change. 2007-06-15 03:35:38 +00:00
Walter Dörwald 42748a8d6d Rip out all codecs that can't work in a unicode/bytes world:
base64, uu, zlib, rot_13, hex, quopri, bz2, string_escape.

However codecs.escape_encode() and codecs.escape_decode()
still exist, as they are used for pickling str8 objects
(so those two functions can go, when the str8 type is removed).
2007-06-12 16:40:17 +00:00
Guido van Rossum 1325790b93 Merged revisions 55795-55816 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/p3yk

........
  r55797 | neal.norwitz | 2007-06-07 00:00:57 -0700 (Thu, 07 Jun 2007) | 3 lines

  Get rid of some remnants of classic classes.  types.ClassType == type.
  Also get rid of almost all uses of the types module and use the builtin name.
........
  r55798 | neal.norwitz | 2007-06-07 00:12:36 -0700 (Thu, 07 Jun 2007) | 1 line

  Remove a use of types, verify commit hook works
........
  r55809 | guido.van.rossum | 2007-06-07 11:11:29 -0700 (Thu, 07 Jun 2007) | 2 lines

  Fix syntax error introduced by Neal in last checkin.
........
2007-06-07 23:15:56 +00:00
Guido van Rossum d59da4b432 Merged revisions 55407-55513 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/p3yk

................
  r55413 | fred.drake | 2007-05-17 12:30:10 -0700 (Thu, 17 May 2007) | 1 line

  fix argument name in documentation; match the implementation
................
  r55430 | jack.diederich | 2007-05-18 06:39:59 -0700 (Fri, 18 May 2007) | 1 line

  Implements class decorators, PEP 3129.
................
  r55432 | guido.van.rossum | 2007-05-18 08:09:41 -0700 (Fri, 18 May 2007) | 2 lines

  obsubmit.
................
  r55434 | guido.van.rossum | 2007-05-18 09:39:10 -0700 (Fri, 18 May 2007) | 3 lines

  Fix bug in test_inspect.  (I presume this is how it should be fixed;
  Jack Diedrich, please verify.)
................
  r55460 | brett.cannon | 2007-05-20 00:31:57 -0700 (Sun, 20 May 2007) | 4 lines

  Remove the imageop module.  With imgfile already removed in Python 3.0 and
  rgbimg gone in Python 2.6 the unit tests themselves were made worthless.  Plus
  third-party libraries perform the same function much better.
................
  r55469 | neal.norwitz | 2007-05-20 11:28:20 -0700 (Sun, 20 May 2007) | 118 lines

  Merged revisions 55324-55467 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r55348 | georg.brandl | 2007-05-15 13:19:34 -0700 (Tue, 15 May 2007) | 4 lines

    HTML-escape the plain traceback in cgitb's HTML output, to prevent
    the traceback inadvertently or maliciously closing the comment and
    injecting HTML into the error page.
  ........
    r55372 | neal.norwitz | 2007-05-15 21:33:50 -0700 (Tue, 15 May 2007) | 6 lines

    Port rev 55353 from Guido:
    Add what looks like a necessary call to PyErr_NoMemory() when PyMem_MALLOC()
    fails.

    Will backport.
  ........
    r55377 | neal.norwitz | 2007-05-15 22:06:33 -0700 (Tue, 15 May 2007) | 1 line

    Mention removal of some directories for obsolete platforms
  ........
    r55380 | brett.cannon | 2007-05-15 22:50:03 -0700 (Tue, 15 May 2007) | 2 lines

    Change the maintainer of the BeOS port.
  ........
    r55383 | georg.brandl | 2007-05-16 06:44:18 -0700 (Wed, 16 May 2007) | 2 lines

    Bug #1719995: don't use deprecated method in sets example.
  ........
    r55386 | neal.norwitz | 2007-05-16 13:05:11 -0700 (Wed, 16 May 2007) | 5 lines

    Fix bug in marshal where bad data would cause a segfault due to
    lack of an infinite recursion check.

    Contributed by Damien Miller at Google.
  ........
    r55389 | brett.cannon | 2007-05-16 15:42:29 -0700 (Wed, 16 May 2007) | 6 lines

    Remove the gopherlib module.  It has been raising a DeprecationWarning since
    Python 2.5.

    Also remove gopher support from urllib/urllib2.  As both imported gopherlib the
    usage of the support would have raised a DeprecationWarning.
  ........
    r55394 | raymond.hettinger | 2007-05-16 18:08:04 -0700 (Wed, 16 May 2007) | 1 line

    calendar.py gets no benefit from xrange() instead of range()
  ........
    r55395 | brett.cannon | 2007-05-16 19:02:56 -0700 (Wed, 16 May 2007) | 3 lines

    Complete deprecation of BaseException.message.  Some subclasses were directly
    accessing the message attribute instead of using the descriptor.
  ........
    r55396 | neal.norwitz | 2007-05-16 23:11:36 -0700 (Wed, 16 May 2007) | 4 lines

    Reduce the max stack depth to see if this fixes the segfaults on
    Windows and some other boxes.  If this is successful, this rev should
    be backported.  I'm not sure how close to the limit we should push this.
  ........
    r55397 | neal.norwitz | 2007-05-16 23:23:50 -0700 (Wed, 16 May 2007) | 4 lines

    Set the depth to something very small to try to determine if the
    crashes on Windows are really due to the stack size or possibly
    some other problem.
  ........
    r55398 | neal.norwitz | 2007-05-17 00:04:46 -0700 (Thu, 17 May 2007) | 4 lines

    Last try for tweaking the max stack depth.  5000 was the original value,
    4000 didn't work either.  1000 does work on Windows.  If 2000 works,
    that will hopefully be a reasonable balance.
  ........
    r55412 | fred.drake | 2007-05-17 12:29:58 -0700 (Thu, 17 May 2007) | 1 line

    fix argument name in documentation; match the implementation
  ........
    r55427 | neal.norwitz | 2007-05-17 22:47:16 -0700 (Thu, 17 May 2007) | 1 line

    Verify neither dumps or loads overflow the stack and segfault.
  ........
    r55446 | collin.winter | 2007-05-18 16:11:24 -0700 (Fri, 18 May 2007) | 1 line

    Backport PEP 3110's new 'except' syntax to 2.6.
  ........
    r55448 | raymond.hettinger | 2007-05-18 18:11:16 -0700 (Fri, 18 May 2007) | 1 line

    Improvements to NamedTuple's implementation, tests, and documentation
  ........
    r55449 | raymond.hettinger | 2007-05-18 18:50:11 -0700 (Fri, 18 May 2007) | 1 line

    Fix beginner mistake -- don't mix spaces and tabs.
  ........
    r55450 | neal.norwitz | 2007-05-18 20:48:47 -0700 (Fri, 18 May 2007) | 1 line

    Clear data so random memory does not get freed.  Will backport.
  ........
    r55452 | neal.norwitz | 2007-05-18 21:34:55 -0700 (Fri, 18 May 2007) | 3 lines

    Whoops, need to pay attention to those test failures.
    Move the clear to *before* the first use, not after.
  ........
    r55453 | neal.norwitz | 2007-05-18 21:35:52 -0700 (Fri, 18 May 2007) | 1 line

    Give some clue as to what happened if the test fails.
  ........
    r55455 | georg.brandl | 2007-05-19 11:09:26 -0700 (Sat, 19 May 2007) | 2 lines

    Fix docstring for add_package in site.py.
  ........
    r55458 | brett.cannon | 2007-05-20 00:09:50 -0700 (Sun, 20 May 2007) | 2 lines

    Remove the rgbimg module.  It has been deprecated since Python 2.5.
  ........
    r55465 | nick.coghlan | 2007-05-20 04:12:49 -0700 (Sun, 20 May 2007) | 1 line

    Fix typo in example (should be backported, but my maintenance branch is woefully out of date)
  ........
................
  r55472 | brett.cannon | 2007-05-20 12:06:18 -0700 (Sun, 20 May 2007) | 2 lines

  Remove imageop from the Windows build process.
................
  r55486 | neal.norwitz | 2007-05-20 23:59:52 -0700 (Sun, 20 May 2007) | 1 line

  Remove callable() builtin
................
  r55506 | neal.norwitz | 2007-05-22 00:43:29 -0700 (Tue, 22 May 2007) | 78 lines

  Merged revisions 55468-55505 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r55468 | neal.norwitz | 2007-05-20 11:06:27 -0700 (Sun, 20 May 2007) | 1 line

    rotor is long gone.
  ........
    r55470 | neal.norwitz | 2007-05-20 11:43:00 -0700 (Sun, 20 May 2007) | 1 line

    Update directories/files at the top-level.
  ........
    r55471 | brett.cannon | 2007-05-20 12:05:06 -0700 (Sun, 20 May 2007) | 2 lines

    Try to remove rgbimg from Windows builds.
  ........
    r55474 | brett.cannon | 2007-05-20 16:17:38 -0700 (Sun, 20 May 2007) | 4 lines

    Remove the macfs module.  This led to the deprecation of macostools.touched();
    it completely relied on macfs and is a no-op on OS X according to code
    comments.
  ........
    r55476 | brett.cannon | 2007-05-20 16:56:18 -0700 (Sun, 20 May 2007) | 3 lines

    Move imgfile import to the global namespace to trigger an import error ASAP to
    prevent creation of a test file.
  ........
    r55477 | brett.cannon | 2007-05-20 16:57:38 -0700 (Sun, 20 May 2007) | 3 lines

    Cause posixfile to raise a DeprecationWarning.  Documented as deprecated since
    Ptyhon 1.5.
  ........
    r55479 | andrew.kuchling | 2007-05-20 17:03:15 -0700 (Sun, 20 May 2007) | 1 line

    Note removed modules
  ........
    r55481 | martin.v.loewis | 2007-05-20 21:35:47 -0700 (Sun, 20 May 2007) | 2 lines

    Add Alexandre Vassalotti.
  ........
    r55482 | george.yoshida | 2007-05-20 21:41:21 -0700 (Sun, 20 May 2007) | 4 lines

    fix against r55474 [Remove the macfs module]

    Remove "libmacfs.tex" from Makefile.deps and mac/mac.tex.
  ........
    r55487 | raymond.hettinger | 2007-05-21 01:13:35 -0700 (Mon, 21 May 2007) | 1 line

    Replace assertion with straight error-checking.
  ........
    r55489 | raymond.hettinger | 2007-05-21 09:40:10 -0700 (Mon, 21 May 2007) | 1 line

    Allow all alphanumeric and underscores in type and field names.
  ........
    r55490 | facundo.batista | 2007-05-21 10:32:32 -0700 (Mon, 21 May 2007) | 5 lines


    Added timeout support to HTTPSConnection, through the
    socket.create_connection function. Also added a small
    test for this, and updated NEWS file.
  ........
    r55495 | georg.brandl | 2007-05-21 13:34:16 -0700 (Mon, 21 May 2007) | 2 lines

    Patch #1686487: you can now pass any mapping after '**' in function calls.
  ........
    r55502 | neal.norwitz | 2007-05-21 23:03:36 -0700 (Mon, 21 May 2007) | 1 line

    Document new params to HTTPSConnection
  ........
    r55504 | neal.norwitz | 2007-05-22 00:16:10 -0700 (Tue, 22 May 2007) | 1 line

    Stop using METH_OLDARGS
  ........
    r55505 | neal.norwitz | 2007-05-22 00:16:44 -0700 (Tue, 22 May 2007) | 1 line

    Stop using METH_OLDARGS implicitly
  ........
................
2007-05-22 18:11:13 +00:00
Guido van Rossum a8add0ec5e Merged revisions 55270-55324 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/p3yk

........
  r55271 | fred.drake | 2007-05-11 10:14:47 -0700 (Fri, 11 May 2007) | 3 lines

  remove jpeg, panel libraries for SGI; there is more IRIX stuff left over,
  I guess that should be removed too, but will leave for someone who is sure
........
  r55280 | fred.drake | 2007-05-11 19:11:37 -0700 (Fri, 11 May 2007) | 1 line

  remove mention of file that has been removed
........
  r55301 | brett.cannon | 2007-05-13 17:38:05 -0700 (Sun, 13 May 2007) | 4 lines

  Remove rexec and Bastion from the stdlib.  This also eliminates the need for
  f_restricted on frames.  This in turn negates the need for
  PyEval_GetRestricted() and PyFrame_IsRestricted().
........
  r55303 | brett.cannon | 2007-05-13 19:22:22 -0700 (Sun, 13 May 2007) | 2 lines

  Remove the md5 and sha modules.
........
  r55305 | george.yoshida | 2007-05-13 19:45:55 -0700 (Sun, 13 May 2007) | 2 lines

  fix markup
........
  r55306 | neal.norwitz | 2007-05-13 19:47:57 -0700 (Sun, 13 May 2007) | 1 line

  Get the doc building again after some removals.
........
  r55307 | neal.norwitz | 2007-05-13 19:50:45 -0700 (Sun, 13 May 2007) | 1 line

  Get test_pyclbr passing again after getstatus was removed from commands.  This "test case" was weird since it was just importing a seemingly random module.  Remove the import
........
  r55322 | brett.cannon | 2007-05-14 14:09:20 -0700 (Mon, 14 May 2007) | 3 lines

  Remove the compiler package.  Will eventually need a mechanism to byte compile
  an AST.
........
2007-05-14 22:03:55 +00:00
Guido van Rossum cfe5f20fe8 Got test_pickletools and test_pickle working.
(Alas, test_cpickle is still broken.)
2007-05-08 21:26:54 +00:00
Guido van Rossum b8142c3e64 Got test_exceptions.py working. 2007-05-08 17:49:10 +00:00
Guido van Rossum 805365ee39 Merged revisions 55007-55179 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/p3yk

........
  r55077 | guido.van.rossum | 2007-05-02 11:54:37 -0700 (Wed, 02 May 2007) | 2 lines

  Use the new print syntax, at least.
........
  r55142 | fred.drake | 2007-05-04 21:27:30 -0700 (Fri, 04 May 2007) | 1 line

  remove old cruftiness
........
  r55143 | fred.drake | 2007-05-04 21:52:16 -0700 (Fri, 04 May 2007) | 1 line

  make this work with the new Python
........
  r55162 | neal.norwitz | 2007-05-06 22:29:18 -0700 (Sun, 06 May 2007) | 1 line

  Get asdl code gen working with Python 2.3.  Should continue to work with 3.0
........
  r55164 | neal.norwitz | 2007-05-07 00:00:38 -0700 (Mon, 07 May 2007) | 1 line

  Verify checkins to p3yk (sic) branch go to 3000 list.
........
  r55166 | neal.norwitz | 2007-05-07 00:12:35 -0700 (Mon, 07 May 2007) | 1 line

  Fix this test so it runs again by importing warnings_test properly.
........
  r55167 | neal.norwitz | 2007-05-07 01:03:22 -0700 (Mon, 07 May 2007) | 8 lines

  So long xrange.  range() now supports values that are outside
  -sys.maxint to sys.maxint.  floats raise a TypeError.

  This has been sitting for a long time.  It probably has some problems and
  needs cleanup.  Objects/rangeobject.c now uses 4-space indents since
  it is almost completely new.
........
  r55171 | guido.van.rossum | 2007-05-07 10:21:26 -0700 (Mon, 07 May 2007) | 4 lines

  Fix two tests that were previously depending on significant spaces
  at the end of a line (and before that on Python 2.x print behavior
  that has no exact equivalent in 3.0).
........
2007-05-07 22:24:25 +00:00
Guido van Rossum 1255ed62bf Much more pickling now works. 2007-05-04 20:30:19 +00:00
Guido van Rossum 2e6a4b37ba Checkpoint. Make pickle.py read/write bytes.
Fix strict.py so it reads/writes bytes and accepts both string types as format.
2007-05-04 19:56:22 +00:00
Guido van Rossum ef87d6ed94 Rip out all the u"..." literals and calls to unicode(). 2007-05-02 19:09:54 +00:00
Georg Brandl a18af4e7a2 PEP 3114: rename .next() to .__next__() and add next() builtin. 2007-04-21 15:47:16 +00:00
Nick Coghlan 650f0d06d3 Hide list comp variables and support set comprehensions 2007-04-15 12:05:43 +00:00
Guido van Rossum 634e53fad4 Fix a bizarre error where test_pickletools fails if preceded by test_pyclbr.
The fix is in neither, but in pickle.py where a loop over sys.modules.items()
could modify sys.modules, occasionally.
2007-02-26 07:07:02 +00:00
Guido van Rossum cc2b016125 - PEP 3106: dict.iterkeys(), .iteritems(), .itervalues() are now gone;
and .keys(), .items(), .values() return dict views.

The dict views aren't fully functional yet; in particular, they can't
be compared to sets yet.  but they are useful as "iterator wells".

There are still 27 failing unit tests; I expect that many of these
have fairly trivial fixes, but there are so many, I could use help.
2007-02-11 06:12:03 +00:00
Guido van Rossum e2a383d062 Rip out 'long' and 'L'-suffixed integer literals.
(Rough first cut.)
2007-01-15 16:59:06 +00:00
Guido van Rossum ddefaf31b3 Merged the int/long unification branch, by very crude means (sorry Thomas!).
I banged on the code (beyond what's in that branch) to make fewer tests fail;
the only tests that fail now are:
  test_descr -- can't pickle ints?!
  test_pickletools -- ???
  test_socket -- See python.org/sf/1619659
  test_sqlite -- ???
I'll deal with those later.
2007-01-14 03:31:43 +00:00
Guido van Rossum b940e113bf SF patch 1631942 by Collin Winter:
(a) "except E, V" -> "except E as V"
(b) V is now limited to a simple name (local variable)
(c) V is now deleted at the end of the except block
2007-01-10 16:19:56 +00:00
Guido van Rossum e2b70bcf74 Get rid of dict.has_key(). Boy this has a lot of repercussions!
Not all code has been fixed yet; this is just a checkpoint...
The C API still has PyDict_HasKey() and _HasKeyString(); not sure
if I want to change those just yet.
2006-08-18 22:13:04 +00:00
Guido van Rossum 65810fee5e SF patch 1495675: Remove types.InstanceType and new.instance
(Collin Winter)
2006-05-26 19:12:38 +00:00
Raymond Hettinger fe59dc1bd8 Revert previous checkin. 2005-02-07 15:28:45 +00:00
Raymond Hettinger f715366f23 Reduce the usage of the types module. 2005-02-07 14:16:21 +00:00
Raymond Hettinger a6b45cc31d Eliminate the deprecated option to return None instead of a tuple of arguments in __reduce__(). 2004-12-07 07:05:57 +00:00
Raymond Hettinger 3489cad30a Removed the deprecated bin parameter from the pickle module. 2004-12-05 05:20:42 +00:00
Andrew M. Kuchling dc54f2be3f Make 'bin' argument trigger DeprecationWarning 2004-08-07 16:27:24 +00:00
Walter Dörwald 70a6b49821 Replace backticks with repr() or "%r"
From SF patch #852334.
2004-02-12 17:35:32 +00:00
Jeremy Hylton e3a565eed0 Fix grammar in comment. 2003-06-29 16:59:59 +00:00
Guido van Rossum 443ada47c3 Remove unused _better_reduce (which will disappear soon) and
_reconstructor (whose import here is a mystery to me).
2003-02-18 22:49:10 +00:00
Guido van Rossum c53f009f94 Introducing __reduce_ex__, which is called with a protocol number argument
if it exists in preference over __reduce__.  Now Tim can go implement this
in cPickle.c.
2003-02-18 22:05:12 +00:00
Tim Peters 080c88b912 cPickle.c, load_build(): Taught cPickle how to pick apart
the optional proto 2 slot state.

pickle.py, load_build():  CAUTION:  Noted that cPickle's
load_build and pickle's load_build really don't do the same
things with the state, and didn't before this patch either.
cPickle never tries to do .update(), and has no backoff if
instance.__dict__ can't be retrieved.  There are no tests
that can tell the difference, and part of what cPickle's
load_build() did looked accidental to me, so I don't know
what the true intent is here.

pickletester.py, test_pickle.py:  Got rid of the hack for
exempting cPickle from running some of the proto 2 tests.

dictobject.c, PyDict_Next():  documented intended use.
2003-02-15 03:01:11 +00:00
Tim Peters 8587b3c073 Added a HIGHEST_PROTOCOL module attribute to pickle and cPickle. 2003-02-13 15:44:41 +00:00
Tim Peters 42f08ac1e3 Implemented batching for dicts in cPickle. This is after two failed
attempts to merge the C list-batch and dict-batch code -- they worked, but
it was a godawful mess to read.
2003-02-11 22:43:24 +00:00
Tim Peters 1092d64002 Implemented list batching in cPickle. 2003-02-11 21:06:20 +00:00
Guido van Rossum cf117b0b40 Rename 'proto' keyword arg to 'protocol' . Greg Ward's suggestion. 2003-02-09 17:19:41 +00:00
Guido van Rossum 5aac4e6312 Move _better_reduce from copy.py to copy_reg.py, and also use it in
pickle.py, where it makes save_newobj() unnecessary.  Tests pass.
2003-02-06 22:57:00 +00:00
Guido van Rossum 98b922c0ec Remove a debug print statement. 2003-02-06 19:30:38 +00:00
Jeremy Hylton cc1fccbc1c No need for a continuation line. 2003-02-06 16:23:01 +00:00
Jeremy Hylton 656d4519b5 Replace hasattr() + getattr() with single getattr() and default value. 2003-02-06 16:22:01 +00:00
Guido van Rossum d4b920c8bd Rename the extension registry variables to have leading underscores --
this clarifies that they are part of an internal API (albeit shared
between pickle.py, copy_reg.py and cPickle.c).

I'd like to do the same for copy_reg.dispatch_table, but worry that it
might be used by existing code.  This risk doesn't exist for the
extension registry.
2003-02-04 01:54:49 +00:00
Guido van Rossum 93fe564217 _slotnames(): this is a fairly expensive calculation. Cache the
outcome as __slotnames__ on the class.  (Like __slots__, it's not safe
to ask for this as an attribute -- you must look for it in the
specific class's __dict__.  But it must be set using attribute
notation, because __dict__ is a read-only proxy.)
2003-02-03 19:46:54 +00:00
Guido van Rossum 868ecc22ab _slotnames(): exclude __dict__ and __weakref__; these aren't real
slots even though they can be listed in __slots__.
2003-02-03 18:10:09 +00:00
Guido van Rossum 795ea89cb5 Support keyword argument 'bin', with a pending deprecation warning. 2003-02-03 16:59:48 +00:00
Tim Peters 1d63c9f151 cPickle support for TUPLE[123]. Incidentally plugged several undetected
overflow holes in Pdata_grow().
2003-02-02 20:29:39 +00:00
Tim Peters bf2674be0e long(string, base) now takes time linear in len(string) when base is a
power of 2.  Enabled the tail end of test_long() in pickletester.py
because it no longer takes forever when run from test_pickle.py.
2003-02-02 07:51:32 +00:00
Tim Peters ee1a53cbb1 cPickle.c: Full support for the new LONG1 and LONG4. Added comments.
Assorted code cleanups; e.g., sizeof(char) is 1 by definition, so there's
no need to do things like multiply by sizeof(char) in hairy malloc
arguments.  Fixed an undetected-overflow bug in readline_file().

longobject.c:  Fixed a really stupid bug in the new _PyLong_NumBits.

pickle.py:  Fixed stupid bug in save_long():  When proto is 2, it
wrote LONG1 or LONG4, but forgot to return then -- it went on to
append the proto 1 LONG opcode too.
Fixed equally stupid cancelling bugs in load_long1() and
load_long4():  they *returned* the unpickled long instead of pushing
it on the stack.  The return values were ignored.  Tests passed
before only because save_long() pickled the long twice.

Fixed bugs in encode_long().

Noted that decode_long() is quadratic-time despite our hopes,
because long(string, 16) is still quadratic-time in len(string).
It's hex() that's linear-time.  I don't know a way to make decode_long()
linear-time in Python, short of maybe transforming the 256's-complement
bytes into marshal's funky internal format, and letting marshal decode
that.  It would be more valuable to make long(string, 16) linear time.

pickletester.py:  Added a global "protocols" vector so tests can try
all the protocols in a sane way.  Changed test_ints() and test_unicode()
to do so.  Added a new test_long(), but the tail end of it is disabled
because it "takes forever" under pickle.py (but runs very quickly under
cPickle:  cPickle proto 2 for longs is linear-time).
2003-02-02 02:57:53 +00:00
Tim Peters 5bd2a79b22 The C pickle now knows how to deal with a proto= argument. Assorted
code cleanups, and purged more references to text-vs-binary modes.
2003-02-01 16:45:06 +00:00
Guido van Rossum 7eff63abce Change the default protocol back to 0.
Add a feature suggested by Tim: a negative protocol value means to use
the largest protocol value supported.
2003-01-31 19:42:31 +00:00
Guido van Rossum 25cb7dfb0f Another extension to reduce(). It can return a 4- or 5-tuple now.
The 4th item can be None or an iterator yielding list items, which are
used to append() or extend() the object.  The 5th item can be None or
an iterator yielding a dict's (key, value) pairs, which are stuffed
into the object using __setitem__.

Also (as a separate, though related, feature) add "batching" for list
and dict items.  If you pickled a dict or list with a million items in
the past, it would push a million items onto the stack.  It now pushes
only 1000 items at a time on the stack, using repeated APPENDS or
SETITEMS opcodes.  (For lists, I hope that using many short extend()
calls doesn't exhibit quadratic behavior.)
2003-01-31 18:53:21 +00:00
Jeremy Hylton 4f0dcc9a9a Provide __module__ attributes for functions defined in C and Python.
__module__ is the string name of the module the function was defined
in, just like __module__ of classes.  In some cases, particularly for
C functions, the __module__ may be None.

Change PyCFunction_New() from a function to a macro, but keep an
unused copy of the function around so that we don't change the binary
API.

Change pickle's save_global() to use whichmodule() if __module__ is
None, but add the __module__ logic to whichmodule() since it might be
used outside of pickle.
2003-01-31 18:33:18 +00:00
Guido van Rossum f7f4517fae Pass the object to save_reduce(), so the memoize() call can go into
save_reduce(), before the state is pickled.  This makes it possible
for an object to be referenced from its own (mutable) state.
2003-01-31 17:17:49 +00:00
Guido van Rossum d053b4b416 Add a magical feature to save_reduce so that __reduce__ can cause
NEWOBJ to be generated.
2003-01-31 16:51:45 +00:00
Tim Peters 4b23f2b44b It's Official: for LONG1/LONG4, a "byte count" of 0 is taken as a
shortcut meaning 0L.  This allows LONG1 to encode 0L in two bytes
total.
2003-01-31 16:43:39 +00:00
Neal Norwitz d17406830c Fix typo 2003-01-31 04:04:23 +00:00
Tim Peters 91149821d3 Linear-time implementations of {encode,decode}_long. 2003-01-31 03:43:58 +00:00
Tim Peters d01c1e91c4 load_inst(), load_obj(): Put the bulk of these into a common new
_instantiate() method.
2003-01-30 15:41:46 +00:00
Guido van Rossum 9b40e804c7 There was a subtle big in save_newobj(): it used self.save_global(t)
on the type instead of self.save(t).  This defeated the purpose of
NEWOBJ, because it didn't generate a BINGET opcode when t was already
memoized; but moreover, it would generate multiple BINPUT opcodes for
the same type!  pickletools.dis() doesn't like this.

How I found this?  I was playing with picklesize.py in the datetime
sandbox, and noticed that protocol 2 pickles for multiple objects were
in fact larger than protocol 1 pickles!  That was suspicious, so I
decided to disassemble one of the pickles.

This really needs a unit test, but I'm exhausted.  I'll be late for
work as it is. :-(
2003-01-30 06:37:41 +00:00
Guido van Rossum 4fba220f4a Slight code rearrangement to avoid testing getstate twice. 2003-01-30 05:41:19 +00:00
Guido van Rossum 45486176ea In save_newobj(), if an object's __getnewargs__ and __getstate__ are
the same function, don't save the state or write a BUILD opcode.  This
is so that a type (e.g. datetime :-) can support protocol 2 using
__getnewargs__ while also supporting protocol 0 and 1 using
__getstate__.  (Without this, the state would be pickled twice with
protocol 2, unless __getstate__ is defined to return None, which
breaks protocol 0 and 1.)
2003-01-30 05:39:04 +00:00
Guido van Rossum ba884f3d22 Use %c rather than chr() to turn some ints into chars. 2003-01-29 20:14:23 +00:00
Guido van Rossum 5d9113d8be Implement appropriate __getnewargs__ for all immutable subclassable builtin
types.  The special handling for these can now be removed from save_newobj().
Add some testing for this.

Also add support for setting the 'fast' flag on the Python Pickler class,
which suppresses use of the memo.
2003-01-29 17:58:45 +00:00
Guido van Rossum 586c9e813c Declare Protocol 2 as implemented. 2003-01-29 06:16:12 +00:00
Guido van Rossum 255f3ee0a5 Support for extension codes. (By accident I checked in the tests first.) 2003-01-29 06:14:11 +00:00
Tim Peters c0c12b5707 pickle: Comment repair.
pickletools:  Import decode_long from pickle instead of duplicating it.
2003-01-29 00:56:17 +00:00
Guido van Rossum 4e2491dbb1 Add a comment about how some built-in types should grow a
__getnewargs__ method.
2003-01-28 22:31:25 +00:00
Guido van Rossum b26a97aa50 Get rid of __safe_for_unpickling__ and safe_constructors.
Also tidied up a few lines, got rid of apply(), added a comment.
2003-01-28 22:29:13 +00:00
Guido van Rossum ac5b5d2e8b Instead of bad hacks trying to worm around the inherited
object.__reduce__, do a getattr() on the class so we can explicitly
test for it.  The reduce()-calling code becomes a bit more regular as
a result.

Also add support slots: if an object has slots, the default state is
(dict, slots) where dict is the __dict__ or None, and slots is a dict
mapping slot names to slot values.  We do a best-effort approach to
find slot names, assuming the __slots__ fields of classes aren't
modified after class definition time to misrepresent the actual list
of slots defined by a class.
2003-01-28 22:01:16 +00:00
Guido van Rossum 3d8c01b31c The default __reduce__ on the base object type obscured any
possibility of calling save_reduce().  Add a special hack for this.
The tests for this are much simpler now (no __getstate__ or
__getnewargs__ needed).
2003-01-28 19:48:18 +00:00
Guido van Rossum 54fb192508 Move the NEWOBJ-generating code to a separate function, and invoke it
after checking for __reduce__.
2003-01-28 18:22:35 +00:00
Guido van Rossum 533dbcf250 Some experimental support for generating NEWOBJ with proto=2, and
fixed a bug in load_newobj().
2003-01-28 17:55:05 +00:00
Tim Peters a6ae9a2128 save_empty_tuple(): Comment on why we can't get rid of this. 2003-01-28 16:58:41 +00:00
Tim Peters 82ca59e002 save_dict(): Added a comment about the control flow NealN missed. 2003-01-28 16:47:59 +00:00
Tim Peters 13a25fb8e6 _is_string_secure(): This method is no longer used; removed it. (It
was used before string-escape codecs were added to the core.)
2003-01-28 16:42:22 +00:00
Guido van Rossum bc64e22ed6 Made save() fit on a page, while adding comments. (I moved some type
checks to save_reduce(), which can also be called from a subclass.)

Also tweaked some more comments.
2003-01-28 16:34:19 +00:00
Tim Peters ad5a771fae Got rid of the _quotes global. Used only once, and is trivial. 2003-01-28 16:23:33 +00:00
Guido van Rossum 1be3175992 Add a few comments. Change the way the protocol is checked (it must
be one of 0, 1 or 2).

I should note that the previous checkin also added NEWOBJ support to
the unpickler -- but there's nothing yet that generates this.
2003-01-28 15:19:53 +00:00
Guido van Rossum 3a41c61dd4 Rename all variables 'object' to 'obj' to avoid conflicts with the
type 'object'.  Also minor docstring tweakage, and rearranged a few
lines in save().
2003-01-28 15:10:22 +00:00
Guido van Rossum cbe2dbddda Don't memoize the empty tuple in protocol 0. 2003-01-28 14:40:16 +00:00
Tim Peters d97da80dd5 save_tuple(): So long as the charter is rewriting for clarity, the snaky
control flow had to be simplified.
2003-01-28 05:48:29 +00:00
Tim Peters ff57bff16e save_tuple(): I believe the new code for TUPLE{1,2,3} in proto 2 was
incorrect for recursive tuples.  Tried to repair; seems to work OK, but
there are no checked-in tests for this yet.
2003-01-28 05:34:53 +00:00
Guido van Rossum 7d97d31a1b OK, this is really the last one tonight!
NEWFALSE and NEWTRUE.
2003-01-28 04:25:27 +00:00
Guido van Rossum 44f0ea5f73 More protocol 2: TUPLE1, TUPLE2, TUPLE3.
Also moved the special case for empty tuples from save() to save_tuple().
2003-01-28 04:14:51 +00:00
Tim Peters 3b769835ca save_inst(): Rewrote to have only one branch on self.bin. Also got rid
of my recent XXX comment, taking a (what appears to be vanishingly small)
chance and calling self.memoize() instead.
2003-01-28 03:51:36 +00:00
Guido van Rossum d6c9e63af9 First baby steps towards implementing protocol 2: PROTO, LONG1 and LONG4. 2003-01-28 03:49:52 +00:00
Tim Peters d95c2df3a9 Fixed odd whitespace after "if", which I believe I introduced long ago. 2003-01-28 03:41:54 +00:00
Tim Peters 8fda7bc48d save_int(): Fixed two new off-by-1 glitches. 2003-01-28 03:40:52 +00:00
Guido van Rossum e0b904232f Add a comment explaining that struct.pack() beats marshal.dumps(), but
marshal.loads() beats struct.unpack()!  Possibly because the latter
creates a one-tuple. :-(
2003-01-28 03:17:21 +00:00
Guido van Rossum 5c938d00a1 Got rid of mdumps; I timed it, and struct.pack("<i", x) is more than
40% faster than marshal.dumps(x)[1:]!  (That's not counting the
module attribute lookups, which can be avoided in either case.)
2003-01-28 03:03:08 +00:00
Tim Peters f558da0f90 save_tuple(): Minor rewriting, and added a comment about the subtlety
created by recursive tuples.
2003-01-28 02:09:55 +00:00
Tim Peters 209ad95b00 load_appends(): replaced .append() loop with an .extend(). 2003-01-28 01:44:45 +00:00
Tim Peters c23d18a955 Comments. 2003-01-28 01:41:51 +00:00
Tim Peters 064567e41a save_dict(): Untangled most of the bin-vs-not-bin logic. Also used
iteritems() instead of materializing a (possibly giant) list of the
items.
2003-01-28 01:34:43 +00:00
Tim Peters 21c18f0bf5 save_list(): Rewrote, to untangle the proto 0 from the proto 1 cases.
The code is much easier to follow now, and I bet it's faster too.
2003-01-28 01:15:46 +00:00
Tim Peters 22dc6f4f7a save_list(): removed unused local "d". 2003-01-28 01:07:48 +00:00
Tim Peters 58d7846e68 save_list(): removed unused local "memo". 2003-01-28 01:07:10 +00:00
Tim Peters bd1cdb9227 save_pers(): Switched the order of cases, to get rid of a "not", and to
make the bin-vs-not-bin order consistent with what other routines try to
do (they almost all handle the bin case first).
2003-01-28 01:03:10 +00:00
Tim Peters 518df0dae4 Several routines appeared to inline the guts of memoize(), possibly for
some notion of low-level efficiency.  Undid that, but left one routine
alone:  save_inst() claims it has a reason for not using memoize().
I don't understand that comment, so added an XXX comment there.
2003-01-28 01:00:38 +00:00
Tim Peters b32a8317d7 save(): Fiddled the control flow to put the normal case where it
belongs.  This is a much smaller change than it may appear:  the bulk
of the function merely got unindented by one level.
2003-01-28 00:48:09 +00:00
Tim Peters c9d7c4a656 Added XXX about save()'s special-casing of tuples -- I don't get it. 2003-01-28 00:43:26 +00:00
Tim Peters 22987e3cbd save_bool(): simplified. 2003-01-28 00:26:14 +00:00
Tim Peters cbd0a32d09 Repaired grammar in new comment. 2003-01-28 00:24:43 +00:00
Tim Peters b377f8ac3e clear_memo(): Repaired grammar in docstring. 2003-01-28 00:23:36 +00:00
Tim Peters e105478a0c Added one-line comments to the proto 2 opcodes. 2003-01-28 00:22:12 +00:00
Tim Peters fdb8cfab08 Removed the new LONG2 opcode: it's extravagant. If LONG1 isn't enough,
then the embedded argument consumes at least 256 bytes.  The difference
between a 3-byte prefix (LONG2 + 2 bytes) and a 5-byte prefix (LONG4 +
4 bytes) is at worst less than 1%.  Note that binary strings and binary
Unicode strings also have only "size is 1 byte, or size is 4 bytes?"
flavors, and I expect for the same reason.  The only place a 2-byte
thingie was used was in BININT2, where the 2 bytes make up the *entire*
embedded argument (and now EXT2 also does this); that's a large savings
over 4 bytes, because the total opcode+argument size is so small in
the BININT2/EXT2 case.

Removed the TAKEN_FROM_ARGUMENT "number of bytes" code, and bifurcated it
into TAKEN_FROM_ARGUMENT1 and TAKEN_FROM_ARGUMENT4.  Now there's enough
info in ArgumentDescriptor objects to deduce the # of bytes consumed by
each opcode.

Rearranged the order in which proto2 opcodes are listed in pickle.py.
2003-01-28 00:13:19 +00:00
Guido van Rossum f29d3d6011 Begin the change from 'binary vs. text mode' to 'protocol 0, 1, 2'.
The protocol now defaults to 1.  Protocol 2 is still unimplemented.
2003-01-27 22:47:53 +00:00
Guido van Rossum 5a2d8f5e9a Begin documenting protocol 2. 2003-01-27 21:44:25 +00:00
Tim Peters bb38e306ae Added some comments. 2003-01-27 21:25:41 +00:00
Tim Peters e46b73f5b3 memoize(): Reworded the docs to try to disentangle the Pickler's memo
dict from the Unpickler's memo (which is a different beast!).
2003-01-27 21:22:10 +00:00
Tim Peters bbf63cd05c Using marshal functions to pack & unpack 1-byte ints is an obscure &
expensive way to spell chr() and ord().
2003-01-27 21:15:36 +00:00
Tim Peters 22a449aacc Added a brief comment to each pickle opcode declaration. 2003-01-27 20:16:36 +00:00
Jeremy Hylton 3422c99de1 Raise PicklingError when __reduce__() fails, and
add memoize() helper function to update the memo.

The first element of the tuple returned by __reduce__() must be a
callable.  If it isn't the Unpickler will raise an error.  Catch this
error in the pickler and raise the error there.

The memoize() helper also has a comment explaining how the memo
works.  So methods can't use memoize() because the write funny codes.
2003-01-24 19:29:52 +00:00
Jeremy Hylton 5e0f4e73a9 Remove inst_persistent_id() WANNI (we ain't never needed it).
Add some simple tests of the persistence hooks.
2002-11-13 22:01:27 +00:00
Jeremy Hylton f0cfdf7314 Fiddle comments and variable names in whichmodule(). 2002-09-19 23:00:12 +00:00
Jeremy Hylton 065a5ab8fb whichmodule() should skip dummy package entries in sys.modules.
This fixes the charming, but unhelpful error message for
>>> pickle.dumps(type.__new__)
Can't pickle <built-in method __new__ of type object at 0x812a440>: it's not the same object as datetime.math.__new__

Bugfix candidate.
2002-09-19 22:57:26 +00:00
Martin v. Löwis 8a8da798a5 Patch #505705: Remove eval in pickle and cPickle. 2002-08-14 07:46:28 +00:00
Jeremy Hylton 39c6116483 Given the persistent id code a shot at a class before calling save_global().
Some persistent picklers (well, probably, the *only* persistent
pickler) would like to pickle some classes in a special way.
2002-07-16 19:47:43 +00:00
Raymond Hettinger 46ac8eb3c8 Code modernization. Replace v=s[i]; del s[i] with single lookup v=s.pop(i) 2002-06-30 03:39:14 +00:00
Raymond Hettinger 54f0222547 SF 563203. Replaced 'has_key()' with 'in'. 2002-06-01 14:18:47 +00:00
Neal Norwitz efbb67b1a7 Remove comment about inheritance, look one line up 2002-05-30 12:12:04 +00:00
Raymond Hettinger aef22fb9cd Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were not updated after 2.2). 2002-05-29 16:18:42 +00:00
Tim Peters 8ac1495a6a Whitespace normalization. 2002-05-23 15:15:30 +00:00
Raymond Hettinger 97394bc795 Patch 533291. Deprecate None return form of __reduce__. 2002-05-21 17:22:02 +00:00
Fred Drake 7f781c9aab Add Pickler.clear_memo() so the pickle and cPickle modules are more similar. 2002-05-01 20:33:53 +00:00
Guido van Rossum e276339cea Implement an idea by Paul Rubin:
Change pickling format for bools to use a backwards compatible
encoding.  This means you can pickle True or False on Python 2.3
and Python 2.2 or before will read it back as 1 or 0.  The code
used for pickling bools before would create pickles that could
not be read in previous Python versions.
2002-04-05 19:30:08 +00:00
Guido van Rossum 77f6a65eb0 Add the 'bool' type and its values 'False' and 'True', as described in
PEP 285.  Everything described in the PEP is here, and there is even
some documentation.  I had to fix 12 unit tests; all but one of these
were printing Boolean outcomes that changed from 0/1 to False/True.
(The exception is test_unicode.py, which did a type(x) == type(y)
style comparison.  I could've fixed that with a single line using
issubtype(x, type(y)), but instead chose to be explicit about those
places where a bool is expected.

Still to do: perhaps more documentation; change standard library
modules to return False/True from predicates.
2002-04-03 22:41:51 +00:00
Guido van Rossum 85ee491b3a Fix for SF 502085.
Don't die when issubclass(t, TypeType) fails.

Bugfix candidate (but I think it's too late for 2.2.1).
2002-03-26 00:51:56 +00:00
Neal Norwitz d5ba4aeab0 SF #515018, delete global variable that was apparently used only
in a list comprehension.
2002-02-11 18:12:06 +00:00
Guido van Rossum f048a8f6d7 Pickler.save(): Fix for SF bug #494904: Cannot pickle a class with a
metaclass, reported by Dan Parisien.

Objects that are instances of custom metaclasses, i.e. whose class is
a subclass of 'type', should be pickled the same as new-style classes
(objects whose class is 'type').  This can't be done through a
dispatch table entry, and the __reduce__ trick doesn't work for these,
since it finds the unbound __reduce__ for instances of the class
(inherited from 'object').  So check explicitly using issubclass().
2001-12-19 16:55:02 +00:00
Barry Warsaw bf4d959d28 Two changes:
load_inst(): Implement the security hook that cPickle already had.
When unpickling callables which are not classes, we look to see if the
object has an attribute __safe_for_unpickling__.  If this exists and
has a true value, then we can call it to create the unpickled object.
Otherwise we raise an UnpicklingError.

find_class(): We no longer mask ImportError, KeyError, and
AttributeError by transforming them into SystemError.  The latter is
definitely not the right thing to do, so we let the former three
exceptions simply propagate up if they occur, i.e. we remove the
try/except!
2001-11-15 23:42:58 +00:00
Jeremy Hylton 20747fa167 A better new, unique object 2001-11-09 16:15:04 +00:00
Jeremy Hylton abe2c62bdb Use cStringIO when available.
Remove test code.  It's available in Lib/test/picklertester.py.
2001-10-15 21:29:28 +00:00
Guido van Rossum dbb718fa87 Make these modules work when Python is compiled without Unicode support. 2001-09-21 19:22:34 +00:00
Tim Peters 19ef62d5a9 pickle.py, load_int(): Match cPickle's just-repaired ability to unpickle
64-bit INTs on 32-bit boxes (where they become longs).  Also exploit that
int(str) and long(str) will ignore a trailing newline (saves creating a
new string at the Python level).

pickletester.py:  Simulate reading a pickle produced by a 64-bit box.
2001-08-28 22:21:18 +00:00
Guido van Rossum b0a98e9c94 Address SF #451547. The approach is a bit draconian: any object that
is pickled as a global must now exist by the name under which it is
pickled, otherwise the pickling fails.  Previously, such things would
fail on unpickling, or unpickle as the wrong global object.  I'm
hoping that this won't break existing code that is playing tricks with
this.

I need a volunteer to do this for cPickle too.
2001-08-17 18:49:52 +00:00
Tim Peters 6d6c1a35e0 Merge of descr-branch back into trunk. 2001-08-02 04:15:00 +00:00
Tim Peters 44714007e8 test_pickle works on sizeof(long)==8 boxes again.
pickle.py
    The code implicitly assumed that all ints fit in 4 bytes, causing all
    sorts of mischief (from nonsense results to corrupted pickles).
    Repaired that.
marshal.c
    The int marshaling code assumed that right shifts of signed longs
    sign-extend.  Repaired that.
2001-04-10 05:02:52 +00:00
Tim Peters c32d824467 Mechanical fiddling for easier reading: every "if" test was enclosed in
parens, but no "while" test.  Removed the former.
2001-04-10 02:48:53 +00:00
Skip Montanaro 23bafc6fcd add module-level constants to __all__ 2001-02-18 03:10:09 +00:00
Tim Peters 658cba6706 Whitespace normalization. 2001-02-09 20:06:00 +00:00
Skip Montanaro 352674d01c a few more __all__ lists 2001-02-07 23:14:30 +00:00
Guido van Rossum 31584cb43d Finn Bock (SF patch #103349):
Allow pickle.py to be using with Jython unicode strings
2001-01-22 14:53:29 +00:00
Tim Peters 2344fae6d0 Whitespace normalization. 2001-01-15 00:50:52 +00:00
Guido van Rossum a31b4ef7c5 Minimal fix for the complaints about pickling Unicode objects. (SF
bugs #126161 and 123634).

The solution doesn't use the unicode-escape encoding; that has other
problems (it seems not 100% reversible).  Rather, it transforms the
input Unicode object slightly before encoding it using
raw-unicode-escape, so that the decoding will reconstruct the original
string: backslash and newline characters are translated into their
\uXXXX counterparts.

This is backwards incompatible for strings containing backslashes, but
for some of those strings, the pickling was already broken.
2000-12-19 01:29:00 +00:00
Guido van Rossum ff87174db1 Get rid of string module and string exceptions. 2000-12-13 18:11:56 +00:00
Jeremy Hylton be467e5c69 Fix Bug #114293:
Strings are unpickled by calling eval on the string's repr. This
    change makes pickle work like cPickle; it checks if the pickled
    string is safe to eval and raises ValueError if it is not.

test suite modifications:
    Verify that pickle catches a variety of insecure string pickles
    Make test_pickle and test_cpickle use exactly the same test suite
    Add test for pickling recursive object
2000-09-15 15:14:51 +00:00
Guido van Rossum 26e89d0185 Change string exceptions to class exceptions. 2000-06-29 16:15:52 +00:00
Guido van Rossum b5f2f1bb6f Marc-Andre Lemburg: support pickling Unicode objects, both in text
mode ('V') and in binary mode ('X').
2000-03-10 23:20:09 +00:00
Guido van Rossum 54f22ed30b More trivial comment -> docstring transformations by Ka-Ping Yee,
who writes:

Here is batch 2, as a big collection of CVS context diffs.
Along with moving comments into docstrings, i've added a
couple of missing docstrings and attempted to make sure more
module docstrings begin with a one-line summary.

I did not add docstrings to the methods in profile.py for
fear of upsetting any careful optimizations there, though
i did move class documentation into class docstrings.

The convention i'm using is to leave credits/version/copyright
type of stuff in # comments, and move the rest of the descriptive
stuff about module usage into module docstrings.  Hope this is
okay.
2000-02-04 15:10:34 +00:00