Commit Graph

99 Commits

Author SHA1 Message Date
Olivier Grisel 3cd7c6e6eb bpo-31993: Do not allocate large temporary buffers in pickle dump. (#4353)
The picklers do no longer allocate temporary memory when dumping large
bytes and str objects into a file object. Instead the data is
directly streamed into the underlying file object.

Previously the C implementation would buffer all content and issue a
single call to file.write() at the end of the dump. With protocol 4
this behavior has changed to issue one call to file.write() per frame.

The Python pickler with protocol 4 now dumps each frame content as a
memoryview to an IOBytes instance that is never reused and the
memoryview is no longer released after the call to write. This makes it
possible for the file object to delay access to the memoryview of
previous frames without forcing any additional memory copy as was
already possible with the C pickler.
2018-01-06 17:18:54 +02:00
Serhiy Storchaka 3daaafb700
bpo-32037: Use the INT opcode for 32-bit integers in protocol 0 pickles. (#4407) 2017-11-16 09:44:43 +02:00
Jelle Zijlstra 5a4e3d8f9c bpo-30174: Remove duplicate definition from pickletools (#1301)
There were two almost identical definitions of bytes1.
2017-04-27 19:05:26 +03:00
Serhiy Storchaka 5f1a5187f7 Use sequence repetition instead of bytes constructor with integer argument. 2016-09-11 14:41:02 +03:00
Martin Panter 1ce738e08f Merge typo fixes from 3.5 2016-05-08 14:02:35 +00:00
Martin Panter 4c35964b76 Corrections for a/an in code comments and documentation 2016-05-08 13:53:41 +00:00
Benjamin Peterson 860c8a404a merge 3.5 (#25502) 2015-10-28 23:15:22 -07:00
Benjamin Peterson 669ff66c32 remove duplicated imports (closes #25502) 2015-10-28 23:15:13 -07:00
Serhiy Storchaka dbc517c2dd Issue #25382: pickletools.dis() now outputs implicit memo index for the
MEMOIZE opcode.
2015-10-13 21:20:14 +03:00
Serhiy Storchaka 5805ddeedb Issue #25380: Fixed protocol for the STACK_GLOBAL opcode in
pickletools.opcodes.
2015-10-13 21:12:32 +03:00
Serhiy Storchaka 05dadcfb28 Issue #19858: pickletools.optimize() now aware of the MEMOIZE opcode, can
produce more compact result and no longer produces invalid output if input
data contains MEMOIZE opcodes together with PUT or BINPUT opcodes.
2014-12-16 18:00:56 +02:00
Alexandre Vassalotti d05c9ff845 Issue #6784: Strings from Python 2 can now be unpickled as bytes objects.
Initial patch by Merlijn van Deen.

I've added a few unrelated docstring fixes in the patch while I was at
it, which makes the documentation for pickle a bit more consistent.
2013-12-07 01:09:27 -08:00
Alexandre Vassalotti 02066d3fbd Issue #19754: Fix typo. 2013-12-01 16:31:49 -08:00
Alexandre Vassalotti 28d271ef6b Issue #19754: Make pickletools.optimize respect the frame size target. 2013-12-01 16:27:46 -08:00
Gregory P. Smith 057e58dc01 Fix test_pickletools.py doctest's on 32-bit platforms. I hate doctests. 2013-11-23 20:40:46 +00:00
Antoine Pitrou c9dc4a2a8a Issue #17810: Implement PEP 3154, pickle protocol 4.
Most of the work is by Alexandre.
2013-11-23 18:59:12 +01:00
Serhiy Storchaka 46e1ce214b Issue #18783: Removed existing mentions of Python long type in docstrings,
error messages and comments.
2013-08-27 20:17:03 +03:00
Serhiy Storchaka 9594942716 Issue #18783: Removed existing mentions of Python long type in docstrings,
error messages and comments.
2013-08-27 19:40:23 +03:00
Ezio Melotti 9a3777e525 #18705: merge with 3.3. 2013-08-17 15:53:55 +03:00
Ezio Melotti 30b9d5d3af #18705: fix a number of typos. Patch by Févry Thibault. 2013-08-17 15:50:46 +03:00
Alexandre Vassalotti 96d6a78ce0 Merge 3.3 2013-04-14 03:44:47 -07:00
Alexandre Vassalotti 8db89ca56c Issue #16550: Update the opcode descriptions of pickletools to use unsigned
integers where appropriate.  Initial patch by Serhiy Storchaka.
2013-04-14 03:30:35 -07:00
Victor Stinner 765531d2d0 Issue #17516: use comment syntax for comments, instead of multiline string 2013-03-26 01:11:54 +01:00
Alexandre Vassalotti a23d65ccfe Merge 3.2 2011-12-13 13:22:42 -05:00
Alexandre Vassalotti 3bfc65a25b Issue #13505: Make pickling of bytes object compatible with Python 2.
Initial patch by sbt.
2011-12-13 13:08:09 -05:00
Florent Xicluna aa6c1d240f Issue #13575: there is only one class type. 2011-12-12 18:54:29 +01:00
Florent Xicluna 2bb96f593a Cleanup code: remove int/long idioms and simplify a while statement. 2011-10-23 22:11:00 +02:00
Ezio Melotti 3b3499ba69 #11565: Merge with 3.1. 2011-03-16 11:35:38 +02:00
Ezio Melotti 13925008dc #11565: Fix several typos. Patch by Piotr Kasprzyk. 2011-03-16 11:05:33 +02:00
Florent Xicluna b4efb3d81e Merged revisions 83212,83829,83833,83838-83839,83878,84019,84025,84028,84032,84036 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r83212 | florent.xicluna | 2010-07-28 18:39:41 +0200 (mer., 28 juil. 2010) | 2 lines

  Syntax cleanup.
........
  r83829 | florent.xicluna | 2010-08-08 18:16:07 +0200 (dim., 08 août 2010) | 2 lines

  Use unittest specific methods for some urllib test cases.  And replace urllib2 with urllib.request in comments.
........
  r83833 | florent.xicluna | 2010-08-08 18:25:27 +0200 (dim., 08 août 2010) | 2 lines

  Add test case for the HTTPResponse being an iterable.  Follow-up of issue #4608.
........
  r83838 | florent.xicluna | 2010-08-08 20:03:44 +0200 (dim., 08 août 2010) | 2 lines

  Typo.
........
  r83839 | florent.xicluna | 2010-08-08 20:06:13 +0200 (dim., 08 août 2010) | 2 lines

  Issue #7564: Skip test_ioctl if another process is attached to /dev/tty.
........
  r83878 | florent.xicluna | 2010-08-09 10:29:08 +0200 (lun., 09 août 2010) | 1 line

  Merge the 2to3 script from /sandbox/trunk/2to3/2to3, revision 72867 (latest).
........
  r84019 | florent.xicluna | 2010-08-14 17:56:42 +0200 (sam., 14 août 2010) | 11 lines

  Merged manually from 2.7 branch to 3.x trunk.

    ------------------------------------------------------------------------
    r79925 | nick.coghlan | 2010-04-10 16:24:36 +0200 (sam. 10 avril 2010)

    Try to turn some buildbots green by allowing test_multiprocessing to
    pass even if it hits the sys.exc_clear code in the threading module, and
    improve the test coverage by making the ctypes dependencies a bit more
    granular (two of the cited ctypes objects don't exist on my system)
    ------------------------------------------------------------------------
........
  r84025 | florent.xicluna | 2010-08-14 18:56:27 +0200 (sam., 14 août 2010) | 1 line

  List Misc/python-config.in in Misc/README.  Fix few typos.
........
  r84028 | florent.xicluna | 2010-08-14 19:02:49 +0200 (sam., 14 août 2010) | 1 line

  Fix order.
........
  r84032 | florent.xicluna | 2010-08-14 19:15:31 +0200 (sam., 14 août 2010) | 1 line

  Convert to spaces.
........
  r84036 | florent.xicluna | 2010-08-14 20:03:19 +0200 (sam., 14 août 2010) | 1 line

  Remove bad merge (from svnmerge r82301)
........
2010-08-14 18:24:40 +00:00
Florent Xicluna 02ea12b291 Syntax cleanup. 2010-07-28 16:39:41 +00:00
Alexander Belopolsky 929d384058 Issue #9268: Add annotation option to pickletools.dis 2010-07-17 15:51:21 +00:00
Alexander Belopolsky bd39f9553a Merged revisions 82915 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r82915 | alexander.belopolsky | 2010-07-16 10:39:45 -0400 (Fri, 16 Jul 2010) | 1 line

  Corrected TUPLE<N> opcodes' docs.
........
2010-07-16 14:52:25 +00:00
Alexander Belopolsky 44c2ffd38f Corrected TUPLE<N> opcodes' docs. 2010-07-16 14:39:45 +00:00
Alexander Belopolsky 60c762b77a Issue #9094: Make python -m pickletools disassemble pickles given in
the command line.
2010-07-03 20:35:53 +00:00
Victor Stinner f7351b40b5 Merged revisions 80031 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r80031 | victor.stinner | 2010-04-13 13:07:24 +0200 (mar., 13 avril 2010) | 4 lines

  Issue #8383: pickle and pickletools use surrogatepass error handler when
  encoding unicode as utf8 to support lone surrogates and stay compatible with
  Python 2.x and 3.0
........
2010-04-13 11:09:22 +00:00
Victor Stinner 485fb56eb8 Issue #8383: pickle and pickletools use surrogatepass error handler when
encoding unicode as utf8 to support lone surrogates and stay compatible with
Python 2.x and 3.0
2010-04-13 11:07:24 +00:00
Antoine Pitrou d9dfaa9487 Issue #6137: The pickle module now translates module names when loading
or dumping pickles with a 2.x-compatible protocol, in order to make data
sharing and migration easier. This behaviour can be disabled using the
new `fix_imports` optional argument.
2009-06-04 20:32:06 +00:00
Collin Winter e61d4372ed Merge r72792: POP_MARK was not in pickle protocol 0, SHORT_BINBYTES was not in protocol 1. 2009-05-20 17:46:47 +00:00
Mark Dickinson cddcf444b2 Merged revisions 68903,68906 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r68903 | mark.dickinson | 2009-01-24 16:40:29 +0000 (Sat, 24 Jan 2009) | 5 lines

  Issue #1672332: Fix unpickling of subnormal floats, which was raising
  ValueError on some platforms as a result of the platform strtod setting
  errno on underflow.
........
  r68906 | mark.dickinson | 2009-01-24 21:08:38 +0000 (Sat, 24 Jan 2009) | 2 lines

  Issue #3657: fix occasional test_pickletools failures.
........
2009-01-24 21:46:33 +00:00
Mark Dickinson 8dd05147d6 Issue #4842, patch 1/2: fix pickle in Python 3.x so that pickling with the
'L' opcode always appends an 'L' on output, just as 2.x does.  When
unpickling, remove the trailing 'L' (if present) before passing the
result to PyLong_FromString.
2009-01-20 20:43:58 +00:00
Alexandre Vassalotti ca2d610dba Restore _pickle module accelerator module.
Removed Windows support temporarily.
64bit bug with integer unpickling is now fixed.
2008-06-12 18:26:05 +00:00
Benjamin Peterson 75f25f2c9a revert the addition of _pickle because it was causing havok with 64-bit 2008-06-12 03:10:02 +00:00
Alexandre Vassalotti cc313061a5 Issue 2917: Merge the pickle and cPickle module. 2008-06-11 22:43:06 +00:00
Alexandre Vassalotti f7fa63dd55 Rename copy_reg module to copyreg.
Updated documentation.
Merged revisions 63042 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r63042 | alexandre.vassalotti | 2008-05-11 04:25:28 -0400 (Sun, 11 May 2008) | 5 lines

  Added module stub for copy_reg renaming in 3.0.
  Renamed copy_reg to copyreg in the standard library, to avoid
  spurious warnings and ease later merging to py3k branch. Public
  documentation remains intact.
........
2008-05-11 08:55:36 +00:00
Guido van Rossum f416981691 - A new pickle protocol (protocol 3) is added with explicit support
for bytes.  This is the default protocol.  It intentionally cannot
  be unpickled by Python 2.x.

- When a pickle	written	by Python 2.x contains an (8-bit) str
  instance, this is now decoded to a (Unicode) str instance.  The
  encoding used to do this defaults to ASCII, but can be overridden
  via two new keyword arguments to the Unpickler class.  Previously
  this would create bytes instances, which is usually wrong: str
  instances are often used to pickle attribute names etc., and text is
  more common than binary data anyway.
2008-03-17 22:56:06 +00:00
Christian Heimes 126d29abf8 Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60569,60571-60572,60574,60576-60583,60585-60586,60589,60591,60594-60595,60597-60598,60600-60601,60606-60612,60615,60617,60619-60621,60623-60625,60627-60629,60631,60633,60635,60647,60650,60652,60654,60656,60658-60659,60664-60666,60668-60670,60672,60676,60678,60680-60683,60685-60686,60688,60690,60692-60694,60697-60700,60705-60706,60708,60711,60714,60720,60724-60734 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r60731 | raymond.hettinger | 2008-02-11 19:51:08 +0100 (Mon, 11 Feb 2008) | 1 line

  No need to register classes that already inherit from ABCs.
........
  r60733 | lars.gustaebel | 2008-02-11 20:17:10 +0100 (Mon, 11 Feb 2008) | 2 lines

  Make sure that xstar headers are read correctly.
........
  r60734 | raymond.hettinger | 2008-02-11 21:05:53 +0100 (Mon, 11 Feb 2008) | 1 line

  Add tests for pickletools.optimize().
........
2008-02-11 22:57:17 +00:00
Christian Heimes 3feef61742 Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60569,60571-60572,60574,60576-60583,60585-60586,60589,60591,60594-60595,60597-60598,60600-60601,60606-60612,60615,60617,60619-60621,60623-60625,60627-60629,60631,60633,60635,60647,60650,60652,60654,60656,60658-60659,60664-60666,60668-60670,60672,60676,60678,60680-60683,60685-60686,60688,60690,60692-60694,60697-60706,60708-60712,60714-60724 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r60701 | georg.brandl | 2008-02-09 22:36:15 +0100 (Sat, 09 Feb 2008) | 2 lines

  Needs only 2.4 now.
........
  r60702 | georg.brandl | 2008-02-09 22:38:54 +0100 (Sat, 09 Feb 2008) | 2 lines

  Docs are rst now.
........
  r60703 | georg.brandl | 2008-02-09 23:00:00 +0100 (Sat, 09 Feb 2008) | 2 lines

  Fix link.
........
  r60704 | georg.brandl | 2008-02-10 00:09:25 +0100 (Sun, 10 Feb 2008) | 2 lines

  Fix for newest doctools.
........
  r60709 | raymond.hettinger | 2008-02-10 08:21:09 +0100 (Sun, 10 Feb 2008) | 1 line

  Clarify that decimal also supports fixed-point arithmetic.
........
  r60710 | nick.coghlan | 2008-02-10 08:32:52 +0100 (Sun, 10 Feb 2008) | 1 line

  Add missing NEWS entry for r60695
........
  r60712 | mark.dickinson | 2008-02-10 15:58:38 +0100 (Sun, 10 Feb 2008) | 3 lines

  Turn classmethods into staticmethods, and avoid calling the constructor
  of subclasses of Rational.  (See discussion in issue #1682.)
........
  r60715 | mark.dickinson | 2008-02-10 16:19:58 +0100 (Sun, 10 Feb 2008) | 2 lines

  Typos in decimal comment and documentation
........
  r60716 | skip.montanaro | 2008-02-10 16:31:54 +0100 (Sun, 10 Feb 2008) | 2 lines

  Get the saying right. ;-)
........
  r60717 | skip.montanaro | 2008-02-10 16:32:16 +0100 (Sun, 10 Feb 2008) | 2 lines

  whoops - revert
........
  r60718 | mark.dickinson | 2008-02-10 20:23:36 +0100 (Sun, 10 Feb 2008) | 2 lines

  Remove reference to Rational
........
  r60719 | raymond.hettinger | 2008-02-10 21:35:16 +0100 (Sun, 10 Feb 2008) | 1 line

  Complete an open todo on pickletools -- add a pickle optimizer.
........
  r60721 | mark.dickinson | 2008-02-10 22:29:51 +0100 (Sun, 10 Feb 2008) | 3 lines

  Rename rational.Rational to fractions.Fraction, to avoid name clash
  with numbers.Rational.  See issue #1682 for related discussion.
........
  r60722 | christian.heimes | 2008-02-11 03:26:22 +0100 (Mon, 11 Feb 2008) | 1 line

  The test requires the network resource
........
  r60723 | mark.dickinson | 2008-02-11 04:11:55 +0100 (Mon, 11 Feb 2008) | 3 lines

  Put an extra space into the repr of a Fraction:
  Fraction(1, 2) instead of Fraction(1,2).
........
2008-02-11 06:19:17 +00:00
Georg Brandl 1a3284ed69 #1535: rename __builtin__ module to builtins. 2007-12-02 09:40:06 +00:00
Guido van Rossum 98297ee781 Merging the py3k-pep3137 branch back into the py3k branch.
No detailed change log; just check out the change log for the py3k-pep3137
branch.  The most obvious changes:

  - str8 renamed to bytes (PyString at the C level);
  - bytes renamed to buffer (PyBytes at the C level);
  - PyString and PyUnicode are no longer compatible.

I.e. we now have an immutable bytes type and a mutable bytes type.

The behavior of PyString was modified quite a bit, to make it more
bytes-like.  Some changes are still on the to-do list.
2007-11-06 21:34:58 +00:00