3cd7c6e6eb
bpo-31993: Do not allocate large temporary buffers in pickle dump. ( #4353 )
...
The picklers do no longer allocate temporary memory when dumping large
bytes and str objects into a file object. Instead the data is
directly streamed into the underlying file object.
Previously the C implementation would buffer all content and issue a
single call to file.write() at the end of the dump. With protocol 4
this behavior has changed to issue one call to file.write() per frame.
The Python pickler with protocol 4 now dumps each frame content as a
memoryview to an IOBytes instance that is never reused and the
memoryview is no longer released after the call to write. This makes it
possible for the file object to delay access to the memoryview of
previous frames without forcing any additional memory copy as was
already possible with the C pickler.
2018-01-06 17:18:54 +02:00
3daaafb700
bpo-32037: Use the INT opcode for 32-bit integers in protocol 0 pickles. ( #4407 )
2017-11-16 09:44:43 +02:00
6545256df9
bpo-32032: Test both implementations of module-level pickle API. ( #4401 )
2017-11-15 14:01:08 +02:00
8c663fd60e
Replace KB unit with KiB ( #4293 )
...
kB (*kilo* byte) unit means 1000 bytes, whereas KiB ("kibibyte")
means 1024 bytes. KB was misused: replace kB or KB with KiB when
appropriate.
Same change for MB and GB which become MiB and GiB.
Change the output of Tools/iobench/iobench.py.
Round also the size of the documentation from 5.5 MB to 5 MiB.
2017-11-08 14:44:44 -08:00
3410c01d83
Issue #17711 : Fixed unpickling by the persistent ID with protocol 0.
...
Original patch by Alexandre Vassalotti.
2016-07-17 11:35:35 +03:00
dec25afab1
Issue #17711 : Fixed unpickling by the persistent ID with protocol 0.
...
Original patch by Alexandre Vassalotti.
2016-07-17 11:24:17 +03:00
96c058b4de
Fixed typo in pickle tests.
2016-03-04 09:40:03 +02:00
be70002866
Fixed typo in pickle tests.
2016-03-04 09:39:47 +02:00
58f8833e4c
Make catched exception more specific and correct a comment.
2015-12-30 21:00:29 +02:00
beb652ccc1
Make catched exception more specific and correct a comment.
2015-12-30 21:00:08 +02:00
59fb6342a4
Issue #25761 : Improved detecting errors in broken pickle data.
2015-12-06 22:01:35 +02:00
92d5fbaf8f
Issue #25761 : Added more test cases for testing unpickling broken data.
...
Output raised exception at verbose level 2 (-vv).
2015-11-29 13:14:00 +02:00
3c49710e9f
Issue #25761 : Added more test cases for testing unpickling broken data.
...
Output raised exception at verbose level 2 (-vv).
2015-11-29 13:13:24 +02:00
7279befccb
Issue #25761 : Added more test cases for testing unpickling broken data.
...
Output raised exception at verbose level 2 (-vv).
2015-11-29 13:12:10 +02:00
b1ce30281d
Issue #23914 : Fixed SystemError raised by unpickler on broken pickle data.
2015-11-23 15:20:21 +02:00
4409c6cfae
Issue #23914 : Fixed SystemError raised by unpickler on broken pickle data.
2015-11-23 15:19:23 +02:00
e9b3074cf9
Issue #23914 : Fixed SystemError raised by unpickler on broken pickle data.
2015-11-23 15:17:43 +02:00
41a87637c0
Issue #892902 : Added new tests for pickling recursive collections.
2015-11-07 11:17:15 +02:00
838b7cc0e7
Issue #892902 : Added new tests for pickling recursive collections.
2015-11-07 11:17:02 +02:00
608c213530
Issue #892902 : Added new tests for pickling recursive collections.
2015-11-07 11:16:10 +02:00
0d554d7ef1
Issue #24164 : Objects that need calling ``__new__`` with keyword arguments,
...
can now be pickled using pickle protocols older than protocol version 4.
2015-10-10 22:42:18 +03:00
525faaeffc
Issue #25262 . Added support for BINBYTES8 opcode in Python implementation of
...
unpickler. Highest 32 bits of 64-bit size for BINUNICODE8 and BINBYTES8
opcodes no longer silently ignored on 32-bit platforms in C implementation.
2015-09-29 22:12:29 +03:00
e060619d4b
Issue #25262 . Added support for BINBYTES8 opcode in Python implementation of
...
unpickler. Highest 32 bits of 64-bit size for BINUNICODE8 and BINBYTES8
opcodes no longer silently ignored on 32-bit platforms in C implementation.
2015-09-29 22:10:07 +03:00
cb764960c9
Added additional unpickling tests.
2015-09-29 15:50:45 +03:00
b8b951f6ee
Added additional unpickling tests.
2015-09-29 15:49:58 +03:00
104de66b76
Moved unpickling tests with prepickled data to separate class.
2015-09-29 15:34:53 +03:00
c6b54b45ea
Moved unpickling tests with prepickled data to separate class.
2015-09-29 15:33:24 +03:00
b32b998bf7
merge 3.4
2015-07-02 16:58:31 -05:00
d3a2a95125
use correct __new__ method ( closes #24552 )
2015-07-02 16:58:22 -05:00
42e913aa3a
merge 3.4 ( #24552 )
2015-07-02 16:18:58 -05:00
80f78a3efc
fix use after free ( closes #24552 )
2015-07-02 16:18:38 -05:00
58e4134a1c
Issue #23611 : Serializing more "lookupable" objects (such as unbound methods
...
or nested classes) now are supported with pickle protocols < 4.
2015-03-31 14:07:24 +03:00
d5d818d40b
Issue #18473 : Fixed 2to3 and 3to2 compatible pickle mappings.
...
Fixed ambigious reverse mappings. Added many new mappings. Import mapping
is no longer applied to modules already mapped with full name mapping.
Added tests for compatible pickling and unpickling and for consistency of
_compat_pickle mappings.
2015-03-31 13:17:10 +03:00
bfe1824d08
Issue #18473 : Fixed 2to3 and 3to2 compatible pickle mappings.
...
Fixed ambigious reverse mappings. Added many new mappings. Import mapping
is no longer applied to modules already mapped with full name mapping.
Added tests for compatible pickling and unpickling and for consistency of
_compat_pickle mappings.
2015-03-31 13:12:37 +03:00
d362c21264
Use pickled data compatible with Python 2 for testing protocols 0-2.
2015-02-15 14:10:24 +02:00
fa310ee3a9
Use pickled data compatible with Python 2 for testing protocols 0-2.
2015-02-15 14:10:03 +02:00
230586739c
Issue #23094 : Fixed readline with frames in Python implementation of pickle.
2015-01-26 10:37:44 +02:00
21d7533c4c
Issue #23094 : Fixed readline with frames in Python implementation of pickle.
2015-01-26 10:37:01 +02:00
707b5ccde5
Issue #22783 : Pickling now uses the NEWOBJ opcode instead of the NEWOBJ_EX
...
opcode if possible.
2014-12-16 19:43:46 +02:00
6cd5eda0dd
Fix uninitialized variable after #22676 .
2014-12-02 00:20:03 +01:00
8cf7c1cff0
Issue #22775 : Fixed unpickling of http.cookies.SimpleCookie with protocol 2
...
and above. Patch by Tim Graham.
2014-11-02 22:18:25 +02:00
786ac7b27d
Issue #19886 : Use better estimated memory requirements for bigmem tests.
...
Incorrect requirements can cause memory swapping.
2014-01-10 13:39:27 +02:00
4847e4e1f4
Issue #19886 : Use better estimated memory requirements for bigmem tests.
...
Incorrect requirements can cause memory swapping.
2014-01-10 13:37:54 +02:00
c1764dd350
Issue #19648 : implement empty tests in pickletester. Patch by Gennadiy Zlobin.
2013-12-28 16:57:37 +01:00
d05c9ff845
Issue #6784 : Strings from Python 2 can now be unpickled as bytes objects.
...
Initial patch by Merlijn van Deen.
I've added a few unrelated docstring fixes in the patch while I was at
it, which makes the documentation for pickle a bit more consistent.
2013-12-07 01:09:27 -08:00
6e73ff1a31
Issue #19881 : Fix bad pickling of large bytes in cpickle.
2013-12-05 19:29:32 -08:00
6e8bc50317
Issue #19800 : make the pickle framing tests more precise.
2013-12-03 09:51:40 +01:00
69359f6207
Fix #19834 : merge with 3.3.
2013-12-02 11:43:20 +01:00
9d1dbca5e2
Fix issue #19834 : Support unpickling of exceptions pickled by Python 2.
2013-12-02 11:41:01 +01:00
28d271ef6b
Issue #19754 : Make pickletools.optimize respect the frame size target.
2013-12-01 16:27:46 -08:00
3c23e7a5dc
Issue #6477 : Merge with 3.3.
2013-11-30 16:21:20 -08:00
19b6fa6ebb
Issue #6477 : Added support for pickling the types of built-in singletons.
2013-11-30 16:06:39 -08:00
a1eedf9ff0
Merge with 3.3.
2013-11-30 13:55:39 -08:00
896414fedf
Fixed _pickle.Unpickler to handle empty persistent IDs correctly.
2013-11-30 13:52:35 -08:00
3ab9cfc501
Selectively re-enable framing tests
2013-11-24 14:33:37 +01:00
5e411b7035
Disable annoying tests which doesn't work optimized pickles.
2013-11-23 20:58:24 -08:00
b6a2f2a0d1
Make framing optional in pickle protocol 4.
...
This will allow us to control in the future whether to use framing or not.
For example, we may want to turn it off for tiny pickle where it doesn't
help.
The change also improves performance slightly:
### fastpickle ###
Min: 0.608517 -> 0.557358: 1.09x faster
Avg: 0.798892 -> 0.694738: 1.15x faster
Significant (t=3.45)
Stddev: 0.17145 -> 0.12704: 1.3496x smaller
Timeline: http://goo.gl/3xQE1J
### pickle_dict ###
Min: 0.669920 -> 0.615271: 1.09x faster
Avg: 0.733633 -> 0.645058: 1.14x faster
Significant (t=5.05)
Stddev: 0.12041 -> 0.02961: 4.0662x smaller
Timeline: http://goo.gl/LpLSXI
### pickle_list ###
Min: 0.397583 -> 0.368112: 1.08x faster
Avg: 0.412784 -> 0.397223: 1.04x faster
Significant (t=2.78)
Stddev: 0.01518 -> 0.03653: 2.4068x larger
Timeline: http://goo.gl/v39E59
### unpickle_list ###
Min: 0.692935 -> 0.594870: 1.16x faster
Avg: 0.730012 -> 0.628395: 1.16x faster
Significant (t=17.76)
Stddev: 0.02720 -> 0.02995: 1.1012x larger
Timeline: http://goo.gl/2P9AEt
The following not significant results are hidden, use -v to show them:
fastunpickle.
2013-11-23 20:30:03 -08:00
d5df19461d
test_pickle: speed up test_long
2013-11-23 21:20:49 +01:00
c9dc4a2a8a
Issue #17810 : Implement PEP 3154, pickle protocol 4.
...
Most of the work is by Alexandre.
2013-11-23 18:59:12 +01:00
48e6a8c88a
Issue #18743 : Fix references to non-existant "StringIO" module
...
in docstrings and comments.
2013-08-29 11:39:48 +03:00
50254c57cd
Issue #18743 : Fix references to non-existant "StringIO" module
...
in docstrings and comments.
2013-08-29 11:35:43 +03:00
f3bc946628
Merge 3.3
2013-04-20 13:25:55 -07:00
1f7492c28a
Isuse #17720 : Fix APPENDS handling in the Python implementation of Unpickler
...
to correctly process the opcode when it is used on non-list objects.
2013-04-20 13:19:46 -07:00
7c5e094cbf
Make C and Python implementations of pickle load STRING opcodes the same way.
...
The C version tried to remove trailing whitespace between the last quote and
the newline character. I am not sure why it had this because pickle never
generated such pickles---for this to happen repr(some_string) would need to
return trailing whitespace. It was maybe there to make it easier for people
to write pickles in text editors. Anyhow, the Python version doesn't do this
so there is no point keeping this around anymore.
Also, I've changed the exception raised when a bad pickle is encountered.
Again this unlikely to make much difference to anyone though it does make
testing slightly nicer for us.
2013-04-15 23:14:55 -07:00
3034efdd29
Issue #17710 : Fix pickle raising a SystemError on bogus input.
2013-04-15 21:51:09 +02:00
296069301a
#17346 : merge with 3.2.
2013-03-04 15:19:02 +02:00
aaef34483c
#17346 : make sure pickle tests are run against all protocols. Initial patch by Marius Gedminas.
2013-03-04 15:17:56 +02:00
9facaf4f65
Issue #15079 : make a test applicable to both C and Python versions of the pickle module.
...
Patch by Stefan Mihaila.
2012-06-26 23:05:27 +02:00
79035bd71f
Issue #15079 : make a test applicable to both C and Python versions of the pickle module.
...
Patch by Stefan Mihaila.
2012-06-26 23:04:48 +02:00
f3078fbee2
Fixes #13842 : cannot pickle Ellipsis or NotImplemented.
...
Thanks for James Sanders for the bug report and the patch.
2012-03-12 19:46:12 +01:00
8d3c290de4
Issue #14166 : Pickler objects now have an optional `dispatch_table` attribute which allows to set custom per-pickler reduction functions.
...
Patch by sbt.
2012-03-04 18:31:48 +01:00
a23d65ccfe
Merge 3.2
2011-12-13 13:22:42 -05:00
3bfc65a25b
Issue #13505 : Make pickling of bytes object compatible with Python 2.
...
Initial patch by sbt.
2011-12-13 13:08:09 -05:00
b7591d4780
Also fix pickletester
2011-10-04 16:18:15 +02:00
1a3ff48c55
Start fixing test_bigmem:
...
- bigmemtest is replaced by precisionbigmemtest
- add a poor man's watchdog thread to print memory consumption
2011-10-04 10:28:37 +02:00
94190bb6e7
Start fixing test_bigmem:
...
- bigmemtest is replaced by precisionbigmemtest
- add a poor man's watchdog thread to print memory consumption
2011-10-04 10:22:36 +02:00
5a688dbf97
Issue #7689 : Allow pickling of dynamically created classes when their
...
metaclass is registered with copyreg. Patch by Nicolas M. Thiéry and
Craig Citro.
2011-10-04 09:25:28 +02:00
ffd41d9f10
Issue #7689 : Allow pickling of dynamically created classes when their
...
metaclass is registered with copyreg. Patch by Nicolas M. Thiéry and
Craig Citro.
2011-10-04 09:23:04 +02:00
2d243494c6
Try to fix one of the bigmem tests in test_pickle
2011-08-30 23:39:59 +02:00
e897e95880
Try to fix one of the bigmem tests in test_pickle
2011-08-30 23:39:34 +02:00
a514eb95f3
Issue #12847 : Fix a crash with negative PUT and LONG_BINPUT arguments in
...
the C pickle implementation.
2011-08-30 00:28:40 +02:00
55549ec476
Issue #12847 : Fix a crash with negative PUT and LONG_BINPUT arguments in
...
the C pickle implementation.
2011-08-30 00:27:10 +02:00
ee763e2acc
Issue #11564 : Avoid crashes when trying to pickle huge objects or containers
...
(more than 2**31 items). Instead, in most cases, an OverflowError is raised.
2011-08-29 23:14:53 +02:00
82be19f889
Issue #11564 : Avoid crashes when trying to pickle huge objects or containers
...
(more than 2**31 items). Instead, in most cases, an OverflowError is raised.
2011-08-29 23:09:33 +02:00
85674936c7
Monotonic, not monotonous
2011-08-14 01:51:52 +02:00
3c7e928098
Issue #12744 : Fix inefficient representation of integers
...
between 2**31 and 2**63 on systems with a 64-bit C "long".
2011-08-13 20:15:19 +02:00
fdf4a27422
Issue #12687 : Fix a possible buffering bug when unpickling text mode (protocol 0, mostly) pickles.
2011-08-11 21:15:53 +02:00
f6c7a8595e
Issue #12687 : Fix a possible buffering bug when unpickling text mode (protocol 0, mostly) pickles.
2011-08-11 21:04:02 +02:00
16c4ce1903
Issue #9935 : Speed up pickling of instances of user-defined classes.
2011-03-11 21:30:43 +01:00
ec8f0df229
Merged revisions 88546 via svnmerge from
...
svn+ssh://pythondev@svn.python.org/python/branches/py3k
........
r88546 | alexander.belopolsky | 2011-02-24 14:40:09 -0500 (Thu, 24 Feb 2011) | 3 lines
Issue #11286 : Fixed unpickling of empty 2.x strings.
........
2011-02-24 20:34:38 +00:00
1ce92dc20d
Issue #11286 : Fixed unpickling of empty 2.x strings.
2011-02-24 19:40:09 +00:00
31f5929c1e
Issue #10990 : Prevent tests from clobbering a set trace function.
...
Many tests simply didn't care if they unset a pre-existing trace function. This
made test coverage impossible. This patch fixes various tests to put back any
pre-existing trace function. It also introduces test.support.no_tracing as a
decorator which will temporarily unset the trace function for tests which
simply fail otherwise.
Thanks to Kristian Vlaardingerbroek for helping to find the cause of various
trace function unsets.
2011-02-21 19:29:56 +00:00
b3aedd4862
#9424 : Replace deprecated assert* methods in the Python test suite.
2010-11-20 19:04:17 +00:00
ff150f2921
Revert r85797 (and r85798): it broke the Windows buildbots because of
...
test_multiprocessing's misbehaviour.
2010-10-22 21:41:05 +00:00
7eecffd05d
Issue #9935 : Speed up pickling of instances of user-defined classes.
2010-10-22 19:43:59 +00:00
04248a8d99
Issue #3873 : Speed up unpickling from file objects which have a peek()
...
method.
2010-10-12 20:51:21 +00:00
ea99c5c949
Issue #9410 : Various optimizations to the pickle module, leading to
...
speedups up to 4x (depending on the benchmark). Mostly ported from
Unladen Swallow; initial patch by Alexandre Vassalotti.
2010-09-09 18:33:21 +00:00
d92f04062a
Issue #5180 : Fixed a bug that prevented loading 2.x pickles in 3.x
...
python when they contain instances of old-style classes.
2010-07-17 22:50:45 +00:00
485fb56eb8
Issue #8383 : pickle and pickletools use surrogatepass error handler when
...
encoding unicode as utf8 to support lone surrogates and stay compatible with
Python 2.x and 3.0
2010-04-13 11:07:24 +00:00
e96159335f
Merged revisions 77727 via svnmerge from
...
svn+ssh://pythondev@svn.python.org/python/trunk
........
r77727 | ezio.melotti | 2010-01-24 18:58:36 +0200 (Sun, 24 Jan 2010) | 1 line
use assert[Not]IsInstance where appropriate
........
2010-01-24 19:26:24 +00:00
Olivier Grisel
Serhiy Storchaka
Victor Stinner
Benjamin Peterson
Antoine Pitrou
Alexandre Vassalotti
Walter Doerwald
Ezio Melotti
Łukasz Langa
Alexander Belopolsky
Brett Cannon
Victor Stinner