Some portions of the pickle documentation hadn't been updated for the pickle protocol changes in Python 3.8 (new protocol 5, default protocol 4). This PR fixes those docs.
https://bugs.python.org/issue39426
The previous code was raising a `KeyError` for both the Python and C implementation.
This was caused by the specified index of an invalid input which did not exist
in the memo structure, where the pickle stores what objects it has seen.
The malformed input would have caused either a `BINGET` or `LONG_BINGET` load
from the memo, leading to a `KeyError` as the determined index was bogus.
https://bugs.python.org/issue38876https://bugs.python.org/issue38876
Allow pure Python implementation of pickle to work
even when the C _pickle module is unavailable.
Fix test_pickle when _pickle is missing: declare PyPicklerHookTests
outside "if has_c_implementation:" block.
Allow reduction methods to return a 6-item tuple where the 6th item specifies a
custom state-setting method that's called instead of the regular
``__setstate__`` method.
* When doing getattr lookups with a default of "None", it now
uses an "is" comparison against None which is more correct
* Removed outdated code
Patch by Brandon Rhodes.
This makes performance better and produces shorter pickles. This change is backwards compatible up to the oldest currently supported version of Python (3.4).
The picklers do no longer allocate temporary memory when dumping large
bytes and str objects into a file object. Instead the data is
directly streamed into the underlying file object.
Previously the C implementation would buffer all content and issue a
single call to file.write() at the end of the dump. With protocol 4
this behavior has changed to issue one call to file.write() per frame.
The Python pickler with protocol 4 now dumps each frame content as a
memoryview to an IOBytes instance that is never reused and the
memoryview is no longer released after the call to write. This makes it
possible for the file object to delay access to the memoryview of
previous frames without forcing any additional memory copy as was
already possible with the C pickler.
This changes the main documentation, doc strings, source code comments, and a
couple error messages in the test suite. In some cases the word was removed
or edited some other way to fix the grammar.
Fixed ambigious reverse mappings. Added many new mappings. Import mapping
is no longer applied to modules already mapped with full name mapping.
Added tests for compatible pickling and unpickling and for consistency of
_compat_pickle mappings.
Initial patch by Merlijn van Deen.
I've added a few unrelated docstring fixes in the patch while I was at
it, which makes the documentation for pickle a bit more consistent.
The C version tried to remove trailing whitespace between the last quote and
the newline character. I am not sure why it had this because pickle never
generated such pickles---for this to happen repr(some_string) would need to
return trailing whitespace. It was maybe there to make it easier for people
to write pickles in text editors. Anyhow, the Python version doesn't do this
so there is no point keeping this around anymore.
Also, I've changed the exception raised when a bad pickle is encountered.
Again this unlikely to make much difference to anyone though it does make
testing slightly nicer for us.
Mark Dickinson
Claudiu Popa
Victor Stinner
Serhiy Storchaka
Antoine Pitrou
Steve Dower
Pierre Glaser
Berker Peksag
Łukasz Langa
Olivier Grisel
Martin Panter
Antoine Pitrou
Victor Stinner
Alexandre Vassalotti
Brett Cannon