gh-123431: Harmonize extension code checks in pickle (GH-123434)
This checks are redundant in normal circumstances and can only work if
the extension registry was intentionally broken.
* The Python implementation now raises exception for the extension code
with false boolean value.
* Simplify the C code. RuntimeError is now raised in explicit checks.
* Add many tests.
(cherry picked from commit 0c3ea30238)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Serializing objects with complex __qualname__ (such as unbound methods and
nested classes) by name no longer involves serializing parent objects by value
in pickle protocols < 4.
(cherry picked from commit dc07f65a53)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
gh-120380: fix Python implementation of `pickle.Pickler` for `bytes` and `bytearray` objects in protocol version 5. (GH-120422)
(cherry picked from commit 7595e6743a)
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
This fixes a divergence between the Python and C implementations of pickle
for protocol 0, such that it pickle.py fails to re-use the first pickled
representation of strings involving characters that have to be escaped.
copy.copy() and copy.deepcopy() now always raise a TypeError if
__reduce__() returns a tuple with length 6 instead of silently ignore
the 6th item or produce incorrect result.
* bpo-42406: Fix whichmodule() with multiprocessing
Signed-off-by: Renato L. de F. Cunha <renatoc@br.ibm.com>
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Some portions of the pickle documentation hadn't been updated for the pickle protocol changes in Python 3.8 (new protocol 5, default protocol 4). This PR fixes those docs.
https://bugs.python.org/issue39426
The previous code was raising a `KeyError` for both the Python and C implementation.
This was caused by the specified index of an invalid input which did not exist
in the memo structure, where the pickle stores what objects it has seen.
The malformed input would have caused either a `BINGET` or `LONG_BINGET` load
from the memo, leading to a `KeyError` as the determined index was bogus.
https://bugs.python.org/issue38876https://bugs.python.org/issue38876
Allow pure Python implementation of pickle to work
even when the C _pickle module is unavailable.
Fix test_pickle when _pickle is missing: declare PyPicklerHookTests
outside "if has_c_implementation:" block.
Allow reduction methods to return a 6-item tuple where the 6th item specifies a
custom state-setting method that's called instead of the regular
``__setstate__`` method.
* When doing getattr lookups with a default of "None", it now
uses an "is" comparison against None which is more correct
* Removed outdated code
Patch by Brandon Rhodes.
This makes performance better and produces shorter pickles. This change is backwards compatible up to the oldest currently supported version of Python (3.4).
The picklers do no longer allocate temporary memory when dumping large
bytes and str objects into a file object. Instead the data is
directly streamed into the underlying file object.
Previously the C implementation would buffer all content and issue a
single call to file.write() at the end of the dump. With protocol 4
this behavior has changed to issue one call to file.write() per frame.
The Python pickler with protocol 4 now dumps each frame content as a
memoryview to an IOBytes instance that is never reused and the
memoryview is no longer released after the call to write. This makes it
possible for the file object to delay access to the memoryview of
previous frames without forcing any additional memory copy as was
already possible with the C pickler.
This changes the main documentation, doc strings, source code comments, and a
couple error messages in the test suite. In some cases the word was removed
or edited some other way to fix the grammar.
Fixed ambigious reverse mappings. Added many new mappings. Import mapping
is no longer applied to modules already mapped with full name mapping.
Added tests for compatible pickling and unpickling and for consistency of
_compat_pickle mappings.
Miss Islington (bot)
Serhiy Storchaka
Jeff Allen
Pieter Eendebak
Irit Katriel
Nikita Sobolev
Christian Clauss
Carl Friedrich Bolz-Tereick
Renato Cunha
Raymond Hettinger
Mark Dickinson
Claudiu Popa
Victor Stinner
Antoine Pitrou
Steve Dower
Pierre Glaser
Berker Peksag
Łukasz Langa
Olivier Grisel
Martin Panter
Antoine Pitrou
Victor Stinner