Commit Graph

63 Commits

Author SHA1 Message Date
Inada Naoki 4827483f47
bpo-43510: Implement PEP 597 opt-in EncodingWarning. (GH-19481)
See [PEP 597](https://www.python.org/dev/peps/pep-0597/).

* Add `-X warn_default_encoding` and `PYTHONWARNDEFAULTENCODING`.
* Add EncodingWarning
* Add io.text_encoding()
* open(), TextIOWrapper() emits EncodingWarning when encoding is omitted and warn_default_encoding is enabled.
* _pyio.TextIOWrapper() uses UTF-8 as fallback default encoding used when failed to import locale module. (used during building Python)
* bz2, configparser, gzip, lzma, pathlib, tempfile modules use io.text_encoding().
* What's new entry
2021-03-29 12:28:14 +09:00
Hai Shi 883bc63833
bpo-40275: Use new test.support helper submodules in tests (GH-21314) 2020-07-06 11:12:49 +02:00
Victor Stinner 63ab4ba07b
bpo-37210: Fix pure Python pickle when _pickle is unavailable (GH-14016)
Allow pure Python implementation of pickle to work
even when the C _pickle module is unavailable.

Fix test_pickle when _pickle is missing: declare PyPicklerHookTests
outside "if has_c_implementation:" block.
2019-06-13 13:58:51 +02:00
Antoine Pitrou 91f4380ced
bpo-36785: PEP 574 implementation (GH-7076) 2019-05-26 17:10:09 +02:00
Pierre Glaser 289f1f80ee bpo-35900: Enable custom reduction callback registration in _pickle (GH-12499)
Enable custom reduction callback registration for functions and classes in
_pickle.c, using the new Pickler's attribute ``reducer_override``.
2019-05-08 23:08:25 +02:00
Sergey Fedoseev 86b89916d1 Fix upsizing of marks stack in pickle module. (GH-8860)
Previously marks stack was upsized even there was space for additional item.
2018-08-25 10:54:40 +03:00
Serhiy Storchaka 0a2da50e18
bpo-31993: Do not create frames for large bytes and str objects (#5114)
when serialize into memory buffer with C pickle implementations.

This optimization already is performed when serialize into memory
with Python pickle implementations or into a file with both
implementations.
2018-01-11 13:03:20 +02:00
Serhiy Storchaka 986375ebde
bpo-28416: Break reference cycles in Pickler and Unpickler subclasses (#4080)
with the persistent_id() and persistent_load() methods.
2017-11-30 22:48:31 +02:00
Serhiy Storchaka 6545256df9
bpo-32032: Test both implementations of module-level pickle API. (#4401) 2017-11-15 14:01:08 +02:00
Eric Snow c943265ba5 Issue #15767: Add ModuleNotFoundError. 2016-09-07 15:42:32 -07:00
Serhiy Storchaka 90493ab30c Issue #25761: Improved error reporting about truncated pickle data in
C implementation of unpickler.  UnpicklingError is now raised instead of
AttributeError and ValueError in some cases.
2016-09-06 23:55:11 +03:00
Serhiy Storchaka 3410c01d83 Issue #17711: Fixed unpickling by the persistent ID with protocol 0.
Original patch by Alexandre Vassalotti.
2016-07-17 11:35:35 +03:00
Serhiy Storchaka dec25afab1 Issue #17711: Fixed unpickling by the persistent ID with protocol 0.
Original patch by Alexandre Vassalotti.
2016-07-17 11:24:17 +03:00
Serhiy Storchaka 9d4f3d81c7 Added exceptins for testing non-reversible import mapping for Issue #26013. 2016-01-18 22:34:19 +02:00
Serhiy Storchaka 5c1d9d2ff7 Added exceptins for testing non-reversible import mapping for Issue #26013. 2016-01-18 22:33:44 +02:00
Serhiy Storchaka 59fb6342a4 Issue #25761: Improved detecting errors in broken pickle data. 2015-12-06 22:01:35 +02:00
Serhiy Storchaka 3c49710e9f Issue #25761: Added more test cases for testing unpickling broken data.
Output raised exception at verbose level 2 (-vv).
2015-11-29 13:13:24 +02:00
Serhiy Storchaka 7279befccb Issue #25761: Added more test cases for testing unpickling broken data.
Output raised exception at verbose level 2 (-vv).
2015-11-29 13:12:10 +02:00
Serhiy Storchaka 4409c6cfae Issue #23914: Fixed SystemError raised by unpickler on broken pickle data. 2015-11-23 15:19:23 +02:00
Serhiy Storchaka e9b3074cf9 Issue #23914: Fixed SystemError raised by unpickler on broken pickle data. 2015-11-23 15:17:43 +02:00
Serhiy Storchaka 456a27013e Issue #25365: test_pickle now works in threads disabled builds. 2015-10-10 20:10:54 +03:00
Serhiy Storchaka 7b2cfc4465 Issue #25365: test_pickle now works in threads disabled builds. 2015-10-10 20:10:07 +03:00
Serhiy Storchaka 104de66b76 Moved unpickling tests with prepickled data to separate class. 2015-09-29 15:34:53 +03:00
Serhiy Storchaka c6b54b45ea Moved unpickling tests with prepickled data to separate class. 2015-09-29 15:33:24 +03:00
Yury Selivanov f488fb422a Issue #19235: Add new RecursionError exception. Patch by Georg Brandl. 2015-07-03 01:04:23 -04:00
Yury Selivanov 7544508f02 PEP 0492 -- Coroutines with async and await syntax. Issue #24017. 2015-05-11 22:57:16 -04:00
Serhiy Storchaka b9100e5d9d Issue #18473: Fixed pickle compatibility tests for optional modules.
Added WindowsError to compatibility mappings.
2015-03-31 16:49:26 +03:00
Serhiy Storchaka bfe1824d08 Issue #18473: Fixed 2to3 and 3to2 compatible pickle mappings.
Fixed ambigious reverse mappings.  Added many new mappings.  Import mapping
is no longer applied to modules already mapped with full name mapping.

Added tests for compatible pickling and unpickling and for consistency of
_compat_pickle mappings.
2015-03-31 13:12:37 +03:00
Serhiy Storchaka 5bbd231f27 Issue #15513: Added a __sizeof__ implementation for pickle classes. 2014-12-16 19:39:08 +02:00
Alexandre Vassalotti d05c9ff845 Issue #6784: Strings from Python 2 can now be unpickled as bytes objects.
Initial patch by Merlijn van Deen.

I've added a few unrelated docstring fixes in the patch while I was at
it, which makes the documentation for pickle a bit more consistent.
2013-12-07 01:09:27 -08:00
Christian Heimes 2178248858 Issue #18339: use with self.assertRaises() to make test case more readable 2013-07-01 23:00:13 +02:00
Christian Heimes a24b4d260b Issue #18339: Negative ints keys in unpickler.memo dict no longer cause a
segfault inside the _pickle C extension.
2013-07-01 15:17:45 +02:00
Antoine Pitrou 8d3c290de4 Issue #14166: Pickler objects now have an optional `dispatch_table` attribute which allows to set custom per-pickler reduction functions.
Patch by sbt.
2012-03-04 18:31:48 +01:00
Antoine Pitrou 82be19f889 Issue #11564: Avoid crashes when trying to pickle huge objects or containers
(more than 2**31 items).  Instead, in most cases, an OverflowError is raised.
2011-08-29 23:09:33 +02:00
Alexander Belopolsky ec8f0df229 Merged revisions 88546 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r88546 | alexander.belopolsky | 2011-02-24 14:40:09 -0500 (Thu, 24 Feb 2011) | 3 lines

  Issue #11286: Fixed unpickling of empty 2.x strings.
........
2011-02-24 20:34:38 +00:00
Antoine Pitrou ea99c5c949 Issue #9410: Various optimizations to the pickle module, leading to
speedups up to 4x (depending on the benchmark).  Mostly ported from
Unladen Swallow; initial patch by Alexandre Vassalotti.
2010-09-09 18:33:21 +00:00
Collin Winter 771d8341cd Port r71408 to py3k: issue 5665, add more pickling tests. 2009-04-16 03:18:06 +00:00
Alexandre Vassalotti ca2d610dba Restore _pickle module accelerator module.
Removed Windows support temporarily.
64bit bug with integer unpickling is now fixed.
2008-06-12 18:26:05 +00:00
Benjamin Peterson 75f25f2c9a revert the addition of _pickle because it was causing havok with 64-bit 2008-06-12 03:10:02 +00:00
Alexandre Vassalotti cc313061a5 Issue 2917: Merge the pickle and cPickle module. 2008-06-11 22:43:06 +00:00
Benjamin Peterson ee8712cda4 #2621 rename test.test_support to test.support 2008-05-20 21:35:26 +00:00
Guido van Rossum f416981691 - A new pickle protocol (protocol 3) is added with explicit support
for bytes.  This is the default protocol.  It intentionally cannot
  be unpickled by Python 2.x.

- When a pickle	written	by Python 2.x contains an (8-bit) str
  instance, this is now decoded to a (Unicode) str instance.  The
  encoding used to do this defaults to ASCII, but can be overridden
  via two new keyword arguments to the Unpickler class.  Previously
  this would create bytes instances, which is usually wrong: str
  instances are often used to pickle attribute names etc., and text is
  more common than binary data anyway.
2008-03-17 22:56:06 +00:00
Christian Heimes 05e8be17fd Merged revisions 60990-61002 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r60990 | eric.smith | 2008-02-23 17:05:26 +0100 (Sat, 23 Feb 2008) | 1 line

  Removed duplicate Py_CHARMASK define.  It's already defined in Python.h.
........
  r60991 | andrew.kuchling | 2008-02-23 17:23:05 +0100 (Sat, 23 Feb 2008) | 4 lines

  #1330538: Improve comparison of xmlrpclib.DateTime and datetime instances.
  Remove automatic handling of datetime.date and datetime.time.
  This breaks backward compatibility, but python-dev discussion was strongly
  against this automatic conversion; see the bug for a link.
........
  r60994 | andrew.kuchling | 2008-02-23 17:39:43 +0100 (Sat, 23 Feb 2008) | 1 line

  #835521: Add index entries for various pickle-protocol methods and attributes
........
  r60995 | andrew.kuchling | 2008-02-23 18:10:46 +0100 (Sat, 23 Feb 2008) | 2 lines

  #1433694: minidom's .normalize() failed to set .nextSibling for last element.
  Fix by Malte Helmert
........
  r61000 | christian.heimes | 2008-02-23 18:40:11 +0100 (Sat, 23 Feb 2008) | 1 line

  Patch #2167 from calvin: Remove unused imports
........
  r61001 | christian.heimes | 2008-02-23 18:42:31 +0100 (Sat, 23 Feb 2008) | 1 line

  Patch #1957: syslogmodule: Release GIL when calling syslog(3)
........
  r61002 | christian.heimes | 2008-02-23 18:52:07 +0100 (Sat, 23 Feb 2008) | 2 lines

  Issue #2051 and patch from Alexander Belopolsky:
  Permission for pyc and pyo files are inherited from the py file.
........
2008-02-23 18:30:17 +00:00
Guido van Rossum 98297ee781 Merging the py3k-pep3137 branch back into the py3k branch.
No detailed change log; just check out the change log for the py3k-pep3137
branch.  The most obvious changes:

  - str8 renamed to bytes (PyString at the C level);
  - bytes renamed to buffer (PyBytes at the C level);
  - PyString and PyUnicode are no longer compatible.

I.e. we now have an immutable bytes type and a mutable bytes type.

The behavior of PyString was modified quite a bit, to make it more
bytes-like.  Some changes are still on the to-do list.
2007-11-06 21:34:58 +00:00
Guido van Rossum cfe5f20fe8 Got test_pickletools and test_pickle working.
(Alas, test_cpickle is still broken.)
2007-05-08 21:26:54 +00:00
Walter Dörwald 21d3a32b99 Combine the functionality of test_support.run_unittest()
and test_support.run_classtests() into run_unittest()
and use it wherever possible.

Also don't use "from test.test_support import ...", but
"from test import test_support" in a few spots.

From SF patch #662807.
2003-05-01 17:45:56 +00:00
Tim Peters 080c88b912 cPickle.c, load_build(): Taught cPickle how to pick apart
the optional proto 2 slot state.

pickle.py, load_build():  CAUTION:  Noted that cPickle's
load_build and pickle's load_build really don't do the same
things with the state, and didn't before this patch either.
cPickle never tries to do .update(), and has no backoff if
instance.__dict__ can't be retrieved.  There are no tests
that can tell the difference, and part of what cPickle's
load_build() did looked accidental to me, so I don't know
what the true intent is here.

pickletester.py, test_pickle.py:  Got rid of the hack for
exempting cPickle from running some of the proto 2 tests.

dictobject.c, PyDict_Next():  documented intended use.
2003-02-15 03:01:11 +00:00
Tim Peters e14295cf5f pickle.py has a few doctest'ed internal functions, so run their tests. 2003-01-30 21:27:37 +00:00
Guido van Rossum 5d9113d8be Implement appropriate __getnewargs__ for all immutable subclassable builtin
types.  The special handling for these can now be removed from save_newobj().
Add some testing for this.

Also add support for setting the 'fast' flag on the Python Pickler class,
which suppresses use of the memo.
2003-01-29 17:58:45 +00:00
Tim Peters 47a6b13988 Temporary hacks to arrange that the pickle tests relying on protocol 2
only get run by test_pickle.py now (& not by test_cpickle.py).  This
should be undone when protocol 2 is implemented in cPickle too.
test_cpickle should pass again.
2003-01-28 22:34:11 +00:00