Add more examples in pickle documentation.

This commit is contained in:
Alexandre Vassalotti 2009-04-03 06:13:29 +00:00
parent ffcec434ce
commit 9d7665df93
1 changed files with 79 additions and 27 deletions

View File

@ -14,6 +14,7 @@
.. sectionauthor:: Jim Kerr <jbkerr@sr.hp.com>.
.. sectionauthor:: Barry Warsaw <barry@zope.com>
The :mod:`pickle` module implements a fundamental, but powerful algorithm for
serializing and de-serializing a Python object structure. "Pickling" is the
process whereby a Python object hierarchy is converted into a byte stream, and
@ -282,23 +283,9 @@ The :mod:`pickle` module exports two classes, :class:`Pickler` and
.. attribute:: memo
Dictionary holding previously pickled objects to allow shared or
recursive objects to pickled by reference as opposed to by value.
.. XXX Move these comments to somewhere more appropriate.
It is possible to make multiple calls to the :meth:`dump` method of the same
:class:`Pickler` instance. These must then be matched to the same number of
calls to the :meth:`load` method of the corresponding :class:`Unpickler`
instance. If the same object is pickled by multiple :meth:`dump` calls, the
:meth:`load` will all yield references to the same object.
Please note, this is intended for pickling multiple objects without intervening
modifications to the objects or their parts. If you modify an object and then
pickle it again using the same :class:`Pickler` instance, the object is not
pickled again --- a reference to it is pickled and the :class:`Unpickler` will
return the old value, not the modified one.
Dictionary-like object holding previously pickled objects to allow
shared or recursive objects to pickled by reference as opposed to
by value.
.. class:: Unpickler(file, [\*, encoding="ASCII", errors="strict"])
@ -345,6 +332,11 @@ return the old value, not the modified one.
how they can be loaded, potentially reducing security risks. Refer to
:ref:`pickle-restrict` for details.
.. attribute:: memo
Dictionary holding previously unpickled objects to allow shared or
recursive objects to unpickled by reference as opposed to by value.
.. _pickle-picklable:
@ -593,7 +585,7 @@ reading resumes from the last location. The :meth:`__setstate__` and
line = self.file.readline()
if not line:
return None
if line.endswith("\n"):
if line.endswith('\n'):
line = line[:-1]
return "%i: %s" % (self.lineno, line)
@ -707,25 +699,24 @@ A sample usage of our unpickler working has intended::
As our examples shows, you have to be careful with what you allow to be
unpickled. Therefore if security is a concern, you may want to consider
alternatives such as the marshalling API in :mod:`xmlrpc.client` or third-party
solutions.
alternatives such as the marshalling API in :mod:`xmlrpc.client` or
third-party solutions.
.. _pickle-example:
Usage Examples
--------------
Examples
--------
For the simplest code, use the :func:`dump` and :func:`load` functions. Note
that a self-referencing list is pickled and restored correctly. ::
For the simplest code, use the :func:`dump` and :func:`load` functions. ::
import pickle
# An arbitrary collection of objects supported by pickle.
data = {
'a': [1, 2.0, 3, 4+6j],
'b': ("character string", b"byte string"),
'c': set([None, True, False])
'a': [1, 2.0, 3, 4+6j],
'b': ("character string", b"byte string"),
'c': set([None, True, False])
}
with open('data.pickle', 'wb') as f:
@ -743,11 +734,72 @@ The following example reads the resulting pickled data. ::
data = pickle.load(f)
Reusing Pickler Instances
^^^^^^^^^^^^^^^^^^^^^^^^^
It is possible to make multiple calls to the :meth:`dump` method of the same
:class:`Pickler` instance. These must then be matched to the same number of
calls to the :meth:`load` method of the corresponding :class:`Unpickler`
instance. If the same object is pickled by multiple :meth:`dump` calls, the
:meth:`load` will all yield references to the same object.
Please note, this is intended for pickling multiple objects without intervening
modifications to the objects or their parts. If you modify an object and then
pickle it again using the same :class:`Pickler` instance, the object is not
pickled again --- a reference to it is pickled and the :class:`Unpickler` will
return the old value, not the modified one. ::
import io
import pickle
data = {"hello": 0, "spam": 1}
# Create a binary file-like object to which the Pickler instance will
# write the pickles.
f = io.BytesIO()
p = pickle.Pickler(f)
p.dump(data)
# This second call appends a new pickle to the file. The modification we
# make is lost because objects are pickled by reference when seen again.
data["eggs"] = 2
p.dump(data)
# Now, we load the pickles saved in our file-like object.
f.seek(0)
u = pickle.Unpickler(f)
data1 = u.load()
data2 = u.load()
if data1 is data2:
print("data1 and data2 are the same object")
else:
print("data1 and data2 are not the same object")
if "eggs" in data2:
print("The modification was pickled.")
else:
print("The modification was not pickled.")
Reusing a :class:`Pickler` instance like we shown can be a useful
optimization. For example, a multi-process application could use this feature
to reduce the size of the pickles transmitted across processes over time
(assuming the pickles exchanged are containers sharing common immutable
objects). However, you should take special care to regularly clear
:attr:``Pickler.memo`` and :attr:``Unpickler.memo`` to avoid memory-leaks.
.. XXX: Add examples showing how to optimize pickles for size (like using
.. pickletools.optimize() or the gzip module).
.. seealso::
Module :mod:`copyreg`
Pickle interface constructor registration for extension types.
Module :mod:`pickletools`
Tools for working with and analyzing pickled data.
Module :mod:`shelve`
Indexed databases of objects; uses :mod:`pickle`.