cpython/Doc/library/marshal.rst

134 lines
5.5 KiB
ReStructuredText
Raw Normal View History

2007-08-15 11:28:01 -03:00
:mod:`marshal` --- Internal Python object serialization
=======================================================
.. module:: marshal
:synopsis: Convert Python objects to streams of bytes and back (with different
constraints).
This module contains functions that can read and write Python values in a binary
format. The format is specific to Python, but independent of machine
architecture issues (e.g., you can write a Python value to a file on a PC,
transport the file to a Sun, and read it back there). Details of the format are
undocumented on purpose; it may change between Python versions (although it
rarely does). [#]_
.. index::
module: pickle
module: shelve
object: code
This is not a general "persistence" module. For general persistence and
transfer of Python objects through RPC calls, see the modules :mod:`pickle` and
:mod:`shelve`. The :mod:`marshal` module exists mainly to support reading and
writing the "pseudo-compiled" code for Python modules of :file:`.pyc` files.
Therefore, the Python maintainers reserve the right to modify the marshal format
in backward incompatible ways should the need arise. If you're serializing and
de-serializing Python objects, use the :mod:`pickle` module instead -- the
performance is comparable, version independence is guaranteed, and pickle
supports a substantially wider range of objects than marshal.
2007-08-15 11:28:01 -03:00
.. warning::
The :mod:`marshal` module is not intended to be secure against erroneous or
maliciously constructed data. Never unmarshal data received from an
untrusted or unauthenticated source.
Not all Python object types are supported; in general, only objects whose value
is independent from a particular invocation of Python can be written and read by
Merged revisions 74492,74531,74545-74550,74553-74555,74588,74603,74608,74614,74616-74618,74631-74633,74652-74653,74666,74671,74737,74739,74779,74781-74782,74784,74791,74793,74818-74820,74822,74832 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r74492 | r.david.murray | 2009-08-17 21:26:49 +0200 (Mo, 17 Aug 2009) | 2 lines Issue 6685: 'toupper' -> 'upper' in cgi doc example explanation. ........ r74531 | vinay.sajip | 2009-08-21 00:04:32 +0200 (Fr, 21 Aug 2009) | 1 line Added section on exceptions raised during logging. ........ r74545 | georg.brandl | 2009-08-24 19:14:29 +0200 (Mo, 24 Aug 2009) | 1 line #6772: mention utf-8 as utf8 alias. ........ r74546 | georg.brandl | 2009-08-24 19:20:40 +0200 (Mo, 24 Aug 2009) | 1 line #6725: spell "namespace" consistently. ........ r74547 | georg.brandl | 2009-08-24 19:22:05 +0200 (Mo, 24 Aug 2009) | 1 line #6718: fix example. ........ r74548 | georg.brandl | 2009-08-24 19:24:27 +0200 (Mo, 24 Aug 2009) | 1 line #6677: mention "deleting" as an alias for removing files. ........ r74549 | benjamin.peterson | 2009-08-24 19:42:36 +0200 (Mo, 24 Aug 2009) | 1 line fix pdf building by teaching latex the right encoding package ........ r74550 | georg.brandl | 2009-08-24 19:48:40 +0200 (Mo, 24 Aug 2009) | 1 line #6677: note that rmdir only removes empty directories. ........ r74553 | r.david.murray | 2009-08-27 03:04:59 +0200 (Do, 27 Aug 2009) | 2 lines Remove leftover text from end of sentence. ........ r74554 | georg.brandl | 2009-08-27 20:59:02 +0200 (Do, 27 Aug 2009) | 1 line Typo fix. ........ r74555 | georg.brandl | 2009-08-27 21:02:43 +0200 (Do, 27 Aug 2009) | 1 line #6787: reference fix. ........ r74588 | georg.brandl | 2009-08-30 10:35:01 +0200 (So, 30 Aug 2009) | 1 line #6803: fix old name. ........ r74603 | georg.brandl | 2009-08-31 08:38:29 +0200 (Mo, 31 Aug 2009) | 1 line other -> others where multiple arguments are accepted. ........ r74608 | senthil.kumaran | 2009-08-31 18:40:27 +0200 (Mo, 31 Aug 2009) | 3 lines Doc fix for the issue2637. ........ r74614 | georg.brandl | 2009-09-01 09:40:54 +0200 (Di, 01 Sep 2009) | 1 line #6813: better documentation for numberless string formats. ........ r74616 | georg.brandl | 2009-09-01 09:46:26 +0200 (Di, 01 Sep 2009) | 1 line #6808: clarification. ........ r74617 | georg.brandl | 2009-09-01 09:53:37 +0200 (Di, 01 Sep 2009) | 1 line #6765: hint that log(x, base) is not very sophisticated. ........ r74618 | georg.brandl | 2009-09-01 10:00:47 +0200 (Di, 01 Sep 2009) | 1 line #6810: add a link to the section about frame objects instead of just a description where to find it. ........ r74631 | georg.brandl | 2009-09-02 22:37:16 +0200 (Mi, 02 Sep 2009) | 1 line #6821: fix signature of PyBuffer_Release(). ........ r74632 | georg.brandl | 2009-09-03 09:27:26 +0200 (Do, 03 Sep 2009) | 1 line #6828: fix wrongly highlighted blocks. ........ r74633 | georg.brandl | 2009-09-03 14:31:39 +0200 (Do, 03 Sep 2009) | 1 line #6757: complete the list of types that marshal can serialize. ........ r74652 | georg.brandl | 2009-09-04 13:25:37 +0200 (Fr, 04 Sep 2009) | 1 line #6756: add some info about the "acct" parameter. ........ r74653 | georg.brandl | 2009-09-04 13:32:18 +0200 (Fr, 04 Sep 2009) | 1 line #6777: dont discourage usage of Exception.args or promote usage of Exception.message. ........ r74666 | georg.brandl | 2009-09-05 11:04:09 +0200 (Sa, 05 Sep 2009) | 1 line #6841: remove duplicated word. ........ r74671 | georg.brandl | 2009-09-05 18:47:17 +0200 (Sa, 05 Sep 2009) | 1 line #6843: add link from filterwarnings to where the meaning of the arguments is covered. ........ r74737 | georg.brandl | 2009-09-09 18:49:13 +0200 (Mi, 09 Sep 2009) | 1 line Properly document copy and deepcopy as functions. ........ r74739 | georg.brandl | 2009-09-11 09:55:20 +0200 (Fr, 11 Sep 2009) | 1 line Move function back to its section. ........ r74779 | michael.foord | 2009-09-13 18:13:36 +0200 (So, 13 Sep 2009) | 1 line Change to tutorial wording for reading text / binary files on Windows. Issue #6301. ........ r74781 | michael.foord | 2009-09-13 18:46:19 +0200 (So, 13 Sep 2009) | 1 line Note that sys._getframe is not guaranteed to exist in all implementations of Python, and a corresponding note in inspect.currentframe. Issue 6712. ........ r74782 | michael.foord | 2009-09-13 19:07:46 +0200 (So, 13 Sep 2009) | 1 line Tutorial tweaks. Issue 6849. ........ r74784 | georg.brandl | 2009-09-13 20:15:07 +0200 (So, 13 Sep 2009) | 1 line Typo fix. ........ r74791 | georg.brandl | 2009-09-14 16:08:54 +0200 (Mo, 14 Sep 2009) | 1 line #6574: list the future features in a table. ........ r74793 | georg.brandl | 2009-09-14 16:50:47 +0200 (Mo, 14 Sep 2009) | 1 line #6908: fix association of hashlib hash attributes. ........ r74818 | georg.brandl | 2009-09-16 11:23:04 +0200 (Mi, 16 Sep 2009) | 1 line #6880: add reference to classes section in exceptions section, which comes earlier. ........ r74819 | georg.brandl | 2009-09-16 11:24:57 +0200 (Mi, 16 Sep 2009) | 1 line #6876: fix base class constructor invocation in example. ........ r74820 | georg.brandl | 2009-09-16 11:30:48 +0200 (Mi, 16 Sep 2009) | 1 line #6891: comment out dead link to Unicode article. ........ r74822 | georg.brandl | 2009-09-16 12:12:06 +0200 (Mi, 16 Sep 2009) | 1 line #5621: refactor description of how class/instance attributes interact on a.x=a.x+1 or augassign. ........ r74832 | georg.brandl | 2009-09-16 17:57:46 +0200 (Mi, 16 Sep 2009) | 1 line Rewrap long lines. ........
2009-10-27 11:50:20 -03:00
this module. The following types are supported: booleans, integers, long
integers, floating point numbers, complex numbers, strings, Unicode objects,
tuples, lists, sets, frozensets, dictionaries, and code objects, where it should
be understood that tuples, lists, sets, frozensets and dictionaries are only
supported as long as the values contained therein are themselves supported; and
recursive lists, sets and dictionaries should not be written (they will cause
infinite loops). The singletons :const:`None`, :const:`Ellipsis` and
:exc:`StopIteration` can also be marshalled and unmarshalled.
2007-08-15 11:28:01 -03:00
.. warning::
Merged revisions 68133-68134,68141-68142,68145-68146,68148-68149,68159-68162,68166,68171-68174,68179,68195-68196,68210,68214-68215,68217-68222 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r68133 | antoine.pitrou | 2009-01-01 16:38:03 +0100 (Thu, 01 Jan 2009) | 1 line fill in actual issue number in tests ........ r68134 | hirokazu.yamamoto | 2009-01-01 16:45:39 +0100 (Thu, 01 Jan 2009) | 2 lines Issue #4797: IOError.filename was not set when _fileio.FileIO failed to open file with `str' filename on Windows. ........ r68141 | benjamin.peterson | 2009-01-01 17:43:12 +0100 (Thu, 01 Jan 2009) | 1 line fix highlighting ........ r68142 | benjamin.peterson | 2009-01-01 18:29:49 +0100 (Thu, 01 Jan 2009) | 2 lines welcome to 2009, Python! ........ r68145 | amaury.forgeotdarc | 2009-01-02 01:03:54 +0100 (Fri, 02 Jan 2009) | 5 lines #4801 _collections module fails to build on cygwin. _PyObject_GC_TRACK is the macro version of PyObject_GC_Track, and according to documentation it should not be used for extension modules. ........ r68146 | ronald.oussoren | 2009-01-02 11:44:46 +0100 (Fri, 02 Jan 2009) | 2 lines Fix for issue4472: "configure --enable-shared doesn't work on OSX" ........ r68148 | ronald.oussoren | 2009-01-02 11:48:31 +0100 (Fri, 02 Jan 2009) | 2 lines Forgot to add a NEWS item in my previous checkin ........ r68149 | ronald.oussoren | 2009-01-02 11:50:48 +0100 (Fri, 02 Jan 2009) | 2 lines Fix for issue4780 ........ r68159 | ronald.oussoren | 2009-01-02 15:48:17 +0100 (Fri, 02 Jan 2009) | 2 lines Fix for issue 1627952 ........ r68160 | ronald.oussoren | 2009-01-02 15:52:09 +0100 (Fri, 02 Jan 2009) | 2 lines Fix for issue r1737832 ........ r68161 | ronald.oussoren | 2009-01-02 16:00:05 +0100 (Fri, 02 Jan 2009) | 3 lines Fix for issue 1149804 ........ r68162 | ronald.oussoren | 2009-01-02 16:06:00 +0100 (Fri, 02 Jan 2009) | 3 lines Fix for issue 4472 is incompatible with Cygwin, this patch should fix that. ........ r68166 | benjamin.peterson | 2009-01-02 19:26:23 +0100 (Fri, 02 Jan 2009) | 1 line document PyMemberDef ........ r68171 | georg.brandl | 2009-01-02 21:25:14 +0100 (Fri, 02 Jan 2009) | 3 lines #4811: fix markup glitches (mostly remains of the conversion), found by Gabriel Genellina. ........ r68172 | martin.v.loewis | 2009-01-02 21:32:55 +0100 (Fri, 02 Jan 2009) | 2 lines Issue #4075: Use OutputDebugStringW in Py_FatalError. ........ r68173 | martin.v.loewis | 2009-01-02 21:40:14 +0100 (Fri, 02 Jan 2009) | 2 lines Issue #4051: Prevent conflict of UNICODE macros in cPickle. ........ r68174 | benjamin.peterson | 2009-01-02 21:47:27 +0100 (Fri, 02 Jan 2009) | 1 line fix compilation on non-Windows platforms ........ r68179 | raymond.hettinger | 2009-01-02 22:26:45 +0100 (Fri, 02 Jan 2009) | 1 line Issue #4615. Document how to use itertools for de-duping. ........ r68195 | georg.brandl | 2009-01-03 14:45:15 +0100 (Sat, 03 Jan 2009) | 2 lines Remove useless string literal. ........ r68196 | georg.brandl | 2009-01-03 15:29:53 +0100 (Sat, 03 Jan 2009) | 2 lines Fix indentation. ........ r68210 | georg.brandl | 2009-01-03 20:10:12 +0100 (Sat, 03 Jan 2009) | 2 lines Set eol-style correctly for mp_distributing.py. ........ r68214 | georg.brandl | 2009-01-03 20:44:48 +0100 (Sat, 03 Jan 2009) | 2 lines Make indentation consistent. ........ r68215 | georg.brandl | 2009-01-03 21:15:14 +0100 (Sat, 03 Jan 2009) | 2 lines Fix role name. ........ r68217 | georg.brandl | 2009-01-03 21:30:15 +0100 (Sat, 03 Jan 2009) | 2 lines Add rstlint, a little tool to find subtle markup problems and inconsistencies in the Doc sources. ........ r68218 | georg.brandl | 2009-01-03 21:38:59 +0100 (Sat, 03 Jan 2009) | 2 lines Recognize usage of the default role. ........ r68219 | georg.brandl | 2009-01-03 21:47:01 +0100 (Sat, 03 Jan 2009) | 2 lines Fix uses of the default role. ........ r68220 | georg.brandl | 2009-01-03 21:55:06 +0100 (Sat, 03 Jan 2009) | 2 lines Remove trailing whitespace. ........ r68221 | georg.brandl | 2009-01-03 22:04:55 +0100 (Sat, 03 Jan 2009) | 2 lines Remove tabs from the documentation. ........ r68222 | georg.brandl | 2009-01-03 22:11:58 +0100 (Sat, 03 Jan 2009) | 2 lines Disable the line length checker by default. ........
2009-01-03 17:55:17 -04:00
On machines where C's ``long int`` type has more than 32 bits (such as the
DEC Alpha), it is possible to create plain Python integers that are longer
than 32 bits. If such an integer is marshaled and read back in on a machine
where C's ``long int`` type has only 32 bits, a Python long integer object
is returned instead. While of a different type, the numeric value is the
same. (This behavior is new in Python 2.2. In earlier versions, all but the
least-significant 32 bits of the value were lost, and a warning message was
printed.)
2007-08-15 11:28:01 -03:00
There are functions that read/write files as well as functions operating on
strings.
The module defines these functions:
.. function:: dump(value, file[, version])
Write the value on the open file. The value must be a supported type. The
file must be an open file object such as ``sys.stdout`` or returned by
:func:`open` or :func:`os.popen`. It must be opened in binary mode (``'wb'``
or ``'w+b'``).
If the value has (or contains an object that has) an unsupported type, a
:exc:`ValueError` exception is raised --- but garbage data will also be written
to the file. The object will not be properly read back by :func:`load`.
.. versionadded:: 2.4
The *version* argument indicates the data format that ``dump`` should use
(see below).
.. function:: load(file)
Read one value from the open file and return it. If no valid value is read
(e.g. because the data has a different Python version's incompatible marshal
format), raise :exc:`EOFError`, :exc:`ValueError` or :exc:`TypeError`. The
file must be an open file object opened in binary mode (``'rb'`` or
``'r+b'``).
.. note::
2007-08-15 11:28:01 -03:00
If an object containing an unsupported type was marshalled with :func:`dump`,
:func:`load` will substitute ``None`` for the unmarshallable type.
.. function:: dumps(value[, version])
Return the string that would be written to a file by ``dump(value, file)``. The
value must be a supported type. Raise a :exc:`ValueError` exception if value
has (or contains an object that has) an unsupported type.
.. versionadded:: 2.4
The *version* argument indicates the data format that ``dumps`` should use
(see below).
.. function:: loads(string)
Convert the string to a value. If no valid value is found, raise
:exc:`EOFError`, :exc:`ValueError` or :exc:`TypeError`. Extra characters in the
string are ignored.
In addition, the following constants are defined:
.. data:: version
Indicates the format that the module uses. Version 0 is the historical format,
version 1 (added in Python 2.4) shares interned strings and version 2 (added in
Python 2.5) uses a binary format for floating point numbers. The current version
is 2.
.. versionadded:: 2.4
.. rubric:: Footnotes
.. [#] The name of this module stems from a bit of terminology used by the designers of
Modula-3 (amongst others), who use the term "marshalling" for shipping of data
around in a self-contained form. Strictly speaking, "to marshal" means to
convert some data from internal to external form (in an RPC buffer for instance)
and "unmarshalling" for the reverse process.