cpython/Doc/library/email.parser.rst

225 lines
9.7 KiB
ReStructuredText
Raw Normal View History

2007-08-15 11:28:01 -03:00
:mod:`email`: Parsing email messages
------------------------------------
.. module:: email.parser
:synopsis: Parse flat text email messages to produce a message object structure.
Message object structures can be created in one of two ways: they can be created
from whole cloth by instantiating :class:`~email.message.Message` objects and
stringing them together via :meth:`attach` and :meth:`set_payload` calls, or they
can be created by parsing a flat text representation of the email message.
2007-08-15 11:28:01 -03:00
The :mod:`email` package provides a standard parser that understands most email
document structures, including MIME documents. You can pass the parser a string
or a file object, and the parser will return to you the root
:class:`~email.message.Message` instance of the object structure. For simple,
non-MIME messages the payload of this root object will likely be a string
containing the text of the message. For MIME messages, the root object will
return ``True`` from its :meth:`is_multipart` method, and the subparts can be
accessed via the :meth:`get_payload` and :meth:`walk` methods.
2007-08-15 11:28:01 -03:00
There are actually two parser interfaces available for use, the classic
:class:`Parser` API and the incremental :class:`FeedParser` API. The classic
:class:`Parser` API is fine if you have the entire text of the message in memory
as a string, or if the entire message lives in a file on the file system.
:class:`FeedParser` is more appropriate for when you're reading the message from
a stream which might block waiting for more input (e.g. reading an email message
from a socket). The :class:`FeedParser` can consume and parse the message
incrementally, and only returns the root object when you close the parser [#]_.
Note that the parser can be extended in limited ways, and of course you can
implement your own parser completely from scratch. There is no magical
connection between the :mod:`email` package's bundled parser and the
:class:`~email.message.Message` class, so your custom parser can create message
object trees any way it finds necessary.
2007-08-15 11:28:01 -03:00
FeedParser API
^^^^^^^^^^^^^^
.. versionadded:: 2.4
The :class:`FeedParser`, imported from the :mod:`email.feedparser` module,
provides an API that is conducive to incremental parsing of email messages, such
as would be necessary when reading the text of an email message from a source
that can block (e.g. a socket). The :class:`FeedParser` can of course be used
to parse an email message fully contained in a string or a file, but the classic
:class:`Parser` API may be more convenient for such use cases. The semantics
and results of the two parser APIs are identical.
The :class:`FeedParser`'s API is simple; you create an instance, feed it a bunch
of text until there's no more to feed it, then close the parser to retrieve the
root message object. The :class:`FeedParser` is extremely accurate when parsing
standards-compliant messages, and it does a very good job of parsing
non-compliant messages, providing information about how a message was deemed
broken. It will populate a message object's *defects* attribute with a list of
any problems it found in a message. See the :mod:`email.errors` module for the
list of defects that it can find.
Here is the API for the :class:`FeedParser`:
.. class:: FeedParser([_factory])
Create a :class:`FeedParser` instance. Optional *_factory* is a no-argument
callable that will be called whenever a new message object is needed. It
defaults to the :class:`email.message.Message` class.
.. method:: feed(data)
2007-08-15 11:28:01 -03:00
Feed the :class:`FeedParser` some more data. *data* should be a string
containing one or more lines. The lines can be partial and the
:class:`FeedParser` will stitch such partial lines together properly. The
lines in the string can have any of the common three line endings,
carriage return, newline, or carriage return and newline (they can even be
mixed).
2007-08-15 11:28:01 -03:00
.. method:: close()
2007-08-15 11:28:01 -03:00
Closing a :class:`FeedParser` completes the parsing of all previously fed
data, and returns the root message object. It is undefined what happens
if you feed more data to a closed :class:`FeedParser`.
2007-08-15 11:28:01 -03:00
Parser class API
^^^^^^^^^^^^^^^^
The :class:`Parser` class, imported from the :mod:`email.parser` module,
provides an API that can be used to parse a message when the complete contents
of the message are available in a string or file. The :mod:`email.parser`
module also provides a second class, called :class:`HeaderParser` which can be
used if you're only interested in the headers of the message.
:class:`HeaderParser` can be much faster in these situations, since it does not
attempt to parse the message body, instead setting the payload to the raw body
as a string. :class:`HeaderParser` has the same API as the :class:`Parser`
class.
.. class:: Parser([_class])
The constructor for the :class:`Parser` class takes an optional argument
*_class*. This must be a callable factory (such as a function or a class), and
it is used whenever a sub-message object needs to be created. It defaults to
:class:`~email.message.Message` (see :mod:`email.message`). The factory will
be called without arguments.
2007-08-15 11:28:01 -03:00
The optional *strict* flag is ignored.
.. deprecated:: 2.4
Because the :class:`Parser` class is a backward compatible API wrapper
around the new-in-Python 2.4 :class:`FeedParser`, *all* parsing is
effectively non-strict. You should simply stop passing a *strict* flag to
the :class:`Parser` constructor.
.. versionchanged:: 2.2.2
The *strict* flag was added.
.. versionchanged:: 2.4
The *strict* flag was deprecated.
The other public :class:`Parser` methods are:
2007-08-15 11:28:01 -03:00
.. method:: parse(fp[, headersonly])
2007-08-15 11:28:01 -03:00
Read all the data from the file-like object *fp*, parse the resulting
text, and return the root message object. *fp* must support both the
:meth:`readline` and the :meth:`read` methods on file-like objects.
2007-08-15 11:28:01 -03:00
The text contained in *fp* must be formatted as a block of :rfc:`2822`
style headers and header continuation lines, optionally preceded by a
envelope header. The header block is terminated either by the end of the
data or by a blank line. Following the header block is the body of the
message (which may contain MIME-encoded subparts).
2007-08-15 11:28:01 -03:00
Optional *headersonly* is as with the :meth:`parse` method.
2007-08-15 11:28:01 -03:00
.. versionchanged:: 2.2.2
The *headersonly* flag was added.
2007-08-15 11:28:01 -03:00
.. method:: parsestr(text[, headersonly])
2007-08-15 11:28:01 -03:00
Similar to the :meth:`parse` method, except it takes a string object
instead of a file-like object. Calling this method on a string is exactly
equivalent to wrapping *text* in a :class:`StringIO` instance first and
calling :meth:`parse`.
2007-08-15 11:28:01 -03:00
Optional *headersonly* is a flag specifying whether to stop parsing after
reading the headers or not. The default is ``False``, meaning it parses
the entire contents of the file.
2007-08-15 11:28:01 -03:00
.. versionchanged:: 2.2.2
The *headersonly* flag was added.
2007-08-15 11:28:01 -03:00
Since creating a message object structure from a string or a file object is such
a common task, two functions are provided as a convenience. They are available
in the top-level :mod:`email` package namespace.
Merged revisions 66801,66803-66804,66813,66854-66856,66866,66870-66872,66874,66887,66903,66905,66911,66913,66927,66932,66938,66942,66962,66964,66973-66974,66977,66992,66998-66999,67002,67005,67007,67028,67040-67041,67044,67070,67089,67091,67101,67117-67119,67123-67124 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ................ r66801 | andrew.kuchling | 2008-10-04 23:51:59 +0200 (Sat, 04 Oct 2008) | 1 line Punctuation fix; expand dict.update docstring to be clearer ................ r66803 | benjamin.peterson | 2008-10-05 00:15:31 +0200 (Sun, 05 Oct 2008) | 1 line fix typo ................ r66804 | andrew.kuchling | 2008-10-05 02:11:56 +0200 (Sun, 05 Oct 2008) | 1 line #1415508 from Rocky Bernstein: add docstrings for enable_interspersed_args(), disable_interspersed_args() ................ r66813 | andrew.kuchling | 2008-10-06 14:07:04 +0200 (Mon, 06 Oct 2008) | 3 lines Per Greg Ward, optparse is no longer being externally maintained. I'll look at the bugs in the Optik bug tracker and copy them to the Python bug tracker if they're still relevant. ................ r66854 | georg.brandl | 2008-10-08 19:20:20 +0200 (Wed, 08 Oct 2008) | 2 lines #4059: patch up some sqlite docs. ................ r66855 | georg.brandl | 2008-10-08 19:30:55 +0200 (Wed, 08 Oct 2008) | 2 lines #4058: fix some whatsnew markup. ................ r66856 | georg.brandl | 2008-10-08 20:47:17 +0200 (Wed, 08 Oct 2008) | 3 lines #3935: properly support list subclasses in the C impl. of bisect. Patch reviewed by Raymond. ................ r66866 | benjamin.peterson | 2008-10-09 22:54:43 +0200 (Thu, 09 Oct 2008) | 1 line update paragraph about __future__ for 2.6 ................ r66870 | armin.rigo | 2008-10-10 10:40:44 +0200 (Fri, 10 Oct 2008) | 2 lines Typo: "ThreadError" is the name in the C source. ................ r66871 | benjamin.peterson | 2008-10-10 22:38:49 +0200 (Fri, 10 Oct 2008) | 1 line fix a small typo ................ r66872 | benjamin.peterson | 2008-10-10 22:51:37 +0200 (Fri, 10 Oct 2008) | 1 line talk about how you can unzip with zip ................ r66874 | benjamin.peterson | 2008-10-11 00:23:41 +0200 (Sat, 11 Oct 2008) | 1 line PyGILState_Acquire -> PyGILState_Ensure ................ r66887 | benjamin.peterson | 2008-10-13 23:51:40 +0200 (Mon, 13 Oct 2008) | 1 line document how to disable fixers ................ r66903 | benjamin.peterson | 2008-10-15 22:34:09 +0200 (Wed, 15 Oct 2008) | 1 line don't recurse into directories that start with '.' ................ r66905 | benjamin.peterson | 2008-10-15 23:05:55 +0200 (Wed, 15 Oct 2008) | 1 line support the optional line argument for idle ................ r66911 | benjamin.peterson | 2008-10-16 01:10:28 +0200 (Thu, 16 Oct 2008) | 41 lines Merged revisions 66805,66841,66860,66884-66886,66893,66907,66910 via svnmerge from svn+ssh://pythondev@svn.python.org/sandbox/trunk/2to3/lib2to3 ........ r66805 | benjamin.peterson | 2008-10-04 20:11:02 -0500 (Sat, 04 Oct 2008) | 1 line mention what the fixes directory is for ........ r66841 | benjamin.peterson | 2008-10-07 17:48:12 -0500 (Tue, 07 Oct 2008) | 1 line use assertFalse and assertTrue ........ r66860 | benjamin.peterson | 2008-10-08 16:05:07 -0500 (Wed, 08 Oct 2008) | 1 line instead of abusing the pattern matcher, use start_tree to find a next binding ........ r66884 | benjamin.peterson | 2008-10-13 15:50:30 -0500 (Mon, 13 Oct 2008) | 1 line don't print tokens to stdout when -v is given ........ r66885 | benjamin.peterson | 2008-10-13 16:28:57 -0500 (Mon, 13 Oct 2008) | 1 line add the -x option to disable fixers ........ r66886 | benjamin.peterson | 2008-10-13 16:33:53 -0500 (Mon, 13 Oct 2008) | 1 line cut down on some crud ........ r66893 | benjamin.peterson | 2008-10-14 17:16:54 -0500 (Tue, 14 Oct 2008) | 1 line add an optional set literal fixer ........ r66907 | benjamin.peterson | 2008-10-15 16:59:41 -0500 (Wed, 15 Oct 2008) | 1 line don't write backup files by default ........ r66910 | benjamin.peterson | 2008-10-15 17:43:10 -0500 (Wed, 15 Oct 2008) | 1 line add the -n option; it stops backupfiles from being written ........ ................ r66913 | benjamin.peterson | 2008-10-16 20:52:14 +0200 (Thu, 16 Oct 2008) | 1 line document that deque indexing is O(n) #4123 ................ r66927 | andrew.kuchling | 2008-10-16 22:15:47 +0200 (Thu, 16 Oct 2008) | 1 line Fix wording (2.6.1 backport candidate) ................ r66932 | benjamin.peterson | 2008-10-16 23:09:28 +0200 (Thu, 16 Oct 2008) | 1 line check for error conditions in _json #3623 ................ r66938 | benjamin.peterson | 2008-10-16 23:27:54 +0200 (Thu, 16 Oct 2008) | 1 line fix possible ref leak ................ r66942 | benjamin.peterson | 2008-10-16 23:48:06 +0200 (Thu, 16 Oct 2008) | 1 line fix more possible ref leaks in _json and use Py_CLEAR ................ r66962 | benjamin.peterson | 2008-10-17 22:01:01 +0200 (Fri, 17 Oct 2008) | 1 line clarify CALL_FUNCTION #4141 ................ r66964 | georg.brandl | 2008-10-17 23:41:49 +0200 (Fri, 17 Oct 2008) | 2 lines Fix duplicate word. ................ r66973 | armin.ronacher | 2008-10-19 10:27:43 +0200 (Sun, 19 Oct 2008) | 3 lines Fixed #4067 by implementing _attributes and _fields for the AST root node. ................ r66974 | benjamin.peterson | 2008-10-19 15:59:01 +0200 (Sun, 19 Oct 2008) | 1 line fix compiler warning ................ r66977 | benjamin.peterson | 2008-10-19 21:39:16 +0200 (Sun, 19 Oct 2008) | 1 line mention -n ................ r66992 | benjamin.peterson | 2008-10-21 22:51:13 +0200 (Tue, 21 Oct 2008) | 1 line make sure to call iteritems() ................ r66998 | benjamin.peterson | 2008-10-22 22:57:43 +0200 (Wed, 22 Oct 2008) | 1 line fix a few typos ................ r66999 | benjamin.peterson | 2008-10-22 23:05:30 +0200 (Wed, 22 Oct 2008) | 1 line and another typo... ................ r67002 | hirokazu.yamamoto | 2008-10-23 02:37:33 +0200 (Thu, 23 Oct 2008) | 1 line Issue #4183: Some tests didn't run with pickle.HIGHEST_PROTOCOL. ................ r67005 | walter.doerwald | 2008-10-23 15:11:39 +0200 (Thu, 23 Oct 2008) | 2 lines Use the correct names of the stateless codec functions (Fixes issue 4178). ................ r67007 | benjamin.peterson | 2008-10-23 23:43:48 +0200 (Thu, 23 Oct 2008) | 1 line only nonempty __slots__ don't work ................ r67028 | benjamin.peterson | 2008-10-26 01:27:07 +0200 (Sun, 26 Oct 2008) | 1 line don't use a catch-all ................ r67040 | armin.rigo | 2008-10-28 18:01:21 +0100 (Tue, 28 Oct 2008) | 5 lines Fix one of the tests: it relied on being present in an "output test" in order to actually test what it was supposed to test, i.e. that the code in the __del__ method did not crash. Use instead the new helper test_support.captured_output(). ................ r67041 | benjamin.peterson | 2008-10-29 21:33:00 +0100 (Wed, 29 Oct 2008) | 1 line mention the version gettempdir() was added ................ r67044 | amaury.forgeotdarc | 2008-10-30 00:15:57 +0100 (Thu, 30 Oct 2008) | 3 lines Correct error message in io.open(): closefd=True is the only accepted value with a file name. ................ r67070 | benjamin.peterson | 2008-10-31 21:41:44 +0100 (Fri, 31 Oct 2008) | 1 line rephrase has_key doc ................ r67089 | benjamin.peterson | 2008-11-03 21:43:20 +0100 (Mon, 03 Nov 2008) | 1 line clarify by splitting into multiple paragraphs ................ r67091 | benjamin.peterson | 2008-11-03 23:34:57 +0100 (Mon, 03 Nov 2008) | 1 line move a FileIO test to test_fileio ................ r67101 | georg.brandl | 2008-11-04 21:49:35 +0100 (Tue, 04 Nov 2008) | 2 lines #4167: fix markup glitches. ................ r67117 | georg.brandl | 2008-11-06 11:17:58 +0100 (Thu, 06 Nov 2008) | 2 lines #4268: Use correct module for two toplevel functions. ................ r67118 | georg.brandl | 2008-11-06 11:19:11 +0100 (Thu, 06 Nov 2008) | 2 lines #4267: small fixes in sqlite3 docs. ................ r67119 | georg.brandl | 2008-11-06 11:20:49 +0100 (Thu, 06 Nov 2008) | 2 lines #4245: move Thread section to the top. ................ r67123 | georg.brandl | 2008-11-06 19:49:15 +0100 (Thu, 06 Nov 2008) | 2 lines #4247: add "pass" examples to tutorial. ................ r67124 | andrew.kuchling | 2008-11-06 20:23:02 +0100 (Thu, 06 Nov 2008) | 1 line Fix grammar error; reword two paragraphs ................
2008-11-07 04:56:27 -04:00
.. currentmodule:: email
2007-08-15 11:28:01 -03:00
.. function:: message_from_string(s[, _class[, strict]])
Return a message object structure from a string. This is exactly equivalent to
``Parser().parsestr(s)``. Optional *_class* and *strict* are interpreted as
with the :class:`Parser` class constructor.
.. versionchanged:: 2.2.2
The *strict* flag was added.
.. function:: message_from_file(fp[, _class[, strict]])
Return a message object structure tree from an open file object. This is
exactly equivalent to ``Parser().parse(fp)``. Optional *_class* and *strict*
are interpreted as with the :class:`Parser` class constructor.
.. versionchanged:: 2.2.2
The *strict* flag was added.
Here's an example of how you might use this at an interactive Python prompt::
>>> import email
>>> msg = email.message_from_string(myString)
Additional notes
^^^^^^^^^^^^^^^^
Here are some notes on the parsing semantics:
* Most non-\ :mimetype:`multipart` type messages are parsed as a single message
object with a string payload. These objects will return ``False`` for
:meth:`is_multipart`. Their :meth:`get_payload` method will return a string
object.
* All :mimetype:`multipart` type messages will be parsed as a container message
object with a list of sub-message objects for their payload. The outer
container message will return ``True`` for :meth:`is_multipart` and their
:meth:`get_payload` method will return the list of :class:`~email.message.Message`
subparts.
2007-08-15 11:28:01 -03:00
* Most messages with a content type of :mimetype:`message/\*` (e.g.
:mimetype:`message/delivery-status` and :mimetype:`message/rfc822`) will also be
parsed as container object containing a list payload of length 1. Their
:meth:`is_multipart` method will return ``True``. The single element in the
list payload will be a sub-message object.
* Some non-standards compliant messages may not be internally consistent about
their :mimetype:`multipart`\ -edness. Such messages may have a
:mailheader:`Content-Type` header of type :mimetype:`multipart`, but their
:meth:`is_multipart` method may return ``False``. If such messages were parsed
with the :class:`FeedParser`, they will have an instance of the
:class:`MultipartInvariantViolationDefect` class in their *defects* attribute
list. See :mod:`email.errors` for details.
.. rubric:: Footnotes
.. [#] As of email package version 3.0, introduced in Python 2.4, the classic
:class:`Parser` was re-implemented in terms of the :class:`FeedParser`, so the
semantics and results are identical between the two parsers.