multipart/digest isn't a message/rfc822. This is legal, but counter
to recommended practice in RFC 2046, $5.1.5.
The fix is to look at the content type after setting the default
content type. If the maintype is then message or multipart, attach
the parsed subobject, otherwise use set_payload() to set the data of
the other object.
Ben. If s is a byte string, make sure it can be converted to unicode
with the input codec, and from unicode with the output codec, or raise
a UnicodeError exception early. Skip this test (and the unicode->byte
string conversion) when the charset is our faux 8bit raw charset.
must be a Charset instance, not a string. The bug here was that
self._charset wasn't being converted to a Charset instance so later
.append() calls which used the default charset would break.
_split(): If the charset of the chunk is '8bit', return the chunk
unchanged. We can't safely split it, so this is the avenue of least
harm.
8-bit data, we cannot split it safely, so return the original string
unchanged.
_is8bitstring(): Helper function which returns True when we have a
byte string that contains non-ascii characters (i.e. mysterious 8-bit
data).
Also, it fixes a really egregious error in Header.encode() (really
in Header._encode_chunks()) that could cause a header to grow and
grow each time encode() was called if output_codec was different
from input_codec.
Also, fix a typo.
the change in revision 1.11 (test_email.py) in response to SF bug
#609988. We now think that was the wrong fix and that WinZip was the
real culprit there.
get_type(). Also, one of the regular expressions is constant so might
as well make it a module global. And, when splitting up digests,
handle lineseps that are longer than 1 character in length
(e.g. \r\n).
semantics of header chunks using byte and Unicode strings.
Specifically,
append(): When the given string is a byte string, charset (whether
specified explicitly in the argument list or implicitly via the
constructor default) is the encoding of the byte string, and a
UnicodeError will be raised if the string cannot be decoded with that
charset. If s is a Unicode string, then charset is a hint specifying
the character set of the characters in the string. In this case, when
producing an RFC 2822 compliant header using RFC 2047 rules, the
Unicode string will be encoded using the following charsets in order:
us-ascii, the charset hint, utf-8.
__init__(): Use the global USASCII Charset instance when the charset
argument is None. Also, clarification in the docstring.
Also, use True/False where appropriate.
Python 2.1.3. However it's required by the email tests suite, so poke
it into the encodings aliases if it's missing. The is apparently the
approved API for doing so.
Now we can remove the hexversion shortcircuits in the test suite.
encoding flag SHORTEST means to return the shortest encoding between
base64 and qp. This is used for the header_enc for utf-8. SHORTEST
isn't legal for body_enc.
Also some code cleanup:
- use True/False everywhere
- use == instead of `is' in a few places
- added _unicode() and make consistent the "is unicode" checks
- update docstrings
project, and with assistance from Oleg Broytmann. Specifically,
added some new tests to make sure we handle RFC 2231 encoded
parameters correctly. Two new data files were added which contain RFC
2231 encoded parameters.
project, and with assistance from Oleg Broytmann. Specifically,
get_param(), get_params(): Document that these methods may return
parameter values that are either strings, or 3-tuples in the case of
RFC 2231 encoded parameters. The application should be prepared to
deal with such return values.
get_boundary(): Be prepared to deal with RFC 2231 encoded boundary
parameters. It makes little sense to have boundaries that are
anything but ascii, so if we get back a 3-tuple from get_param() we
will decode it into ascii and let any failures percolate up.
get_content_charset(): New method which treats the charset parameter
just like the boundary parameter in get_boundary(). Note that
"get_charset()" was already taken to return the default Charset
object.
get_charsets(): Rewrite to use get_content_charset().
Move the imports of Parser and Message inside the
message_from_string() and message_from_file() functions. This way
just "import email" won't suck in most of the submodules of the
package.
Note: this will break code that relied on "import email" giving you a
bunch of the submodules, but that was never documented and should not
have been relied on.
_handle_text(): Use _isstring() for stringiness test.
_handle_multipart(): Add a test before the ListType test, checking for
stringiness of the payload. String payloads for multitypes means a
message with broken MIME chrome was parsed by a lax parser. Instead
of raising a BoundaryError in those cases, the entire body is assigned
to the message payload (but since the content type is still
multipart/*, the Generator needs to be updated too).
Broytmann in SF patch #600096. Specifically, the former function now
encodes the triplets, while the latter adds optional charset and
language arguments.
2045, section 5.2 states that if the Content-Type: header is
syntactically invalid, the default type should be text/plain.
Implement minimal sanity checking of the header -- it must have
exactly one slash in it. This closes SF patch #597593 by Skip, but in
a different way.
Note that these methods used to raise ValueError for invalid ctypes,
but now they won't.
get the MIME main and sub types, instead of getting the whole ctype
and splitting it here. The two more specific methods now correctly
implement RFC 2045, section 5.2.
imports e.g. test_support must do so using an absolute package name
such as "import test.test_support" or "from test import test_support".
This also updates the README in Lib/test, and gets rid of the
duplicate data dirctory in Lib/test/data (replaced by
Lib/email/test/data).
Now Tim and Jack can have at it. :)
from test.test_support import TestSkipped, run_unittest
to
from test_support import TestSkipped, run_unittest
Otherwise, if the Japanese codecs aren't installed, regrtest doesn't
believe the TestSkipped exception raised by this test matches the
except (ImportError, test_support.TestSkipped), msg:
it's looking for, and reports the skip as a crash failure instead of
as a skipped test.
I suppose this will make it harder to run this test outside of
regrtest, but under the assumption only Barry does that, better to
make it skip cleanly for everyone else.
(i.e. email.test), so move the guts of them here from Lib/test. The
latter directory will retain stubs to run the email.test tests using
Python's standard regression test.
test_email_torture.py is a torture tester which will not run under
Python's test suite because I don't want to commit megs of data to
that project (it will fail cleanly there). When run under the mimelib
project it'll stress test the package with megs of message samples
collected from various locations in the wild.
(i.e. email.test), so move the guts of them here from Lib/test. The
latter directory will retain stubs to run the email.test tests using
Python's standard regression test.
test_email_torture.py is a torture tester which will not run under
Python's test suite because I don't want to commit megs of data to
that project (it will fail cleanly there). When run under the mimelib
project it'll stress test the package with megs of message samples
collected from various locations in the wild.
email/test/data is a copy of Lib/test/data. The fate of the latter is
still undecided.
backwards compatibility, we're silently deprecating get_type(),
get_subtype() and get_main_type(). We may eventually noisily
deprecate these. For now, we'll just fix a bug in the splitting of
the main and subtypes.
get_content_type(), get_content_maintype(), get_content_subtype(): New
methods which replace the above. These /always/ return a content type
string and do not take a failobj, because an email message always at
least has a default content type.
set_default_type(): Someday there may be additional default content
types, so don't hard code an assertion about the value of the ctype
argument.
quoting:
in non-strict mode, messages don't require a blank line at the end
with a missing end-terminator. A single newline is sufficient now.
Handle trailing whitespace at the end of a boundary. Had to switch
from using string.split() to re.split()
Handle whitespace on the end of a parameter list for Content-type.
Handle whitespace on the end of a plain content-type header.
Specifically,
get_type(): Strip the content type string.
_get_params_preserve(): Strip the parameter names and values on both
sides.
_parsebody(): Lots of changes as described above, with some stylistic
changes by Barry (who hopefully didn't screw things up ;).
create a Header instance. Closes feature request #539481.
Header.__init__(): Allow the initial string to be omitted.
__eq__(), __ne__(): Support rich comparisons for equality of Header
instances withy Header instances or strings.
Also, update a bunch of docstrings.
argument to the constructor -- defaulting to true -- which is
different than Anthony's approach of using global state.
parse(), parsestr(): Grow a `headersonly' argument which stops parsing
once the header block has been seen, i.e. it does /not/ parse or even
read the body of the message. This is used for parsing message/rfc822
type messages.
We need test cases for the non-strict parsing. Anthony will supply
these.
_parsebody(): We can get rid of the isdigest end-of-line kludges,
although we still need to know if we're parsing a multipart/digest so
we can set the default type accordingly.
text/plain but the RFCs state that inside a multipart/digest, the
default type is message/rfc822. To preserve idempotency, we need a
separate place to define the default type than the Content-Type:
header.
get_default_type(), set_default_type(): Accessor and mutator methods
for the default type.
recursive generation).
_dispatch(): If the message object doesn't have a Content-Type:
header, check its default type instead of assuming it's text/plain.
This makes for correct generation of message/rfc822 containers.
_handle_multipart(): We can get rid of the isdigest kludge. Just
print the message as normal and everything will work out correctly.
_handle_mulitpart_digest(): We don't need this anymore either.
Barry Warsaw
Tim Peters