[ 1752723 ] email.message_from_string: initial line gets discarded
I added a test to assert that when the first line of text passed to
message_from_string() contains a leading space, the message ends up with the
appropriate FirstHeaderLineIsContinuationDefect on its defects list.
The bug is invalid.
2047-like headers where there is no whitespace between encoded words. This
fix changes the matching regexp to include a trailing lookahead assertion that
the closing ?= must be followed by whitespace, newline, or end-of-string.
This also changes the regexp to add the MULTILINE flag.
it into email 4.0. Specifically, in Message.get_content_charset(), handle RFC
2231 headers that contain an encoding not known to Python, or a character in
the data that isn't in the charset encoding. Also forward port the
appropriate unit tests.
points out there are really two types of continued headers defined in this
RFC (i.e. "encoded" parameters with the form "name*0*=" and unencoded
parameters with the form "name*0="), but we were were handling them both the
same way and that isn't correct.
This patch should be much more RFC compliant in that only encoded params are
%-decoded and the charset/language information is only extract if there are
any encoded params in the segments. If there are no encoded params then the
RFC says that there will be no charset/language parts.
Note however that this will change the return value for Message.get_param() in
some cases. For example, whereas before if you had all unencoded param
continuations you would have still gotten a 3-tuple back from this method
(with charset and language == None), you will now get just a string. I don't
believe this is a backward incompatible change though because the
documentation for this method already indicates that either return value is
possible and that you must do an isinstance(val, tuple) check to discriminate
between the two. (Yeah that API kind of sucks but we can't change /that/
without breaking code.)
Test cases, some documentation updates, and a NEWS item accompany this patch.
Specifically, instead of raising a ValueError when there is a single tick in
the parameter, simply return that the entire string unquoted, with None for
both the charset and the language. Also, if there are more than 2 ticks in
the parameter, interpret the first three parts as the standard RFC 2231 parts,
then the rest of the parts as the encoded string.
Test cases added.
Original fewer-than-3-parts fix by Tokio Kikuchi.
Resolves SF bug # 1218081. I will back port the fix and tests to Python 2.4
(email 3.0) and Python 2.3 (email 2.5).
Also, bump the version number to email 4.0.1, removing the 'alpha' moniker.
Patch #1464708 from William McVey: fixed handling of nested comments in mail
addresses. E.g.
"Foo ((Foo Bar)) <foo@example.com>"
Fixes for both rfc822.py and email package. This patch needs to be back
ported to Python 2.3 for email 2.5.
SF bug #1403349 solution for email 3.0; some MUAs use the 'file' parameter
name in the Content-Distribution header, so Message.get_filename() should fall
back to using that. Will port to the Python 2.5 trunk.
Also, bump the email package version to 3.0.1 for eventual release. Of
course, add a test case too.
XXX Need to update the documentation.
caused by a self._input.readline() call that wasn't checking for the
NeedsMoreData marker.
msg_43.txt contains a message that illustrates the problem, when
email.message_from_*() is called. That interface uses the Parser API, which
splits reads into 8192 byte chunks. It so happens that for the test message,
the 8192 chunk falls inside a message/delivery-status, which is where in the
FeedParser the readline() call was that didn't check for NeedsMoreData.
I also added an assert to unreadline() so it'll be more evident if an attempt
to push back NeedsMoreData ever happens again.
Bump the email package version number.
in a newline, and it's an end boundary, the FeedParser wasn't recognizing it
as such. Tweak the regexp to make the ending linesep optional.
For grins, clear self._partial when closing the BufferedSubFile.
Added a test case.
capturing_preamble but we found a StartBoundaryNotFoundDefect, we need to
consume all lines from the current position to the EOF, which we'll set as the
epilogue of the current message. If we're not at EOF when we return from
here, the outer message's capturing_preamble assertion will fail.
calling .lower() on it. This fixes the problem described in SF patch # 866982
where in the tr_TR.ISO-8859-9 locale, 'I'.lower() isn't 'i'. unicodes are
locale insensitive.
Briefly (from the NEWS file):
- Updates for the email package:
+ All deprecated APIs that in email 2.x issued warnings have been removed:
_encoder argument to the MIMEText constructor, Message.add_payload(),
Utils.dump_address_pair(), Utils.decode(), Utils.encode()
+ New deprecations: Generator.__call__(), Message.get_type(),
Message.get_main_type(), Message.get_subtype(), the 'strict' argument to
the Parser constructor. These will be removed in email 3.1.
+ Support for Python earlier than 2.3 has been removed (see PEP 291).
+ All defect classes have been renamed to end in 'Defect'.
+ Some FeedParser fixes; also a MultipartInvariantViolationDefect will be
added to messages that claim to be multipart but really aren't.
+ Updates to documentation.
Specifically, time.strftime() no longer accepts a 0 in the yday position of a
time tuple, since that can crash some platform strftime() implementations.
parsedate_tz(): Change the return value to return 1 in the yday position.
Update tests in test_rfc822.py and test_email.py
\r\n only get the \n stripped, not the \r (unless it's the last header which
does get the \r stripped). Patch by Tony Meyer.
test_whitespace_continuation_last_header(),
test_strip_line_feed_and_carriage_return_in_headers(): New tests.
_parse_headers(): Be sure to strip \r\n from the right side of header lines.
parser must recognize outer boundaries in inner parts. So cruise through the
EOF stack backwards testing each predicate against the current line.
There's still some discussion about whether this is (always) the best thing to
do. Anthony would rather parse these messages as if the outer boundaries were
ignored. I think that's counter to the RFC, but might be practically more
useful. Can you say behavior flag? (ug).
Barry Warsaw
Martin v. Löwis
Georg Brandl
Anthony Baxter
Tim Peters