Commit Graph

306 Commits

Author SHA1 Message Date
Barry Warsaw b8b57e75b2 A bunch of new tests, and updated tests for the email 3.0 FeedParser. 2004-05-09 03:16:03 +00:00
Anthony Baxter 39a0f04421 New parser. Next up, making the current parser use this parser 2004-03-22 00:33:28 +00:00
Thomas Wouters 1e1305491b test_email: comment out two fail-test cases that no longer fail with the new
parser -- for now. Failure behaviour of the new parser(s) will change in any
case, so this will be revisited later anyway.
2004-03-20 20:29:50 +00:00
Thomas Wouters 0813d76cb0 Merge in Anthony's new parser code, from the anthony-parser-branch:
> ----------------------------
> revision 1.20.4.4
> date: 2003/06/12 09:14:17;  author: anthonybaxter;  state: Exp;  lines: +13 -6
> preamble is None when missing, not ''.
> Handle a couple of bogus formatted messages - now parses my main testsuite.
> Handle message/external-body.
> ----------------------------
> revision 1.20.4.3
> date: 2003/06/12 07:16:40;  author: anthonybaxter;  state: Exp;  lines: +6 -4
> epilogue-processing is now the same as the old parser - the newline at the
> end of the line with the --endboundary-- is included as part of the epilogue.
> Note that any whitespace after the boundary is _not_ part of the epilogue.
> ----------------------------
> revision 1.20.4.2
> date: 2003/06/12 06:39:09;  author: anthonybaxter;  state: Exp;  lines: +6 -4
> message/delivery-status fixed.
> HeaderParser fixed.
> ----------------------------
> revision 1.20.4.1
> date: 2003/06/12 06:08:56;  author: anthonybaxter;  state: Exp;  lines: +163 -129
> A work-in-progress snapshot of the new parser. A couple of known problems:
>
> - first (blank) line of MIME epilogues is being consumed
> - message/delivery-status isn't quite right
>
> It still needs a lot of cleanup, but right now it parses a whole lot of
> badness that the old parser failed on. I also need to think about adding
> back the old 'strict' flag in some way.
> =============================================================================
2004-03-20 17:31:29 +00:00
Hye-Shik Chang 3e2a306920 Add CJK codecs support as discussed on python-dev. (SF #873597)
Several style fixes are suggested by Martin v. Loewis and
Marc-Andre Lemburg. Thanks!
2004-01-17 14:29:29 +00:00
Barry Warsaw 4a44293735 Fixes to support CJKCodecs as per SF bug #852347. Actually, this
patch removes dependencies on the old unsupported KoreanCodecs package
and the alternative JapaneseCodecs package.  Since both of those
provide aliases for their codecs, this removal just makes the generic
codec names work.

We needed to make slight changes to __init__() as well.

This will be backported to Python 2.3 when its branch freeze is over.
2003-12-30 16:52:25 +00:00
Barry Warsaw 339270e6f5 Bump version number to 2.5.5 2003-12-30 16:49:40 +00:00
Barry Warsaw b03136ad52 __init__(): The docstring was incorrect regarding how header wrapping
gets done when maxheaderlen <> 0.  The header really gets wrapped via
the email.Header.Header class, which has a more sophisticated
algorithm than just splitting on semi-colons.
2003-11-19 02:23:01 +00:00
Walter Dörwald f0dfc7ac5c Fix a bunch of typos in documentation, docstrings and comments.
(From SF patch #810751)
2003-10-20 14:01:56 +00:00
Barry Warsaw 2e31ce2514 test_get_param_with_semis_in_quotes(): Test case for SF bug #794466.
Backport candidate.
2003-09-03 04:10:52 +00:00
Barry Warsaw a74e868857 A fix for parsing parameters when there are semicolons inside the
quotes.  Fixes SF bug #794466, with the essential patch provided by
Stuart D. Gathman.  Specifically,

_parseparam(), _get_params_preserve(): Use the parsing function that
takes quotes into account, as given (essentially) in the bug report's
test program.

Backport candidate.
2003-09-03 04:08:13 +00:00
Barry Warsaw c86c24e44d Bump the version number for the standalone mimelib package. 2003-08-19 04:05:25 +00:00
Barry Warsaw 622d60b5a2 test_rfc2231_no_language_or_charset_in_filename(),
test_rfc2231_no_language_or_charset_in_boundary(),
test_rfc2231_no_language_or_charset_in_charset(): New tests for proper
decoding of some RFC 2231 headers.

Backport candidate (as was the Utils.py 1.25 change) to both Python
2.3.1 and 2.2.4 -- will do momentarily.
2003-08-19 03:54:24 +00:00
Barry Warsaw 6208369ff3 get_param(): Update the docstring to explain how CHARSET and LANGUAGE
can be None, and what to do in that situation.

get_filename(), get_boundary(), get_content_charset(): Make sure these
handle RFC 2231 headers without a CHARSET field.

Backport candidate (as was the Utils.py 1.25 change) to both Python
2.3.1 and 2.2.4 -- will do momentarily.
2003-08-19 03:53:02 +00:00
Barry Warsaw 0b6f0d8810 decode_rfc2231(): We need to urllib.unquote() the value even if the
charset and language fields are not present, e.g. as in:

    title*0="This%20is%20encoded"
2003-08-19 03:49:34 +00:00
Barry Warsaw 663219a8cc _make_boundary(): A minor optimization suggested by the Timbot. 2003-06-24 20:19:34 +00:00
Barry Warsaw efad5880ba AddressList.__str__(): Get rid of useless, and broken method. Closes
SF #753617.  Back port candidate (but low priority).
2003-06-13 21:16:06 +00:00
Barry Warsaw 6754d52521 get_payload(): Improve the TypeError message when the payload isn't of
the expected type.  In response to SF #751451.
2003-06-10 16:31:55 +00:00
Barry Warsaw db6888b7df _make_boundary(): Fix for SF bug #745478, broken boundary calculation
in some locales.  This code simplifies the boundary algorithm to use
randint() which is what we wanted anyway.

Bump package version to 2.5.3.

Backport candidate for Python 2.2.3
2003-05-29 19:39:33 +00:00
Barry Warsaw 362310df81 Bump version number 2003-05-08 03:34:58 +00:00
Barry Warsaw f8b3e1f76e A couple of new parsedate test cases. 2003-05-08 03:34:01 +00:00
Barry Warsaw b5dc39f02c parsedate_tz(): Be slightly more lenient when there's no day of the
week.  Patch given by Daniel Berlin in SF bug # 732761.  Also closes
SF bug # 727719.

Backport candidate.
2003-05-08 03:33:15 +00:00
Barry Warsaw 0822ff7cca Get rid of some hard coded tabs 2003-04-24 15:58:47 +00:00
Barry Warsaw 482c5f7eb7 as_string(): Added some text to the docstring to make it clear that
it's a convenience only and give hints on what to do for more
flexibility.
2003-04-18 23:04:35 +00:00
Barry Warsaw 7ba256f039 Fix a comment 2003-04-02 04:51:33 +00:00
Barry Warsaw 1a99cf045d Bump to version 2.5.1 2003-03-30 20:47:48 +00:00
Barry Warsaw 9c505ae3da test_whitespace_eater_unicode_2(): Test case for SF bug #710498. 2003-03-30 20:47:22 +00:00
Barry Warsaw ba1548a736 __unicode__(): Fix the logic for calculating whether to add a
separating space or not between encoded chunks.  Closes SF bug
#710498.
2003-03-30 20:46:47 +00:00
Barry Warsaw e159d584d2 Temporary bump of the version number. 2003-03-26 17:58:11 +00:00
Barry Warsaw cd7051f698 typed_subpart_iterator(): Fix these to use non-deprecated APIs,
i.e. get_content_maintype() and get_content_subtype().

Also, add True, False for Python 2.2.x where x < 2 compatibility.
2003-03-26 17:57:25 +00:00
Barry Warsaw 8af56778fd typed_subpart_iterator(): Fix these to use non-deprecated APIs,
i.e. get_content_maintype() and get_content_subtype().
2003-03-26 17:56:21 +00:00
Barry Warsaw 5fe9ead82c Email version 2.5 -- I will now backport this to Python 2.2.3. 2003-03-21 18:57:59 +00:00
Barry Warsaw 6613fb8412 _encode_chunks(): Throw out empty chunks. 2003-03-17 20:36:20 +00:00
Barry Warsaw 240754933e test_long_lines_with_different_header(): Another test from Jason. 2003-03-17 20:35:14 +00:00
Barry Warsaw ab75840cd0 test_getaddresses_nasty(): A test for mimelib SF bug # 697641. 2003-03-17 18:36:37 +00:00
Barry Warsaw fa348c876f getaddrlist(): Make sure this consumes all the data, and if there is
no address there (perhaps because of invalid characters, it appends
('', '') to the result set.

Closes mimelib SF bug # 697641.
2003-03-17 18:35:42 +00:00
Barry Warsaw ea8f6fa094 test_whitespace_eater_unicode(): Make this test Python 2.1 compatible. 2003-03-12 03:14:11 +00:00
Barry Warsaw ca53c12c8b Python 2.1 doesn't have True and False 2003-03-12 02:54:17 +00:00
Barry Warsaw f9e0bd8df8 Adjust tests for no newline appending to MIMEText.__init__()'s _text
argument.
2003-03-11 05:10:46 +00:00
Barry Warsaw df6c70b454 beta 1 2003-03-11 05:05:21 +00:00
Barry Warsaw bd757ba1ed Adjust tests for no newline appending to MIMEText.__init__()'s _text
argument.
2003-03-11 05:04:54 +00:00
Barry Warsaw cbec700b49 __init__(): Don't add a newline to _text if it doesn't already end in
one.  Possibly controversial.
2003-03-11 05:04:09 +00:00
Barry Warsaw 12dc230c00 body_line_iterator(): Accept optional decode argument, pass through to
Message.get_payload().
2003-03-11 04:41:35 +00:00
Barry Warsaw 08898499b2 get_payload(): Teach this about various uunencoded
Content-Transfer-Encodings
2003-03-11 04:33:30 +00:00
Barry Warsaw 3840b49d9c test_get_decoded_uu_payload(): A new test for
Content-Transfer-Encoding: x-uuencode
2003-03-11 04:31:37 +00:00
Barry Warsaw a2369928b5 specialsre, escapesre: In SF bug #663369, Matthew Woodcraft points out
that backslashes must be escaped in character sets.
2003-03-10 19:20:18 +00:00
Barry Warsaw a2e64702ca test_escape_backslashes(): A test for SF bug #663369 by Matthew Woodcraft. 2003-03-10 19:18:34 +00:00
Barry Warsaw 59e98ae1c5 _bdecode(): Remove redundant check. 2003-03-10 17:36:04 +00:00
Barry Warsaw 513af770d7 Fix base class 2003-03-10 17:00:43 +00:00
Barry Warsaw e1ff4bbce6 Use ndiffAssertEqual in a couple of places for better error reporting. 2003-03-10 16:59:34 +00:00
Barry Warsaw 21191d3e31 get_payload(): If we get a low-level binascii.Error when base64
decoding the payload, just return it as-is.
2003-03-10 16:13:14 +00:00
Barry Warsaw 3efb651ea3 test_broken_base64_payload(): Test for crash in low-level binascii
module when decoding a message with broken base64.
2003-03-10 16:09:51 +00:00
Barry Warsaw 5b8c69f11e _split_ascii() [method and function]: Don't join the lines just to
split them again.  Simply return them as chunk lists.

_encode_chunks(): Don't add more folding whitespace than necessary.
2003-03-10 15:14:08 +00:00
Barry Warsaw 796376338f test_another_long_multiline_header(): Yet another formatting test. 2003-03-10 15:11:29 +00:00
Barry Warsaw 33975eac3d _split_ascii(): lstrip the individual lines in the ascii split lines,
since we'll be adding our own continuation whitespace later.
2003-03-07 23:24:34 +00:00
Barry Warsaw 28ffcef4e6 test_long_unbreakable_lines_with_continuation(): Another funky example
from Jason Mastaler :)
2003-03-07 23:23:04 +00:00
Barry Warsaw 8e1e7f5468 decode_rfc2231(): RFC 2231 allows leaving out both the charset and
language without including any single quotes.
2003-03-07 22:46:41 +00:00
Barry Warsaw 21fcc4e287 test_rfc2231_no_language_or_charset(): RFC 2231 allows leaving out
both the charset and language without including any single quotes.
2003-03-07 22:45:55 +00:00
Barry Warsaw bf7e241397 whitespace normalization 2003-03-07 15:58:51 +00:00
Barry Warsaw ce6bf59b2d _write_headers(), _split_header(): All of the smarts for splitting
long header lines is now (properly) in the Header class.  So we no
longer need _split_header() and we'll just defer to Header.encode()
when we have a plain string.
2003-03-07 15:43:17 +00:00
Barry Warsaw 9f3fcd9c23 More internal refinements of the ascii splitting algorithm.
_encode_chunks(): Pass maxlinelen in instead of always using
self._maxlinelen, so we can adjust for shorter initial lines.
Pass this value through to _max_append().

encode(): Weave maxlinelen through to the _encode_chunks() call.

_split_ascii(): When recursively splitting a line on spaces
(i.e. lower level syntactic split), don't append the whole returned
string.  Instead, split it on linejoiners and extend the lines up to
the last line (for proper packing).  Calculate the linelen based on
the last element in the this list.
2003-03-07 15:39:37 +00:00
Barry Warsaw 82783e6f33 test_string_headerinst_eq(): Another Jason test :) 2003-03-07 15:35:47 +00:00
Tim Peters 2b4821347f Repaired a misleading comment Barry inherited from me. 2003-03-06 23:41:58 +00:00
Barry Warsaw bd836dfba3 _split_ascii(): In the clause where curlen + partlen > maxlen, if the
part itself is longer than maxlen, and we aren't already splitting on
whitespace, then we recursively split the part on whitespace and
append that to the this list.
2003-03-06 20:33:04 +00:00
Barry Warsaw f0d3585669 test_long_received_header(): Another test case for folding long
Received headers (first on semis then on whitespace), given by Jason
Mastaler.
2003-03-06 20:31:02 +00:00
Barry Warsaw c79ffb022f test_whitespace_eater_unicode(): Test of the last outstanding bug in
SF # 640110.
2003-03-06 16:11:14 +00:00
Barry Warsaw 4848805341 __unicode__(): When converting to a unicode string, we need to
preserve spaces in the encoded/unencoded word boundaries.  RFC 2047 is
ambiguous here, but most people expect the space to be preserved.
Really closes SF bug # 640110.
2003-03-06 16:10:30 +00:00
Barry Warsaw 28ffcb6f84 test_rfc2047_multiline(): Test case for SF bug #640110. 2003-03-06 06:38:29 +00:00
Barry Warsaw 671c3e6373 decode_header(): Typo when appending an unencoded chunk to the
previous unencoded chunk (e.g. when they appear on separate lines).
Closes the 2nd bug in SF #640110 (the first one's already been
fixed).
2003-03-06 06:37:42 +00:00
Barry Warsaw 10627ba9b8 Merge of the folding-reimpl-branch. Specific changes,
Update tests for email 2.5.
2003-03-06 05:41:07 +00:00
Barry Warsaw e899e51c06 Merge of the folding-reimpl-branch. Specific changes,
_split(): New implementation of ASCII line splitting which should do a
better job and not be subject to the various weird artifacts (bugs)
reported.  This should also do a better job of higher-level syntactic
splits by trying first to split on semis, then commas, then
whitespace.

Use a Timbot-ly binary search for optimal non-ASCII split points for
better packing of header lines.  This also lets us remove one
recursion call.  Don't pass in firstline, but instead pass in the
actual line length we're shooting for.  Also pass in the list of split
characters.

encode(): Pass in the list of split characters so applications can
have some control over what "higher level syntactic breaks" are.

Also,

decode_header(): Transform binascii.Errors which can occur when
decoding a base64 RFC 2047 header with bogus data, into an
email.Errors.HeaderParseError.  Closes SF bug #696712.
2003-03-06 05:39:46 +00:00
Barry Warsaw 0e4570bcb0 Merge of the folding-reimpl-branch. Specific changes,
Rename a constant.
2003-03-06 05:25:35 +00:00
Barry Warsaw 5c2f1536d0 Merge of the folding-reimpl-branch. Specific changes,
Remove a senseless comment.
2003-03-06 05:25:00 +00:00
Barry Warsaw 5d384ef069 Merge of the folding-reimpl-branch. Specific changes,
_handle_multipart(): Ensure that if the preamble exists but does not
end in a newline, a newline is still added.  Without this, the
boundary separator will end up on the preamble line, breaking the MIME
structure.

_make_boundary(): Handle differences in the decimal point character
based on the locale.
2003-03-06 05:22:02 +00:00
Barry Warsaw 784cf6ae88 Merge of the folding-reimpl-branch. Specific changes,
Charset: Alias __repr__ to __str__ for debugging.

header_encode(): When calling quopriMIME.header_encode(), set
maxlinelen=None so that the lower level function doesn't (also) try to
wrap/fold the line.
2003-03-06 05:16:29 +00:00
Barry Warsaw 0ed81c35a7 Merge of the folding-reimpl-branch. Specific changes,
_max_append(): Change the comparison so that the new string is
concatenated if it's less than or equal to the max length.

header_encode(): Allow for maxlinelen == None to mean, don't do any
line splitting.  This is because this module is mostly used by higher
level abstractions (Header.py) which already ensures line lengths.  We
do this in a cheapo way by setting the max_encoding to some insanely
<100k wink> large number.
2003-03-06 05:14:20 +00:00
Barry Warsaw 4e68a1ec6c CHARSETS, ALIASES, CODEC_MAP: SF feature request 633543, Korean
support and other charset defaults.  See also:

http://article.gmane.org/gmane.comp.python.mime.devel/250

(this just commits the last bit of the article that wasn't part of
email 2.4.3.)
2003-01-07 00:29:07 +00:00
Barry Warsaw 3d597812b6 Jack complained that on test_crlf_separation() was failing on MacOS9
because the test file, msg_26.txt which has \r\n line endings, was
getting munged by cvs, which knows to do line ending conversions for
text files.  But we want \r\n to be preserved on all platforms, so we
cvs admin'd the file to be -kb (binary), which means we have to open
the file in binary mode to preserve these line ends.  Hopefully this
will be the end of the thrashing on this issue (but probably not).

Test passes on *nix now, and Tim confirms it passes on Windows.  We'll
leave it to Jack to test MacOS.
2003-01-02 22:48:36 +00:00
Barry Warsaw 10ee7a7f15 test_bad_8bit_header(): Tests for optional argument `errors'. See SF
bug #648119.
2002-12-30 19:14:38 +00:00
Barry Warsaw f4fdff715a Header.__init__(), .append(): Add an optional argument `errors' which
is passed straight through to the unicode() and ustr.encode() calls.
I think it's the best we can do to address the UnicodeErrors in badly
encoded headers such as is described in SF bug #648119.
2002-12-30 19:13:00 +00:00
Barry Warsaw 72261c9dfb Actually, make this 2.5a1 since it will include API changes that may
need more vetting, and it will be included in Python 2.3a1.
2002-12-30 19:08:38 +00:00
Barry Warsaw 207d1c2065 Bump to 2.5 2002-12-30 17:45:41 +00:00
Barry Warsaw f29ffbdbf5 TestMIMEAudio.setUp(): Use the email package's copy of the audio test
file, needed because some binary distros (read RPMs) don't include the
test module in their standard Python package.  This eliminates an
external dependency and closes SF bug # 650441.
2002-12-30 17:45:02 +00:00
Barry Warsaw c99c08c764 A copy of the audio test file from Lib/test, needed because some
binary distros (read RPMs) don't include the test module in their
standard Python package.  This eliminates an external dependency and
closes SF bug # 650441.
2002-12-30 17:44:27 +00:00
Barry Warsaw ba97659f5f parsedate_tz(): Fix SF bug #552345, optional FWS between the comma and
the day in an RFC 2822 date.
2002-12-30 17:21:36 +00:00
Barry Warsaw 795833fbc6 test_parsedate_compact(): A test for optional FWS between the comma
and the day number in an RFC 2822 date specification.  See bug
#552345.
2002-12-30 17:20:53 +00:00
Barry Warsaw 5c8fef903d A code cleansing pass 2002-12-30 16:43:42 +00:00
Barry Warsaw 1fb22bb24f Port rfc822.py changes that didn't make it into this copy,
specifically that dots are allowed in obs-phrase.  This fixes parsing
of dots in realnames.
2002-12-30 16:21:07 +00:00
Barry Warsaw edb59c1ee8 test_name_with_dots(): A new test to ensure that we're implementing
RFC 2822's rules w.r.t. dots in the realname part of address fields.
2002-12-30 16:19:52 +00:00
Tim Peters 6578dc925f Whitespace normalization. 2002-12-24 18:31:27 +00:00
Barry Warsaw da2525ed2a parse(), _parseheaders(), _parsebody(): A fix for SF bug #633527,
where in lax parsing, the first non-header line after a header block
(e.g. the first line not containing a colon, and not a continuation),
can be treated as the first body line, even without the RFC mandated
blank line separator.

rfc822 had this behavior, and I vaguely remember problems with this,
but can't remember details.  In any event, all the tests still pass,
so I guess we'll find out. ;/

This patch works by returning the non-header, non-continuation line
from _parseheader() and using that as the first header line prepended
to fp.read() if given.  It's usually None.

We use this approach instead of trying to seek/tell the file-like
object.
2002-11-05 21:44:06 +00:00
Barry Warsaw a0a00761a5 test_no_separating_blank_line(): A test for SF bug #633527, no
separating blank line between a header block and body text.

Tests both lax and strict parsing.
2002-11-05 21:36:17 +00:00
Barry Warsaw 847fdbbe71 A message with no separating blank line between the headers and the
body.  A test message for SF bug #633527.
2002-11-05 21:29:47 +00:00
Barry Warsaw 48b0a1c603 test_text_plain_in_a_multipart_digest(): A test of the fix for SF bug
#631350, where a subobject in a multipart/digest isn't a
message/rfc822.
2002-11-05 21:04:52 +00:00
Barry Warsaw 5c9130ec46 _parsebody(): A fix for SF bug #631350, where a subobject in a
multipart/digest isn't a message/rfc822.  This is legal, but counter
to recommended practice in RFC 2046, $5.1.5.

The fix is to look at the content type after setting the default
content type.  If the maintype is then message or multipart, attach
the parsed subobject, otherwise use set_payload() to set the data of
the other object.
2002-11-05 20:54:37 +00:00
Barry Warsaw 00e6a02ef8 Test case, distilled from SF bug #631350, where a subobject in a
multipart/digest isn't a message/rfc822.  This is legal, but counter
to recommended practice in RFC 2046, $5.1.5.
2002-11-05 20:53:18 +00:00
Barry Warsaw 8f4dcbd3f6 Bump __version__ (yes, to 2.5 "minus") 2002-11-05 19:56:47 +00:00
Barry Warsaw 030ddf794f Jason Mastaler's patch to break the dependence on rfc822.py for the
address parsing routines.  Closes SF patch #613434.
2002-11-05 19:54:52 +00:00
Barry Warsaw 4111804548 test_body_encoding(): a new test for Charset.body_encode(), especially
one that tests the obscure bug reported in SF # 625509.
2002-10-21 05:43:58 +00:00
Barry Warsaw 34aa44538d test_body_encoding(): a new test 2002-10-21 05:31:08 +00:00