Commit Graph

189 Commits

Author SHA1 Message Date
R David Murray c4bb599be0 Merge: #23745: handle duplicate MIME parameter names in new parser. 2015-03-29 21:54:27 -04:00
R David Murray 7d0325d6c8 #23745: handle duplicate MIME parameter names in new parser.
This mimics get_param's error handling for the most part.  It is slightly
better in some regards as get_param can produce some really weird results for
duplicate *0* parts.  It departs from get_param slightly in that if we have a
mix of non-extended and extended pieces for the same parameter name, the new
parser assumes they were all supposed to be extended and concatenates all the
values, whereas get_param always picks the non-extended parameter value.  All
of this error recovery is pretty much arbitrary decisions...
2015-03-29 21:53:05 -04:00
Benjamin Peterson 60a2f49c8c merge 3.4 (#19996) 2015-01-25 23:31:58 -05:00
Benjamin Peterson 155ceaa454 handle headers with no key (closes #19996)
Patch by Cory Benfield.
2015-01-25 23:30:30 -05:00
Serhiy Storchaka 65ee4674e2 Issue #22777: Test pickling with all protocols. 2014-12-15 14:06:02 +02:00
Serhiy Storchaka bad1257c96 Issue #22777: Test pickling with all protocols. 2014-12-15 14:03:42 +02:00
R David Murray d2ff243b38 Merge: #21991: make headerregistry params property MappingProxyType. 2014-10-17 19:32:08 -04:00
R David Murray 685b3495e1 #21991: make headerregistry params property MappingProxyType.
It is unlikely anyone is using the fact that the dictionary returned
by the 'params' attribute was previously writable, but even if someone
is the API is provisional so this kind of change is acceptable (and
needed, to get the API "right" before it becomes official).

Patch by Stéphane Wirtel.
2014-10-17 19:30:13 -04:00
Berker Peksag fe21e4d4d7 Issue #16324: _charset parameter of MIMEText now also accepts email.charset.Charset instances.
Initial patch by Claude Paroz.
2014-09-27 00:57:29 +03:00
R David Murray 7c4093ca95 Merge: #21091: make is_attachment a method. 2014-09-20 18:16:39 -04:00
R David Murray 8a97896a76 #21091: make is_attachment a method.
Since EmailMessage is a provisional API we can fix API bugs in a
maintenance release, but I used a trick suggested by Serhiy to
maintain backward compatibility with 3.4.0/1.
2014-09-20 18:05:28 -04:00
R David Murray 97dfad7856 #21079: is_attachment now looks only at the value, ignoring parameters. 2014-09-20 17:44:53 -04:00
Serhiy Storchaka 4d83192ea0 Decreased memory requirements of new tests added in issue21448. 2014-08-12 20:22:48 +03:00
Serhiy Storchaka 320a1c0ff7 Issue #21448: Fixed FeedParser feed() to avoid O(N**2) behavior when parsing long line.
Original patch by Raymond Hettinger.
2014-08-12 13:59:11 +03:00
Ezio Melotti d577480197 #20977: fix undefined name in the email module. Patch by Rose Ames. 2014-08-04 17:16:49 +03:00
Zachary Ware f012ba42fe Issue #22002: Make full use of test discovery in test sub-packages.
Adds `load_package_tests` function to test.support, uses it in test_asyncio,
test_email, test_json, test_tools, test_importlib and all test_importlib
sub-packages to implement test discovery.
2014-07-23 12:00:29 -05:00
R David Murray c6772c4d59 #21476: Unwrap fp in BytesParser so the file isn't unexpectedly closed.
This makes the behavior match that of Parser.  Patch by Vajrasky Kok.
2014-06-26 13:31:43 -04:00
R David Murray b8bf9951ad #20871: improve email policy test coverage.
More tests of the concrete policy methods would probably be
a good idea, but this is a start, and it brings line coverage
up to 100% for the policy module.

Patch by Milan Oberkirch.
2014-03-09 15:29:24 -04:00
R David Murray 5dda12491e #11558: Better message if attach called on non-multipart.
Original patch by Varun Sharma.
2014-03-06 11:44:17 -05:00
R David Murray 66c3d18af1 Merge: #14983: always add a line end after a MIME boundary marker. 2014-02-08 17:56:17 -05:00
R David Murray e9c31470e1 #14983: always add a line end after a MIME boundary marker.
This is more RFC compliant (see issue) and fixes a problem with
signature verifiers rejecting the part when signed.  There is some
amount of backward compatibility concern here since it changes
the output, but the RFC issue coupled with fixing the problem
with signature verifiers seems worth the small risk of breaking
code that depends on the current incorrect output.
2014-02-08 17:54:56 -05:00
R David Murray 01e46ee7e2 Merge: #16983: Apply postel's law to encoded words inside quoted strings. 2014-02-08 13:13:01 -05:00
R David Murray 0400d33928 #16983: Apply postel's law to encoded words inside quoted strings.
This applies only to the new parser.  The old parser decodes encoded words
inside quoted strings already, although it gets the whitespace wrong
when it does so.

This version of the patch only handles the most common case (a single encoded
word surrounded by quotes), but I haven't seen any other variations of this in
the wild yet, so its good enough for now.
2014-02-08 13:12:00 -05:00
R David Murray ff9616bbf7 Merge #19772: Do not mutate message when downcoding to 7bit. 2014-02-08 11:51:18 -05:00
R David Murray 905c8c3d8d #19772: Do not mutate message when downcoding to 7bit.
This is a bit of an ugly hack because of the way generator pieces together the
output message.  The deepcopys aren't too expensive, though, because we know it
is only called on messages that are not multiparts, and the payload (the thing
that could be large) is an immutable object.

Test and preliminary work on patch by Vajrasky Kok.
2014-02-08 11:48:20 -05:00
R David Murray c489e83432 Merge: #17369: Improve handling of broken RFC2231 values in get_filename. 2014-02-07 15:04:26 -05:00
R David Murray 1e949890f6 #17369: Improve handling of broken RFC2231 values in get_filename.
This fixes a regression relative to python2.
2014-02-07 15:02:19 -05:00
R David Murray 15a693a6f8 #20531: Apply the 3.3 version of the #19063 fix.
So passing unicode to set_payload works again (but still doesn't
do what you want when the message is serialized).
2014-02-07 12:46:17 -05:00
R David Murray 27e9de669b #20531: Revert e20f98a8ed71, the 3.4 version of the #19063 fix. 2014-02-07 12:40:37 -05:00
R David Murray aa21297457 #20476: use EmailMessage as factory if non-compat32 policy is used.
In 3.5 I will fix this right by adding a message_factory attribute
to the policy.
2014-02-07 10:44:16 -05:00
R David Murray 44fcaae90d Merge #20206, #5803: more efficient algorithm that doesn't truncate output.
(No idea why test_tarfile is listed as changed...it isn't.)
2014-01-13 13:30:13 -05:00
R David Murray 2313e15578 #20206, #5803: more efficient algorithm that doesn't truncate output.
This fixes an edge case (20206) where if the input ended in a character
needing encoding but there was no newline on the string, the last byte
of the encoded character would be dropped.  The fix is to use a more
efficient algorithm, provided by Serhiy Storchaka (5803), that does not
have the bug.
2014-01-13 13:19:21 -05:00
Benjamin Peterson eaedaeccf2 update Barry's email (#19563) 2013-12-22 19:45:38 -06:00
Benjamin Peterson 8fb007f329 merge 3.3 (#19563) 2013-12-22 19:45:53 -06:00
R David Murray 775632ba10 #19957: Simplify encode_7or8bit now that _payload is always str.
Patch by Vajrasky Kok, test enhancement by me.
2013-12-12 21:40:20 -05:00
R David Murray 50bfbb9903 #19063: fix set_payload handling of non-ASCII string input.
This version of the fix raises an error instead of accepting the invalid
input (ie: if a non-ASCII string is used but no charset is specified).
2013-12-11 16:52:11 -05:00
R David Murray d5c4c7411a #19063: partially fix set_payload handling of non-ASCII string input.
This is a backward compatible partial fix, the complete fix requires raising
an error instead of accepting the invalid input, so the real fix is only
suitable for 3.4.
2013-12-11 16:34:34 -05:00
Serhiy Storchaka a7a34a83f3 Issue #19590: Use specific asserts in email tests. 2013-11-16 12:56:54 +02:00
Serhiy Storchaka 328cf3cbdf Issue #19590: Use specific asserts in email tests. 2013-11-16 12:56:23 +02:00
R David Murray 3da240fd01 #18891: Complete new provisional email API.
This adds EmailMessage and, MIMEPart subclasses of Message
with new API methods, and a ContentManager class used by
the new methods.  Also a new policy setting, content_manager.

Patch was reviewed by Stephen J. Turnbull and Serhiy Storchaka,
and reflects their feedback.

I will ideally add some examples of using the new API to the
documentation before the final release.
2013-10-16 22:48:40 -04:00
Terry Jan Reedy 7e7cf8bc51 Issue #12037: Fix test_email for desktop Windows. 2013-08-31 17:16:45 -04:00
Terry Jan Reedy 740d6b6f39 Issue #12037: Fix test_email for desktop Windows. 2013-08-31 17:12:21 -04:00
R David Murray b8c537094d Merge #18324: set_payload now correctly handles binary input. 2013-08-21 21:13:51 -04:00
R David Murray 00ae435dee #18324: set_payload now correctly handles binary input.
This also backs out the previous fixes for for #14360, #1717, and #16564.
Those bugs were actually caused by the fact that set_payload didn't decode to
str, thus rendering the model inconsistent.  This fix does mean the data
processed by the encoder functions goes through an extra encode/decode cycle,
but it means the model is always consistent.  Future API updates will provide
a better way to encode payloads, which will bypass this minor de-optimization.

Tests by Vajrasky Kok.
2013-08-21 21:10:31 -04:00
Ezio Melotti e0c69161bc #18741: merge with 3.3. 2013-08-17 16:13:22 +03:00
Ezio Melotti b5bc353b88 #18741: fix more typos. Patch by Févry Thibault. 2013-08-17 16:11:40 +03:00
Ezio Melotti a1e639a0f4 #18505: merge with 3.3. 2013-08-10 18:57:52 +03:00
Ezio Melotti 1c4810b57b #18505: fix duplicate name and remove duplicate test. Patch by Vajrasky Kok. 2013-08-10 18:57:12 +03:00
R David Murray bb17d2b857 #18600: add policy to add_string, and as_bytes and __bytes__ methods.
This was triggered by wanting to make the doctest in email.policy.rst pass;
as_bytes and __bytes__ are clearly useful now that we have BytesGenerator.
Also updated the Message docs to document the policy keyword that was
added in 3.3.
2013-08-09 16:15:28 -04:00
R David Murray 4d0f808c24 #18503: small cleanups in test_email.
Patch by Vajrasky Kok.
2013-07-25 12:15:28 -04:00
R David Murray 271ade87ac #18503: small cleanups in test_email.
Patch by Vajrasky Kok.
2013-07-25 12:11:55 -04:00
R David Murray 743e3d5139 Merge: Closes #18475: add unittest.main() to test_email/test_inversions. 2013-07-16 11:46:00 -04:00
R David Murray b5c9dfdab3 Closes #18475: add unittest.main() to test_email/test_inversions.
Patch by Vajrasky Kok.
2013-07-16 11:45:31 -04:00
R David Murray 48830035e5 Closes #18464: fix typo in test name. 2013-07-15 12:09:47 -04:00
R David Murray 7f93b36e73 Merge: Closes #18464: fix typo in test name. 2013-07-15 12:10:29 -04:00
R David Murray 1f9d24a18d Merge: #18431: Decode encoded words in atoms in new email parser. 2013-07-12 16:01:10 -04:00
R David Murray 923512f327 #18431: Decode encoded words in atoms in new email parser.
There is more to be done here in terms of accepting RFC invalid
input that some mailers accept, but this covers the valid
RFC places where encoded words can occur in structured headers.
2013-07-12 16:00:28 -04:00
R David Murray 63194a774e Merge: #18044: Fix parsing of encoded words of the form =?utf8?q?=XX...?= 2013-07-11 15:58:07 -04:00
R David Murray 65171b28e7 #18044: Fix parsing of encoded words of the form =?utf8?q?=XX...?=
The problem was I was only checking for decimal digits after the third '?',
not for *hex* digits :(.

This changeset also fixes a couple of comment typos, deletes an unused
function relating to encoded word parsing, and removed an invalid
'if' test from the folding function that was revealed by the tests
written to validate this issue.
2013-07-11 15:52:57 -04:00
Ezio Melotti e0a39de647 #18380: merge with 3.3. 2013-07-06 17:17:45 +02:00
Ezio Melotti 2a99d5df63 #18380: pass regex flags to the right argument. Patch by Valentina Mukhamedzhanova. 2013-07-06 17:16:04 +02:00
R David Murray c723da361a Merge #14360: make encoders.encode_quopri work. 2013-06-27 18:38:36 -04:00
R David Murray f6069f9f22 #14360: make encoders.encode_quopri work.
There were no tests for the encoders module.  encode_base64 worked
because it is the default and so got tested implicitly elsewhere, and
we use encode_7or8bit internally, so that worked, too.  I previously
fixed encode_noop, so this fix means that everythign in the encoders
module now works, hopefully correctly.  Also added an explicit test
for encode_base64.
2013-06-27 18:37:00 -04:00
Ezio Melotti d62a514386 #17877: merge with 3.3. 2013-05-08 11:16:33 +03:00
Ezio Melotti 6bcc0f1b51 #17877: skip test if the Olson's TZ database is missing. 2013-05-08 11:16:02 +03:00
R David Murray 5efee58014 Merge: #17431: Fix missing import of BytesFeedParser in email.parser. 2013-03-15 20:45:11 -04:00
R David Murray 8093d6f822 Merge: #17431: Fix missing import of BytesFeedParser in email.parser. 2013-03-15 20:42:29 -04:00
R David Murray 857b24b0d5 Merge: PEP8 fixup on previous patch, remove unused imports in test_email. 2013-03-07 18:17:19 -05:00
R David Murray 965794ed58 Merge: PEP8 fixup on previous patch, remove unused imports in test_email. 2013-03-07 18:16:47 -05:00
R David Murray 2e78cd9b5e Merge: #14645: Generator now emits correct linesep for all parts.
Previously the parts of the message retained whatever linesep they had on
read, which means if the messages weren't read in univeral newline mode, the
line endings could well be inconsistent.  In general sending it via smtplib
would result in them getting fixed, but it is better to generate them
correctly to begin with.  Also, the new send_message method of smtplib does
not do the fixup, so that method is producing rfc-invalid output without this
fix.
2013-03-07 17:31:21 -05:00
R David Murray addb0be63e Merge: #14645: Generator now emits correct linesep for all parts.
Previously the parts of the message retained whatever linesep they had on
read, which means if the messages weren't read in univeral newline mode, the
line endings could well be inconsistent.  In general sending it via smtplib
would result in them getting fixed, but it is better to generate them
correctly to begin with.  Also, the new send_message method of smtplib does
not do the fixup, so that method is producing rfc-invalid output without this
fix.
2013-03-07 16:43:58 -05:00
R David Murray 64634eb321 Merge: #17171: fix email.encoders.encode_7or8bit when applied to binary data. 2013-02-11 10:54:22 -05:00
R David Murray 66383b2e0a Merge: #17171: fix email.encoders.encode_7or8bit when applied to binary data. 2013-02-11 10:53:35 -05:00
R David Murray c9b4e60683 Merge: #16564: Fix regression in use of encoders.encode_noop with binary data. 2013-02-09 13:13:14 -05:00
R David Murray 6cb1d67eb3 Merge: #16564: Fix regression in use of encoders.encode_noop with binary data. 2013-02-09 13:10:54 -05:00
R David Murray c44911f49a Merge: #16948: Fix quopri encoding of non-latin1 character sets. 2013-02-05 11:34:39 -05:00
R David Murray e201e9d584 Merge: #16948: Fix quopri encoding of non-latin1 character sets. 2013-02-05 10:55:27 -05:00
R David Murray 2c4f6e8693 Merge #16811: Fix folding of headers with no value in provisional policies. 2013-02-04 15:25:06 -05:00
R David Murray 844b0e6971 #16811: Fix folding of headers with no value in provisional policies. 2013-02-04 15:22:53 -05:00
Ezio Melotti 3a62e45b97 Merge typo fixes from 3.3. 2013-01-27 06:20:51 +02:00
Ezio Melotti 3f5db3940f Fix a few typos and a double semicolon. Patch by Eitan Adler. 2013-01-27 06:20:14 +02:00
Andrew Svetlov f7a17b48d7 Replace IOError with OSError (#16715) 2012-12-25 16:47:37 +02:00
Victor Stinner 9d95cbeaf9 Close #15165: Fix test_email on Windows 64 bits
Use year 1990 instead of year 1970 to avoid issues with negative timestamps.
2012-11-05 02:06:13 +01:00
Georg Brandl 1aca31e8f3 Closes #15925: fix regression in parsedate() and parsedate_tz() that should return None if unable to parse the argument. 2012-09-22 09:03:56 +02:00
R David Murray ad2a7d528a Merge #15249: Mangle From lines correctly when body contains invalid bytes.
Fix by Colin Su.  Test by me, based on a test written by Petri Lehtinen.
2012-08-24 11:23:50 -04:00
Alexander Belopolsky 1f924d21d3 Added test for a bug fixed in issue #665194. 2012-08-22 22:06:37 -04:00
R David Murray b8687df653 #665194: Update email.utils.localtime to use astimezone, and fix bug.
The new code correctly handles historic changes in UTC offsets.
A test for this should follow.

Original patch by Alexander Belopolsky.
2012-08-22 21:34:00 -04:00
R David Murray 970bef295d Merge #15232: correctly mangle From lines in MIME preamble and epilogue 2012-07-22 21:53:54 -04:00
R David Murray 97f43c019f #15160: Extend the new email parser to handle MIME headers.
This code passes all the same tests that the existing RFC mime header
parser passes, plus a bunch of additional ones.

There are a couple of commented out tests where there are issues with the
folding.  The folding doesn't normally get invoked for headers parsed from
source, and the cases are marginal anyway (headers with invalid binary data)
so I'm not worried about them, but will fix them after the beta.

There are things that can be done to make this API even more convenient, but I
think this is a solid foundation worth having.  And the parser is a full RFC
parser, so it handles cases that the current parser doesn't.  (There are also
probably cases where it fails when the current parser doesn't, but I haven't
found them yet ;)

Oh, yeah, and there are some really ugly bits in the parser for handling some
'postel' cases that are unfortunately common.

I hope/plan to to eventually refactor a lot of the code in the parser which
should reduce the line count...but there is no escaping the fact that the
error recovery is welter of special cases.
2012-06-24 05:03:27 -04:00
Alexander Belopolsky 76935b9c8c Issue #14653: email.utils.mktime_tz() no longer relies on system
mktime() when timezone offest is supplied.
2012-06-21 20:48:23 -04:00
R David Murray 82ffabdfa4 #2658: Add test for issue fixed by fix for #1079. 2012-06-03 12:27:07 -04:00
R David Murray 07ea53cb21 #1079: Fix parsing of encoded words.
This is a behavior change: before this leading and trailing spaces were
stripped from ASCII parts, now they are preserved.  Without this fix we didn't
parse the examples in the RFC correctly, so I think breaking backward
compatibility here is justified.

Patch by Ralf Schlatterbeck.
2012-06-02 17:56:49 -04:00
R David Murray 1be413e366 Don't use metaclasses when class decorators can do the job.
Thanks to Nick Coghlan for pointing out that I'd forgotten about class
decorators.
2012-05-31 18:00:45 -04:00
R David Murray 56517e5cb9 Make parameterized tests in email less hackish.
Or perhaps more hackish, depending on your perspective.  But at least this
way it is now possible to run the individual tests using the unittest CLI.
2012-05-30 21:53:40 -04:00
R David Murray 026ba312d4 #10839: add new test file that was omitted from checkin 2012-05-29 12:31:11 -04:00
R David Murray 6bed342b58 Refactor test_email/test_pickleable and add tests for date headers 2012-05-28 21:09:04 -04:00
R David Murray a7c9ddb59c Regularize test_email/test_headerregistry's references to policy. 2012-05-28 20:22:37 -04:00
R David Murray d41595b920 Refactor test_email/test_defect_handling. 2012-05-28 20:14:10 -04:00
R David Murray 7ef3ff3f2e #12515: email now registers a defect if the MIME end boundary is missing.
This commit also restores the news item for 167256 that it looks like
Terry inadvertently deleted.  (Either that, or I don't understand
now merging works...which is equally possible.)
2012-05-27 22:20:42 -04:00
R David Murray 80e0aee95b #1672568: email now registers defects for base64 payload format errors.
Which also means that it is now producing *something* for any base64
payload, which is what leads to the couple of older test changes in
test_email.  This is a slightly backward incompatible behavior change,
but the new behavior is so much more useful than the old (you can now
*reliably* detect errors, and any program that was detecting errors by
sniffing for a base64 return from get_payload(decode=True) and then doing
its own error-recovery decode will just get the error-recovery decode
right away).  So this seems to me to be worth the small risk inherent
in this behavior change.

This patch also refactors the defect tests into a separate test file,
since they are no longer just parser tests.
2012-05-27 21:23:34 -04:00