Commit Graph

725 Commits

Author SHA1 Message Date
Terry Jan Reedy 8b53559a89 Merge with 3.3, issue #17047: remove doubled words added in 3.3,
as reported by Serhiy Storchaka and Matthew Barnett.
2013-03-11 18:36:38 -04:00
Terry Jan Reedy 0f84764a09 Issue #17047: remove doubled words added in 3.3
as reported by Serhiy Storchaka and Matthew Barnett.
2013-03-11 18:34:00 -04:00
R David Murray 857b24b0d5 Merge: PEP8 fixup on previous patch, remove unused imports in test_email. 2013-03-07 18:17:19 -05:00
R David Murray 965794ed58 Merge: PEP8 fixup on previous patch, remove unused imports in test_email. 2013-03-07 18:16:47 -05:00
R David Murray b9534f4ed5 PEP8 fixup on previous patch, remove unused import in test_email. 2013-03-07 18:15:13 -05:00
R David Murray 2e78cd9b5e Merge: #14645: Generator now emits correct linesep for all parts.
Previously the parts of the message retained whatever linesep they had on
read, which means if the messages weren't read in univeral newline mode, the
line endings could well be inconsistent.  In general sending it via smtplib
would result in them getting fixed, but it is better to generate them
correctly to begin with.  Also, the new send_message method of smtplib does
not do the fixup, so that method is producing rfc-invalid output without this
fix.
2013-03-07 17:31:21 -05:00
R David Murray addb0be63e Merge: #14645: Generator now emits correct linesep for all parts.
Previously the parts of the message retained whatever linesep they had on
read, which means if the messages weren't read in univeral newline mode, the
line endings could well be inconsistent.  In general sending it via smtplib
would result in them getting fixed, but it is better to generate them
correctly to begin with.  Also, the new send_message method of smtplib does
not do the fixup, so that method is producing rfc-invalid output without this
fix.
2013-03-07 16:43:58 -05:00
R David Murray e67c6c545b #14645: Generator now emits correct linesep for all parts.
Previously the parts of the message retained whatever linesep they had on
read, which means if the messages weren't read in univeral newline mode, the
line endings could well be inconsistent.  In general sending it via smtplib
would result in them getting fixed, but it is better to generate them
correctly to begin with.  Also, the new send_message method of smtplib does
not do the fixup, so that method is producing rfc-invalid output without this
fix.
2013-03-07 16:38:03 -05:00
R David Murray 2940e71add #15220: simplify and speed up feedparser's line splitting.
Original patch submitted by QNX, modified for clarity by me (mostly comments).
QNX reports a 30% speed up in average email parsing time.
2013-02-13 21:17:13 -05:00
R David Murray 64634eb321 Merge: #17171: fix email.encoders.encode_7or8bit when applied to binary data. 2013-02-11 10:54:22 -05:00
R David Murray 66383b2e0a Merge: #17171: fix email.encoders.encode_7or8bit when applied to binary data. 2013-02-11 10:53:35 -05:00
R David Murray ec317a8985 #17171: fix email.encoders.encode_7or8bit when applied to binary data. 2013-02-11 10:51:28 -05:00
R David Murray c9b4e60683 Merge: #16564: Fix regression in use of encoders.encode_noop with binary data. 2013-02-09 13:13:14 -05:00
R David Murray 6cb1d67eb3 Merge: #16564: Fix regression in use of encoders.encode_noop with binary data. 2013-02-09 13:10:54 -05:00
R David Murray ceaa8b1d75 #16564: Fix regression in use of encoders.encode_noop with binary data. 2013-02-09 13:02:58 -05:00
R David Murray c44911f49a Merge: #16948: Fix quopri encoding of non-latin1 character sets. 2013-02-05 11:34:39 -05:00
R David Murray e201e9d584 Merge: #16948: Fix quopri encoding of non-latin1 character sets. 2013-02-05 10:55:27 -05:00
R David Murray f581b37200 #16948: Fix quopri encoding of non-latin1 character sets. 2013-02-05 10:49:49 -05:00
R David Murray 2c4f6e8693 Merge #16811: Fix folding of headers with no value in provisional policies. 2013-02-04 15:25:06 -05:00
R David Murray 844b0e6971 #16811: Fix folding of headers with no value in provisional policies. 2013-02-04 15:22:53 -05:00
Andrew Svetlov a191959849 Issue #16714: use 'raise' exceptions, don't 'throw'.
Patch by Serhiy Storchaka.
2012-12-18 21:27:16 +02:00
Andrew Svetlov 5b89840d9c Issue #16714: use 'raise' exceptions, don't 'throw'.
Patch by Serhiy Storchaka.
2012-12-18 21:26:36 +02:00
Andrew Svetlov 737fb89dd1 Issue #16714: use 'raise' exceptions, don't 'throw'.
Patch by Serhiy Storchaka.
2012-12-18 21:14:22 +02:00
Philip Jenvey 4993cc0a5b utilize yield from 2012-10-01 12:53:43 -07:00
Georg Brandl 1aca31e8f3 Closes #15925: fix regression in parsedate() and parsedate_tz() that should return None if unable to parse the argument. 2012-09-22 09:03:56 +02:00
R David Murray ad2a7d528a Merge #15249: Mangle From lines correctly when body contains invalid bytes.
Fix by Colin Su.  Test by me, based on a test written by Petri Lehtinen.
2012-08-24 11:23:50 -04:00
R David Murray 638d40b433 #15249: Mangle From lines correctly when body contains invalid bytes.
Fix by Colin Su.  Test by me, based on a test written by Petri Lehtinen.
2012-08-24 11:14:13 -04:00
Alexander Belopolsky f9bd9141c5 Issue #665194: Added a small optimization 2012-08-22 23:02:36 -04:00
R David Murray 097a1208bc #665194: fix variable name in exception code path.
It was correct in the original patch and I foobared it
when I restructured part of the code.
2012-08-22 21:52:31 -04:00
R David Murray b8687df653 #665194: Update email.utils.localtime to use astimezone, and fix bug.
The new code correctly handles historic changes in UTC offsets.
A test for this should follow.

Original patch by Alexander Belopolsky.
2012-08-22 21:34:00 -04:00
R David Murray 970bef295d Merge #15232: correctly mangle From lines in MIME preamble and epilogue 2012-07-22 21:53:54 -04:00
R David Murray 6a31bc6d81 #15232: correctly mangle From lines in MIME preamble and epilogue 2012-07-22 21:47:53 -04:00
R David Murray 97f43c019f #15160: Extend the new email parser to handle MIME headers.
This code passes all the same tests that the existing RFC mime header
parser passes, plus a bunch of additional ones.

There are a couple of commented out tests where there are issues with the
folding.  The folding doesn't normally get invoked for headers parsed from
source, and the cases are marginal anyway (headers with invalid binary data)
so I'm not worried about them, but will fix them after the beta.

There are things that can be done to make this API even more convenient, but I
think this is a solid foundation worth having.  And the parser is a full RFC
parser, so it handles cases that the current parser doesn't.  (There are also
probably cases where it fails when the current parser doesn't, but I haven't
found them yet ;)

Oh, yeah, and there are some really ugly bits in the parser for handling some
'postel' cases that are unfortunately common.

I hope/plan to to eventually refactor a lot of the code in the parser which
should reduce the line count...but there is no escaping the fact that the
error recovery is welter of special cases.
2012-06-24 05:03:27 -04:00
Alexander Belopolsky 76935b9c8c Issue #14653: email.utils.mktime_tz() no longer relies on system
mktime() when timezone offest is supplied.
2012-06-21 20:48:23 -04:00
Alexander Belopolsky a07548e97b Issue #14653: email.utils.mktime_tz() no longer relies on system
mktime() when timezone offest is supplied.
2012-06-21 20:34:09 -04:00
R David Murray 7636b19366 Merge: Update out of date docstring. 2012-06-17 15:27:21 -04:00
R David Murray 3ac8c780f4 Update out of date docstring. 2012-06-17 15:26:35 -04:00
R David Murray 6e50b699ac Now that Defects are Exception subclasses, call super.
The behavior of MessageDefect is legacy behavior.  The chances anyone is
actually using the undocumented 'line' attribute is low, but it costs
little to retain backward compatibility.  Although one of the costs is
having to restore normal exception behavior in HeaderDefect.  On the
other hand, I'll probably add some specialized behavior there later.
2012-06-08 22:45:46 -04:00
R David Murray 07ea53cb21 #1079: Fix parsing of encoded words.
This is a behavior change: before this leading and trailing spaces were
stripped from ASCII parts, now they are preserved.  Without this fix we didn't
parse the examples in the RFC correctly, so I think breaking backward
compatibility here is justified.

Patch by Ralf Schlatterbeck.
2012-06-02 17:56:49 -04:00
R David Murray 1be413e366 Don't use metaclasses when class decorators can do the job.
Thanks to Nick Coghlan for pointing out that I'd forgotten about class
decorators.
2012-05-31 18:00:45 -04:00
R David Murray abfc37491c #10839: raise an error on add of duplicate unique headers in new email policies
This feature was supposed to be part of the initial email6 checkin, but it got
lost in my big refactoring.

In this patch I'm not providing an easy way to turn off the errors, but they
only happen when a header is added programmatically, and it is almost never
the right thing to do to allow the duplicate to be added.  An application that
needs to add duplicates of unique headers can create a policy subclass to
allow it.
2012-05-29 09:14:44 -04:00
R David Murray 7ef3ff3f2e #12515: email now registers a defect if the MIME end boundary is missing.
This commit also restores the news item for 167256 that it looks like
Terry inadvertently deleted.  (Either that, or I don't understand
now merging works...which is equally possible.)
2012-05-27 22:20:42 -04:00
R David Murray 80e0aee95b #1672568: email now registers defects for base64 payload format errors.
Which also means that it is now producing *something* for any base64
payload, which is what leads to the couple of older test changes in
test_email.  This is a slightly backward incompatible behavior change,
but the new behavior is so much more useful than the old (you can now
*reliably* detect errors, and any program that was detecting errors by
sniffing for a base64 return from get_payload(decode=True) and then doing
its own error-recovery decode will just get the error-recovery decode
right away).  So this seems to me to be worth the small risk inherent
in this behavior change.

This patch also refactors the defect tests into a separate test file,
since they are no longer just parser tests.
2012-05-27 21:23:34 -04:00
R David Murray adbdcdbd95 #14925: email now registers a defect for missing header/body separator.
This patch also deprecates the MalformedHeaderDefect.  My best guess is that
this defect was rendered obsolete by a refactoring of the parser, and the
corresponding defect for the new parser (which this patch introduces) was
overlooked.
2012-05-27 20:45:01 -04:00
R David Murray ea9766897b Make headerregistry fully part of the provisional api.
When I made the checkin of the provisional email policy, I knew that
Address and Group needed to be made accessible from somewhere.  The more
I looked at it, though, the more it became clear that since this is a
provisional API anyway, there's no good reason to hide headerregistry as
a private API.  It was designed to ultimately be part of the public API,
and so it should be part of the provisional API.

This patch fully documents the headerregistry API, and deletes the
abbreviated version of those docs I had added to the provisional policy
docs.
2012-05-27 15:03:38 -04:00
R David Murray 36432ea783 Add '__all__' to _encoded_words and mark QByteMap as private. 2012-05-27 12:39:54 -04:00
R David Murray 032eed3c4a Recognize '<>' as a special case of an angle-addr in header_value_parser.
Although '<>' is invalid according to RFC 5322, SMTP uses it for various
things, and it sometimes ends up in email headers.  This patch changes
get_angle_addr to recognize it and just register a Defect instead of raising a
parsing error.
2012-05-26 14:31:12 -04:00
R David Murray d2d521eafd #665194: Add a localtime function to email.utils.
Without this function people would be tempted to use the other date functions
in email.utils to compute an aware localtime, and those functions are not as
good for that purpose as this code.  The code is Alexander Belopolsy's from
his proposed patch for issue 9527, with a fix (and additional tests) by Brian
K. Jones.
2012-05-25 23:22:59 -04:00
R David Murray dcaf2ece6c #12586: Fix a small oversight in the new email policy header setting code.
This is a danger of focusing on unit tests: sometimes you forget
to do the integration tests.
2012-05-25 22:53:12 -04:00
R David Murray 0b6f6c82b5 #12586: add provisional email policy with new header parsing and folding.
When the new policies are used (and only when the new policies are explicitly
used) headers turn into objects that have attributes based on their parsed
values, and can be set using objects that encapsulate the values, as well as
set directly from unicode strings.  The folding algorithm then takes care of
encoding unicode where needed, and folding according to the highest level
syntactic objects.

With this patch only date and time headers are parsed as anything other than
unstructured, but that is all the helper methods in the existing API handle.
I do plan to add more parsers, and complete the set specified in the RFC
before the package becomes stable.
2012-05-25 18:42:14 -04:00
R David Murray c27e52265b #14731: refactor email policy framework.
This patch primarily does two things: (1) it adds some internal-interface
methods to Policy that allow for Policy to control the parsing and folding of
headers in such a way that we can construct a backward compatibility policy
that is 100% compatible with the 3.2 API, while allowing a new policy to
implement the email6 API.  (2) it adds that backward compatibility policy and
refactors the test suite so that the only differences between the 3.2
test_email.py file and the 3.3 test_email.py file is some small changes in
test framework and the addition of tests for bugs fixed that apply to the 3.2
API.

There are some additional teaks, such as moving just the code needed for the
compatibility policy into _policybase, so that the library code can import
only _policybase.  That way the new code that will be added for email6
will only get imported when a non-compatibility policy is imported.
2012-05-25 15:01:48 -04:00
R David Murray 42243c4dca #14380: Make actual default match docs, fix __init__ order.
Éric pointed out that given that the default was documented as None, someone
would reasonably pass that to get the default behavior.  In fixing the code to
use None, I noticed that the change to _charset was being done after it had
already been passed to MIMENonMultipart.  The change to the test verifies that
the order is now correct.
2012-03-22 22:40:44 -04:00
R David Murray 8680bcc5db #14380: Have MIMEText defaults to utf-8 when passed non-ASCII unicode
Previously it would just accept the unicode, which would wind up as unicode in
the transfer-encoded message object, which is just wrong.

Patch by Jeff Knupp.
2012-03-22 22:17:51 -04:00
R David Murray bd53809ee7 #14344: fixed the repr of email.policy objects. 2012-03-17 15:11:59 -04:00
R David Murray 80e22b56d3 Merge #11686: add missing entries to email __all__ lists.
Original patch by Steffen Daode Nurpmeso
2012-03-16 22:46:14 -04:00
R David Murray 1b6c724df0 #11686: add missing entries to email __all__ lists.
Original patch by Steffen Daode Nurpmeso
2012-03-16 22:43:05 -04:00
R David Murray b53319f509 #12818: remove escaping of () in quoted strings in formataddr
The quoting of ()s inside quoted strings is allowed by the RFC, but is not
needed.  There seems to be no reason to add needless escapes.
2012-03-14 15:31:47 -04:00
R David Murray 8d8f110492 #14062: fix BytesParser handling of Header objects
This is a different fix than the 3.2 fix, but the new tests are the same.

This also affected smtplib.SMTP.send_message, which calls BytesParser.
2012-03-14 14:24:22 -04:00
R David Murray 9fd170e2d0 #14062: fix BytesParser handling of linesep for Header objects
This also affected smtplib.SMTP.send_message, which calls BytesParser.
2012-03-14 14:05:03 -04:00
R David Murray e2922835b0 Merge #14291: if a header has non-ascii unicode, default to CTE using utf-8
In Python2, if a unicode string was assigned as the value of a header,
email would automatically CTE encode it using the UTF8 charset.
This capability was lost in the Python3 translation, and this patch
restores it.

Patch by Ali Ikinci, assisted by R. David Murray.

I also added a fix for the mailbox test that was depending (with a comment
that it was a bad idea to so depend) on non-ASCII causing message_from_string
to raise an error.  It now uses support.patch to induce an error during
message serialization.
2012-03-14 03:03:27 -04:00
R David Murray 7441a7aedd #14291: if a header has non-ascii unicode, default to CTE using utf-8
In Python2, if a unicode string was assigned as the value of a header,
email would automatically CTE encode it using the UTF8 charset.
This capability was lost in the Python3 translation, and this patch
restores it.

Patch by Ali Ikinci, assisted by R. David Murray.

I also added a fix for the mailbox test that was depending (with a comment
that it was a bad idea to so depend) on non-ASCII causing message_from_string
to raise an error.  It now uses support.patch to induce an error during
message serialization.
2012-03-14 02:59:51 -04:00
Florent Xicluna e41f0de84d Merge 3.2, fix typos. 2011-11-11 19:39:25 +01:00
Florent Xicluna 992d9e081f Fix few typos. 2011-11-11 19:35:42 +01:00
R David Murray 875048bd4c #665194: support roundtripping RFC2822 date stamps in the email.utils module 2011-07-20 11:41:21 -04:00
R David Murray 749073af13 #1874: detect invalid multipart CTE and report it as a defect. 2011-06-22 13:47:53 -04:00
R David Murray e76ff4081a merge #11584: make Header and make_header handle binary unknown-8bit input 2011-06-18 13:02:42 -04:00
R David Murray e5e366c856 #11584: make Header and make_header handle binary unknown-8bit input
Analogous to the decode_header fix, this fix makes Header.append and
make_header correctly handle the unknown-8bit charset introduced by email5.1,
when the input to them is binary strings.  Previous to this fix the
make_header(decode_header(x)) == x invariant was broken in the face of the
unknown-8bit charset.
2011-06-18 12:57:28 -04:00
R David Murray 7df08379c6 merge #11584: make decode_header handle Header objects correctly
This updates 12e39cd7a0e4 (merge of b21fdfa0019c), which fixed this bug
incorrectly.
2011-06-18 12:32:27 -04:00
R David Murray 6bdb1769dc #11584: make decode_header handle Header objects correctly
This updates b21fdfa0019c, which fixed this bug incorrectly.
2011-06-18 12:30:55 -04:00
R David Murray 3edd22ac95 #11731: simplify/enhance parser/generator API by introducing policy objects.
This new interface will also allow for future planned enhancements
in control over the parser/generator without requiring any additional
complexity in the parser/generator API.

Patch reviewed by Éric Araujo and Barry Warsaw.
2011-04-18 13:59:37 -04:00
R David Murray f3299989a2 Merge: #11492: rewrite header folding algorithm. Less code, more passing tests. 2011-04-18 10:11:06 -04:00
R David Murray 01581ee0b7 #11492: rewrite header folding algorithm. Less code, more passing tests. 2011-04-18 10:04:34 -04:00
R David Murray a2860e8b51 Improve message.py test coverage to 100%.
coverage.py reports 99% on branch coverage, but that appears to be
a bug or limitation in coverage.py.
2011-04-16 09:20:30 -04:00
R David Murray 8437fe2708 Remove unused method from internal class. 2011-04-15 14:55:04 -04:00
R David Murray b35c850a3f #11684: Complete parser bytes interface by adding BytesHeaderParser
Patch by Steffen Daode Nurpmeso.
2011-04-13 16:46:05 -04:00
Ezio Melotti 5e610def7d Merge with 3.2. 2011-04-13 16:45:00 +03:00
Ezio Melotti a3d0be456c Merge with 3.1. 2011-04-13 16:44:18 +03:00
Ezio Melotti ce073cdac7 Fix typo in docstring. 2011-04-13 16:43:21 +03:00
R David Murray b30f1b4106 Merge: Add maxlinelen to docstring, delete obsolete wording 2011-04-12 15:02:07 -04:00
R David Murray 94f90dd4a1 Merge: Add maxlinelen to docstring, delete obsolete wording 2011-04-12 15:01:28 -04:00
R David Murray 308f14aeae Add maxlinelen to docstring, delete obsolete wording 2011-04-12 15:00:44 -04:00
R David Murray 7ede59d77a Merge #11492: fix header truncation on folding when there are runs of split chars.
Not a complete fix for this issue.
2011-04-07 21:00:33 -04:00
R David Murray 80221ed0c0 Merge #11492: fix header truncation on folding when there are runs of split chars.
Not a complete fix for this issue.
2011-04-07 20:56:31 -04:00
R David Murray e1292a25d8 #11492: fix header truncation on folding when there are runs of split chars.
Not a complete fix for this issue.
2011-04-07 20:54:03 -04:00
R David Murray b65df26830 Merge: Improve test coverage of _split_ascii method. 2011-04-07 20:40:01 -04:00
R David Murray 7da4db118e Improve test coverage of _split_ascii method. 2011-04-07 20:37:17 -04:00
R David Murray 8debacb51c #1690608: make formataddr RFC2047 aware.
Patch by Torsten Becker.
2011-04-06 09:35:57 -04:00
R David Murray a0b1c77a19 Merge #11605: don't use set/get_payload in feedparser; they do conversions. 2011-04-06 08:16:13 -04:00
R David Murray c5c1472895 #11605: don't use set/get_payload in feedparser; they do conversions.
Really the whole API needs to be gone over to restore the
separation of concerns; but that's what email6 is about.
2011-04-06 08:13:02 -04:00
R David Murray 86cc82e36f Remove the 'strict' argument to Parser, deprecated since 2.4. 2011-03-29 11:32:35 -04:00
R David Murray 961355a56a Merge #11584: Since __getitem__ returns headers, make decode_header handle them. 2011-03-25 15:31:52 -04:00
R David Murray 041015cc70 #11584: Since __getitem__ returns headers, make decode_header handle them.
Why I consider this a bug rather than an API change: the API change was
to Message, which didn't used to return Headers unless you added them
yourself.  Now it does (for 8bit binary header input), so decode_header
needs to be able to handle them.
2011-03-25 15:10:55 -04:00
R David Murray 73bd0448b9 Merge #11606: improved body_encode algorithm, no longer produces overlong lines 2011-03-24 12:28:39 -04:00
R David Murray c7d28be62f Merge #11606: improved body_encode algorithm, no longer produces overlong lines 2011-03-24 12:27:23 -04:00
R David Murray b938c8c253 #11606: improved body_encode algorithm, no longer produces overlong lines
Algorithm and initial patch by Michael Henry.
2011-03-24 12:19:26 -04:00
R David Murray 5839b9635c Merge #11590: fix quoprimime decode handling of empty strings and line endings. 2011-03-23 15:37:26 -04:00
R David Murray 523b41c4b3 Merge #11590: fix quoprimime decode handling of empty strings and line endings. 2011-03-23 15:27:04 -04:00
R David Murray cafd79d904 #11590: fix quoprimime decode handling of empty strings and line endings. 2011-03-23 15:25:55 -04:00
R David Murray 3d9e973abd Merge #11589: add additional tests for the email quoprimime module. 2011-03-23 14:27:18 -04:00
R David Murray ec1b5b88ee #11589: add additional tests for the email quoprimime module.
Patch by Michael Henry.
2011-03-23 14:19:05 -04:00
R David Murray 9aaba789b3 Correctly move email package tests to Lib/test. 2011-03-21 17:17:06 -04:00
R David Murray a215023b78 #11243: tests and fixes for handling of 'dirty data' in additional methods 2011-03-16 21:11:23 -04:00
R David Murray 4e4326829f Merge #11401 fix from 3.1. 2011-03-16 18:27:34 -04:00
R David Murray de91276a5c #11401: handle headers with no value. 2011-03-16 18:26:23 -04:00
R David Murray 78099bb153 Merge #9298 fix. 2011-03-16 16:13:07 -04:00
R David Murray 6d94bd470e #9298: restore proper folding of base64 encoded bodies.
Patch by Yves Dorfsman.
2011-03-16 15:52:22 -04:00
Ezio Melotti 3b3499ba69 #11565: Merge with 3.1. 2011-03-16 11:35:38 +02:00
Ezio Melotti 13925008dc #11565: Fix several typos. Patch by Piotr Kasprzyk. 2011-03-16 11:05:33 +02:00
R David Murray 4a0b60c527 Merge #11555 as_string doc fix from 3.1. 2011-03-15 14:02:53 -04:00
R David Murray 7dedcb4644 #11555: update doc for 3.x change to as_string mangle_from default. 2011-03-15 14:01:18 -04:00
R David Murray 106f8e3ea4 Remove obsolete comment. 2011-03-15 12:48:41 -04:00
R David Murray 56a9d7e3da #11554: reactivate test_email_codecs, and make it pass.
The fix is to charset.py, which was not doing the encoding to the
correct output character set when doing a body_encode for either
the shift-jis or euc-jp charsets.  There's also a fix for handling
a bytes input in encoders.py.

Patch by Michael Henry, comment changes by me.
2011-03-15 12:20:02 -04:00
Ezio Melotti 4969f709cc #11515: Merge with 3.1. 2011-03-15 05:59:46 +02:00
Ezio Melotti 42da663e6f #11515: fix several typos. Patch by Piotr Kasprzyk. 2011-03-15 05:18:48 +02:00
R David Murray 3a6152f3b3 Fix header encoding of long headers when using euc-jp and shift_jis.
When a header was long enough to need to be split across lines, the
input charset name was used instead of the output charset name in
the encoded words.  This make a difference only for the two charsets
above.
2011-03-14 21:13:03 -04:00
R David Murray cd37dfcfac Harmonize linesep docstrings, and fix the quoting of \r\n 2011-03-14 18:35:56 -04:00
Nick Coghlan ce4ee1f34d Merge from 3.2 2011-03-16 21:48:30 -04:00
R David Murray cfec350e63 Merge #11401 fix from 3.2. 2011-03-16 18:28:07 -04:00
R David Murray bb35299fcd Merge #9298 fix. 2011-03-16 16:14:43 -04:00
Ezio Melotti b88ed1549e #11565: Merge with 3.2. 2011-03-16 11:38:59 +02:00
R David Murray ab1dfd68e1 Merge #11555 as_string doc fix from 3.2. 2011-03-15 14:04:00 -04:00
R David Murray e1773cd3cb Merge obsolete comment removal from 3.2. 2011-03-15 12:49:33 -04:00
R David Murray df1d00a1e3 Merge #11554 test_email_codecs activation from 3.2. 2011-03-15 12:24:47 -04:00
Ezio Melotti 373089239b #11515: Merge with 3.2. 2011-03-15 06:03:08 +02:00
R David Murray 7d9129c147 Merge long Japanese header fix from 3.2. 2011-03-14 21:14:53 -04:00
R David Murray 4572572718 Merge linesep docstring changes from 3.2. 2011-03-14 18:39:41 -04:00
R David Murray accd1c040f #1162477: accept '.' in addition to ':' when parsing time in date header.
Some non-compliant MUAs use '.'s, so by the Postel Principle we should
accept them.  Patch by Thomas Herve.
2011-03-13 20:06:23 -04:00
Antoine Pitrou d1b1991fd4 Lib/email/test/data/msg_26.txt had its line endings wrong in the SVN repository.
(they were probably converted right in working copies through an SVN property)
2011-03-05 21:49:50 +01:00
Georg Brandl 49857f8a93 Add updated .hgeol file and fix newlines in the 3.2 branch. 2011-03-05 15:11:35 +01:00
Georg Brandl 2774310c27 Merged revisions 87627,87638,87739,87760,87771,87787,87984,87986,88108,88115,88144,88165,88329,88364-88365,88369-88370,88423-88424 via svnmerge from
svn+ssh://svn.python.org/python/branches/py3k

........
  r87627 | georg.brandl | 2011-01-02 15:23:43 +0100 (So, 02 Jan 2011) | 1 line

  #1665333: add more docs for optparse.OptionGroup.
........
  r87638 | georg.brandl | 2011-01-02 20:07:51 +0100 (So, 02 Jan 2011) | 1 line

  Fix code indentation.
........
  r87739 | georg.brandl | 2011-01-04 18:27:13 +0100 (Di, 04 Jan 2011) | 1 line

  Fix exception catching.
........
  r87760 | georg.brandl | 2011-01-05 11:59:48 +0100 (Mi, 05 Jan 2011) | 1 line

  Fix duplicate end tag.
........
  r87771 | georg.brandl | 2011-01-05 22:47:47 +0100 (Mi, 05 Jan 2011) | 1 line

  On Py3k, -tt and -3 are no-op and unsupported respectively.
........
  r87787 | georg.brandl | 2011-01-06 10:15:45 +0100 (Do, 06 Jan 2011) | 1 line

  Remove doc for nonexisting parameter.
........
  r87984 | georg.brandl | 2011-01-13 08:24:40 +0100 (Do, 13 Jan 2011) | 1 line

  Add semicolon for consistency.
........
  r87986 | georg.brandl | 2011-01-13 08:31:18 +0100 (Do, 13 Jan 2011) | 1 line

  Fix the example output of count().
........
  r88108 | georg.brandl | 2011-01-19 09:42:03 +0100 (Mi, 19 Jan 2011) | 1 line

  Suppress trailing spaces in table paragraphs.
........
  r88115 | georg.brandl | 2011-01-19 21:05:49 +0100 (Mi, 19 Jan 2011) | 1 line

  #10944: add c_bool to types table.
........
  r88144 | georg.brandl | 2011-01-22 23:06:24 +0100 (Sa, 22 Jan 2011) | 1 line

  #10983: fix several bugs in the _tunnel implementation that seem to have missed while porting between branches.  A unittest is needed!
........
  r88165 | georg.brandl | 2011-01-24 20:53:18 +0100 (Mo, 24 Jan 2011) | 1 line

  Typo fix.
........
  r88329 | georg.brandl | 2011-02-03 08:08:25 +0100 (Do, 03 Feb 2011) | 1 line

  Punctuation typos.
........
  r88364 | georg.brandl | 2011-02-07 13:10:46 +0100 (Mo, 07 Feb 2011) | 1 line

  #11138: fix order of fill and align specifiers.
........
  r88365 | georg.brandl | 2011-02-07 13:13:58 +0100 (Mo, 07 Feb 2011) | 1 line

  #8691: document that right alignment is default for numbers.
........
  r88369 | georg.brandl | 2011-02-07 16:30:45 +0100 (Mo, 07 Feb 2011) | 1 line

  Consistent heading spacing, and fix two typos.
........
  r88370 | georg.brandl | 2011-02-07 16:44:27 +0100 (Mo, 07 Feb 2011) | 1 line

  Spelling fixes.
........
  r88423 | georg.brandl | 2011-02-15 13:41:17 +0100 (Di, 15 Feb 2011) | 1 line

  Apply logging SocketHandler doc update by Vinay.
........
  r88424 | georg.brandl | 2011-02-15 13:44:43 +0100 (Di, 15 Feb 2011) | 1 line

  Remove editing slip.
........
2011-02-25 10:18:11 +00:00
R. David Murray 43b2f457a0 Merged revisions 87136,87221,87256,87337-87338,87571,87839,88164 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k


........
  r87136 | r.david.murray | 2010-12-08 17:53:00 -0500 (Wed, 08 Dec 2010) | 6 lines

  Have script_helper._assert_python strip refcount strings from stderr.

  This makes the output of the function and those that depend on it
  independent of whether or not they are being run under a debug
  build.
........
  r87221 | r.david.murray | 2010-12-13 19:55:46 -0500 (Mon, 13 Dec 2010) | 4 lines

  #10699: fix docstring for tzset: it does not take a parameter

  Thanks to Garrett Cooper for the fix.
........
  r87256 | r.david.murray | 2010-12-14 21:19:14 -0500 (Tue, 14 Dec 2010) | 2 lines

  #10705: document what the values of debuglevel are and mean.
........
  r87337 | r.david.murray | 2010-12-17 11:11:40 -0500 (Fri, 17 Dec 2010) | 2 lines

  #10559: provide instructions for accessing sys.argv when first mentioned.
........
  r87338 | r.david.murray | 2010-12-17 11:29:07 -0500 (Fri, 17 Dec 2010) | 2 lines

  #10454: clarify the compileall docs and help messages.
    [compileall.py changes not backported.]
........
  r87571 | r.david.murray | 2010-12-29 14:06:48 -0500 (Wed, 29 Dec 2010) | 2 lines

  Fix same typo in docs.
........
  r87839 | r.david.murray | 2011-01-07 16:57:25 -0500 (Fri, 07 Jan 2011) | 9 lines

  Fix formatting of values with embedded newlines when rfc2047 encoding

  Before this patch if a value being encoded had an embedded newline,
  the line following the newline would have no leading whitespace,
  and the whitespace it did have was encoded into the word.  Now
  the existing whitespace gets turned into a blank, the way it does
  in other header reformatting, and the _continuation_ws gets added
  at the beginning of the encoded line.
........
  r88164 | r.david.murray | 2011-01-24 14:34:58 -0500 (Mon, 24 Jan 2011) | 12 lines

  #10960: fix 'stat' links, link to lstat from stat, general tidy of stat doc.

  Original patch by Michal Nowikowski, with some additions and wording
  fixes by me.

  I changed the wording from 'Performs a stat system call' to 'Performs
  the equivalent of a stat system call', since on Windows there are no
  stat/lstat system calls involved.  I also extended Michal's breakout
  of the attributes into a list to the other paragraphs, and rearranged
  the order of the paragraphs in the 'stat' docs to make it flow
  better and put it in what I think is a more logical/useful order.
........
2011-02-11 03:13:19 +00:00
R. David Murray 7372a07fd0 #11019: Make BytesGenerator handle Message with None body.
Bug discovery and initial patch by Victor Stinner.
2011-01-26 21:21:32 +00:00
R. David Murray bdd2d93bfe Revert r88197. I'll refix correctly once there is a test. 2011-01-26 02:31:37 +00:00
Victor Stinner 3a7ee3ab5a Fix BytesGenerator._handle_text() if the message has no payload (None) 2011-01-26 00:39:19 +00:00
Victor Stinner 0c4cc559cb fix import in email.message_from_binary_file()
Parser => BytesParser
2011-01-25 23:15:47 +00:00
R. David Murray 389af00371 Merged revisions 87873 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r87873 | r.david.murray | 2011-01-08 21:35:24 -0500 (Sat, 08 Jan 2011) | 12 lines

  #5871: protect against header injection attacks.

  This makes Header.encode throw a HeaderParseError if it winds up
  formatting a header such that a continuation line has no leading
  whitespace and looks like a header.  Since Header accepts values
  containing newlines and preserves them (and this is by design), without
  this fix any program that took user input (say, a subject in a web form)
  and passed it to the email package as a header was vulnerable to header
  injection attacks.  (As far as we know this has never been exploited.)

  Thanks to Jakub Wilk for reporting this vulnerability.
........
2011-01-09 02:48:04 +00:00
R. David Murray 5b2d9ddf69 #5871: protect against header injection attacks.
This makes Header.encode throw a HeaderParseError if it winds up
formatting a header such that a continuation line has no leading
whitespace and looks like a header.  Since Header accepts values
containing newlines and preserves them (and this is by design), without
this fix any program that took user input (say, a subject in a web form)
and passed it to the email package as a header was vulnerable to header
injection attacks.  (As far as we know this has never been exploited.)

Thanks to Jakub Wilk for reporting this vulnerability.
2011-01-09 02:35:24 +00:00
R. David Murray 9253214fd9 #10686: recode non-ASCII headers to 'unknown-8bit' instead of ?s.
This applies only when generating strings from non-RFC compliant binary
input; it makes the existing recoding behavior more consistent (ie:
now no data is lost when recoding).
2011-01-07 23:25:30 +00:00
R. David Murray 6f0022d84a Fix formatting of values with embedded newlines when rfc2047 encoding
Before this patch if a value being encoded had an embedded newline,
the line following the newline would have no leading whitespace,
and the whitespace it did have was encoded into the word.  Now
the existing whitespace gets turned into a blank, the way it does
in other header reformatting, and the _continuation_ws gets added
at the beginning of the encoded line.
2011-01-07 21:57:25 +00:00
Georg Brandl e98628d60c Remove doc for nonexisting parameter. 2011-01-06 09:15:45 +00:00
R. David Murray f9844c8292 Merged revisions 87750 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r87750 | r.david.murray | 2011-01-04 20:39:32 -0500 (Tue, 04 Jan 2011) | 5 lines

  #10790: make append work when output codec is different from input codec

  There's still a bug here (the encode call shouldn't use the 'errors'
  paramter), but I'll fix that later.
........
2011-01-05 01:47:38 +00:00
R. David Murray 477efb3944 #10790: make append work when output codec is different from input codec
There's still a bug here (the encode call shouldn't use the 'errors'
paramter), but I'll fix that later.
2011-01-05 01:39:32 +00:00
R. David Murray 5723d225f8 Merged revisions 87567 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r87567 | r.david.murray | 2010-12-29 11:57:24 -0500 (Wed, 29 Dec 2010) | 2 lines

  Fix a comment typo and update another comment to match Python3 reality
........
2010-12-29 19:03:53 +00:00
R. David Murray 4c1da4c2a8 Fix a comment typo and update another comment to match Python3 reality 2010-12-29 16:57:24 +00:00
R. David Murray dfd7eb0ba2 #1693546: don't add quotes around RFC 2231 encoded values.
The RFC is bit hard to understand on this point, but the examples
clearly show that parameter values that are encoded according
to its charset/language rules don't have surrounding quotes, and
the ABNF does not allow for quotes.  So when we produce such
encoded values, we no longer add quotes.
2010-12-24 22:36:49 +00:00
R. David Murray 4a62e89728 #1155362: allow hh:mm:ss-uuuu like we allow hh:mm:ss+uuuu in parsedate_tz
Original patch by Thomas Herve.
2010-12-23 20:35:46 +00:00
R. David Murray 27c1914c1b Merged revisions 87415 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r87415 | r.david.murray | 2010-12-21 13:07:59 -0500 (Tue, 21 Dec 2010) | 4 lines

  Fix the change made for issue 1243654.

  Surprisingly, it turns out there was no test that exercised this code path.
........
2010-12-21 18:11:01 +00:00
R. David Murray 73a559d0c9 Fix the change made for issue 1243654.
Surprisingly, it turns out there was no test that exercised this code path.
2010-12-21 18:07:59 +00:00
R. David Murray 63563cdf9d #9286: Fix the rfc822 parser to preserve whitespace in address local part.
Such addresses are not RFC compliant except under the 'obsolete syntax'
rules, but before this fix the whitespace was dropped from the input,
concatenating the pieces.  That breaks one of the principles of the
email package, that of preserving the input as much as possible.
It also denies the application program the opportunity to apply its
own heuristics to interpretation of such non-compliant addresses.

It is possible users of the email package were depending on the local
part always being a single token, so this fix will not be backported.
2010-12-18 18:25:38 +00:00
R. David Murray ccb9d05b6c Merged revisions 87217 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r87217 | r.david.murray | 2010-12-13 18:51:19 -0500 (Mon, 13 Dec 2010) | 5 lines

  #1078919: make add_header automatically do RFC2231 encoding when needed.

  Also document the use of three-tuples if control of the charset
  and language is desired.
........
2010-12-13 23:57:01 +00:00
R. David Murray 7ec754b7da #1078919: make add_header automatically do RFC2231 encoding when needed.
Also document the use of three-tuples if control of the charset
and language is desired.
2010-12-13 23:51:19 +00:00
R. David Murray 0101a3a827 Merged revisions 87191 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r87191 | r.david.murray | 2010-12-12 15:06:19 -0500 (Sun, 12 Dec 2010) | 6 lines

  #243654: only create a new MIME boundary if we don't already have one.

  The rearranged code should do exactly what the old code did, but
  the new code avoids a potentially costly re computation in the case
  where a boundary already exists.
........
2010-12-12 20:28:13 +00:00
R. David Murray 5260a9bbdc #243654: only create a new MIME boundary if we don't already have one.
The rearranged code should do exactly what the old code did, but
the new code avoids a potentially costly re computation in the case
where a boundary already exists.
2010-12-12 20:06:19 +00:00
R. David Murray abb10753c5 Merged revisions 85146 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r85146 | r.david.murray | 2010-10-01 16:38:33 -0400 (Fri, 01 Oct 2010) | 3 lines

  Fix docstring typo.
........
2010-12-06 18:48:11 +00:00
R. David Murray c1b3ed522c Merged revisions 85130 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r85130 | r.david.murray | 2010-09-30 22:08:02 -0400 (Thu, 30 Sep 2010) | 2 lines

  Fix a couple spelling errors in comments and delete redundant __len__ def.
........
2010-12-06 18:39:32 +00:00
R. David Murray a0b44b5adb #8989: add 'domain' keyword to make_msgid.
Patch by Adrian von Bidder.
2010-12-02 21:47:19 +00:00
R. David Murray 719a449ba6 Fix TestBytesGeneratorIdempotent tests and a couple bugs they revealed.
The tests that were failing on (some) windows machines, where the
msg_XX.txt files used native \r\n lineseps are now also run on machines
that use \n natively, and conversely the \n tests are run on Windows.
The failing tests revealed one place where linesep needed to be added
to a flatten call in generator.  There was also another that the tests
didn't catch, so I added a test for that case as well.
2010-11-21 16:53:48 +00:00
Ezio Melotti 19f2aeba67 Merged revisions 86596 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r86596 | ezio.melotti | 2010-11-20 21:04:17 +0200 (Sat, 20 Nov 2010) | 1 line

  #9424: Replace deprecated assert* methods in the Python test suite.
........
2010-11-21 01:30:29 +00:00
Ezio Melotti b3aedd4862 #9424: Replace deprecated assert* methods in the Python test suite. 2010-11-20 19:04:17 +00:00
R. David Murray e5db2636f3 Improve TestBytesGeneratorIdempotent using by using linesep.
Also corrects a typo from a previous commit.  Unfortunately
this does *not* fix issue #10134.
2010-11-20 15:10:13 +00:00
Brett Cannon 384917a922 Properly close files used by test_email. 2010-10-29 23:08:36 +00:00
Brett Cannon 06407b35b1 Properly close a temporary TextIOWrapper in 'email'. 2010-10-29 23:08:13 +00:00
R. David Murray 8451c4b6e0 #1349106: add linesep argument to generator.flatten and header.encode. 2010-10-23 22:19:56 +00:00
R. David Murray 96fd54eaec #4661: add bytes parsing and generation to email (email version bump to 5.1.0)
The work on this is not 100% complete, but everything is present to
allow real-world testing of the code.  The only remaining major todo
item is to (hopefully!) enhance the handling of non-ASCII bytes in headers
converted to unicode by RFC2047 encoding them rather than replacing them with
'?'s.
2010-10-08 15:55:28 +00:00
R. David Murray 7f8199a997 Merged revisions 85179 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r85179 | r.david.murray | 2010-10-02 11:58:26 -0400 (Sat, 02 Oct 2010) | 6 lines

  #1050268: make parseaddr 'quote' the contents of quoted strings in addresses.

  Also made the doc string for email._parseaddr's 'quote' function more
  accurate; I'd love to make the function match the old docstring instead,
  but other code uses it according the existing semantics.
........
2010-10-02 16:04:44 +00:00
R. David Murray 5397e862e2 #1050268: make parseaddr 'quote' the contents of quoted strings in addresses.
Also made the doc string for email._parseaddr's 'quote' function more
accurate; I'd love to make the function match the old docstring instead,
but other code uses it according the existing semantics.
2010-10-02 15:58:26 +00:00
R. David Murray 70a9993bfd Fix docstring typo. 2010-10-01 20:38:33 +00:00
R. David Murray f9c957f2c6 Merged revisions 85142 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r85142 | r.david.murray | 2010-10-01 11:40:20 -0400 (Fri, 01 Oct 2010) | 5 lines

  #10004: in Q encoded word ignore '=xx' when xx is not valid hex.

  Bug report and fix by Thomas Guettler.
........
2010-10-01 15:45:48 +00:00
R. David Murray 31e984c39c #10004: in Q encoded word ignore '=xx' when xx is not valid hex.
Bug report and fix by Thomas Guettler.
2010-10-01 15:40:20 +00:00
R. David Murray d2c310f0b7 Fix a couple spelling errors in comments and delete redundant __len__ def. 2010-10-01 02:08:02 +00:00
R. David Murray 1061f18beb Merged revisions 84310 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r84310 | r.david.murray | 2010-08-24 20:45:55 -0400 (Tue, 24 Aug 2010) | 8 lines

  #1194222: make parsedate always return RFC2822 four character years.

  Two character years are now converted to four character years using
  the Posix standard rule (<68 == 2000, >=68==1900).  This makes the
  parsed date RFC2822 compliant even if the input is not.

  Patch and test by Jeffrey Finkelstein.
........
2010-08-25 01:55:24 +00:00
R. David Murray 219d1c8ae3 #1194222: make parsedate always return RFC2822 four character years.
Two character years are now converted to four character years using
the Posix standard rule (<68 == 2000, >=68==1900).  This makes the
parsed date RFC2822 compliant even if the input is not.

Patch and test by Jeffrey Finkelstein.
2010-08-25 00:45:55 +00:00
R. David Murray e06528c64b Merged revisions 83690 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r83690 | r.david.murray | 2010-08-03 18:14:10 -0400 (Tue, 03 Aug 2010) | 10 lines

  #3196: if needed pad a short base64 encoded word before trying to decode.

  The RFCs encourage following Postel's law: be liberal in what you accept.
  So if someone forgot to pad the base64 encoded word payload to an
  even four bytes, we add the padding before handing it to base64mime.decode.
  Previously, missing padding resulted in a HeaderParseError.

  Patch by Jason Williams.
........
2010-08-03 23:35:44 +00:00
R. David Murray c4e69cc1d8 #3196: if needed pad a short base64 encoded word before trying to decode.
The RFCs encourage following Postel's law: be liberal in what you accept.
So if someone forgot to pad the base64 encoded word payload to an
even four bytes, we add the padding before handing it to base64mime.decode.
Previously, missing padding resulted in a HeaderParseError.

Patch by Jason Williams.
2010-08-03 22:14:10 +00:00
Florent Xicluna f1046ca817 Issue #4770: Restrict binascii module to accept only bytes (as specified).
And fix the email package to encode to ASCII instead of ``raw-unicode-escape`` before ASCII-to-binary decoding.
2010-07-27 21:20:15 +00:00
R. David Murray 6d4a06c91e Merged revisions 82922 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r82922 | r.david.murray | 2010-07-16 21:19:57 -0400 (Fri, 16 Jul 2010) | 4 lines

  #1555570: correctly handle a \r\n that is split by the read buffer.

  Patch and test by Tony Nelson.
........
2010-07-17 01:28:04 +00:00
R. David Murray 45bf773f60 #1555570: correctly handle a \r\n that is split by the read buffer.
Patch and test by Tony Nelson.
2010-07-17 01:19:57 +00:00
R. David Murray 71df9d9216 Merged revisions 82011 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r82011 | r.david.murray | 2010-06-15 22:19:40 -0400 (Tue, 15 Jun 2010) | 17 lines

  Merged revisions 81675 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r81675 | r.david.murray | 2010-06-03 11:43:20 -0400 (Thu, 03 Jun 2010) | 10 lines

    #5610: use \Z not $ so we don't eat extra chars when body part ends with \r\n.

    If a body part ended with \r\n, feedparser, using '$' to terminate its
    search for the newline, would match on the \r\n, and think that it needed
    to strip two characters in order to account for the line end before the
    boundary.  That made it chop one too many characters off the end of
    the body part.  Using \Z makes the match correct.

    Patch and test by Tony Nelson.
  ........
................
2010-06-16 02:22:56 +00:00
R. David Murray 45e0e1444b Merged revisions 81675 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81675 | r.david.murray | 2010-06-03 11:43:20 -0400 (Thu, 03 Jun 2010) | 10 lines

  #5610: use \Z not $ so we don't eat extra chars when body part ends with \r\n.

  If a body part ended with \r\n, feedparser, using '$' to terminate its
  search for the newline, would match on the \r\n, and think that it needed
  to strip two characters in order to account for the line end before the
  boundary.  That made it chop one too many characters off the end of
  the body part.  Using \Z makes the match correct.

  Patch and test by Tony Nelson.
........
2010-06-16 02:19:40 +00:00
R. David Murray 99147c40b5 Merged revisions 81685 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r81685 | r.david.murray | 2010-06-04 12:11:08 -0400 (Fri, 04 Jun 2010) | 4 lines

  #4768: store base64 encoded email body parts as text, not binary.

  Patch and tests by Forest Bond.
........
2010-06-04 16:15:34 +00:00
R. David Murray 7da8f06df0 #4768: store base64 encoded email body parts as text, not binary.
Patch and tests by Forest Bond.
2010-06-04 16:11:08 +00:00
R. David Murray d2d08c6872 Merged revisions 81660 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r81660 | r.david.murray | 2010-06-02 21:58:28 -0400 (Wed, 02 Jun 2010) | 25 lines

  Fix Charset.body_encode to encode to output_charset before calling base64mime.

  This means that what gets encoded in base64 is the encoded version of the
  unicode payload.  This bug was revealed by a forward port of the tests from
  Issue 1368247, but the fix was completely different.

  Note that the merge is only of the tests, the doc changes were inappropriate
  since email5 expects unicode, not bytes.  I'm also not convinced that
  quopri works correctly in email5, but that's a different issue.

  Merged revisions 81658 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r81658 | r.david.murray | 2010-06-02 18:03:15 -0400 (Wed, 02 Jun 2010) | 9 lines

    #1368247: make set_charset/MIMEText automatically encode unicode _payload.

    Fixes (mysterious, to the end user) UnicodeErrors when using utf-8 as
    the charset and unicode as the _text argument.  Also makes the way in
    which unicode gets encoded to quoted printable for other charsets more
    sane (it only worked by accident previously).  The _payload now is encoded
    to the charset.output_charset if it is unicode.
  ........
................
2010-06-03 02:05:47 +00:00
R. David Murray 850fc85e69 Fix Charset.body_encode to encode to output_charset before calling base64mime.
This means that what gets encoded in base64 is the encoded version of the
unicode payload.  This bug was revealed by a forward port of the tests from
Issue 1368247, but the fix was completely different.

Note that the merge is only of the tests, the doc changes were inappropriate
since email5 expects unicode, not bytes.  I'm also not convinced that
quopri works correctly in email5, but that's a different issue.

Merged revisions 81658 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r81658 | r.david.murray | 2010-06-02 18:03:15 -0400 (Wed, 02 Jun 2010) | 9 lines

  #1368247: make set_charset/MIMEText automatically encode unicode _payload.

  Fixes (mysterious, to the end user) UnicodeErrors when using utf-8 as
  the charset and unicode as the _text argument.  Also makes the way in
  which unicode gets encoded to quoted printable for other charsets more
  sane (it only worked by accident previously).  The _payload now is encoded
  to the charset.output_charset if it is unicode.
........
2010-06-03 01:58:28 +00:00
Matthias Klose 06f018d54d Merged revisions 80795 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

........
  r80795 | barry.warsaw | 2010-05-05 18:18:31 +0200 (Mi, 05 Mai 2010) | 5 lines

  Bug 7755: audiotest.au is arguably copyrighted material, but definitely makes
  Debian unhappy.  The actual contents of the audio clip are unimportant, so
  replace it with something that we know is okay.  Guido likes woodpeckers.
........
2010-05-21 13:11:00 +00:00
R. David Murray f870d8736a Merged revisions 79996,80855 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r79996 | r.david.murray | 2010-04-12 10:48:58 -0400 (Mon, 12 Apr 2010) | 15 lines

  Merged revisions 79994 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r79994 | r.david.murray | 2010-04-12 10:26:06 -0400 (Mon, 12 Apr 2010) | 9 lines

    Issue #7472: ISO-2022 charsets now consistently use 7bit CTE.

    Fixed a typo in the email.encoders module so that messages output using
    an ISO-2022 character set will use a content-transfer-encoding of
    7bit consistently.  Previously if the input data had any eight bit
    characters the output data would get marked as 8bit even though it
    was actually 7bit.
  ........
................
  r80855 | r.david.murray | 2010-05-05 21:41:14 -0400 (Wed, 05 May 2010) | 24 lines

  Merged revisions 80800 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  It turns out that email5 (py3k), because it is using unicode for the
  payload, doesn't do the encoding to the output character set until later
  in the process.  Specifically, charset.body_encode no longer does the
  input-to-output charset conversion.  So the if test in the exception
  clause in encoders.encode_7or8bit really is needed in email5.

  So, this merge only merges the test, not the removal of the 'if'.

  ........
    r80800 | r.david.murray | 2010-05-05 13:31:03 -0400 (Wed, 05 May 2010) | 9 lines

    Issue #7472: remove unused code from email.encoders.encode_7or8bit.

    Yukihiro Nakadaira noticed a typo in encode_7or8bit that was trying
    to special case iso-2022 codecs.  It turns out that the code in
    question is never used, because whereas it was designed to trigger
    if the payload encoding was eight bit but its output encoding was
    7 bit, in practice the payload is always converted to the 7bit
    encoding before encode_7or8bit is called.  Patch by Shawat Anand.
  ........
................
2010-05-06 01:53:03 +00:00
R. David Murray e85200d555 Merged revisions 80800 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

It turns out that email5 (py3k), because it is using unicode for the
payload, doesn't do the encoding to the output character set until later
in the process.  Specifically, charset.body_encode no longer does the
input-to-output charset conversion.  So the if test in the exception
clause in encoders.encode_7or8bit really is needed in email5.

So, this merge only merges the test, not the removal of the 'if'.

........
  r80800 | r.david.murray | 2010-05-05 13:31:03 -0400 (Wed, 05 May 2010) | 9 lines

  Issue #7472: remove unused code from email.encoders.encode_7or8bit.

  Yukihiro Nakadaira noticed a typo in encode_7or8bit that was trying
  to special case iso-2022 codecs.  It turns out that the code in
  question is never used, because whereas it was designed to trigger
  if the payload encoding was eight bit but its output encoding was
  7 bit, in practice the payload is always converted to the 7bit
  encoding before encode_7or8bit is called.  Patch by Shawat Anand.
........
2010-05-06 01:41:14 +00:00
Barry Warsaw 83d8998390 Bug 7755: audiotest.au is arguably copyrighted material, but definitely makes
Debian unhappy.  The actual contents of the audio clip are unimportant, so
replace it with something that we know is okay.  Guido likes woodpeckers.
2010-05-05 16:18:31 +00:00
Ezio Melotti c30bb7d5ca Merged revisions 80359 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r80359 | ezio.melotti | 2010-04-22 14:57:12 +0300 (Thu, 22 Apr 2010) | 9 lines

  Merged revisions 80357 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r80357 | ezio.melotti | 2010-04-22 14:53:21 +0300 (Thu, 22 Apr 2010) | 1 line

    Rephrase comment.
  ........
................
2010-04-22 11:58:06 +00:00
Ezio Melotti c303c12160 Merged revisions 80357 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r80357 | ezio.melotti | 2010-04-22 14:53:21 +0300 (Thu, 22 Apr 2010) | 1 line

  Rephrase comment.
........
2010-04-22 11:57:12 +00:00
Ezio Melotti ba47a37194 Merged revisions 80353 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r80353 | ezio.melotti | 2010-04-22 14:29:27 +0300 (Thu, 22 Apr 2010) | 9 lines

  Merged revisions 80350 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r80350 | ezio.melotti | 2010-04-22 14:23:31 +0300 (Thu, 22 Apr 2010) | 1 line

    #8474: fix duplicate test in test_email.
  ........
................
2010-04-22 11:30:51 +00:00
Ezio Melotti 5e001f65da Merged revisions 80350 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r80350 | ezio.melotti | 2010-04-22 14:23:31 +0300 (Thu, 22 Apr 2010) | 1 line

  #8474: fix duplicate test in test_email.
........
2010-04-22 11:29:27 +00:00
R. David Murray 84ee3103a3 Merged revisions 80078 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r80078 | r.david.murray | 2010-04-14 14:59:18 -0400 (Wed, 14 Apr 2010) | 9 lines

  Merged revisions 80062 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r80062 | r.david.murray | 2010-04-13 16:57:40 -0400 (Tue, 13 Apr 2010) | 2 lines

    Issue #5277: Fix quote counting when parsing RFC 2231 encoded parameters.
  ........
................
2010-04-14 19:05:38 +00:00
R. David Murray d48739f32d Merged revisions 80062 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r80062 | r.david.murray | 2010-04-13 16:57:40 -0400 (Tue, 13 Apr 2010) | 2 lines

  Issue #5277: Fix quote counting when parsing RFC 2231 encoded parameters.
........
2010-04-14 18:59:18 +00:00
R. David Murray ef3d6bd25f Merged revisions 79994 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r79994 | r.david.murray | 2010-04-12 10:26:06 -0400 (Mon, 12 Apr 2010) | 9 lines

  Issue #7472: ISO-2022 charsets now consistently use 7bit CTE.

  Fixed a typo in the email.encoders module so that messages output using
  an ISO-2022 character set will use a content-transfer-encoding of
  7bit consistently.  Previously if the input data had any eight bit
  characters the output data would get marked as 8bit even though it
  was actually 7bit.
........
2010-04-12 14:48:58 +00:00
Georg Brandl 89fad14944 Merged revisions 78018,78035-78040,78042-78043,78046,78048-78052,78054,78059,78075-78080 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r78018 | georg.brandl | 2010-02-06 11:08:21 +0100 (Sa, 06 Feb 2010) | 1 line

  #7864: make deprecation notices a bit clearer.
........
  r78035 | georg.brandl | 2010-02-06 23:44:17 +0100 (Sa, 06 Feb 2010) | 1 line

  Fix duplicate import.
........
  r78036 | georg.brandl | 2010-02-06 23:49:47 +0100 (Sa, 06 Feb 2010) | 1 line

  Remove unused import.
........
  r78037 | georg.brandl | 2010-02-06 23:59:15 +0100 (Sa, 06 Feb 2010) | 1 line

  No need to assign the results of expressions used only for side effects.
........
  r78038 | georg.brandl | 2010-02-07 00:02:29 +0100 (So, 07 Feb 2010) | 1 line

  Add a missing import.
........
  r78039 | georg.brandl | 2010-02-07 00:06:24 +0100 (So, 07 Feb 2010) | 1 line

  Add missing imports.
........
  r78040 | georg.brandl | 2010-02-07 00:08:00 +0100 (So, 07 Feb 2010) | 1 line

  Fix a few UnboundLocalErrors in test_long.
........
  r78042 | georg.brandl | 2010-02-07 00:12:12 +0100 (So, 07 Feb 2010) | 1 line

  Add missing import.
........
  r78043 | georg.brandl | 2010-02-07 00:12:19 +0100 (So, 07 Feb 2010) | 1 line

  Remove duplicate test method.
........
  r78046 | georg.brandl | 2010-02-07 00:18:00 +0100 (So, 07 Feb 2010) | 1 line

  Fix various missing import/unbound name errors.
........
  r78048 | georg.brandl | 2010-02-07 00:23:45 +0100 (So, 07 Feb 2010) | 1 line

  We heard you like test failures so we put unbound locals in your test so that you can fail while you fail.
........
  r78049 | georg.brandl | 2010-02-07 00:33:33 +0100 (So, 07 Feb 2010) | 1 line

  Fix import/access for some identifiers.  _TestSharedCTypes does not seem to be executed?
........
  r78050 | georg.brandl | 2010-02-07 00:34:10 +0100 (So, 07 Feb 2010) | 1 line

  Fix more unbound locals in code paths that do not seem to be used.
........
  r78051 | georg.brandl | 2010-02-07 00:53:52 +0100 (So, 07 Feb 2010) | 1 line

  Add missing import when running these tests standalone.
........
  r78052 | georg.brandl | 2010-02-07 00:54:04 +0100 (So, 07 Feb 2010) | 1 line

  Add missing import when running these tests standalone.
........
  r78054 | georg.brandl | 2010-02-07 00:58:25 +0100 (So, 07 Feb 2010) | 1 line

  Add missing import.
........
  r78059 | georg.brandl | 2010-02-07 12:34:15 +0100 (So, 07 Feb 2010) | 1 line

  Use "regexp" consistently.
........
  r78075 | georg.brandl | 2010-02-07 13:16:12 +0100 (So, 07 Feb 2010) | 1 line

  Fix another duplicated test method.
........
  r78076 | georg.brandl | 2010-02-07 13:19:43 +0100 (So, 07 Feb 2010) | 1 line

  Fix wrong usage of "except X, Y:".
........
  r78077 | georg.brandl | 2010-02-07 13:25:50 +0100 (So, 07 Feb 2010) | 1 line

  Fix two redefined test methods.
........
  r78078 | georg.brandl | 2010-02-07 13:27:06 +0100 (So, 07 Feb 2010) | 1 line

  Fix a redefined test method.
........
  r78079 | georg.brandl | 2010-02-07 13:34:26 +0100 (So, 07 Feb 2010) | 1 line

  Add a minimal test for fnmatchcase().
........
  r78080 | georg.brandl | 2010-02-07 13:55:12 +0100 (So, 07 Feb 2010) | 1 line

  Remove duplicate test method.
........
2010-03-14 10:23:39 +00:00
R. David Murray 57a4b983a6 bdecode was already gone in email 5. This merge adds the test from
the trunk patch, and removes the last trace of bdecode, which was
a commented out call in message.py.

Merged revisions 78778 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r78778 | r.david.murray | 2010-03-07 21:04:06 -0500 (Sun, 07 Mar 2010) | 9 lines

  Issue #7143: get_payload used to strip any trailing newline from a
  base64 transfer-encoded payload *after* decoding it; it no longer does.
  email had a special method in utils, _bdecode, specifically to do this,
  so it must have served a purpose at some point, yet it is clearly wrong
  per RFC.  Fixed with Barry's approval, but no backport.  Email package
  minor version number is bumped, now version 4.0.1.

  Patch by Joaquin Cuenca Abela.
........
2010-03-08 02:17:03 +00:00
R. David Murray d0a04ff95e Merged revisions 78276 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r78276 | r.david.murray | 2010-02-20 23:39:40 -0500 (Sat, 20 Feb 2010) | 16 lines

  Merged revisions 78274 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  ........
    r78274 | r.david.murray | 2010-02-20 23:23:00 -0500 (Sat, 20 Feb 2010) | 9 lines

    Issue 7970: When email.Parser.Parser parses a MIME message of type
    message/rfc822 it turns it into an object whose body consists of
    a list containing a single Message object.  HeaderParser, on the
    other hand, just copies the body as a string.  Generator.flatten
    has a special handler for the message mime type that expected the
    body to be the one item list.  This fails if the message was parsed
    by HeaderParser.  So we now check to see if the body is a string
    first, and if so just we just emit it.
  ........
................
2010-02-21 04:48:18 +00:00
R. David Murray 57c45ac5e5 Merged revisions 78274 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r78274 | r.david.murray | 2010-02-20 23:23:00 -0500 (Sat, 20 Feb 2010) | 9 lines

  Issue 7970: When email.Parser.Parser parses a MIME message of type
  message/rfc822 it turns it into an object whose body consists of
  a list containing a single Message object.  HeaderParser, on the
  other hand, just copies the body as a string.  Generator.flatten
  has a special handler for the message mime type that expected the
  body to be the one item list.  This fails if the message was parsed
  by HeaderParser.  So we now check to see if the body is a string
  first, and if so just we just emit it.
........
2010-02-21 04:39:40 +00:00
R. David Murray fa606926d6 Merged revisions 77542 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/branches/py3k

................
  r77542 | r.david.murray | 2010-01-16 13:30:03 -0500 (Sat, 16 Jan 2010) | 18 lines

  Merged revisions 77517,77525 via svnmerge from
  svn+ssh://pythondev@svn.python.org/python/trunk

  Merge adds an additional test for as_string with a maxheaderlen specified.

  ........
    r77517 | r.david.murray | 2010-01-16 00:15:17 -0500 (Sat, 16 Jan 2010) | 6 lines

    Issue #1670765: Prevent email.generator.Generator from re-wrapping
    headers in multipart/signed MIME parts, which fixes one of the sources of
    invalid modifications to such parts by Generator.  Patch and tests by
    Martin von Gagern.
  ........
    r77525 | r.david.murray | 2010-01-16 11:08:32 -0500 (Sat, 16 Jan 2010) | 2 lines

    Fix issue number in comment.
  ........
................
2010-01-16 18:41:00 +00:00
R. David Murray a8f480f545 Merged revisions 77517,77525 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

Merge adds an additional test for as_string with a maxheaderlen specified.

........
  r77517 | r.david.murray | 2010-01-16 00:15:17 -0500 (Sat, 16 Jan 2010) | 6 lines

  Issue #1670765: Prevent email.generator.Generator from re-wrapping
  headers in multipart/signed MIME parts, which fixes one of the sources of
  invalid modifications to such parts by Generator.  Patch and tests by
  Martin von Gagern.
........
  r77525 | r.david.murray | 2010-01-16 11:08:32 -0500 (Sat, 16 Jan 2010) | 2 lines

  Fix issue number in comment.
........
2010-01-16 18:30:03 +00:00